Merge pull request #181 from nils-braun/bugfix/gbasf2-correctly-handl…

…e-multi-extension-outputs

Fix queries for gbasf2 grid files with multiple filename extensions, like `.udst.root` and `.mdst.root`

.pre-commit-config.yaml

@@ -1,7 +1,7 @@
 # basic checks
 repos:
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v4.3.0
+  rev: v4.4.0
   hooks:
   - id: check-yaml
   - id: check-json
@@ -13,7 +13,7 @@ repos:
   - id: check-symlinks
   - id: no-commit-to-branch
 
-- repo: https://gitlab.com/pycqa/flake8
+- repo: https://github.com/pycqa/flake8
   rev: 5.0.4
   hooks:
   - id: flake8

CHANGELOG.md

@@ -5,6 +5,13 @@ Older entries have been generated from github releases.
 New entries aim to adhere to the format proposed by [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+### Unreleased
+
+### Fixed
+
+* **gbasf2**: Fix gbasf2 glob queries (e.g. for downloading) for basf2 output files with multiple extensions, e.g. `<file>.udst.root` `<file>.mdst.root`. [#180](https://github.com/nils-braun/b2luigi/pull/181). Thanks [@schmitca](https://github.com/schmitca) for reporting.
+
+
 ## [0.8.1]  - 2022-11-14
 
 ### Fixed

b2luigi/batch/processes/gbasf2.py

@@ -583,21 +583,25 @@ def _get_gbasf2_dataset_query(self, output_file_name: str) -> str:
         """
         if output_file_name != os.path.basename(output_file_name):
             raise ValueError(
-                f"For grid projects, the output file name must not be a basename, not a path, but is \"{output_file_name}\""
+                f'For grid projects, the output file name must be a basename, not a path, but is "{output_file_name}"'
             )
-        output_file_stem, output_file_ext = os.path.splitext(output_file_name)
-        if output_file_ext != ".root":
+        # split file basename into stem and all extensions (e.g. ".udst.root")
+        output_file_stem, output_file_extensions = output_file_name.split(
+            ".", maxsplit=1
+        )
+        # make sure that last extension is ".root"
+        if os.path.splitext(output_file_extensions)[-1] != ".root":
             raise ValueError(
-                f"Output file name \"{output_file_name}\" does not end with \".root\", "
+                f'Output file name "{output_file_name}" does not end with ".root", '
                 "but gbasf2 batch only supports root outputs"
             )
         output_lpn_dir = f"/belle/user/{self.dirac_user}"
         group_name = get_setting("gbasf2_proxy_group", default="belle")
         if group_name != "belle":
             output_lpn_dir = get_setting("gbasf2_project_lpn_path")
-        dataset_query_string = \
-            f"{output_lpn_dir}/{self.gbasf2_project_name}/sub*/{output_file_stem}_*{output_file_ext}"
-        return dataset_query_string
+        return '.'.join(
+            [f'{output_lpn_dir}/{self.gbasf2_project_name}/sub*/{output_file_stem}_*', output_file_extensions]
+        )
 
     def _local_gb2_dataset_is_complete(self, output_file_name: str, check_temp_dir: bool = False) -> bool:
         """

docs/index.rst

@@ -134,6 +134,7 @@ Features, fixing, help and testing
     * Moritz Bauer (`sognetic`_)
     * Matthias Schnepf  (`mschnepf`_)
     * Artur Gottmann (`ArturAkh`_)
+    * Caspar Schmitt (`schmitca`_)
 
 Stolen ideas
     * Implementation of SGE batch system (`sge`_).
@@ -155,6 +156,7 @@ Stolen ideas
 .. _`sognetic`: https://github.com/sognetic
 .. _`ArturAkh`: https://github.com/ArturAkh
 .. _`mschnepf`: https://github.com/mschnepf
+.. _`schmitca`: https://github.com/schmitca
 .. _`sge`: https://github.com/spotify/luigi/blob/master/luigi/contrib/sge.py
 .. _`lsf`: https://github.com/spotify/luigi/pull/2373/files