Skip to content

Commit

Permalink
Add gbasf2 process setting for disabling the log download from grid
Browse files Browse the repository at this point in the history
Having an automatic download of logs (to be precise: the "job output sandbox")
from the grid is nice, but currently that is quite slow. And people might just
want to use Dirac-WebApp for checking the output of failed jobs.
  • Loading branch information
meliache committed Feb 8, 2021
1 parent bdf97db commit 8587bfa
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 3 deletions.
9 changes: 7 additions & 2 deletions b2luigi/batch/processes/gbasf2.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,9 @@ def output(self):
as they don't need the output locally but take the grid datasets as input. Also useful when you just want
to produce data on the grid for other people to use.
- ``gbasf2_download_logs``: Whether to automatically download the log output of gbasf2 projects when the
task succeeds or fails. Having the logs is important for reproducibility, but k
The following optional settings correspond to the equally named ``gbasf`` command line options
(without the ``gbasf_`` prefix) that you can set to customize your gbasf2 project:
Expand Down Expand Up @@ -300,7 +303,8 @@ def _on_first_success_action(self):
"""
Things to do after all jobs in the project had been successful, e.g. downloading the dataset and logs
"""
self._download_logs()
if get_setting("gbasf2_download_logs", default=True, task=self.task):
self._download_logs()
if get_setting("gbasf2_download_dataset", default=True, task=self.task):
self._download_dataset()

Expand All @@ -313,7 +317,8 @@ def _on_failure_action(self):
if job_info["Status"] == "Failed"}
n_failed = len(failed_job_dict)
print(f"{n_failed} failed jobs:\n{failed_job_dict}")
self._download_logs()
if get_setting("gbasf2_download_logs", default=True, task=self.task):
self._download_logs()

def _reschedule_failed_jobs(self):
"""
Expand Down
3 changes: 2 additions & 1 deletion examples/gbasf2/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@
"gbasf2_install_directory": "~/gbasf2KEK",
"gbasf2_print_status_updates": true,
"gbasf2_max_retries": 0,
"gbasf2_cputime": 5
"gbasf2_cputime": 5,
"gbasf2_download_logs": true
}

0 comments on commit 8587bfa

Please sign in to comment.