Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

webhook-handler: ignore install times for failed jobs #1015

Merged
merged 2 commits into from
Dec 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/custom_docker_builds.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
- docker-image: ./images/cache-indexer
image-tags: ghcr.io/spack/cache-indexer:0.0.3
- docker-image: ./analytics
image-tags: ghcr.io/spack/django:0.3.27
image-tags: ghcr.io/spack/django:0.3.28
- docker-image: ./images/ci-prune-buildcache
image-tags: ghcr.io/spack/ci-prune-buildcache:0.0.4
- docker-image: ./images/protected-publish
Expand Down
3 changes: 2 additions & 1 deletion analytics/analytics/job_processor/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -140,5 +140,6 @@ def process_job(job_input_data_json: str):

# Create build timing facts in a separate transaction, in case this fails
with transaction.atomic():
if job.job.job_type == JobDataDimension.JobType.BUILD:
job_data = job.job
if job_data.job_type == JobDataDimension.JobType.BUILD and job_data.status == 'success':
create_build_timing_facts(job_fact=job, gljob=gl_job)
8 changes: 7 additions & 1 deletion analytics/analytics/job_processor/artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@
from gitlab.v4.objects import ProjectJob


class JobArtifactDownloadFailed(Exception):
def __init__(self, job: ProjectJob) -> None:
message = f"Job {job.id} artifact download failed"
super().__init__(message)


class JobArtifactFileNotFound(Exception):
def __init__(self, job: ProjectJob, filename: str):
message = f"File {filename} not found in job artifacts of job {job.id}"
Expand Down Expand Up @@ -37,7 +43,7 @@ def get_job_artifacts_file(job: ProjectJob, filename: str):
with open(artifacts_file, "wb") as f:
job.artifacts(streamed=True, action=f.write)
except GitlabGetError:
raise JobArtifactFileNotFound(job, filename)
raise JobArtifactDownloadFailed(job)

# Open specific file within artifacts zip
with zipfile.ZipFile(artifacts_file) as zfile:
Expand Down
3 changes: 2 additions & 1 deletion analytics/analytics/job_processor/metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from gitlab.v4.objects import ProjectJob

from analytics.job_processor.artifacts import (
JobArtifactDownloadFailed,
JobArtifactFileNotFound,
JobArtifactVariablesNotFound,
get_job_artifacts_data,
Expand Down Expand Up @@ -180,7 +181,7 @@ def retrieve_job_info(gljob: ProjectJob, is_build: bool) -> JobInfo:
# If the build is failed, this is not unexpected. Otherwise, raise the error
try:
artifacts = get_job_artifacts_data(gljob)
except (JobArtifactFileNotFound, JobArtifactVariablesNotFound):
except (JobArtifactDownloadFailed, JobArtifactFileNotFound, JobArtifactVariablesNotFound):
if gljob.status == "failed":
return JobInfo()

Expand Down
4 changes: 2 additions & 2 deletions k8s/production/custom/webhook-handler/deployments.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ spec:
serviceAccountName: webhook-handler
containers:
- name: webhook-handler
image: ghcr.io/spack/django:0.3.27
image: ghcr.io/spack/django:0.3.28
imagePullPolicy: Always
resources:
requests:
Expand Down Expand Up @@ -146,7 +146,7 @@ spec:
serviceAccountName: webhook-handler
containers:
- name: webhook-handler-worker
image: ghcr.io/spack/django:0.3.27
image: ghcr.io/spack/django:0.3.28
command:
[
"celery",
Expand Down
Loading