Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

manual deleting for volume (PV) causes the snapshot pod to fail #98

Open
halradaideh opened this issue Jun 8, 2020 · 2 comments
Open

Comments

@halradaideh
Copy link

i noticed that when i deleted a namespace volumes with it's snapshots manually from GCP console

this caused the tool to fail. i suggest to pass volumes that don't exists and continue the tool work with some error message, the BUG is FETAL for the tool and it cause it to enter crush loop back.

oogleapis.com/compute/v1/projects/X/zones/us-central1-a/disks/gke-X-d039b866-d-pvc-9749b085-9548-11ea-8048-4201ac100009/createSnapshot?alt=json returned "The resource \'projects/X/zones/us-central1-a/disks/gke-X-d039b866-d-pvc-9749b085-9548-11ea-8048-4201ac100009\' was not found">\n']}, {'type': 'SnapshotCreateError', 'message': 'Error creating snapshot', 'data': {}, 'readable': ['Traceback (most recent call last):\n', '  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/__main__.py", line 58, in main\n    loop.run_until_complete(main_task)\n', '  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 488, in run_until_complete\n    return future.result()\n', '  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/core.py", line 610, in daemon\n    await asyncio.gather(*tasks)\n', '  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/core.py", line 575, in backuper\n    await make_backup(ctx, current_target_rule)\n', '  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/snapshot.py", line 125, in make_backup\n    ) from exc\n', 'k8s_snapshots.errors.SnapshotCreateError: SnapshotCreateError: Error creating snapshot {}\n']}]
2020-05-31T06:05:10.716336Z Shutdown complete              [k8s_snapshots.__main__] message=Shutdown complete severity=INFO
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/snapshot.py", line 101, in make_backup
  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/snapshot.py", line 180, in create_snapshot
  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/asyncutils.py", line 12, in run_in_executor
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/snapshot.py", line 184, in <lambda>
  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/backends/google.py", line 255, in create_snapshot
  File "/usr/local/lib/python3.6/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/googleapiclient/http.py", line 840, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://compute.googleapis.com/compute/v1/projects/X/zones/us-central1-a/disks/gke-X-d039b866-d-pvc-9749b085-9548-11ea-8048-4201ac100009/createSnapshot?alt=json returned "The resource 'projects/X/zones/us-central1-a/disks/gke-X-d039b866-d-pvc-9749b085-9548-11ea-8048-4201ac100009' was not found">

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/k8s-snapshots", line 11, in <module>
    load_entry_point('k8s-snapshots==0.0.0', 'console_scripts', 'k8s-snapshots')()
  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/__main__.py", line 58, in main
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 488, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/core.py", line 610, in daemon
  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/core.py", line 575, in backuper
  File "/usr/local/lib/python3.6/site-packages/k8s_snapshots-0.0.0-py3.6.egg/k8s_snapshots/snapshot.py", line 125, in make_backup
k8s_snapshots.errors.SnapshotCreateError: SnapshotCreateError: Error creating snapshot {}
@umomany
Copy link

umomany commented Jul 27, 2020

@miracle2k is this tool still maintained ?

@miracle2k
Copy link
Owner

@umomany I don't have a lot of time to invest to fix this bug (or others), but I do review and merge pull requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants