You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In v0.20.0, I'm running into an issue where, when trying to read an S3 file from multiple readers in a parallel scenario (multiple readers, no write access), I am passing force_overwrite_from_cloud as instructed in the error report, but still getting an OverwriteNewerLocalError.
I will try to set different caches for each process, but this could be wasteful as I am only reading and will have to copy already downloaded files.
Related to isses: #283 and #128 , but not exactly.
The error reads like this:
...
fp = file_path.open('rb', force_overwrite_from_cloud=True)
File "/opt/conda/lib/python3.10/site-packages/cloudpathlib/cloudpath.py", line 686, in open
self._refresh_cache(force_overwrite_from_cloud=force_overwrite_from_cloud)
File "/opt/conda/lib/python3.10/site-packages/cloudpathlib/cloudpath.py", line 1303, in _refresh_cache raise OverwriteNewerLocalError(
cloudpathlib.exceptions.OverwriteNewerLocalError: Local file (/tmp/tmp9h3c_4qk/s3://|bucket|/|ObjectPath|/GT_32.h5) for cloud path (s3://|bucket|/|ObjectPath|/GT_32.h5) is newer on disk, but is being requested for download from cloud. Either (1) push your changes to the cloud, (2) r:emove the local file, or (3) pass force_overwrite_from_cloud=True to overwrite; or set env var CLOUDPATHLIB_FORCE_OVERWRITE_FROM_CLOUD=1.
The text was updated successfully, but these errors were encountered:
Hi,
In v0.20.0, I'm running into an issue where, when trying to read an S3 file from multiple readers in a parallel scenario (multiple readers, no write access), I am passing force_overwrite_from_cloud as instructed in the error report, but still getting an OverwriteNewerLocalError.
I will try to set different caches for each process, but this could be wasteful as I am only reading and will have to copy already downloaded files.
Related to isses: #283 and #128 , but not exactly.
The error reads like this:
...
fp = file_path.open('rb', force_overwrite_from_cloud=True)
File "/opt/conda/lib/python3.10/site-packages/cloudpathlib/cloudpath.py", line 686, in open
self._refresh_cache(force_overwrite_from_cloud=force_overwrite_from_cloud)
File "/opt/conda/lib/python3.10/site-packages/cloudpathlib/cloudpath.py", line 1303, in _refresh_cache
raise OverwriteNewerLocalError(
cloudpathlib.exceptions.OverwriteNewerLocalError: Local file (/tmp/tmp9h3c_4qk/s3://|bucket|/|ObjectPath|/GT_32.h5) for cloud path (s3://|bucket|/|ObjectPath|/GT_32.h5) is newer on disk, but is being requested for download from cloud. Either (1) push your changes to the cloud, (2) r:emove the local file, or (3) pass
force_overwrite_from_cloud=True
to overwrite; or set env var CLOUDPATHLIB_FORCE_OVERWRITE_FROM_CLOUD=1.The text was updated successfully, but these errors were encountered: