GoogleApiToS3Operator started throwing an error on airflow MWAA after update

[HammadASiddiqui]

I have been using the follwing task to extract the details from google shhet and push to s3

task_google_sheets_values_to_s3 = GoogleApiToS3Operator(
    task_id="google_sheet_data_to_s3",
    google_api_service_name="sheets",
    google_api_service_version="v4",
    google_api_endpoint_path="sheets.spreadsheets.values.get",
    google_api_endpoint_params={"spreadsheetId": GOOGLE_SHEET_ID, "range": GOOGLE_SHEET_RANGE},
    s3_destination_key=f"s3://{s3_bucket}/{s3_key}",
)

it was working fine but then I updated my MWAA environment to 2.4.3 and somehow the package dependency has messed up and the operator now has started to throw the following error
TypeError: GoogleDiscoveryApiHook.__init__() got an unexpected keyword argument 'delegate_to'
can someone please guide me on what package requirements are there to fix the issue

[HammadASiddiqui]

I have the following packages installed in my env

aiobotocore==2.5.4
aiofiles==23.2.1
aiohttp==3.8.5
aioitertools==0.11.0
aiosignal==1.3.1
alembic==1.8.1
amqp==5.1.1
anyio==3.6.2
apache-airflow==2.4.3
apache-airflow-providers-amazon==6.0.0
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-common-sql==1.7.1
apache-airflow-providers-databricks==4.4.0
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-google==10.7.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-mysql==5.3.0
apache-airflow-providers-postgres==5.2.2
apache-airflow-providers-slack==8.0.0
apache-airflow-providers-sqlite==3.2.1
apache-airflow-providers-ssh==3.7.2
apispec==3.3.2
argcomplete==2.0.0
asgiref==3.7.2
asn1crypto==1.5.1
async-timeout==4.0.3
attrs==22.1.0
awswrangler==3.3.0
Babel==2.11.0
backoff==2.2.1
bcrypt==4.0.1
beautifulsoup4==4.11.1
billiard==3.6.4.0
blinker==1.5
boto3==1.28.17
botocore==1.31.17
CacheControl==0.12.12
cached-property==1.5.2
cachelib==0.9.0
cachetools==5.3.1
cattrs==22.2.0
celery==5.2.7
certifi==2022.9.24
cffi==1.15.1
chardet==5.2.0
charset-normalizer==2.1.1
cleo==2.0.1
click==8.1.3
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
clickclick==20.10.2
colorama==0.4.6
colorlog==4.8.0
commonmark==0.9.1
ConfigUpdater==3.1.1
connexion==2.14.1
crashtest==0.4.1
cron-descriptor==1.2.31
croniter==1.3.7
cryptography==41.0.3
databricks-sql-connector==2.9.3
db-dtypes==1.1.1
decorator==5.1.1
Deprecated==1.2.13
dill==0.3.1.1
distlib==0.3.6
dnspython==2.2.1
docutils==0.19
dulwich==0.20.50
email-validator==1.3.0
et-xmlfile==1.1.0
exceptiongroup==1.0.1
filelock==3.8.0
Flask==2.2.2
Flask-AppBuilder==4.1.4
Flask-Babel==2.0.0
Flask-Caching==2.0.1
Flask-JWT-Extended==4.4.4
Flask-Login==0.6.2
Flask-Session==0.4.0
Flask-SQLAlchemy==2.5.1
Flask-WTF==1.0.1
flower==1.2.0
frozenlist==1.4.0
fsspec==2023.6.0
gcloud-aio-auth==4.2.3
gcloud-aio-bigquery==6.3.0
gcloud-aio-storage==8.3.0
google-ads==21.3.0
google-api-core==2.11.1
google-api-python-client==2.97.0
google-auth==2.22.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-cloud-aiplatform==1.31.1
google-cloud-appengine-logging==1.3.1
google-cloud-audit-log==0.2.5
google-cloud-automl==2.11.2
google-cloud-batch==0.16.0
google-cloud-bigquery==3.11.4
google-cloud-bigquery-datatransfer==3.12.0
google-cloud-bigquery-storage==2.22.0
google-cloud-bigtable==2.21.0
google-cloud-build==3.20.0
google-cloud-compute==1.14.0
google-cloud-container==2.30.0
google-cloud-core==2.3.3
google-cloud-datacatalog==3.15.0
google-cloud-dataflow-client==0.8.4
google-cloud-dataform==0.5.2
google-cloud-dataplex==1.6.2
google-cloud-dataproc==5.4.3
google-cloud-dataproc-metastore==1.12.0
google-cloud-dlp==3.12.2
google-cloud-kms==2.19.1
google-cloud-language==2.11.0
google-cloud-logging==3.6.0
google-cloud-memcache==1.7.2
google-cloud-monitoring==2.15.1
google-cloud-orchestration-airflow==1.9.1
google-cloud-os-login==2.10.0
google-cloud-pubsub==2.18.3
google-cloud-redis==2.13.1
google-cloud-resource-manager==1.10.3
google-cloud-run==0.9.1
google-cloud-secret-manager==2.16.3
google-cloud-spanner==3.40.1
google-cloud-speech==2.21.0
google-cloud-storage==2.10.0
google-cloud-storage-transfer==1.9.1
google-cloud-tasks==2.14.1
google-cloud-texttospeech==2.14.1
google-cloud-translate==3.12.0
google-cloud-videointelligence==2.11.3
google-cloud-vision==3.4.4
google-cloud-workflows==1.11.0
google-crc32c==1.5.0
google-resumable-media==2.5.0
googleapis-common-protos==1.60.0
graphviz==0.20.1
greenlet==2.0.1
grpc-google-iam-v1==0.12.6
grpcio==1.57.0
grpcio-gcp==0.2.2
grpcio-status==1.57.0
gunicorn==20.1.0
h11==0.12.0
html5lib==1.1
httpcore==0.15.0
httplib2==0.22.0
httpx==0.23.0
humanize==4.4.0
idna==3.4
importlib-metadata==5.0.0
inflection==0.5.1
itsdangerous==2.1.2
jaraco.classes==3.2.3
jeepney==0.8.0
Jinja2==3.1.2
jmespath==0.10.0
json-merge-patch==0.2
jsonpath-ng==1.5.3
jsonschema==4.17.0
keyring==23.11.0
kombu==5.2.4
lazy-object-proxy==0.0.0
linkify-it-py==2.0.0
lockfile==0.12.2
looker-sdk==23.14.1
lxml==4.9.1
lz4==4.3.2
Mako==1.2.3
Markdown==3.4.1
markdown-it-py==2.1.0
MarkupSafe==2.1.1
marshmallow==3.18.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==3.0.1
marshmallow-sqlalchemy==0.26.1
mdit-py-plugins==0.3.1
mdurl==0.1.2
more-itertools==8.14.0
msgpack==1.0.4
multidict==6.0.4
mypy-boto3-appflow==1.26.0.post1
mypy-boto3-rds==1.26.0.post1
mypy-boto3-redshift-data==1.26.0.post1
mysql-connector-python==8.1.0
mysqlclient==2.2.0
numpy==1.22.4
oauthlib==3.2.2
openpyxl==3.1.2
packaging==21.3
pandas==1.5.1
pandas-gbq==0.19.2
paramiko==3.3.1
pathspec==0.9.0
pendulum==2.1.2
pexpect==4.8.0
pkginfo==1.8.3
platformdirs==2.5.3
pluggy==1.0.0
ply==3.11
poetry==1.3.1
poetry-core==1.4.0
poetry-plugin-export==1.2.0
prison==0.2.1
prometheus-client==0.15.0
prompt-toolkit==3.0.32
proto-plus==1.22.3
protobuf==4.21.12
psutil==5.9.4
psycopg2==2.9.5
ptyprocess==0.7.0
pyarrow==13.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pycurl==7.44.1
pydata-google-auth==1.8.2
Pygments==2.13.0
PyJWT==2.6.0
PyMySQL==1.1.0
PyNaCl==1.5.0
pyOpenSSL==23.2.0
pyparsing==3.0.9
pyrsistent==0.19.2
python-daemon==2.3.2
python-dateutil==2.8.2
python-nvd3==0.15.0
python-slugify==6.1.2
pytz==2022.6
pytzdata==2020.1
PyYAML==6.0
rapidfuzz==2.13.7
ratelimit==2.2.1
redshift-connector==2.0.909
requests==2.28.1
requests-oauthlib==1.3.1
requests-toolbelt==0.9.1
rfc3986==1.5.0
rich==12.6.0
rsa==4.9
s3fs==2023.6.0
s3transfer==0.6.0
scramp==1.4.4
SecretStorage==3.3.3
setproctitle==1.3.2
Shapely==1.8.5.post1
shellingham==1.5.0.post1
six==1.16.0
slack-sdk==3.21.3
sniffio==1.3.0
soupsieve==2.3.2.post1
SQLAlchemy==1.4.27
sqlalchemy-bigquery==1.8.0
SQLAlchemy-JSONField==1.0.0
sqlalchemy-redshift==0.8.11
sqlalchemy-spanner==1.6.2
SQLAlchemy-Utils==0.38.3
sqlparse==0.4.4
sshtunnel==0.4.0
statsd==4.0.1
swagger-ui-bundle==0.0.9
tabulate==0.9.0
tenacity==8.1.0
termcolor==2.1.0
text-unidecode==1.3
thrift==0.16.0
throttler==1.2.2
tomli==2.0.1
tomlkit==0.11.6
tornado==6.1
trove-classifiers==2022.12.22
typing_extensions==4.4.0
uc-micro-py==1.0.1
unicodecsv==0.14.1
uritemplate==4.1.1
urllib3==1.26.12
vine==5.0.0
virtualenv==20.16.6
watchtower==2.0.1
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==2.2.2
wrapt==1.14.1
WTForms==3.0.1
yarl==1.9.2
zipp==3.10.0

[potiuk]

You need to upgrade apache-airflow-providers-amazon I believe.

[HammadASiddiqui]

to what version?

[HammadASiddiqui]

i am currently on apache-airflow-providers-amazon==6.0.0 @potiuk

[potiuk]

See in https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/changelog.html

[potiuk]

Generally release notes is good source to check