Fusion doesn't work on AWS China region, the AWS access key id you provided does not exist in our records.

[waynewang117]

Bug report

Fusion local execution with AWS China S3.
got below error:
the AWS access key id you provided does not exist in our records.

Expected behavior and actual behavior

Fusion can connect to s3.

Steps to reproduce the problem

env vars:
AWS_ACCESS_KEY_ID=xxx
AWS_SECRET_ACCESS_KEY=xxx
AWS_SESSION_TOKEN=xxx
AWS_S3_ENDPOINT=https://s3.cn-north-1.amazonaws.com.cn
AWS_DEFAULT_REGION=cn-north-1
FUSION_AWS_REGION=cn-north-1

Program output

statusCode: 403. the AWS access key id you provided does not exist in our records.

Environment

• Nextflow version: 24.10.3.5933
• Java version: 17.0.10
• Operating system: Linux
• Bash version: bash 4.2.46

[pditommaso]

Hi there, this is tough to troubleshot because we don't have access to china region. Could you please provide the .nextflow.log file?

[waynewang117]

hi @pditommaso, my company's computer cannot connect to Github,so I have to use my phone.
I have added AWS_S3_ENDPOINT into container's env vars. But it seems the property AWS_S3_ENDPOINT was not configured correctly in Fusion.
9c81104 . So the fusion cannot connect to Github and got the 403 error.
"statusCode: 403. the AWS access key id you provided does not exist in our records."

for the nexflow AWS plugin, the endpoint has been configured. But not sure the endpoint is used in Fusion correctly.
https://github.com/nextflow-io/nextflow/blob/master/plugins/nf-amazon/src/main/nextflow/cloud/aws/AwsClientFactory.groovy#L255

[fntlnz]

Hi @waynewang117 you should be able to workaround that can you try to add this line to your nextflow config file?

docker.containerOptions = '-e FUSION_AWS_REGION=cn-north-1'

[waynewang117]

Hi @waynewang117 you should be able to workaround that can you try to add this line to your nextflow config file?

docker.containerOptions = '-e FUSION_AWS_REGION=cn-north-1'

Hi @fntlnz , I have added this into container's env vars, but doesn't work.

[waynewang117]

@fntlnz

My test case:

export AWS_ACCESS_KEY_ID=xxx
export AWS_SECRET_ACCESS_KEY=xxx
export AWS_SESSION_TOKEN=xxx
export AWS_DEFAULT_REGION=cn-north-1
export AWS_S3_ENDPOINT=https://s3.cn-north-1.amazonaws.com.cn

nextflow.config

docker.enabled = true
wave.enabled = true
fusion.enabled = true
fusion.exportStorageCredentials = true

process.container = 'quay.io/nextflow/bash'
workDir = 's3://nextflow-ci/scratch'

docker.containerOptions = '-e FUSION_AWS_REGION=cn-north-1 -e AWS_S3_ENDPOINT=https://s3.cn-north-1.amazonaws.com.cn -e AWS_DEFAULT_REGION=cn-north-1'

nextflow run nextflow-io/hello -c nextflow.config

Test result:
The nextflow cli can create folder on AWS China s3 bucket and start several containers to run pipeline.
But these containers cannot connect to AWS China s3 bucket with Fusion.
Checked the logs in containes, got below error:
"statusCode: 403. the AWS access key id you provided does not exist in our records."

[pditommaso]

Is there any chance you can share some temporary AWS keys to access to this China region? as mentioned we are not able to create an AWS account in China and therefore we cannot test it.