Bug: SQS MessageAttribute Schema invalid

[hrzbrg]

Expected Behavior

Using the SQS Envelope and my custom body schema, I expected to be able to parse SQS events generated by Symfony Messenger.

Current Behavior

Parsing fails, because binaryValue is null. The SQS SqsMsgAttributeSchema defines it as optional string, but not nullish.

Code snippet

        "messageAttributes": {
          "X-Symfony-Messenger": {
            "stringValue": "{\"X-Message-Stamp-Symfony\\\\Component\\\\Messenger\\\\Stamp\\\\BusNameStamp\":\"[{\\\"busName\\\":\\\"external.event.bus\\\"}]\"}",
            "binaryValue": null,
            "stringListValues": [],
            "binaryListValues": [],
            "dataType": "String"
          },
          "type": {
            "stringValue": "App\\Event\\ChangeLogEvent",
            "binaryValue": null,
            "stringListValues": [],
            "binaryListValues": [],
            "dataType": "String"
          },
          "Content-Type": {
            "stringValue": "application/json",
            "binaryValue": null,
            "stringListValues": [],
            "binaryListValues": [],
            "dataType": "String"
          }
        },

Steps to Reproduce

1. Use the Powertools Parser with envelope: SqsEnvelope to parse an SQS event with the above mentioned messageAttributes.

Possible Solution

BinaryValue could be set to be nullish: binaryValue: z.string().nullish()

Powertools for AWS Lambda (TypeScript) version

2.10

AWS Lambda function runtime

20.x

Packaging format used

npm

Execution logs

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #typescript channel on our Powertools for AWS Lambda Discord: Invite link

[am29d]

Hey @hrzbrg thanks for dropping the issue, I will take a look why we missed it in the tests. Feel free to open a PR, if you want.

[dreamorosi]

Hi, could you please share more info on the API call used to generate these SQS messages and how the event gets to the Lambda function?

I tried to send a message to SQS with attribute of type String, and the binary value field is undefined instead of null when I poll the queue:

{
    "Messages": [
        {
            "MessageId": "92f0ce33-f118-4a6a-8041-3f5016ab16e3",
            "ReceiptHandle": "AQEBsoWTnNZVxAUZOcJ....",
            "MD5OfBody": "5dea26020e811def89cbf972425f0e46",
            "Body": "two two",
            "MD5OfMessageAttributes": "2db1f7ff6cc285162639816522a868fb",
            "MessageAttributes": {
                "type": {
                    "StringValue": "application/json",
                    "DataType": "String"
                }
            }
        }
    ]
}

[github-actions]

This issue has not received a response in 2 weeks. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

[hrzbrg]

Hi @dreamorosi
we use Symfony Messenger to output messages to SQS from our PHP application with no special settings around it.

[leandrodamascena]

Hello @hrzbrg! Thanks for reporting this behavior. What surprises me here is because data and field validation happen at the AWS SQS ControlPlane level and not in the library/local level. I tried to make a request to SQS with StringValue with value and BinaryValue with None/null/empty and I couldn't because SQS responded InvalidParameterValue

app.py

sqs = boto3.client('sqs')

# Replace with your SQS queue URL
queue_url = "https://sqs.us-east-1.amazonaws.com/xxxxx/myqueue"

# Message body
message_body = "Your message body here"

# Message attributes
message_attributes = {
    'xyz': {
        'DataType': 'String',
        'StringValue': "leo",
        'BinaryValue': b""  # Null byte
    },
}

# Send the message
response = sqs.send_message(
    QueueUrl=queue_url,
    MessageBody=message_body,
    #MessageGroupId="1",
    MessageAttributes=message_attributes
)

logs

2024-12-29 22:41:10,411 urllib3.connectionpool [DEBUG] Starting new HTTPS connection (1): sqs.us-east-1.amazonaws.com:443
2024-12-29 22:41:10,949 urllib3.connectionpool [DEBUG] https://sqs.us-east-1.amazonaws.com:443 "POST / HTTP/1.1" 400 160
2024-12-29 22:41:10,950 botocore.parsers [DEBUG] Response headers: {'x-amzn-RequestId': '4089c382-034b-509c-a583-9c69d10c24b3', 'x-amzn-query-error': 'InvalidParameterValue;Sender', 'Date': 'Sun, 29 Dec 2024 22:41:11 GMT', 'Content-Type': 'application/x-amz-json-1.0', 'Content-Length': '160', 'connection': 'keep-alive'}

I also tested with pure PHP (I miss you, PHP) + aws sdk and symphony + messenger and using a messagebus to send the message and I couldn't reproduce the problem, since it was also validated at the control plane level. I'm not saying that your report is not valid and this can't happen, but this is new for me. I'm wondering if this is perhaps not a local message before sending to SQS and is failing your tests? Or perhaps a fake/test payload? Can you give us more details about this and perhaps a small project where I can run locally here and reproduce? If you prefer, we can also set up a meeting and you can show me this happening.

I'm super interested in this issue, as I don't see a situation where this should be allowed, but if it is happening, this issue can be used to open a ticket to the SQS team requesting more details about this behavior.

Thank you.

[github-actions]

This issue has not received a response in 2 weeks. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.