SQS: Support Async JSON SQS Protocol & Message Attributes

[hfern]

@auvipy 👋 Hi there! There is no ticket for this issue -- I found this bug myself and thought I'd fix it.

---

Adds support for consuming botocore SQS json models in the SQS async code. This only affects consumer code.

Several parameters are supplied incorrectly as they have changed in the query to json botocore change. If you have a newer version of botocore, you will get the json service model instead of the old query model. I found several places where kombu was generating calls to the new json API (indicated by the x-amz-json-1.0 content-type) but providing parameter names consistent with the query protocol.

This PR makes kombu aware of both the json and query protocol query names and provide them both. When the AWS request is later constructed from the service model, we apply protocol-specific and common parameters. Unfortunately, AWS has not maintained consistency in the parameter names, so I manually reviewed each API call and supplied both.

The only things that broke were compound data types for ReceiveMessage -- those are attribute names. Other things did not break because the code path for post-process message deletion is through the synchronous branch, which uses standard boto and not the async request builder (kombu.transport.SQS.Channel.basic_ack). Nevertheless, I fixed all the parameters in the async client.

Logged request after PR (note new usage of AttributeNames instead of AttributeName.1):

POST https://sqs.us-west-2.amazonaws.com 200
Content-Type: application/x-amz-json-1.0
X-Amz-Target: AmazonSQS.ReceiveMessage
X-Amz-Date: 20250115T190022Z
Authorization: AWS4-HMAC-SHA256 XXX
Content-Length: 177
User-Agent: Mozilla/5.0 (compatible; urllib3)
Accept-Encoding: gzip,deflate

{"MaxNumberOfMessages": 2, "WaitTimeSeconds": 20, "AttributeNames": ["ApproximateReceiveCount"], "QueueUrl": "https://sqs.us-west-2.amazonaws.com/XXX/identity-hunterq"}



200
x-amzn-RequestId: 05a8dffc-0ce3-5cc7-85c0-bffe04e5c2b2
Date: Wed, 15 Jan 2025 19:00:42 GMT
Content-Type: application/x-amz-json-1.0
Content-Length: 2
connection: keep-alive

{}

This code supporting both protocols can be cleaned up if Kombu is willing to require botocore>=1.34.90, which was released in April 2024 (boto/botocore/pull/3165). It can take quite a while to move forward the boto3/botocore/moto ecosystem, so Kombu may want to give consumers some time before dropping old botocores. Either way, this PR would be required.

---

As I fixed this to get support for AttributeNames back, I also added support for customizing fetched attributes via transport_options.fetch_message_attributes. Now you can supply ["All"] to get better diagnostics from SQS. I added documentation for this.

[codecov]

Codecov Report

Attention: Patch coverage is 96.96970% with 1 line in your changes missing coverage. Please review.

Project coverage is 81.53%. Comparing base (a0175b0) to head (4f4612f).

Files with missing lines	Patch %	Lines
kombu/asynchronous/aws/sqs/connection.py	96.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2226      +/-   ##
==========================================
+ Coverage   81.50%   81.53%   +0.02%     
==========================================
  Files          77       77              
  Lines        9524     9538      +14     
  Branches     1152     1156       +4     
==========================================
+ Hits         7763     7777      +14     
  Misses       1569     1569              
  Partials      192      192              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.