Skip to content

Commit

Permalink
feat(runtime): add Python 3.13 support (#5527)
Browse files Browse the repository at this point in the history
* Initial commit for Python 3.13

* Initial commit for Python 3.13

* Initial commit for Python 3.13

* Make mypy happy

* Constructor initial changes

* Constructor initial changes

* Docker folder

* More changes

* More changes

* More changes

* Merging from develop

* CDK 2.167.0

* Should not run CDK tests with all python version

* Should not run CDK tests with all python version - Python 3.12
  • Loading branch information
leandrodamascena authored Nov 14, 2024
1 parent b14ebdd commit 6d07589
Show file tree
Hide file tree
Showing 41 changed files with 670 additions and 468 deletions.
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ body:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
validations:
required: true
- type: dropdown
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/static_typing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ body:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
validations:
required: true
- type: input
Expand Down
13 changes: 8 additions & 5 deletions .github/workflows/layer_govcloud.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand Down Expand Up @@ -94,6 +95,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand All @@ -110,7 +112,7 @@ jobs:
- name: Verify Layer Signature
run: |
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}_${{ matrix.arch }}.json')
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
Expand Down Expand Up @@ -144,7 +146,7 @@ jobs:
run: |
REMOTE_SHA=$(aws --region us-gov-east-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-east-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --query 'Content.CodeSha256' --output text)
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}_${{ matrix.arch }}.json')
test "$REMOTE_SHA" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
test "$REMOTE_SHA" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
aws --region us-gov-east-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-east-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --output table
copy_west:
Expand All @@ -162,6 +164,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand All @@ -179,7 +182,7 @@ jobs:
- name: Verify Layer Signature
run: |
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}_${{ matrix.arch }}.json')
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
Expand Down Expand Up @@ -213,5 +216,5 @@ jobs:
run: |
REMOTE_SHA=$(aws --region us-gov-west-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-west-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --query 'Content.CodeSha256' --output text)
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}_${{ matrix.arch }}.json')
test "$REMOTE_SHA" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
aws --region us-gov-west-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-west-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --output table
test "$REMOTE_SHA" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
aws --region us-gov-west-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-west-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --output table
3 changes: 3 additions & 0 deletions .github/workflows/layer_govcloud_verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand Down Expand Up @@ -63,6 +64,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand Down Expand Up @@ -94,6 +96,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish_v3_layer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ jobs:
strategy:
max-parallel: 5
matrix:
python-version: ["3.8","3.9","3.10","3.11","3.12"]
python-version: ["3.8","3.9","3.10","3.11","3.12","3.13"]
defaults:
run:
working-directory: ./layer_v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/quality_check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
strategy:
max-parallel: 4
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.8","3.9","3.10","3.11","3.12","3.13"]
env:
PYTHON: "${{ matrix.python-version }}"
permissions:
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/quality_code_cdk_constructor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,25 @@ name: Code quality - CDK constructor
on:
pull_request:
paths:
- "layer/layer_constructors/**"
- "layer_v3/layer_constructors/**"
branches:
- develop
push:
paths:
- "layer/layer_constructors/**"
- "layer_v3/layer_constructors/**"
branches:
- develop

permissions:
contents: read

jobs:
quality_check:
quality_check_cdk:
runs-on: ubuntu-latest
strategy:
max-parallel: 4
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.12"]
env:
PYTHON: "${{ matrix.python-version }}"
permissions:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/reusable_deploy_v3_layer_stack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ jobs:
"eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3",
"il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1",
"us-east-2", "us-west-1", "us-west-2"]
python-version: ["3.8","3.9","3.10","3.11","3.12"]
python-version: ["3.8","3.9","3.10","3.11","3.12","3.13"]
include:
- region: "af-south-1"
has_arm64_support: "true"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/reusable_deploy_v3_sar.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ name: Deploy V3 SAR
#
# 1. This workflow starts after the layer artifact is produced on `publish_v3_layer`
# 2. We use the same layer artifact to ensure the SAR app is consistent with the published Lambda Layer
# 3. We publish the SAR for 3.8 to 3.12 Python runtime and both x86_64 and arm64 (see `matrix` section)
# 3. We publish the SAR for 3.8 to 3.13 Python runtime and both x86_64 and arm64 (see `matrix` section)
# 4. We use `sam package` and `sam publish` to publish the SAR app
# 5. We remove the previous Canary stack (if present) and deploy a new one to test the SAR App. We retain the Canary in the account for debugging purposes
# 6. Finally the published SAR app is made public on the PROD environment
Expand Down Expand Up @@ -72,7 +72,7 @@ jobs:
strategy:
matrix:
architecture: ["x86_64", "arm64"]
python-version: ["3.8","3.9","3.10","3.11","3.12"]
python-version: ["3.8","3.9","3.10","3.11","3.12","3.13"]
steps:
- name: checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/run-e2e-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
strategy:
fail-fast: false # needed so if a version fails, the others will still be able to complete and cleanup
matrix:
version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
version: ["3.8", "3.9", "3.10", "3.11", "3.12","3.13"]
if: ${{ github.actor != 'dependabot[bot]' && github.repository == 'aws-powertools/powertools-lambda-python' }}
steps:
- name: "Checkout"
Expand Down
1 change: 0 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ target:

dev:
pip install --upgrade pip pre-commit poetry
poetry config --local virtualenvs.in-project true
@$(MAKE) dev-version-plugin
poetry install --extras "all redis datamasking"
pre-commit install
Expand Down
2 changes: 1 addition & 1 deletion aws_lambda_powertools/event_handler/openapi/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -390,7 +390,7 @@ class HTTPBase(SecurityBase):
scheme: str


class HTTPBearer(HTTPBase):
class HTTPBearer(HTTPBase): # type: ignore[override]
scheme: Literal["bearer"] = "bearer"
bearerFormat: Optional[str] = None

Expand Down
2 changes: 1 addition & 1 deletion aws_lambda_powertools/utilities/parameters/dynamodb.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,4 +230,4 @@ def _get_multiple(self, path: str, **sdk_options) -> dict[str, str]:

# maintenance: look for better ways to correctly type DynamoDB multiple return types
# without a breaking change within ABC return type
return {item[self.sort_attr]: item[self.value_attr] for item in items}
return {item[self.sort_attr]: item[self.value_attr] for item in items} # type: ignore[misc]
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,16 @@ class CloudFormationCustomResourceBaseModel(BaseModel):
resource_properties: Union[Dict[str, Any], BaseModel, None] = Field(None, alias="ResourceProperties")


class CloudFormationCustomResourceCreateModel(CloudFormationCustomResourceBaseModel):
class CloudFormationCustomResourceCreateModel(CloudFormationCustomResourceBaseModel): # type: ignore[override]
request_type: Literal["Create"] = Field(..., alias="RequestType")


class CloudFormationCustomResourceDeleteModel(CloudFormationCustomResourceBaseModel):
class CloudFormationCustomResourceDeleteModel(CloudFormationCustomResourceBaseModel): # type: ignore[override]
request_type: Literal["Delete"] = Field(..., alias="RequestType")
physical_resource_id: str = Field(..., alias="PhysicalResourceId")


class CloudFormationCustomResourceUpdateModel(CloudFormationCustomResourceBaseModel):
class CloudFormationCustomResourceUpdateModel(CloudFormationCustomResourceBaseModel): # type: ignore[override]
request_type: Literal["Update"] = Field(..., alias="RequestType")
physical_resource_id: str = Field(..., alias="PhysicalResourceId")
old_resource_properties: Union[Dict[str, Any], BaseModel, None] = Field(None, alias="OldResourceProperties")
2 changes: 1 addition & 1 deletion aws_lambda_powertools/utilities/parser/models/s3.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ class S3EventNotificationEventBridgeDetailModel(BaseModel):
destination_access_tier: Optional[str] = Field(None, alias="destination-access-tier")


class S3EventNotificationEventBridgeModel(EventBridgeModel):
class S3EventNotificationEventBridgeModel(EventBridgeModel): # type: ignore[override]
detail: S3EventNotificationEventBridgeDetailModel


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
from aws_lambda_powertools.utilities.parser.models.sqs import SqsModel, SqsRecordModel


class S3SqsEventNotificationRecordModel(SqsRecordModel):
class S3SqsEventNotificationRecordModel(SqsRecordModel): # type: ignore[override]
body: Json[S3Model]


class S3SqsEventNotificationModel(SqsModel):
class S3SqsEventNotificationModel(SqsModel): # type: ignore[override]
Records: List[S3SqsEventNotificationRecordModel]
8 changes: 7 additions & 1 deletion docs/automation.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ This is a snapshot of our automated checks at a glance.
To build and deploy the Lambda Layers, we run a pipeline with the following steps:

* We fetch the latest PyPi release and use it as the source for our layer.
* We build Python versions ranging from **3.8 to 3.12** for x86_64 and arm64 architectures. This is necessary because we use pre-compiled libraries like **Pydantic** and **Cryptography**, which require specific Python versions for each layer.
* We build Python versions ranging from **3.8 to 3.13** for x86_64 and arm64 architectures. This is necessary because we use pre-compiled libraries like **Pydantic** and **Cryptography**, which require specific Python versions for each layer.
* We provide layer distributions for both the **x86_64** and **arm64** architectures.
* For each Python version, we create a single CDK package containing both x86_64 and arm64 assets to optimize deployment performance.

Expand All @@ -111,6 +111,7 @@ graph LR
Fetch --> P310[<strong>Python 3.10</strong>]
Fetch --> P311[<strong>Python 3.11</strong>]
Fetch --> P312[<strong>Python 3.12</strong>]
Fetch --> P313[<strong>Python 3.13</strong>]
subgraph build ["LAYER BUILD"]
P38 --> P38x86[build x86_64]
Expand All @@ -124,6 +125,8 @@ graph LR
P311 --> P311arm64[build arm64]
P312 --> P312x86[build x86_64]
P312 --> P312arm64[build arm64]
P313 --> P313x86[build x86_64]
P313 --> P313arm64[build arm64]
P38x86 --> CDKP1[CDK Package]
P38arm64 --> CDKP1[CDK Package]
P39x86 --> CDKP2[CDK Package]
Expand All @@ -134,6 +137,8 @@ graph LR
P311arm64 --> CDKP4[CDK Package]
P312x86 --> CDKP5[CDK Package]
P312arm64 --> CDKP5[CDK Package]
P313x86 --> CDKP6[CDK Package]
P313arm64 --> CDKP6[CDK Package]
end
subgraph beta ["BETA (all regions)"]
Expand All @@ -142,6 +147,7 @@ graph LR
CDKP3 --> DeployBeta
CDKP4 --> DeployBeta
CDKP5 --> DeployBeta
CDKP6 --> DeployBeta
DeployBeta --> RunBetaCanary["Beta canary tests<br> <i>(all packages)</i>"]
end
subgraph prod ["PROD (all regions)"]
Expand Down
32 changes: 32 additions & 0 deletions docs/includes/_layer_homepage_arm64.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,3 +160,35 @@
| **`us-east-2`** | **arn:aws:lambda:us-east-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python312-arm64:3**{: .copyMe}:clipboard: |
| **`us-west-1`** | **arn:aws:lambda:us-west-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python312-arm64:3**{: .copyMe}:clipboard: |
| **`us-west-2`** | **arn:aws:lambda:us-west-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python312-arm64:3**{: .copyMe}:clipboard: |

=== "Python 3.13"

| Region | Layer ARN |
| -------------------- | --------------------------------------------------------------------------------------------------------------- |
| **`af-south-1`** | **arn:aws:lambda:af-south-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-east-1`** | **arn:aws:lambda:ap-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-northeast-1`** | **arn:aws:lambda:ap-northeast-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-northeast-2`** | **arn:aws:lambda:ap-northeast-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-northeast-3`** | **arn:aws:lambda:ap-northeast-3:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-south-1`** | **arn:aws:lambda:ap-south-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-south-2`** | **arn:aws:lambda:ap-south-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-southeast-1`** | **arn:aws:lambda:ap-southeast-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-southeast-2`** | **arn:aws:lambda:ap-southeast-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-southeast-3`** | **arn:aws:lambda:ap-southeast-3:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ca-central-1`** | **arn:aws:lambda:ca-central-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-central-1`** | **arn:aws:lambda:eu-central-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-central-2`** | **arn:aws:lambda:eu-central-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-north-1`** | **arn:aws:lambda:eu-north-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-south-1`** | **arn:aws:lambda:eu-south-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-south-2`** | **arn:aws:lambda:eu-south-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-west-1`** | **arn:aws:lambda:eu-west-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-west-2`** | **arn:aws:lambda:eu-west-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-west-3`** | **arn:aws:lambda:eu-west-3:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`il-central-1`** | **arn:aws:lambda:il-central-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`me-central-1`** | **arn:aws:lambda:me-central-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`me-south-1`** | **arn:aws:lambda:me-south-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`sa-east-1`** | **arn:aws:lambda:sa-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`us-east-1`** | **arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`us-east-2`** | **arn:aws:lambda:us-east-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`us-west-1`** | **arn:aws:lambda:us-west-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`us-west-2`** | **arn:aws:lambda:us-west-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
Loading

0 comments on commit 6d07589

Please sign in to comment.