feat(event_handler): generate OpenAPI specifications and validate input/output

[rubenfonseca]

Issue number: #2421

Summary

Changes

Please provide a summary of what's being changed

This PR adds support for generating OpenAPI specifications from an event handler resolver, by inspecting the handler parameters and return types. Additionally, it adds a new middleware that will validate the input and output before/after the handler is invocated, allowing one to use typed parameters in the handler signature.

User experience

Please share what the user experience looks like before and after this change

Creating simple openapi spec - Python code

from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.utilities.typing import LambdaContext

logger = Logger()
app = APIGatewayRestResolver(enable_validation=True)


@app.get("/openapispec")
def get_openapi():
    return app.get_openapi_json_schema(title="My API", version="2.5.0", description="Powertools OpenAPI")


# You can continue to use other utilities just as before
@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
def lambda_handler(event: dict, context: LambdaContext) -> dict:
    return app.resolve(event, context)

Creating simple openapi spec - output

{
  "openapi": "3.1.0",
  "info": {
    "title": "My API",
    "description": "Powertools OpenAPI",
    "version": "2.5.0"
  },
  "servers": [
    {
      "url": "/"
    }
  ],
  "paths": {
    "/openapispec": {
      "get": {
        "summary": "GET /openapispec",
        "operationId": "get_openapi_openapispec_get",
        "responses": {
          "200": {
            "description": "Successful Response",
            "content": {
              "application/json": {
                "schema": {}
              }
            }
          },
          "422": {
            "description": "Validation Error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/HTTPValidationError"
                }
              }
            }
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "HTTPValidationError": {
        "properties": {
          "detail": {
            "items": {
              "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
          }
        },
        "type": "object",
        "title": "HTTPValidationError"
      },
      "ValidationError": {
        "properties": {
          "loc": {
            "items": {
              "anyOf": [
                {
                  "type": "string"
                },
                {
                  "type": "integer"
                }
              ]
            },
            "type": "array",
            "title": "Location"
          },
          "msg": {
            "type": "string",
            "title": "Message"
          },
          "type": {
            "type": "string",
            "title": "Error Type"
          }
        },
        "type": "object",
        "required": [
          "loc",
          "msg",
          "type"
        ],
        "title": "ValidationError"
      }
    }
  }
}

---

Using simple get and post route - Python code

from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.utilities.typing import LambdaContext

logger = Logger()
app = APIGatewayRestResolver(enable_validation=True)


@app.get("/todo/<id>")
def get_todos_using_parameter(id: str) -> dict:
    logger.info("Getting todo", id=id)
    return {"id": id, "name": "get_todos"}

@app.post("/todo")
def post_todo():
    logger.info("Returning todo")
    return {"ok"}

# You can continue to use other utilities just as before
@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
def lambda_handler(event: dict, context: LambdaContext) -> dict:
    print(app.get_openapi_json_schema(title="My API", version="2.5.0", description="Powertools OpenAPI"))
    return app.resolve(event, context)

Using simple get and post route - OpenAPI spec

{
  "openapi": "3.1.0",
  "info": {
    "title": "My API",
    "description": "Powertools OpenAPI",
    "version": "2.5.0"
  },
  "servers": [
    {
      "url": "/"
    }
  ],
  "paths": {
    "/todo/<id>": {
      "get": {
        "summary": "GET /todo/<id>",
        "operationId": "get_todos_using_parameter_todo__id__get",
        "parameters": [
          {
            "required": true,
            "schema": {
              "type": "string",
              "title": "Id"
            },
            "name": "id",
            "in": "path"
          }
        ],
        "responses": {
          "200": {
            "description": "Successful Response",
            "content": {
              "application/json": {
                "schema": {
                  "type": "object",
                  "title": "Return"
                },
                "name": "Return get_todos_using_parameter_todo__id__get"
              }
            }
          },
          "422": {
            "description": "Validation Error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/HTTPValidationError"
                }
              }
            }
          }
        }
      }
    },
    "/todo": {
      "post": {
        "summary": "POST /todo",
        "operationId": "post_todo_todo_post",
        "requestBody": {
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/Todo"
              }
            }
          },
          "required": true
        },
        "responses": {
          "200": {
            "description": "Successful Response",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Todo"
                },
                "name": "Return post_todo_todo_post"
              }
            }
          },
          "422": {
            "description": "Validation Error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/HTTPValidationError"
                }
              }
            }
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "HTTPValidationError": {
        "properties": {
          "detail": {
            "items": {
              "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
          }
        },
        "type": "object",
        "title": "HTTPValidationError"
      },
      "Todo": {
        "properties": {
          "id": {
            "type": "string",
            "title": "Id"
          },
          "name": {
            "type": "string",
            "title": "Name"
          }
        },
        "type": "object",
        "required": [
          "id",
          "name"
        ],
        "title": "Todo"
      },
      "ValidationError": {
        "properties": {
          "loc": {
            "items": {
              "anyOf": [
                {
                  "type": "string"
                },
                {
                  "type": "integer"
                }
              ]
            },
            "type": "array",
            "title": "Location"
          },
          "msg": {
            "type": "string",
            "title": "Message"
          },
          "type": {
            "type": "string",
            "title": "Error Type"
          }
        },
        "type": "object",
        "required": [
          "loc",
          "msg",
          "type"
        ],
        "title": "ValidationError"
      }
    }
  }
}

---

Using get and post route with dataclass - Python code

from dataclasses import dataclass
from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.utilities.typing import LambdaContext

logger = Logger()
app = APIGatewayRestResolver(enable_validation=True)

@dataclass
class Todo:
    id: str
    name: str


@app.get("/todo/<id>")
def get_todos_using_parameter(id: str) -> Todo:
    logger.info("Getting todo", id=id)

    todo = Todo(id=id, name="get_todos")

    return todo

@app.post("/todo")
def post_todo(todo: Todo) -> Todo:
    logger.info("Returning todo")
    return todo

# You can continue to use other utilities just as before
@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
def lambda_handler(event: dict, context: LambdaContext) -> dict:
    print(app.get_openapi_json_schema(title="My API", version="2.5.0", description="Powertools OpenAPI"))
    return app.resolve(event, context)

Using get and post route with dataclass - OpenAPI Schema

{
  "openapi": "3.1.0",
  "info": {
    "title": "My API",
    "description": "Powertools OpenAPI",
    "version": "2.5.0"
  },
  "servers": [
    {
      "url": "/"
    }
  ],
  "paths": {
    "/todo/<id>": {
      "get": {
        "summary": "GET /todo/<id>",
        "operationId": "get_todos_using_parameter_todo__id__get",
        "parameters": [
          {
            "required": true,
            "schema": {
              "type": "string",
              "title": "Id"
            },
            "name": "id",
            "in": "path"
          }
        ],
        "responses": {
          "200": {
            "description": "Successful Response",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Todo"
                },
                "name": "Return get_todos_using_parameter_todo__id__get"
              }
            }
          },
          "422": {
            "description": "Validation Error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/HTTPValidationError"
                }
              }
            }
          }
        }
      }
    },
    "/todo": {
      "post": {
        "summary": "POST /todo",
        "operationId": "post_todo_todo_post",
        "requestBody": {
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/Todo"
              }
            }
          },
          "required": true
        },
        "responses": {
          "200": {
            "description": "Successful Response",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Todo"
                },
                "name": "Return post_todo_todo_post"
              }
            }
          },
          "422": {
            "description": "Validation Error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/HTTPValidationError"
                }
              }
            }
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "HTTPValidationError": {
        "properties": {
          "detail": {
            "items": {
              "$ref": "#/components/schemas/ValidationError"
            },
            "type": "array",
            "title": "Detail"
          }
        },
        "type": "object",
        "title": "HTTPValidationError"
      },
      "Todo": {
        "properties": {
          "id": {
            "type": "string",
            "title": "Id"
          },
          "name": {
            "type": "string",
            "title": "Name"
          }
        },
        "type": "object",
        "required": [
          "id",
          "name"
        ],
        "title": "Todo"
      },
      "ValidationError": {
        "properties": {
          "loc": {
            "items": {
              "anyOf": [
                {
                  "type": "string"
                },
                {
                  "type": "integer"
                }
              ]
            },
            "type": "array",
            "title": "Location"
          },
          "msg": {
            "type": "string",
            "title": "Message"
          },
          "type": {
            "type": "string",
            "title": "Error Type"
          }
        },
        "type": "object",
        "required": [
          "loc",
          "msg",
          "type"
        ],
        "title": "ValidationError"
      }
    }
  }
}

---

Serializing DTO - Python code

from dataclasses import dataclass
from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.utilities.typing import LambdaContext

logger = Logger()
app = APIGatewayRestResolver(enable_validation=True)

@dataclass
class Todo:
    id: str
    name: str


@app.get("/todo/<id>")
def get_todos_using_parameter(id: str) -> Todo:
    logger.info("Getting todo", id=id)
    todo = Todo(id=id, name="get_todos")
    return todo


# You can continue to use other utilities just as before
@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
def lambda_handler(event: dict, context: LambdaContext) -> dict:
    return app.resolve(event, context)

Serializing DTO - Output

❯ curl -X GET http://127.0.0.1:3000/todo/1
{'id': '1', 'name': 'get_todos'}

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• Meet tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

TODO

The following are high level tasks necessary to finish the PR:

• General cleanup, refactor and commenting of the code
• Make sure any part of the spec is extendable/customizable by the user
  • function decorators
  • returns examples
• Handle body parameters
• Write tests
• Make sure the new code won't break existing users that don't have Pydantic installed
• Make sure the code work both with Pydantic v1 and v2

NEXT PR

• Expose Swagger endpoint [postponed as the PR is too big already]
• Documentation [postponed as the PR is too big already]

Is this a breaking change?

RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[codecov-commenter]

Codecov Report

Attention: 123 lines in your changes are missing coverage. Please review.

Comparison is base (14cb407) 95.94% compared to head (45573a0) 95.09%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3109      +/-   ##
===========================================
- Coverage    95.94%   95.09%   -0.85%     
===========================================
  Files          195      202       +7     
  Lines         8387     9407    +1020     
  Branches      1563     1719     +156     
===========================================
+ Hits          8047     8946     +899     
- Misses         277      341      +64     
- Partials        63      120      +57     

Files	Coverage Δ
...da_powertools/event_handler/lambda_function_url.py	100.00% <100.00%> (ø)
...bda_powertools/event_handler/openapi/exceptions.py	100.00% <100.00%> (ø)
aws_lambda_powertools/event_handler/vpc_lattice.py	100.00% <100.00%> (ø)
aws_lambda_powertools/shared/types.py	100.00% <100.00%> (+14.28%)	⬆️
...s_lambda_powertools/event_handler/openapi/types.py	88.88% <88.88%> (ø)
...ambda_powertools/event_handler/openapi/encoders.py	96.62% <96.62%> (ø)
aws_lambda_powertools/event_handler/api_gateway.py	96.45% <92.00%> (-2.33%)	⬇️
...mbda_powertools/event_handler/openapi/dependant.py	80.85% <80.85%> (ø)
..._lambda_powertools/event_handler/openapi/params.py	88.07% <88.07%> (ø)
...ls/event_handler/middlewares/openapi_validation.py	83.47% <83.47%> (ø)
... and 1 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

⚠️Large PR detected⚠️

Please consider breaking into smaller PRs to avoid significant review delays. Ignore if this PR has naturally grown to this size after reviews.

[github-actions]

⚠️Large PR detected⚠️

Please consider breaking into smaller PRs to avoid significant review delays. Ignore if this PR has naturally grown to this size after reviews.

[github-actions]

⚠️Large PR detected⚠️

Please consider breaking into smaller PRs to avoid significant review delays. Ignore if this PR has naturally grown to this size after reviews.

[mattpker]

Is this also going to have support for the Application Load Balancer event handler?

[rubenfonseca]

Is this also going to have support for the Application Load Balancer event handler?

Absolutely! All event handlers will work out of the box :)

[thomasbrizard]

Hi @rubenfonseca, pretty interested by this, would it be possible to get a preview of the UX for a @app.post with a body ? :)

[heitorlessa]

Hi @rubenfonseca, pretty interested by this, would it be possible to get a preview of the UX for a @app.post with a body ? :)

Almost there, Ruben has just nailed this part but pushing changes this week to the PR.

It was a difficult call to make UX wise while following our Progressive tenet and no backwards incompatible change.

You'll be pleased, we hope!!

[mattpker]

I see that we will have the ability to spit out the JSON, but will it also be able to generate the Swagger UI like FastAPI does? Or will we have to host the Swagger UI static assets on S3 or something?

[heitorlessa]

It will auto-generate like FastAPI, including data validation. We're aiming for a FastAPI like experience.

We're also considering a method to generate a fragment of the JSON Schema to allow customers with micro functions to get to the same end result with Swagger UI -- this will need a CLI of sorts, so we will need a RFC for that CLI/alternative solution later (the method is already implemented AFAIK).

Ruben is deeply heads down on handling the final bits of backwards compatibility but the whole experience to enable ala FastAPI will be simply a flag + bring Pydantic as a dep.

[github-actions]

⚠️Large PR detected⚠️

Please consider breaking into smaller PRs to avoid significant review delays. Ignore if this PR has naturally grown to this size after reviews.

[rubenfonseca]

@leandrodamascena had some trouble with the shared types since it broke some older versions of Python, but finally the tests are green again

[leandrodamascena]

Hello @rubenfonseca! I left some comments and I'm missing some tests with optional fields in Pydantic and Dataclasses. Just to make sure we are respecting the API contract and if we parser them as not required.

That's it for me and I'm fine! I'm sure @heitorlessa wants to review it before merging.

Thanks for all the effort here, man.

[heitorlessa]

Quick peer review feedback with Leandro...

Split Data Validation and Serialization from OpenAPI Generation example

It is not clear from the PR Body what the experience looks like BEFORE and AFTER, and what the richer experience brings to the table besides OpenAPI Schema. For example, we can now return a DTO and we will serialize it for them... we couldn't before.. but there's no JSON output here.

If you split it, it becomes clearer, and we can even use this in the documentation.

• Example: Get
• Example: Post
• Example: Query strings / params / request details -- Data coercion
• Both examples have type annotations (return+param)
• Post example return a DTO
• Output example demonstrating Event Handler now serializes DTO to JSON
• Make a separate example for OpenAPI Schema only

---

Increase coverage to at least 90%

JSON Encoder for example is missing several tests

[rubenfonseca]

@leandrodamascena I've elevated the coverage now, and I think it's ready for another review

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
22 Code Smells

No Coverage information
10.0% Duplication

[leandrodamascena]

APPROVED!!!