Feature request: Add support for API doc auto-gen (SwaggerUI/OpenAPI Specification) from routes

[gwlester]

Use case

It would be useful to allow the code to self-document the API.

In particular, it would be helpful to be able to:

1. Autogenerate a SwaggerUI
2. Be able to generate (this could be a separate utility) the Swagger definition file of the API to import into the API Gateway

Solution/User Experience

Something like flasgger but for Powertools.

Alternative solutions

No response

Acknowledgment

• This feature request meets Lambda Powertools Tenets
• Should this be considered in other Lambda Powertools languages? i.e. Java, TypeScript

[michaelbrewer]

@gwlester - this was also raised in a discussion thread. It should not be hard to add, but it might increase the overall size of the library.

[michaelbrewer]

Here is an example closer to what Powertools does https://github.com/vincentsarago/lambda-proxy

[gwlester]

Maybe this should be in a "sister" library to powertool with powertools providing only a stub implementation?

That way the library size would be kept down.

[michaelbrewer]

I have to assume that a modular version of powertools will happen. then api gw handler would just be a small library additional to the core powertools.

[gwlester]

Modular would be nice.

[heitorlessa]

Hi @gwlester thank you for taking the time to create this feature request. I'd like to ask you a few questions to better understand it.

• Would this be at runtime (/swagger, /redoc) or during development? If the latter, could you update the issue to share your thoughts on what the experience would look like?
• Would that be a replacement for what Amazon API Gateway export provides you?
  • If not a replacement, what are your thoughts on ensuring accuracy beyond view routes? e.g authorization, validation, infra decoupling, etc.
  • If it's a replacement, what are your thoughts on mechanisms to prevent unauthorized access?

• Do you see this starting as a spec generation and eventually expanding to other areas covered by flasgger e.g., validation, from_dict, YAML, HTML sanitization, extending schema, code generation (create API from spec), etc.?
• If you were to create an external library, what information would you need us to provide to ease your maintenance?

Also thank you for sharing about flasgger, I wasn't familiar and I will read more about it.

Thanks!

[gwlester]

@heitorlessa ,

This came up when onboarding a new whose previous experience was with Flask. The person asked how to do this with Powertools -- and it seemed like a missing feature to so I wrote it up,

I've actually not thought about the questions you bring up.

[heitorlessa]

thanks for the additional background @gwlester :) I suspected it was the case but didn't want to assume anything. I've added it to our Ideas backlog so we can explore those answers as we hear more from customers.

As of now, we try to separate what API GW/AppSync offers natively vs what you can do better at runtime. This helps with operational excellence along with common security concerns in the long term. That being said, we'll continuously hear from customers on what bridges need to be built to make that transition easier - sometimes it's code, a separate lib, or documentation with a more opinionated guide (e.g., coming from Flask? coming from Django? etc.)

These lines might blur a bit more as soon as we add official support for Lambda Function URL, along with middleware support for Event Handler to ease common concerns with Function URL.

Thanks!

[kmcquade]

@heitorlessa - I'd also love to have a way to get Swagger docs automatically from Powertools. It's one of the downsides of Powertools - I absolutely love Powertools compared to FastAPI for example for usage in Lambda functions, but FastAPI automatically generates Swagger docs. See example: https://fastapi.tiangolo.com/#interactive-api-docs

Some brainstorming:

In the case of FastAPI, they use Pydantic to control API inputs and it's pretty strongly enforced. Perhaps Powertools could generate Swagger docs from a few data sources. Basically one would need (1) Routes, (2) Parameters and optionally a Request Example, (3) HTTP errors, and (4) Response Data.

1. Routes - all the get/post/put/delete routes from APIGatewayHttpResolver or APIGatewayRestResolver
2. Parameters Use JSON Schemas of models to generate the OpenAPI generated schema, taking inspiration from FastAPI here. Let's say that the user is supplying their JSONSchema to the Powertools schema validator.
3. HTTP errors: This could be based on the Powertools HTTP Errors
4. Response Data: Either the typical Response class or the usage of Powertools Fine grained responses

Caveat: I don't know how to do this myself so I'm really just brainstorming here. Just wanted to provide some ideas that would make sense to me as a developer using this library - do with it what you will.

With that being said, the operational and security advantage that this would give customers would be incredibly useful. I know it would require developers to adjust their code and ascribe to certain conventions to take advantage of these features... but the effort in doing so would have huge payoffs so I think developers would be willing to do it (I certainly would be willing to!).

Hope this helps. Thanks again for such an amazing framework.

[heitorlessa]

hey @kmcquade great to hear from you again, and thanks a lot for sharing you're interested in this feature! I left some questions above to help think this through more carefully.

I do see the value of integrated swagger in a traditional web framework but when it comes to Lambda and API Gateway the boundaries are a bit more complex - see this discussion in Chalice too: aws/chalice#36

I think a RFC would be great here to dive into the initial questions I posed plus other angles we're likely missing - from Powertools, we're happy to facilitate changes that could enable this from an external library.

As a background, this discussion often comes from two fronts: 1/ Someone is coming from Flask/FastAPI and want a similar feature set, and 2/ Someone is using API Gateway primarily as a proxy and sending all requests to Lambda. For 1/, it's a mix of enablement to explain the differences in approaches in Serverless (we could create guides here!). For 2/, this feature becomes useful despite a customer not fully utilizing API GW request validation and export to OpenAPI features.

For the record, I also love FastAPI and its capabilities. We could head towards that direction in future 2.0 version, however that doesn't solve the issue of API Gateway having X routes to X Y Z functions, where each function would have a different Swagger URL if we were to implement this feature this way.

Look forward to hearing everyone thoughts on those questions above and a RFC to dive into the approach(es) we could take here.

[heitorlessa]

Sharing an article that @kmcquade sent me about FastAPI + Lambda Powertools. It uses API Gateway mainly as a proxy, mangum library as a ASGI Lambda adaptor, and it show cases FastAPI auto documentation with SwaggerUI and redoc (fancier).

https://www.eliasbrange.dev/posts/observability-with-fastapi-aws-lambda-powertools/

[kmcquade]

@heitorlessa - thanks for the patience - I know I promised you a more detailed example, and the script that we are using at my startup to (badly) document the Lambda functions based on Pydantic models.

I put together a GitHub repository with:

1. The script we are using
2. A detailed README outlining how it works and how to use it
3. More details in the README about "What amazing would look like" - i.e., how Powertools could solve this problem.

https://github.com/kmcquade/powertools-autodoc-proposal/

Note: I have moved away from the opinion that generating Swagger docs/OpenAPI Specs from Lambda functions would be the best option here. In my opinion, a well formatted README on how to call the Lambda function would be sufficient, and also easier to implement.

In my dream world, if a Lambda function used the event_parser utility and fed the event_parser a Pydantic model, you could auto-generate a README document that would tell the reader how to call the Lambda function with the required arguments.

Here's an example of what the generated README would look like:

Let me know if you have any questions! Happy to chat about it here, or hmu in my Twitter DMs :)

[andyreagan]

We have this same request, +1 on adding support for this. Having used FastAPI, having this out of the box was very nice.

@kmcquade I like your idea - using the event parser decorator, but what about the return value?

[leandrodamascena]

Hi @kmcquade and @andyreagan, thanks for sharing ideas and examples of how to implement this.

We are finalizing some other tasks that were in our pipeline. We will revisit this issue soon to continue the discussion and make a decision on which way to go next.

[pharindoko]

This would be amazing.
This is a feature that is missing for powertools to make it a simple api framework for serverless.

The swagger ui could be stored in a s3 bucket and requested using a route e.g. /documentation or /spec on the same api gateway in combination with a s3 integration.

[heitorlessa]

hey everyone, we'd like to give you an update on timelines now that we've got Observability Providers in a good shape.

We're going to prioritize related features: (1) Event Handler Middleware support, (2) Support request data injection via type annotation (FastAPI style), and then (3) OpenAPI auto-generation.

The first two are paramount before we support OpenAPI, so we're prioritizing them in our next iteration: May 8th-May 19th. We keep hearing from more customers about this need, and a new trend of customers migrating to Functions URL - this is therefore a public ACK that we want to prioritize this feature.

We're not entirely sure of all the corner cases of OpenAPI when following the auto-generation route. Hopefully we might be able to use OpenAPI libraries that stood the test of time for Flask or something like Chalice-Spec.

Regardless, it's clear that this is a big endeavour. We will be cautious and do our due diligence in performance, auth, docs, maintenance effort, and developer experience ergonomics as always - so please bear with us and we'd always welcome any help in that regard.

If anyone would like to create a POC in a fork and/or post UX ideas here, please do so! As of now, I think this requires a new Event Handler Resolver as it'll require Pydantic as a dependency and we might need to build out OpenAPI support from scratch if no lib fits.

Thanks a lot for everyone patience, truly.

[heitorlessa]

This would be amazing. This is a feature that is missing for powertools to make it a simple api framework for serverless.

The swagger ui could be stored in a s3 bucket and requested using a route e.g. /documentation or /spec on the same api gateway in combination with a s3 integration.

hey @pharindoko thanks a lot for expressing interest ;)

Quick question on that, by Swagger UI you mean the Spec or the entire UI and its customizations (e.g., HTML, etc.)?

I'm curious on the latter while the former somewhat worries me as this can lead to drift (API changes and the Spec doesn't).

[pharindoko]

Hey @heitorlessa
I mean the swagger ui component (html,css) itself.
But I guess this can be solved in a later step.

The initial step to generate a spec and expose it on a additional route (that can be customized) is the important one.

[FoodyFood]

I'm glad to see this progressing, it is of great interest to me also :)

[gomboc1985]

Hello @heitorlessa, @leandrodamascena suggested me to comment about the priority. To me it looks fundamental for every client-facing application. In my case:

1. Users already write code with a Flask/FastAPI (@kmcquade thanks for quoting it!) user-experience but the lack of automatic documentation make the deliverable of the API to client not sustainable as the process is manual and slow and error-prone. In short: it's not AWS-style :)

2. Swagger-like pages can be stored wherever you like (S3 seem a good choice) - adding related-configuration at programming time could be a good idea: for example config={'doc-target': {'type':'s3', 'bucket-key': 'foo-bar-'baz', ...} ...} that could be fed to APIGatewayRestResolver (app = APIGatewayRestResolver(config = config)).

3. Access rights should be aligned by definition: if you can access the API (e.g. by submitting an AuthorizationToken) then with the same authorization you should access the documentation page.

This would ensure consistency between API and documentation by default and, again, as a AWS client I feel this is a must-have feature otherwise I would always be in a painful position whenever delivering my APIs to clients.

[rubenfonseca]

We just created an initial RFC for adding support for autogenerating OpenAPI docs from routes! Your feedback is very welcome! #2421

[heitorlessa]

Quick update that we continue to work on this and might have a first draft PR by EOW.

[leandrodamascena]

Hey everyone! We are excited to let you know that we have completed the first step to add support for OpenAPI Spec + SwaggerUI. After intense months of work, we have finally added support and plan to release this feature in the next version! We are writing the documentation to explain how to use this new feature.

This first version brings several functionalities, such as:

1 - Integration with Pydantic
2 - Dataclass support
3 - Support for serialization of different types of data in addition to JSON
4 - Data type validation
5 - Inference to know the types of input and output data and create models/bodyRequest
6 - Integration with all available resolvers: Rest, HTTP, ALB, LambdaFunction, VPC Lattice, and in the near future Bedrock Agent.
7 - SwaggerUI with self-hosted UI or built-in in the package.
8 - And several other things.

We know that we have things to improve, we have to develop a CLI to make your lives easier. Still, this first version is a big step in the project and fulfills our main objective: improving the development experience in Lambda environments.

Thank you for all your support, collaboration, and contributions. 🚀 🥇

[heitorlessa]

Resetting status as @rubenfonseca will work on documentation so it's ready to launch at re:Invent

[heitorlessa]

This is now released in 2.28.0 🎉 🎉 THANK YOU everyone who provided feedback on so many levels.

You'll need Pydantic as a dependency. Everything else is taken care of

Docs: https://docs.powertools.aws.dev/lambda/python/latest/core/event_handler/api_gateway/#data-validation

We'll work with the community after re:invent to start gathering ideas on how we could export OpenAPI Fragments to benefit customers following a micro-function architecture pattern.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[kmcquade]

It's great to see this rolled out. Thanks so much @heitorlessa!