Dify on AWS with CDK

Self-host Dify, an LLM app development platform, using AWS managed services with AWS CDK.

Key Features:

• Fully managed services requiring less maintenance effort
  • Aurora servereless v2, ElastiCache, ECS Fargate, etc.
• Cost effective architectural decisions
  • allow to use NAT instances instead of NAT Gateway, and Fargate spot capacity by default
• Easily integrate with Bedrock models and Knowledge Bases

本リポジトリの使い方について、日本語で書かれた記事もあります:

• AWS CDKでDifyを一撃構築 ( CloudShell版 )
• AWSマネージドサービスで Dify のセルフホスティングを試してみた

Prerequisites

You must have the following dependencies installed to deploy this app:

• Node.js (v18 or newer)
• Docker
• AWS CLI and IAM profile with Administrator policy

Deploy

You can adjust configuration parameters such as AWS regions by modifying bin/cdk.ts. Please also check DifyOnAwsStackProps interface for all the available parameters.

Then you can run the following commands to deploy the entire stack.

# install npm dependencies
npm ci
# bootstrap the AWS account (required only once per account and region)
npx cdk bootstrap
# deploy the CDK stack
npx cdk deploy

The initial deployment usually takes about 20 minutes. After a successful deployment, you will get the URL for the app.

 ✅  DifyOnAwsCdkStack

✨  Deployment time: 326.43s

Outputs:
DifyOnAwsStack.DifyUrl = https://dify.example.com

You can open the URL with a browser and get started!

Deploy from CloudShell

When you deploy from CloudShell, you can use a dedicated script that works even in an environment with limited storage space.

In CloudShell, run the following commands:

git clone https://github.com/aws-samples/dify-self-hosted-on-aws.git
./simple-deploy.sh

Then follow the prompts from the shell script. You will finally get the DifyOnAwsStack.DifyUrl output in the CLI.

Tips

Here is the information that might help when you use Dify on AWS.

Setup Dify to use Bedrock

After logged in, you can setup Dify to use Bedrock LLMs.

Important

Before setting up models in Dify, you have to enable models you want to use in Bedrock management console. Please read this document for more details.

Go to settings by clicking the right-top profile, click WORKSPACE -> Model Provider, and select AWS Bedrock model.

IAM policies are already configured properly, so you can just select a correct AWS region (where the models are enabled) to use Bedrock models, and click Save.

Add Python packages available in code execution

You can add Python packages that is available in Dify code execution feature. Edit python-requirements.txt following the Requirements File Format.

In some libraries, you have to allow additonal system calls in Dify sandbox. This CDK project let you to allow all the system calls by allowAnySysCalls flag in bin/cdk.ts.

Warning

If you enable allowAnySysCalls flag, please make sure that code executed in your Dify tenant can be fully trusted.

Please also refer to this blog article for more details: Using any Python libraries in Dify's code block

Connect to Bedrock Knowledge Bases

You can use the External Knowledge Base feature to connect to Amazon Bedrock Knowledge Bases. Because the external knowledge API is deployed as a sidecar of Dify API, you can use the feature immediately with the following steps:

1. Click Dify -> Knowledge -> Add an External Knowledge API button.
   • 
2. Fill the form as below:
   1. Name: any name as you like (e.g. Bedrock Knowledge Bases)
   2. API Endpoint: http://localhost:8000
   3. API Key: dummy-key (you can configure it by editing BEARER_TOKEN environment variable in api.ts.)
3. Click Dify -> Knowledge -> Create Knowledge -> Connect to an External Knowledge Base
   • 
4. Fill the form as below
   1. External Knowledge Name / Knowledge Description: any string
   2. External Knowledge API: the external API you created in the previous step
   3. External Knowledge ID: The Bedrock Knowledge Base ID you want to use. The AWS region is us-west-2 by default, but you can override the AWS region by adding region prefix with colon, e.g. us-east-1:QWERTYASDF.
5. Now you can use the knowledge from Dify tools.

For more information, please refer to this article: Dify can also do RAG on documents with charts and graphs!

Clean up

To avoid incurring future charges, clean up the resources you created.

npx cdk destroy --force
# If you encountered an error during the deletion, please retry. It happens sometimes.

Cost

The following table provides a sample cost breakdown for deploying this system in the us-east-1 (N. Virginia) region for one month (when deployed using cheap configuration).

AWS service	Dimensions	Cost [USD]
RDS Aurora	Postgres Serverless v2 (0 ACU)	$0
ElastiCache	Valkey t4g.micro	$9.2
ECS (Fargate)	Dify-web 1 task running 24/7 (256CPU)	$2.7
ECS (Fargate)	Dify-api 1 task running 24/7 (1024CPU)	$10.7
ECS (Fargate)	Dify-worker 1 task running 24/7 (1024CPU)	$10.7
Application Load Balancer	ALB-hour per month	$17.5
VPC	NAT Instances t4g.nano x1	$3.0
TOTAL	estimate per month	$53.8

Note that you have to pay LLM cost (e.g. Amazon Bedrock ) in addition to the above, which totally depends on your specific use case.

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file. You should also check Dify's license.

Acknowledgement

This CDK code is heavily inspired by dify-aws-terraform.