-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cloud Backend test solution #26
Open
himabindugit
wants to merge
32
commits into
Superformula:main
Choose a base branch
from
himabindugit:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
a7a9456
chore: adding lint tools
himabindugit 9d6485d
chore: add prettier config file
himabindugit c58e2ea
chore: add husky with linting pre-commit pre-push commit-msg hooks
himabindugit 2b7c7e2
chore: add deploy supported scripts
himabindugit ced73c9
chore: added lambda handler
himabindugit 72687e7
chore: package files for dependencies
himabindugit c6e5e5f
chore: added sample.env for reference
himabindugit 8533093
chore: add documentation folder with screenshots
himabindugit c3a7fba
chore: typecript complier config files
himabindugit aa3ff3b
chore: graphql schema file
himabindugit 9781b06
chore: gitignore file
himabindugit 3b85cbf
chore: mocha configuration file
himabindugit cee1771
chore: implementation around env files and global configs
himabindugit 23d4a92
chore: tests utilities
himabindugit f2f5d74
chore: unit tests
himabindugit 2fd5f1b
chore: integration tests
himabindugit cfc64d1
chore: playing around express context and tracing
himabindugit 6579e80
chore: file to calculate coordinates using node-geocoder
himabindugit 0072c8c
chore: starting apollo server file
himabindugit 6acfa44
chore: logging plugin for logs for more info
himabindugit dec91e8
chore: coordinate resolvers
himabindugit 081e625
chore: starting apollo server file
himabindugit ed291c4
chore: typedefs
himabindugit 1c9bb93
chore: logging,types, handler tests
himabindugit 8241fe5
chore: review comments> documentation, more edge cases andtests
himabindugit 99b99da
chore: more documentation changes
himabindugit a610e3d
chore: more tests, used hive plugin, better error handling, more tests
himabindugit 1f0315a
chore: better error handler, tests fixes
himabindugit ce31a0f
chore: consuming Google Geocode API once using data sources and consu…
himabindugit babec8d
chore: shuffling prd and dev dependencies
himabindugit be911c9
chore: tests fix, better jwt unauthorized error handling, unskip tests
himabindugit ce0d627
chore: cleanup and added more tests
himabindugit File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
node_modules/* | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
module.exports = { | ||
parser: '@typescript-eslint/parser', | ||
plugins: ['@typescript-eslint', 'prettier'], | ||
extends: [ | ||
'plugin:@typescript-eslint/eslint-recommended', | ||
'plugin:@typescript-eslint/recommended', | ||
'prettier', | ||
], | ||
parserOptions: { | ||
ecmaVersion: 'latest', | ||
sourceType: 'module', | ||
}, | ||
rules: { | ||
'require-await': 2, | ||
'@typescript-eslint/camelcase': 0, | ||
'prettier/prettier': [ | ||
'error', | ||
{ | ||
singleQuote: true, | ||
trailingComma: 'es5', | ||
arrowParens: 'always', | ||
}, | ||
], | ||
}, | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
node_modules | ||
.env | ||
dist/ | ||
|
||
coverage/* | ||
.nyc_output/* | ||
|
||
# CDK asset staging directory | ||
.cdk.staging | ||
cdk.out | ||
cdk.context.json | ||
|
||
|
||
# Parcel build directories | ||
.cache | ||
.build | ||
nodejs | ||
nodejs.zip | ||
.DS_Store | ||
conf/.env | ||
hive.json |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
#!/bin/sh | ||
. "$(dirname "$0")/_/husky.sh" | ||
|
||
npm run commit-msg |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
#!/bin/sh | ||
. "$(dirname "$0")/_/husky.sh" | ||
|
||
npm run pre-commit |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
#!/bin/sh | ||
. "$(dirname "$0")/_/husky.sh" | ||
|
||
npm run pre-push |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
{ | ||
"diff": true, | ||
"extension": ["ts"], | ||
"opts": false, | ||
"sort": true, | ||
"color": true, | ||
"slow": 75, | ||
"timeout": 99999, | ||
"ui": "bdd", | ||
"spec": "src/test/**/*.test.ts", | ||
"require": "ts-node/register", | ||
"reporter": "spec" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
{ | ||
"singleQuote": true | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
SERVICE_PORT= | ||
API_KEY=<YOUR_API_KEY> | ||
JWT_TOKENS_AUTHORITY= | ||
DEPLOY_REGION= | ||
ENV_SHORT= | ||
STACK_NAME= |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
#!/usr/bin/env node | ||
|
||
import * as cdk from '@aws-cdk/core'; | ||
|
||
import { GraphqlLambdaStack } from '../deploy/graphql-stack'; | ||
import conf from '../src/conf'; | ||
import { Environment } from '../src/conf/config'; | ||
|
||
const account = conf.deployAccount; | ||
const region = conf.deployRegion; | ||
const hostedZone = process.env.DEPLOY_ZONE ?? 'us-east-1'; | ||
|
||
const authority = conf.jwtTokens.authority; | ||
const environment = conf.env as Environment; | ||
const serviceName = 'graphql'; | ||
const apiKey = conf.apiKey as string; | ||
const hiveToken = conf.hiveToken as string; | ||
|
||
class GraphQLCDKApp extends cdk.App { | ||
constructor() { | ||
super(); | ||
|
||
new GraphqlLambdaStack(this, `${conf.stackName}`, { | ||
environment, | ||
env: { | ||
account, | ||
region, | ||
}, | ||
hostedZone: hostedZone, | ||
authority, | ||
serviceName, | ||
apiKey, | ||
hiveToken, | ||
}); | ||
} | ||
} | ||
|
||
const app = new GraphQLCDKApp(); | ||
app.synth(); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
{ | ||
"app": "npx ts-node bin/graphql-cdk.ts", | ||
"requireApproval": "never", | ||
"context": { | ||
"@aws-cdk/core:enableStackNameDuplicates": "true", | ||
"aws-cdk:enableDiffNoFail": "true" | ||
} | ||
} | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
#!/bin/bash | ||
|
||
set -e | ||
|
||
# Installing only prod dependencies because of 250MB limitation | ||
# https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html | ||
echo "## Installing prd dependencies..." | ||
npm install --only=prod --loglevel=warn | ||
npm run build -- --project tsconfig.prod.json | ||
|
||
AWS_CREDS_FILE=~/.aws/credentials | ||
if [[ ! -e ~/.aws/credentials ]]; then | ||
mkdir ~/.aws | ||
touch $AWS_CREDS_FILE | ||
chmod 600 $AWS_CREDS_FILE | ||
fi | ||
|
||
grep -qF -- "[graphql-deploy]" ~/.aws/credentials || cat << EndOfFile >> $AWS_CREDS_FILE | ||
[graphql-deploy] | ||
aws_access_key_id=${AWS_ACCESS_KEY_ID} | ||
aws_secret_access_key=${AWS_SECRET_ACCESS_KEY} | ||
EndOfFile | ||
|
||
echo "## Check for existing nodejs directory..." | ||
if [[ ! -e nodejs ]]; then | ||
echo "## Deleting nodejs zip" | ||
rm -rf nodejs.zip | ||
echo "## Creating nodejs directory..." | ||
mkdir nodejs | ||
fi | ||
cd nodejs | ||
cp -r ../node_modules node_modules | ||
cp ../package.json . | ||
cp ../schema.graphql ../dist | ||
cd .. || exit | ||
npm run zip:nodejs | ||
|
||
npm run cdk -- deploy --profile graphql-deploy -o cdk.out | ||
|
||
rm -r nodejs | ||
rm -r nodejs.zip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,78 @@ | ||
import * as cdk from '@aws-cdk/core'; | ||
import * as lambda from '@aws-cdk/aws-lambda'; | ||
import * as apiGateway from '@aws-cdk/aws-apigateway'; | ||
import { Construct } from '@aws-cdk/core'; | ||
import { Duration } from '@aws-cdk/core'; | ||
import * as gql from './graphql'; | ||
// import _ from 'lodash'; | ||
// import * as ec2 from '@aws-cdk/aws-ec2'; | ||
|
||
const createTagger = | ||
(tags: { [key: string]: string }) => (taggable: Construct) => | ||
Object.keys(tags).map((tag) => cdk.Tags.of(taggable).add(tag, tags[tag])); | ||
|
||
export class GraphqlLambdaStack extends cdk.Stack { | ||
constructor(scope: cdk.Construct, id: string, props: gql.StackProps) { | ||
super(scope, id, props); | ||
const tagIt = createTagger({ | ||
env: props.environment, | ||
'provisioned-by': 'cdk', | ||
service: 'graphql', | ||
Name: `${props.environment}-${props.env?.region}-${props.serviceName}`, | ||
}); | ||
// The code that defines your stack goes here | ||
const layer = new lambda.LayerVersion(this, 'NpmModulesLayer', { | ||
code: new lambda.AssetCode('nodejs.zip'), | ||
compatibleRuntimes: [lambda.Runtime.NODEJS_12_X], | ||
description: 'A layer to package modules', | ||
}); | ||
|
||
// If you want, you can configure own VPC, subnets, SGs when creating | ||
// Lambda function. | ||
|
||
// const vpc = ec2.Vpc.fromLookup(this, 'Vpc', { | ||
// vpcId: props.vpcId, | ||
// }); | ||
// const securityGroup = ec2.SecurityGroup.fromSecurityGroupId( | ||
// this, | ||
// 'GraphQLSG', | ||
// 'sg-0e8485c6cad5d3be7', | ||
// { | ||
// mutable: false, | ||
// } | ||
// ); | ||
const graphqlLambda = new lambda.Function(this, 'MessageHandler', { | ||
functionName: 'GraphQLLambda', | ||
description: 'Process GraphQL requests', | ||
// Where our lambda function is located | ||
code: new lambda.AssetCode('dist'), | ||
// What should be executed once the lambda is invoked | ||
// - in that case, the `handler` function exported by `handler.ts` | ||
handler: 'handler.handler', | ||
runtime: lambda.Runtime.NODEJS_12_X, | ||
timeout: Duration.seconds(300), | ||
environment: { | ||
AUTHORITY: props.authority, | ||
API_KEY: props.apiKey, | ||
HIVE_TOKEN: props.hiveToken, | ||
}, | ||
layers: [layer], | ||
// vpc: vpc, | ||
// vpcSubnets: { | ||
// subnets: _.filter(vpc.privateSubnets, (subnet) => { | ||
// return props.subnetIds.includes(subnet.subnetId); | ||
// }), | ||
// }, | ||
// securityGroups: [securityGroup] | ||
}); | ||
tagIt(graphqlLambda); | ||
const apiGatewayLambdaRestApi = new apiGateway.LambdaRestApi( | ||
this, | ||
'graphqlEndpoint', | ||
{ | ||
handler: graphqlLambda, | ||
} | ||
); | ||
tagIt(apiGatewayLambdaRestApi); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
import * as cdk from '@aws-cdk/core'; | ||
|
||
export enum Environment { | ||
Production = 'prd', | ||
Staging = 'stg', | ||
QA = 'qa', | ||
Development = 'dev', | ||
} | ||
|
||
export interface StackProps extends cdk.StackProps { | ||
apiKey: string; | ||
hiveToken: string; | ||
environment: Environment; | ||
imageTag?: string; | ||
hostedZone: string; | ||
authority: string; | ||
serviceName: string; | ||
} |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
## Deployment guide | ||
|
||
### Welcome to CDK TypeScript project | ||
|
||
You should explore the contents of this project. It demonstrates a CDK app with an instance of a stack (`DevGraphQLStack`) | ||
which contains a Lambda Function and API Gateway. | ||
|
||
The `cdk.json` file tells the CDK Toolkit how to execute your app.<p> </p> | ||
|
||
### What prerequisites do I need to deploy the service? | ||
|
||
1. Install node and awscli, follow this [link](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) to confugure aws. | ||
|
||
2. Install cdk, `npm install aws-cdk` | ||
This same depedency is mentioned in `package.json` as well, but installing ahead for cdk bootstrap purposes | ||
|
||
3. Run `cdk bootstrap aws://<YOUR_AWS_ACCOUNT>/<YOUR_AWS_REGION>` which will set up a deployment environment on AWS you can use with your default AWS credentials. For example, `cdk bootstrap aws://1234567/us-east-1` | ||
rleonr-sf marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Note: Keep in mind not to install extra packages other than prd dependencies in `package.json`, because of lambda 250MB limitation, for more [details](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html) | ||
|
||
|
||
### <u>How to deploy CDK stack</u> | ||
|
||
deploy.sh script to deploy cdk stacks to Dev, QA, STG, PRD environments. | ||
|
||
`deploy.sh` script is to deploy CDK stacks to AWS | ||
- pass these as command line arguments for `deploy.sh` | ||
AWS_ACCESS_KEY_ID - <YOUR_AWS_ACCESS_KEY_ID> | ||
AWS_SECRET_ACCESS_KEY - <YOUR_AWS_SECRET_ACCESS_KEY> | ||
|
||
eg:, from the root dir: | ||
```bash | ||
AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_KEY_ID> AWS_SECRET_ACCESS_KEY=<YOUR_AWS_SECRET_ACCESS_KEY> bash deploy/deploy.sh | ||
``` | ||
Grab GraphQL endpoint from cdk output. | ||
|
||
Example output | ||
```bash | ||
Outputs: | ||
DevGraphQLStack.graphqlEndpoint711CBBBD = https://6holtoh5t4.execute-api.us-east-1.amazonaws.com/prod/ | ||
``` | ||
<p> </p> | ||
|
||
### Troubleshooting | ||
|
||
1.verify cloudformation stack with right resources like lambda, api gaeway deployed in AWS | ||
2. Clod watch logs are helpful to debug any issues | ||
|
||
### Useful commands | ||
|
||
- `npm run build` compile typescript to js | ||
- `npm run watch` watch for changes and compile | ||
- `npm run test` perform the mocha unit tests | ||
- `cdk deploy` deploy this stack to your default AWS account/region | ||
- `cdk diff` compare deployed stack with current state | ||
- `cdk synth` emits the synthesized CloudFormation template | ||
|
||
NOTE: To deploy, you will need to run `cdk bootstrap` which will set up a deployment environment on AWS you can use with your default AWS credentials. | ||
|
||
- cdk deploy deploy this stack to your default AWS account/region, in graphql case, stack is going to be Lambda Function, Lambda Layer, IAM Role, Policy, API Gateway etc.. | ||
- cdk diff compare deployed stack with current state | ||
- cdk synth emits the synthesized CloudFormation template to ./cdk.out | ||
|
||
[Layers](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-readme.html#layers): using lambda layers to provision code to Lambda function which are node modules. | ||
rleonr-sf marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Check [here](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-readme.html#bundling-asset-code) for more details.<p> </p> | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
### How to run | ||
Once Stack is deployed, access API Gateway in Postman. First update Lambda configuration environment variables with the right API key. By providing right jwt token, you can access GraphQL API by appending `/graphql` at the end. | ||
|
||
#### How to generate API_KEY in Google Cloud Console? | ||
|
||
> Follow this [link](https://developers.google.com/maps/documentation/geocoding/get-api-key) to create API key and restrict to use it for Geocoding API | ||
|
||
### GraphQL Endpoint | ||
|
||
For example, you can hit `https://dqe0uhf4s5.execute-api.us-east-1.amazonaws.com/prod/graphql` and make queries with the right JWT token. Please check `documentation/8_HowToGenerateJWT.md` for more details about how to generate jwt token. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @himabindugit! I'm looking forward to hearing your thoughts following my review :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @rleonr-sf thanks for the detailed review, i have tried my best to answer your questions, i can take an another look tomorrow just to make i haven't missed anything.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @himabindugit, I appreciate the time invested in this submission. We'll get in touch with you with an update soon
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @rleonr-sf Just FYI - I have pushed more changes. I have just included screenshots (success and error cases) using my GraphQL API, am happy to upload interactive demo with using GraphQL API.