Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert cron, ceramic and websockets into monorepo apps #4556

Merged
merged 52 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
de44201
clean up sub-apps
mattcasey Sep 7, 2024
bf75c79
add yml for websockets and cron
mattcasey Sep 7, 2024
3ec9de6
fix install
mattcasey Sep 7, 2024
3cbd853
fix types for theme
mattcasey Sep 7, 2024
a0acd34
add test env files
mattcasey Sep 7, 2024
f3ea4db
add test command to sockets
mattcasey Sep 7, 2024
7c11591
fix socket tests
mattcasey Sep 7, 2024
cc45102
add test command for scouts
mattcasey Sep 7, 2024
9526f51
add docker files
mattcasey Sep 7, 2024
65468d5
fix types
mattcasey Sep 7, 2024
3771435
fix dockerfile
mattcasey Sep 7, 2024
474333f
fix install args
mattcasey Sep 7, 2024
0bb515b
fix tpl
mattcasey Sep 7, 2024
75d2cd7
..
mattcasey Sep 7, 2024
25adbff
fix script
mattcasey Sep 7, 2024
e71acf5
fix checkout
mattcasey Sep 7, 2024
42ad90e
add to staging cleanup
mattcasey Sep 7, 2024
2e5ec8d
add to destroy staging
mattcasey Sep 7, 2024
6159a9f
pass secrest in
mattcasey Sep 7, 2024
9292e6f
remove env
mattcasey Sep 7, 2024
4e97d50
ci everything
mattcasey Sep 8, 2024
ea4f8fd
rebuild npm
mattcasey Sep 8, 2024
988f896
try installing again
mattcasey Sep 8, 2024
2a3dc8f
remove use of dotenv
mattcasey Sep 8, 2024
9e14f86
allow custom health check
mattcasey Sep 8, 2024
a34e218
fix env for deploy
mattcasey Sep 8, 2024
0a2b779
copy generated files
mattcasey Sep 8, 2024
5162754
update health check port
mattcasey Sep 8, 2024
13b1385
update prod stack
mattcasey Sep 8, 2024
e92aba9
remove ssl if one is not specified
mattcasey Sep 8, 2024
8b0c77a
fix cofnig
mattcasey Sep 8, 2024
6f09889
fix staging config
mattcasey Sep 8, 2024
283b51b
override port of default process health check
mattcasey Sep 8, 2024
024be09
fix staging github destroy
mattcasey Sep 8, 2024
95d19d2
fix command
mattcasey Sep 8, 2024
24bf7de
fix edxports in cv
mattcasey Sep 8, 2024
955c11a
update health check
mattcasey Sep 8, 2024
6c5cad9
fix health check
mattcasey Sep 8, 2024
58be596
update ports on health check
mattcasey Sep 8, 2024
662852d
update staging
mattcasey Sep 8, 2024
238e89c
remove cron tests
mattcasey Sep 8, 2024
6c0507b
build scokets
mattcasey Sep 8, 2024
c062ebb
try pointing to other socket env
mattcasey Sep 9, 2024
26fb99d
remove sockets from staging
mattcasey Sep 9, 2024
5dcc56d
dont use special port
mattcasey Sep 9, 2024
ac52dca
cleanup webapp build
mattcasey Sep 9, 2024
2b5bb01
fix socket staging
mattcasey Sep 9, 2024
9d2cc3c
..
mattcasey Sep 9, 2024
9a5eef0
fix eslint
mattcasey Sep 9, 2024
6e19f3c
merge main
mattcasey Sep 9, 2024
b3ebbcf
fix e2e for webapp
mattcasey Sep 9, 2024
ac841d2
fix staging again
mattcasey Sep 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
75 changes: 50 additions & 25 deletions .cdk/ProductionStack.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,22 @@
import { CfnTag, Stack, StackProps } from 'aws-cdk-lib';

Check warning on line 1 in .cdk/ProductionStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/ProductionStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/ProductionStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/ProductionStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/ProductionStack.ts

View workflow job for this annotation

GitHub Actions / Test app

File ignored by default.

Check warning on line 1 in .cdk/ProductionStack.ts

View workflow job for this annotation

GitHub Actions / Validate code

File ignored by default.
import * as elasticbeanstalk from 'aws-cdk-lib/aws-elasticbeanstalk';
import * as s3assets from 'aws-cdk-lib/aws-s3-assets';
import { Construct } from 'constructs';

export type Options = {
sslCert: string;
healthCheck?: { port: number; path: string };
sslCert?: string;
};

const defaultHealthCheck = { path: '/api/health', port: 80 };

export class ProductionStack extends Stack {
constructor(scope: Construct, appName: string, props: StackProps, options: Options) {
constructor(
scope: Construct,
appName: string,
props: StackProps,
{ sslCert, healthCheck = defaultHealthCheck }: Options
) {
super(scope, appName, props);

const webAppZipArchive = new s3assets.Asset(this, 'WebAppZip', {
Expand Down Expand Up @@ -45,8 +53,6 @@
Version: 1
};

const sslCert = 'arn:aws:acm:us-east-1:310849459438:certificate/4618b240-08da-4d91-98c1-ac12362be229';

// list of all options: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options-general.html
const optionSettingProperties: elasticbeanstalk.CfnEnvironment.OptionSettingProperty[] = [
{
Expand Down Expand Up @@ -79,26 +85,30 @@
optionName: 'ConfigDocument',
value: JSON.stringify(healthReportingSystemConfig)
},
{
namespace: 'aws:elbv2:listener:443',
optionName: 'Protocol',
value: 'HTTPS'
},
{
namespace: 'aws:elbv2:listener:443',
optionName: 'ListenerEnabled',
value: 'true'
},
{
namespace: 'aws:elbv2:listener:443',
optionName: 'SSLCertificateArns',
value: options.sslCert
},
{
namespace: 'aws:elbv2:listener:443',
optionName: 'SSLPolicy',
value: 'ELBSecurityPolicy-TLS13-1-2-2021-06'
},
...(sslCert
? [
{
namespace: 'aws:elbv2:listener:443',
optionName: 'Protocol',
value: 'HTTPS'
},
{
namespace: 'aws:elbv2:listener:443',
optionName: 'ListenerEnabled',
value: 'true'
},
{
namespace: 'aws:elbv2:listener:443',
optionName: 'SSLCertificateArns',
value: sslCert
},
{
namespace: 'aws:elbv2:listener:443',
optionName: 'SSLPolicy',
value: 'ELBSecurityPolicy-TLS13-1-2-2021-06'
}
]
: []),
{
// add security group to access
namespace: 'aws:autoscaling:launchconfiguration',
Expand Down Expand Up @@ -130,11 +140,26 @@
optionName: 'InstanceTypes',
value: 't3a.small,t3.small'
},
{
namespace: 'aws:elasticbeanstalk:environment:process:default',
optionName: 'HealthCheckPath',
value: healthCheck.path
},
{
namespace: 'aws:elasticbeanstalk:environment:process:default',
optionName: 'MatcherHTTPCode',
value: '200'
},
{
namespace: 'aws:elasticbeanstalk:environment:process:default',
optionName: 'Port',
value: healthCheck.port?.toString() || '80'
},
{
// ALB health check
namespace: 'aws:elasticbeanstalk:application',
optionName: 'Application Healthcheck URL',
value: '/api/health'
value: healthCheck.path
}
];

Expand Down
31 changes: 28 additions & 3 deletions .cdk/StagingStack.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { CfnOutput, CfnTag, Stack, StackProps } from 'aws-cdk-lib';

Check warning on line 1 in .cdk/StagingStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/StagingStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/StagingStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/StagingStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/StagingStack.ts

View workflow job for this annotation

GitHub Actions / Test app

File ignored by default.

Check warning on line 1 in .cdk/StagingStack.ts

View workflow job for this annotation

GitHub Actions / Validate code

File ignored by default.
import * as elasticbeanstalk from 'aws-cdk-lib/aws-elasticbeanstalk';
import * as route53 from 'aws-cdk-lib/aws-route53';
import * as s3assets from 'aws-cdk-lib/aws-s3-assets';
Expand All @@ -6,10 +6,20 @@

const domain = 'charmverse.co';

type CustomOptions = { options?: elasticbeanstalk.CfnEnvironment.OptionSettingProperty[] };
type CustomOptions = {
healthCheck?: { port: number; path: string };
options?: elasticbeanstalk.CfnEnvironment.OptionSettingProperty[];
};

const defaultHealthCheck = { path: '/api/health', port: 80 };

export class StagingStack extends Stack {
constructor(scope: Construct, appName: string, props: StackProps, { options = [] }: CustomOptions = {}) {
constructor(
scope: Construct,
appName: string,
props: StackProps,
{ healthCheck = defaultHealthCheck, options = [] }: CustomOptions = {}
) {
super(scope, appName, props);

const webAppZipArchive = new s3assets.Asset(this, 'WebAppZip', {
Expand Down Expand Up @@ -131,11 +141,26 @@
optionName: 'InstanceTypes',
value: 't3a.small,t3.small'
},
{
namespace: 'aws:elasticbeanstalk:environment:process:default',
optionName: 'HealthCheckPath',
value: healthCheck.path
},
{
namespace: 'aws:elasticbeanstalk:environment:process:default',
optionName: 'MatcherHTTPCode',
value: '200'
},
{
namespace: 'aws:elasticbeanstalk:environment:process:default',
optionName: 'Port',
value: healthCheck.port?.toString() || '80'
},
{
// ALB health check
namespace: 'aws:elasticbeanstalk:application',
optionName: 'Application Healthcheck URL',
value: '/api/health'
value: healthCheck.path
},
{
namespace: 'aws:elasticbeanstalk:application:environment',
Expand Down
74 changes: 37 additions & 37 deletions .cdk/WebappStagingStack.ts
Original file line number Diff line number Diff line change
@@ -1,47 +1,47 @@
import { StagingStack } from './StagingStack';

Check warning on line 1 in .cdk/WebappStagingStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/WebappStagingStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/WebappStagingStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/WebappStagingStack.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/WebappStagingStack.ts

View workflow job for this annotation

GitHub Actions / Test app

File ignored by default.

Check warning on line 1 in .cdk/WebappStagingStack.ts

View workflow job for this annotation

GitHub Actions / Validate code

File ignored by default.
import { StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';

export class WebappStagingStack extends StagingStack {
constructor(scope: Construct, id: string, props: StackProps) {
super(scope, id, props, {
options: [
{
namespace: 'aws:elasticbeanstalk:environment:process:websocket',
optionName: 'HealthCheckPath',
value: '/health_check'
},
{
namespace: 'aws:elasticbeanstalk:environment:process:websocket',
optionName: 'Port',
value: '3002'
},
{
namespace: 'aws:elasticbeanstalk:environment:process:websocket',
optionName: 'Protocol',
value: 'HTTP'
},
{
namespace: 'aws:elbv2:listener:3002',
optionName: 'ListenerEnabled',
value: 'true'
},
{
namespace: 'aws:elbv2:listener:3002',
optionName: 'Protocol',
value: 'HTTPS'
},
{
namespace: 'aws:elbv2:listener:3002',
optionName: 'SSLCertificateArns',
value: 'arn:aws:acm:us-east-1:310849459438:certificate/bfea3120-a440-4667-80fd-d285146f2339'
},
{
namespace: 'aws:elbv2:listener:3002',
optionName: 'DefaultProcess',
value: 'websocket'
}
]
// options: [
// {
// namespace: 'aws:elasticbeanstalk:environment:process:websocket',
// optionName: 'HealthCheckPath',
// value: '/api/health'
// },
// {
// namespace: 'aws:elasticbeanstalk:environment:process:websocket',
// optionName: 'Port',
// value: '3002'
// },
// {
// namespace: 'aws:elasticbeanstalk:environment:process:websocket',
// optionName: 'Protocol',
// value: 'HTTP'
// },
// {
// namespace: 'aws:elbv2:listener:3002',
// optionName: 'ListenerEnabled',
// value: 'true'
// },
// {
// namespace: 'aws:elbv2:listener:3002',
// optionName: 'Protocol',
// value: 'HTTPS'
// },
// {
// namespace: 'aws:elbv2:listener:3002',
// optionName: 'SSLCertificateArns',
// value: 'arn:aws:acm:us-east-1:310849459438:certificate/bfea3120-a440-4667-80fd-d285146f2339'
// },
// {
// namespace: 'aws:elbv2:listener:3002',
// optionName: 'DefaultProcess',
// value: 'websocket'
// }
// ]
});
}
}
11 changes: 11 additions & 0 deletions .cdk/bin/app.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env node

Check warning on line 1 in .cdk/bin/app.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/bin/app.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/bin/app.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/bin/app.ts

View workflow job for this annotation

GitHub Actions / Test apps

File ignored by default.

Check warning on line 1 in .cdk/bin/app.ts

View workflow job for this annotation

GitHub Actions / Test app

File ignored by default.

Check warning on line 1 in .cdk/bin/app.ts

View workflow job for this annotation

GitHub Actions / Validate code

File ignored by default.
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { WebappStagingStack } from '../WebappStagingStack';
Expand Down Expand Up @@ -30,6 +30,13 @@
new ProductionStack(app, stackNameParam, deployProps, {
sslCert: 'arn:aws:acm:us-east-1:310849459438:certificate/b901f27e-5a33-4dea-b4fb-39308a580423'
});
} else if (stackNameParam === 'prd-ceramic') {
new ProductionStack(app, stackNameParam, deployProps, {
healthCheck: {
path: '/graphql',
port: 5001
}
});
}
// Connect webapp and api production
else if (stackNameParam.startsWith('prd')) {
Expand All @@ -38,6 +45,10 @@
});
} else if (stackNameParam.startsWith('stg-webapp')) {
new WebappStagingStack(app, stackNameParam, deployProps);
} else if (stackNameParam.startsWith('stg-websockets')) {
new StagingStack(app, stackNameParam, deployProps, { healthCheck: { path: '/api/health', port: 3002 } });
} else if (stackNameParam.startsWith('stg-ceramic')) {
new StagingStack(app, stackNameParam, deployProps, { healthCheck: { path: '/graphql', port: 5001 } });
} else if (stackNameParam.startsWith('stg-')) {
new StagingStack(app, stackNameParam, deployProps);
} else {
Expand Down
2 changes: 1 addition & 1 deletion .ebextensions/ceramic/00_env_vars.config
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ option_settings:
EBSTALK_ENV_FILE: "ceramic.env"
SERVICE_ENV: "prd" # this sets the value of datadog env tag
SERVICE_NAME: "ceramic"
IMGNAME: "charmverse-web3-workspace"
IMGNAME: "ceramic"
IMGTAG: ""
2 changes: 1 addition & 1 deletion .ebextensions/cron/00_env_vars.config
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ option_settings:
EBSTALK_ENV_FILE: "cron.env"
SERVICE_ENV: "prd"
SERVICE_NAME: "cron"
IMGNAME: "charmverse-web3-workspace"
IMGNAME: "cron"
IMGTAG: ""
2 changes: 1 addition & 1 deletion .ebextensions/websockets/00_env_vars.config
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ option_settings:
EBSTALK_ENV_FILE: "websocket.env"
SERVICE_ENV: "prd" # this sets the value of datadog env tag
SERVICE_NAME: "websockets"
IMGNAME: "charmverse-web3-workspace"
IMGNAME: "websockets"
IMGTAG: ""
2 changes: 1 addition & 1 deletion .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ scripts/
dist/
.next/

background/ceramicServer/generated/
apps/ceramic/src/generated/
4 changes: 2 additions & 2 deletions .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"node": true
},
"settings": {
"import/internal-regex": "^(adapters|background|charmClient|components|config|context|db|hooks|lib|models|pages|public|seedData|testing|theme)",
"import/internal-regex": "^(adapters|charmClient|components|config|context|db|hooks|lib|models|pages|public|seedData|testing|theme)",
"import/resolver": {
"node": {
"moduleDirectory": ["node_modules"]
Expand Down Expand Up @@ -76,7 +76,7 @@
"error",
{
"devDependencies": true,
"packageDir": ["./", "./@connect-shared", "./packages/charmeditor/", "./apps/farcaster/", "./apps/scoutgame/", "./apps/sunnyawards/", "./apps/waitlist/"]
"packageDir": ["./", "./@connect-shared", "./packages/charmeditor/", "./apps/farcaster/", "./apps/scoutgame/", "./apps/sunnyawards/", "./apps/waitlist/", "./apps/websockets/", "./apps/cron/", "./apps/ceramic/"]
}
],
"import/order": [
Expand Down
31 changes: 13 additions & 18 deletions .github/actions/build/action.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
name: 'Build application'
description: ''

inputs:
save_cache:
description: 'Whether to save the cache'
required: false
REACT_APP_APP_ENV:
description: '...'
required: false
Expand All @@ -17,32 +15,29 @@ runs:
# GITHUB_CONTEXT: ${{ toJson(github) }}
# run: echo "$GITHUB_CONTEXT"

- name: Get last Git commit hashes for cache key
shell: bash
# Instead of hasing files, use the latest git commit in a directory to speed up cache key https://github.com/actions/runner/issues/1840#issuecomment-2094847172
run: |
echo "NPM_PACKAGES_COMMIT=$(git log -1 --pretty=format:%H -- package-lock.json)" >> $GITHUB_ENV
echo "SOURCE_CODE_COMMIT=$(git log -1 --pretty=format:%H -- lib components packages public pages hooks charmClient adapters abi config connectors)" >> $GITHUB_ENV

- name: Restore build
id: restore_build
uses: actions/cache/restore@v4
uses: actions/cache@v4
with:
path: |
.next
.env.test.local
dist
background/ceramicServer/generated
key: webapp-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('**/*.ts', '**/*.tsx', '**/*.gql', '**/*.scss', '!node_modules', '!.next', '!dist', '!apps') }}
restore-keys: |
webapp-${{ hashFiles('**/package-lock.json') }}-

- name: Build
shell: bash
if: steps.restore_build.outputs.cache-hit != 'true'
run: npm run build
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we not going to save the build cache anymore 😮


# building cache key takes almost a minute, so we only run this if we are saving the cache
- name: Cache build
if: inputs.save_cache
uses: actions/cache/save@v4
with:
path: |
.next
.env.test.local
dist
background/ceramicServer/generated
key: webapp-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('**/*.ts', '**/*.tsx', '**/*.gql', '**/*.scss', '!node_modules', '!.next', '!dist', '!apps') }}
# Build the sockets for e2e tests to run
run: |
npm run webapp:build
npm run sockets:build
7 changes: 3 additions & 4 deletions .github/actions/build_app/action.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
name: 'Build app'
description: ''

inputs:
app_name:
description: The name of the app'
description: 'The name of the app'
required: true

runs:
Expand All @@ -24,8 +25,6 @@ runs:
path: |
apps/${{ inputs.app_name }}/.next
apps/${{ inputs.app_name }}/public
# Generate a new cache whenever packages or source files change.
# key: app-${{ inputs.app_name }}-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('lib/**/*.ts', 'lib/**/*.tsx', '@connect-shared/**/*.ts', '@connect-shared/**/*.tsx', format('apps/{0}/**/*.ts', inputs.app_name), format('apps/{0}/**/*.tsx', inputs.app_name), format('apps/{0}/**/*.scss', inputs.app_name)) }}
key: app-${{ inputs.app_name }}-${{ env.NPM_PACKAGES_COMMIT }}-${{ env.SOURCE_CODE_COMMIT }}
restore-keys: |
app-${{ inputs.app_name }}-${{ env.NPM_PACKAGES_COMMIT }}-
Expand All @@ -34,4 +33,4 @@ runs:
shell: bash
if: steps.restore_build.outputs.cache-hit != 'true'
run: |
npm run ${{ inputs.app_name }}:build
npm run build -w apps/${{ inputs.app_name }}
Loading
Loading