Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend E2E script to allow CDK customization #651

Merged
merged 5 commits into from
May 15, 2024

Conversation

lewijacn
Copy link
Collaborator

@lewijacn lewijacn commented May 13, 2024

Description

This change includes the following changes to allow more customization with our awsE2ESolutionSetup.sh script as well as minor bug fixes:

  • Allow option to migration CDK to provide a file path for loading CDK context and add tests cases around this
  • Update e2e script to load CDK context for both source and target via a provided context file
  • Add clean-up-all option to e2e script for cleaning up all deployed resources after running
  • Modify spots in e2e script to use absolute path from single variable instead of on-the-fly relative paths
  • Bug fix around Custom Resource for getting MSK broker endpoints which still tried to retrieve these endpoints on 'Delete' lambda events

Issues Resolved

https://opensearch.atlassian.net/browse/MIGRATIONS-1690

Testing

CDK tests and manual AWS testing

Check List

  • New functionality includes testing
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tanner Lewis <lewijacn@amazon.com>
Signed-off-by: Tanner Lewis <lewijacn@amazon.com>
Signed-off-by: Tanner Lewis <lewijacn@amazon.com>
Signed-off-by: Tanner Lewis <lewijacn@amazon.com>
Copy link

codecov bot commented May 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.55%. Comparing base (2ae2959) to head (719cab1).
Report is 46 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #651      +/-   ##
============================================
- Coverage     75.84%   75.55%   -0.30%     
+ Complexity     1545     1541       -4     
============================================
  Files           168      168              
  Lines          6446     6455       +9     
  Branches        574      575       +1     
============================================
- Hits           4889     4877      -12     
- Misses         1177     1195      +18     
- Partials        380      383       +3     
Flag Coverage Δ
unittests 75.55% <ø> (-0.30%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@chelma chelma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, makes sense. I have one questions but it's not a blocker.

Comment on lines +123 to +124
// For a context block to be provided as a string (as in the case of providing via command line) it will need to be properly escaped
// to be captured. This requires JSON to parse twice, 1. Returns a normal JSON string with no escaping 2. Returns a JSON object for use
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the comment!

@@ -3,14 +3,20 @@
# Note: As this script will deploy an E2E solution in AWS, it assumes all the dependencies of the migration solution (e.g. aws cli, cdk cli,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No action necessarily required, just a question for you - are you having trouble with maintaining/extending/debugging this script? Have we considered migrating its functionality to something like Python, etc?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm 1000% onboard with migrating this script to python, so much so that I let AI take a swing at it the other day to see how well it could do with it, and with some parameters it did a decent job and might be worth looking into as a reference

Signed-off-by: Tanner Lewis <lewijacn@amazon.com>
@lewijacn lewijacn merged commit 090a040 into opensearch-project:main May 15, 2024
7 of 8 checks passed
@lewijacn lewijacn deleted the extend-e2e-script branch May 23, 2024 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants