Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

E2E tests - Parallel Run and Improve test Isolation #133

Open
1 task
jpnovais opened this issue Oct 2, 2024 · 1 comment
Open
1 task

E2E tests - Parallel Run and Improve test Isolation #133

jpnovais opened this issue Oct 2, 2024 · 1 comment
Assignees
@jonesho @VGau
Labels
Contracts Smart Contract related Coordinator

Comments

@jpnovais
Copy link
Collaborator

jpnovais commented Oct 2, 2024

Description

End2End tests shall refactored to:

  • allow them to run in parallel and
  • have better isolation

Refactor Suggestions

One pinpoint at the moment is that we need to configure loads of environment variables in multiple files and know them by heart. This does not scale well and is error-prone.

Alternatively, we can he a global setup helper(s) for the test setup, that is responsible:

  • provide the overall configs: l1-endpoint, l2-endpoint, smart contract address etc
  • create l1/l2 accounts on the fly and fund them from whale accounts in the genesis

Motivation

  • Simulate Real-World Scenarios: e.g Coordinator & Message anchoring will benefit from being restarted when other things are happening. This is like in production
  • Speed up CI times
  • Shift-Left - Faster feedback loop - by reducing e2e times developers are more likely to run them locally.

Acceptance criteria

  • All the tests suites run in parallel, including the coordinator-restart
@jpnovais jpnovais added Coordinator Contracts Smart Contract related labels Oct 2, 2024
@VGau VGau self-assigned this Oct 4, 2024
@VGau VGau mentioned this issue Oct 9, 2024
Merged
3 tasks
VGau added a commit that referenced this issue Oct 15, 2024
@VGau
Feat(#133): run e2e tests in parallel (#164)
215b60c 
* feat: run e2e tests in paralell + refactor test setup

* fix: open handle issue

* fix: sequencer finalized tag test issue + clean l2.spec tests

* fix: refactor config structure

* fix: genesis path issue

* fix: pnpm lock file issue

* fix: make each test run concurrently

* fix: remove nested describes in tests

* fix: refactor coordinator tests + add global L2 traffic generation

* fix: change l2 genesis file for getting whale accounts + refactor utils function

* fix: refactor coordinator restart util function

* fix: remove conflation e2e tests

* fix: add environment variable in e2e test ci workflowk

* fix: coordinator restart test issue

* fix: instanciate account manager only once per test file

* fix: add restart option to zkbesu shomei and postman in docker compose file

* fix: remove getAndIncreaseFeeData function and unsed utils functions

* fix: increase messaging L2 to l1 timeout
@Filter94
Copy link
Collaborator

Filter94 commented Oct 16, 2024 •
edited
Loading

I think there are a few things to improve still:

  • NodeJs installation alone takes 24 seconds which becomes a considerable share of total E2E time. Can we speed it up with cache?
  • Env spin up takes 1m 30 seconds. I think we can't speed up docker containers startup by much. I suspect most of the time is taken by contracts deployments. I know we had some effort in the past to speed deployments up, but I'd just like to revisit this, if we make it in an optimal way
  • E2E testsuite itself takes 2 minutes, longest of which is src/submission-finalization.spec.ts (106.366 s). We should see if there is a way to reduce the latency between L2 transaction sending to L1 finalization. As a compromise, we should consider using latest tag instead of finalize. Another thing we should revisist is Coordinator configuration. Some polling intervals and delays might be slowing us down.

@jonesho I think it would be good for you to take a look at that as you're proficient in the stack as a whole, as well as the typescript part of the E2E tests

@VGau VGau mentioned this issue Oct 23, 2024
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jonesho jonesho

@VGau VGau

Labels
Contracts Smart Contract related Coordinator
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@jpnovais @Filter94 @julien-marchand @jonesho @VGau