Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Bulk load cdk: add checkpointing test #46749

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

edgao
Copy link
Contributor

@edgao edgao commented Oct 11, 2024

What

How

Review guide

User Impact

Can this PR be safely reverted and rolled back?

  • YES πŸ’š
  • NO ❌

Copy link

vercel bot commented Oct 11, 2024

The latest updates on your projects. Learn more about Vercel for Git β†—οΈŽ

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Oct 11, 2024 11:53pm

@octavia-squidington-iii octavia-squidington-iii added area/connectors Connector related issues CDK Connector Development Kit connectors/destination/dev-null labels Oct 11, 2024
*
*/
@Test
open fun testMidSyncCheckpointingStreamState() = runTest {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't want to use runTest for tests that wrap the destination process? I'm assuming the delay-suppressing behavior propagates to the application context? If so there are probably a couple of processes that either loop and sleep (like TimedForcedFlush) or busy wait (like awaiting flushes at the end) that are now blocking.

Probably the latter? Because it's taking the lock and checking the checkpoint maps over and over without giving anyone else a chance to flush?

Just a guess, try it with runBlocking

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues CDK Connector Development Kit connectors/destination/dev-null
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants