Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Source-Stripe: Add incremental stream support to accounts stream #46864

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

williamkaper
Copy link

@williamkaper williamkaper commented Oct 12, 2024

What

This PR adds incremental sync support to the accounts stream (I did not create an issue because I figured I'd just submit the change via PR)

How

I followed the pattern for other UpdatedCursorIncrementalStripeStreams and changed it from StripeStream to UpdatedCursorIncrementalStripeStream keying off of the events for a new updated cursor column that I added to the ERD.

Review guide

  1. erd/discovered_catalog.json - Added new columns and new sync types to accouts
  2. erd/source.dbml - Added new columns to erd
  3. integration_tests/configured_catalog.json
  4. source_strip/schemas/accounts.json - added updated column
  5. source.py -- Added new streaming operations for accounts stream
  6. docs/stripe.md -- Fixed a bad link that was in main
  7. PyProject.toml and metadata.yaml updated

**** NOTE:** I did not update the unit_tests/integration for test_accounts.py. I was going to use one of the other tests in the holder to try to rewrite it, but it was beyond me. The tests are passing, but that test suite isn't testing the incremental part of the stream. I'm hoping a member of the Airbyte Stripe source connector team can pick that up.

User Impact

For merchants with large amounts of connected accounts, the stream update won't take hours to keep repulling needlessly.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

Copy link

vercel bot commented Oct 12, 2024

@williamkaper is attempting to deploy a commit to the Airbyte Growth Team on Vercel.

A member of the Team first needs to authorize it.

@williamkaper williamkaper force-pushed the williamkaper/source-stripe-account-stream-incremental branch from 7ec7dc4 to c017906 Compare October 12, 2024 17:16
@williamkaper williamkaper force-pushed the williamkaper/source-stripe-account-stream-incremental branch from c017906 to 5bc1351 Compare October 12, 2024 17:18
@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Oct 12, 2024
@@ -127,7 +127,7 @@ The Stripe source connector supports the following streams:
- [Transactions](https://stripe.com/docs/api/transfers/list) \(Incremental\)
- [Transfers](https://stripe.com/docs/api/transfers/list) \(Incremental\)
- [Transfer Reversals](https://stripe.com/docs/api/transfer_reversals/list)
- [Usage Records](https://stripe.com/docs/api/usage_records/subscription_item_summary_list)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was causing the tests to fail - not sure how this got here...

@natikgadzhi
Copy link
Contributor

natikgadzhi commented Oct 12, 2024

/format-fix

Format-fix job started... Check job output.

🟦 Job completed successfully (no changes).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation community connectors/source/stripe
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants