New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature][CONNECTORS-V2-Paimon] Piamon Sink supports changelog-procuder is lookup and full-compaction mode #7834

Open

hawk9821 wants to merge 3 commits into apache:dev from hawk9821:paimon_change_producer

Contributor

hawk9821 commented Oct 14, 2024 •

edited

Loading

Purpose of this pull request

Piamon Sink supports writing to the Piamon table whose changelog-procuder is lookup and full-compaction mode.

Does this PR introduce any user-facing change?

How was this patch tested?

e2e: PaimonSinkCDCIT#testChangelogLookup PaimonSinkCDCIT#testChangelogFullCompaction

Check list

If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
If you are contributing the connector code, please check that the following files are updated:
1. Update plugin-mapping.properties and add new connector information in it
2. Update the pom file of seatunnel-dist
3. Add ci label in label-scope-conf
4. Add e2e testcase in seatunnel-e2e
5. Update connector plugin_config
Update the release-note.

github-actions bot added document connectors-v2 e2e paimon labels

hawk9821 force-pushed the paimon_change_producer branch from 9a053d0 to 4084633 Compare

October 14, 2024 11:39

Member

Hisoka-X commented Oct 14, 2024

hawk9821 force-pushed the paimon_change_producer branch 2 times, most recently from 9bf3fd5 to dd69259 Compare

October 16, 2024 08:17

hawk9821 changed the title ~~[Feature][CONNECTORS-V2-Paimon] changelog-producer support the lookup and full-compaction mode~~ [Feature][CONNECTORS-V2-Paimon] Piamon Sink supports changelog-procuder is lookup and full-compaction mode

dailai reviewed

View reviewed changes

docs/en/connector-v2/sink/Paimon.md Outdated Show resolved Hide resolved

dailai reviewed

View reviewed changes

docs/zh/connector-v2/sink/Paimon.md Outdated Show resolved Hide resolved

dailai reviewed

View reviewed changes

...main/java/org/apache/seatunnel/connectors/seatunnel/paimon/catalog/PaimonCatalogFactory.java Outdated Show resolved Hide resolved

dailai reviewed

View reviewed changes

.../src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/config/PaimonSinkConfig.java Outdated Show resolved Hide resolved

dailai reviewed

View reviewed changes

docs/en/connector-v2/sink/Paimon.md Outdated Show resolved Hide resolved

dailai reviewed

View reviewed changes

...on/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSinkWriter.java Outdated Show resolved Hide resolved

dailai reviewed

View reviewed changes

...on/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSinkWriter.java Outdated Show resolved Hide resolved

dailai reviewed

View reviewed changes

...on/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSinkWriter.java Outdated Show resolved Hide resolved

dailai reviewed

View reviewed changes

...on/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSinkWriter.java Outdated Show resolved Hide resolved

dailai reviewed

View reviewed changes

...on/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSinkWriter.java Show resolved Hide resolved

hawk9821 force-pushed the paimon_change_producer branch 2 times, most recently from 6bd8322 to 91b055d Compare

October 17, 2024 13:26

dailai reviewed

View reviewed changes

...on/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSinkWriter.java Outdated Show resolved Hide resolved

dailai previously approved these changes

View reviewed changes

Contributor

dailai left a comment

LGTM

github-actions bot added approved reviewed labels

Contributor

dailai commented Oct 18, 2024

Please fix ci.

hawk9821 dismissed dailai’s stale review via

5e4e266

October 18, 2024 01:58

hawk9821 force-pushed the paimon_change_producer branch from 3af0f38 to 5e4e266 Compare

October 18, 2024 01:58

github-actions bot removed approved reviewed labels

dailai previously approved these changes

View reviewed changes

github-actions bot added approved reviewed labels

Hisoka-X reviewed

View reviewed changes

docs/en/connector-v2/sink/Paimon.md Outdated Show resolved Hide resolved

docs/zh/connector-v2/sink/Paimon.md Outdated Show resolved Hide resolved

...ctor-paimon-e2e/src/test/java/org/apache/seatunnel/e2e/connector/paimon/CancelStreamJob.java Outdated Show resolved Hide resolved

hawk9821 dismissed dailai’s stale review via

6cb4867

October 18, 2024 08:25

hailin0 previously approved these changes

View reviewed changes


          [Feature][CONNECTORS-V2-Paimon]changelog-producer support the lookup …

e39701e

…and full-compaction mode

hawk9821 dismissed hailin0’s stale review via

e39701e

October 18, 2024 13:26

hawk9821 force-pushed the paimon_change_producer branch from b957002 to e39701e Compare

October 18, 2024 13:26

github-actions bot removed approved reviewed labels

hawk9821 added 2 commits

October 18, 2024 21:38


          [Feature][CONNECTORS-V2-Paimon]changelog-producer support the lookup …

…and full-compaction mode


          [Feature][CONNECTORS-V2-Paimon]changelog-producer support the lookup …

12816b2

…and full-compaction mode

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

connectors-v2 document e2e paimon