Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Throw TrinoException when Iceberg commit fails #24042

Merged
merged 1 commit into from
Nov 6, 2024

Conversation

pajaks
Copy link
Member

@pajaks pajaks commented Nov 6, 2024

Description

In case Iceberg commit fails throw TrinoException with clear error type.

Additional context and related issues

Example stack trace:

org.apache.iceberg.exceptions.CommitFailedException: Commit failed: Failed to commit to table ...
org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:89)
org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:80)
org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:211)
org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:323)
org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:262)
org.apache.iceberg.rest.HTTPClient.post(HTTPClient.java:368)
org.apache.iceberg.rest.RESTClient.post(RESTClient.java:112)
org.apache.iceberg.rest.RESTTableOperations.commit(RESTTableOperations.java:152)
org.apache.iceberg.BaseTransaction.lambda$commitSimpleTransaction$3(BaseTransaction.java:427)
org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
org.apache.iceberg.BaseTransaction.commitSimpleTransaction(BaseTransaction.java:423)
org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:318)
io.trino.plugin.iceberg.IcebergMetadata.commitTransaction(IcebergMetadata.java:1850)
io.trino.plugin.iceberg.IcebergMetadata.commitUpdateAndTransaction(IcebergMetadata.java:1840)
io.trino.plugin.iceberg.IcebergMetadata.finishInsert(IcebergMetadata.java:1323)

and

org.apache.iceberg.exceptions.CommitStateUnknownException: Service failed: 500: Invalid snapshot with sequence number 73 greater than last sequence number 72
Cannot determine whether the commit was successful or not, the underlying data files may or may not be needed. Manual intervention via the Remove Orphan Files Action can remove these files when a connection to the Catalog can be re-established if the commit was actually unsuccessful.
Please check to see whether or not your commit was successful before retrying this commit. Retrying an already successful operation will result in duplicate records or unintentional modifications.
At this time no files will be deleted including possibly unused manifest lists.
org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:94)
org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:80)
org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:211)
org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:323)
org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:262)
org.apache.iceberg.rest.HTTPClient.post(HTTPClient.java:368)
org.apache.iceberg.rest.RESTClient.post(RESTClient.java:112)
org.apache.iceberg.rest.RESTTableOperations.commit(RESTTableOperations.java:152)
org.apache.iceberg.BaseTransaction.lambda$commitReplaceTransaction$1(BaseTransaction.java:381)
org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
org.apache.iceberg.BaseTransaction.commitReplaceTransaction(BaseTransaction.java:365)
org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:310)
io.trino.plugin.iceberg.IcebergMetadata.commitTransaction(IcebergMetadata.java:1850)
io.trino.plugin.iceberg.IcebergMetadata.commitUpdateAndTransaction(IcebergMetadata.java:1840)
io.trino.plugin.iceberg.IcebergMetadata.finishCreateTable(IcebergMetadata.java:1124)
io.trino.plugin.iceberg.IcebergMetadata.createTable(IcebergMetadata.java:991)

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Nov 6, 2024
@github-actions github-actions bot added the iceberg Iceberg connector label Nov 6, 2024
@pajaks pajaks requested review from losipiuk and ebyhr November 6, 2024 10:50
@losipiuk losipiuk merged commit d79e9be into trinodb:master Nov 6, 2024
42 checks passed
@github-actions github-actions bot added this to the 465 milestone Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed iceberg Iceberg connector
Development

Successfully merging this pull request may close these issues.

3 participants