Skip to content

[fix](load) Make insert cancellation reliable - #67137

Open
0AyanamiRei wants to merge 1 commit into
apache:masterfrom
0AyanamiRei:fix/nereids-insert-cancel-race
Open

[fix](load) Make insert cancellation reliable#67137
0AyanamiRei wants to merge 1 commit into
apache:masterfrom
0AyanamiRei:fix/nereids-insert-cancel-race

Conversation

@0AyanamiRei

@0AyanamiRei 0AyanamiRei commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary:

INSERT INTO SELECT and CTAS expose two query cancellation races:

  1. The insert coordinator is created before transaction initialization, but is published to StmtExecutor only after beginTransaction and sink finalization. A KILL QUERY, JDBC Statement.cancel(), or REST kill request received before publication finds the connection but no coordinator. The old path reports success and loses the cancellation, so the coordinator can later dispatch fragments and commit.
  2. After coordinator.join() reports success, there is a window before onComplete() starts transaction commit. A cancellation in this window only targets an already completed coordinator, reports success, and does not prevent commit.

The same false-success behavior exists in local REST cancellation and follower-to-master forwarding because those layers do not propagate whether the statement accepted cancellation.

This PR:

  • remembers cancellation in StmtExecutor and replays it when an INSERT or CTAS coordinator is published;
  • checks coordinator cancellation while holding the dispatch lock before sending fragments to BEs;
  • establishes a linearization point between cancellation and transaction commit: cancellation wins and aborts the transaction, or commit wins and the cancellation request is rejected;
  • propagates the accepted or rejected result through ConnectContext, local schedulers, SQL KILL, JDBC cancellation, REST kill, Arrow Flight SQL, and forwarded FE cancellation;
  • adds deterministic unit and regression coverage for cancellation before coordinator publication, cancellation before commit, and CTAS cancellation.

Scope: CANCEL LOAD is not changed by this PR. Running InsertLoadJob transaction/query binding is an independent load-job lifecycle issue.

Release note

KILL QUERY, JDBC Statement.cancel(), and the REST query kill API now reliably cancel Nereids INSERT INTO SELECT and CTAS before transaction commit, including coordinator-publication races. Cancellation attempted after commit begins returns an error instead of a false success.

Check List (For Author)

  • Test

    • Regression test
      • ./run-regression-test.sh --run -d insert_p0 -s test_cancel_nereids_insert_query
    • Unit Test
      • ./run-fe-ut.sh --run org.apache.doris.qe.StmtExecutorTest
    • Manual test
      • Built FE with ./build.sh --fe -j48.
      • Verified the regression suite against FE version doris-0.0.0-1731787677f on query port 24030 and HTTP port 23030.
  • Behavior changed:

    • Yes. Query cancellation is sticky until coordinator publication, cancellation is serialized against INSERT commit, and cancellation that arrives after commit begins returns failure.
  • Does this need documentation?

    • No.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Nereids INSERT INTO SELECT and CTAS publish their coordinator only after planning and transaction initialization. A cancellation received before publication was reported as successful but lost, allowing the later coordinator to execute and commit. A second race allowed cancellation after coordinator completion but immediately before transaction commit to report success while the insert still committed. REST and forwarded cancellation also did not propagate whether cancellation was accepted.

Remember cancellation in StmtExecutor and apply it when a command publishes its coordinator, prevent a cancelled coordinator from dispatching fragments, and serialize cancellation against the start of insert completion. Propagate the accepted or rejected result through SQL, JDBC, REST, local scheduling, and forwarded FE cancellation paths so a request received after commit starts is not reported as successful.

### Release note

KILL QUERY, JDBC Statement.cancel(), and the REST query kill API now reliably cancel Nereids INSERT INTO SELECT and CTAS before transaction commit, including coordinator-publication races. Cancellation attempted after commit begins returns an error instead of a false success.

### Check List (For Author)

- Test: Unit Test and Regression Test
    - ./run-fe-ut.sh --run org.apache.doris.qe.StmtExecutorTest
    - ./run-regression-test.sh --run -d insert_p0 -s test_cancel_nereids_insert_query
    - ./build.sh --fe -j48
- Behavior changed: Yes. Query cancellation is sticky until coordinator publication and is rejected once insert commit begins.
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow morrySnow changed the title [fix](fe) Make Nereids insert cancellation reliable [fix](load) Make Nereids insert cancellation reliable Aug 26, 2026
@morrySnow morrySnow changed the title [fix](load) Make Nereids insert cancellation reliable [fix](load) Make insert cancellation reliable Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants