Skip to content

[fix](partition pruning) Preserve partition pruning state in plan copies - #67113

Open
feiniaofeiafei wants to merge 1 commit into
apache:masterfrom
feiniaofeiafei:fix_deep_copy
Open

[fix](partition pruning) Preserve partition pruning state in plan copies#67113
feiniaofeiafei wants to merge 1 commit into
apache:masterfrom
feiniaofeiafei:fix_deep_copy

Conversation

@feiniaofeiafei

@feiniaofeiafei feiniaofeiafei commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Issue Number: #62196

Related PR:

Problem Summary: LogicalOlapScan.withRelationId reset partitionPruned and hasPartitionPredicate as a hidden side effect, which broke default deep-copy semantics. Preserve both fields while replacing the relation ID. Add an explicit DeepCopierContext option that StructInfo enables for MTMV union compensation, so copied OLAP scans invalidate only partitionPruned during the existing copy traversal and the following whole-tree rewrite reapplies partition pruning.

Release note

None

Check List (For Author)

  • Test: Unit Test
    • ./run-fe-ut.sh --run org.apache.doris.nereids.trees.copier.LogicalPlanDeepCopierTest,org.apache.doris.nereids.mv.StructInfoTest
    • mvn checkstyle:check -pl fe-core
  • Behavior changed: Yes. Default plan copies preserve partition pruning metadata, while MTMV union compensation explicitly invalidates partitionPruned during its copy.
  • Does this need documentation: No

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: N/A

Related PR: N/A

Problem Summary: LogicalOlapScan.withRelationId reset partitionPruned and hasPartitionPredicate as a hidden side effect, which broke default deep-copy semantics. Preserve both fields while replacing the relation ID. Add an explicit DeepCopierContext option that StructInfo enables for MTMV union compensation, so copied OLAP scans invalidate only partitionPruned during the existing copy traversal and the following whole-tree rewrite reapplies partition pruning.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - ./run-fe-ut.sh --run org.apache.doris.nereids.trees.copier.LogicalPlanDeepCopierTest,org.apache.doris.nereids.mv.StructInfoTest
    - mvn checkstyle:check -pl fe-core
- Behavior changed: Yes. Default plan copies preserve partition pruning metadata, while MTMV union compensation explicitly invalidates partitionPruned during its copy.
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow morrySnow changed the title [fix](fe) Preserve partition pruning state in plan copies [fix](partition pruning) Preserve partition pruning state in plan copies Aug 25, 2026
@feiniaofeiafei

Copy link
Copy Markdown
Collaborator Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 17094 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e7982dc52f9dfb4562026f3fccdd85b974864efa, data reload: false

------ Round 1 ----------------------------------
============================================
q1	18627	3195	3183	3183
q2	2262	228	179	179
q3	10269	861	518	518
q4	4703	247	201	201
q5	7828	590	389	389
q6	142	117	94	94
q7	552	517	386	386
q8	9311	910	965	910
q9	4039	2423	2429	2423
q10	6556	895	710	710
q11	588	203	179	179
q12	849	266	199	199
q13	18009	1549	1168	1168
q14	160	152	138	138
q15	q16	450	410	371	371
q17	794	778	788	778
q18	3231	2313	2262	2262
q19	1204	912	809	809
q20	374	280	202	202
q21	5342	1763	1859	1763
q22	331	267	232	232
Total cold run time: 95621 ms
Total hot run time: 17094 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3539	3500	3521	3500
q2	212	218	162	162
q3	2233	2333	2221	2221
q4	1208	1194	909	909
q5	2213	2136	2148	2136
q6	172	128	87	87
q7	1052	987	869	869
q8	1651	1454	1454	1454
q9	3186	3159	3161	3159
q10	1951	1821	1630	1630
q11	363	274	256	256
q12	462	435	347	347
q13	1524	1555	1157	1157
q14	180	171	169	169
q15	q16	393	391	370	370
q17	1071	1062	1050	1050
q18	5074	4434	4780	4434
q19	1177	938	867	867
q20	1006	979	818	818
q21	3654	3033	3198	3033
q22	400	348	318	318
Total cold run time: 32721 ms
Total hot run time: 28946 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 82007 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e7982dc52f9dfb4562026f3fccdd85b974864efa, data reload: false

query5	4255	424	321	321
query6	446	135	128	128
query7	4934	427	229	229
query8	733	128	113	113
query9	8666	2929	2926	2926
query10	458	222	188	188
query11	5392	1048	902	902
query12	115	71	71	71
query13	1202	441	316	316
query14	6016	2259	2118	2118
query14_1	2007	1972	1981	1972
query15	182	117	106	106
query16	3156	383	355	355
query17	824	444	362	362
query18	2132	323	225	225
query19	233	135	104	104
query20	74	71	68	68
query21	627	100	88	88
query22	5591	5314	5466	5314
query23	7191	6409	6047	6047
query23_1	6122	6057	5943	5943
query24	7308	1098	760	760
query24_1	786	777	773	773
query25	434	288	236	236
query26	1204	237	129	129
query27	2772	447	263	263
query28	4665	1511	1499	1499
query29	969	472	363	363
query30	379	167	131	131
query31	841	418	333	333
query32	106	51	49	49
query33	483	232	176	176
query34	1035	818	493	493
query35	420	407	348	348
query36	573	563	538	538
query37	125	83	70	70
query38	1019	861	1003	861
query39	526	529	503	503
query39_1	470	482	510	482
query40	291	88	77	77
query41	58	57	55	55
query42	74	74	75	74
query43	248	258	218	218
query44	1057	560	562	560
query45	112	112	105	105
query46	789	847	539	539
query47	786	764	768	764
query48	340	336	237	237
query49	632	299	188	188
query50	762	270	194	194
query51	8251	8273	8152	8152
query52	67	65	56	56
query53	195	194	144	144
query54	234	163	154	154
query55	87	63	52	52
query56	207	171	141	141
query57	963	662	625	625
query58	203	161	150	150
query59	1189	1213	1077	1077
query60	250	173	165	165
query61	111	105	108	105
query62	389	200	170	170
query63	169	139	138	138
query64	2270	662	556	556
query65	1573	1586	1566	1566
query66	1730	260	200	200
query67	9855	10070	9577	9577
query68	2775	1277	765	765
query69	528	232	193	193
query70	665	635	605	605
query71	255	168	166	166
query72	2520	1699	1531	1531
query73	711	600	353	353
query74	1551	1238	1159	1159
query75	1184	1104	963	963
query76	1857	731	556	556
query77	309	261	212	212
query78	3769	3599	3196	3196
query79	2433	768	565	565
query80	1523	325	283	283
query81	652	155	134	134
query82	652	131	95	95
query83	277	210	190	190
query84	371	114	86	86
query85	859	352	295	295
query86	447	182	169	169
query87	1037	983	889	889
query88	2907	2116	2143	2116
query89	326	201	171	171
query90	1987	129	131	129
query91	131	128	101	101
query92	53	49	47	47
query93	1586	1072	678	678
query94	644	256	224	224
query95	530	247	305	247
query96	816	590	279	279
query97	1042	1092	1026	1026
query98	186	146	132	132
query99	549	358	310	310
Total cold run time: 183766 ms
Total hot run time: 82007 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.82 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e7982dc52f9dfb4562026f3fccdd85b974864efa, data reload: false

query1	0.00	0.00	0.00
query2	0.45	0.04	0.04
query3	0.78	0.11	0.11
query4	2.38	0.10	0.10
query5	0.17	0.16	0.15
query6	2.07	0.71	0.69
query7	0.03	0.01	0.00
query8	0.05	0.04	0.04
query9	0.79	0.21	0.22
query10	0.35	0.34	0.34
query11	0.36	0.11	0.12
query12	0.23	0.13	0.12
query13	0.29	0.30	0.32
query14	0.46	0.44	0.46
query15	0.39	0.35	0.35
query16	0.22	0.22	0.22
query17	0.68	0.67	0.68
query18	0.16	0.17	0.17
query19	1.75	1.20	1.13
query20	0.01	0.01	0.01
query21	17.70	0.16	0.11
query22	4.76	0.05	0.04
query23	17.16	0.25	0.10
query24	20.51	0.34	0.30
query25	0.15	0.06	0.03
query26	0.78	0.16	0.13
query27	0.04	0.02	0.04
query28	3.58	0.59	0.34
query29	13.98	3.20	2.61
query30	0.26	0.12	0.12
query31	2.93	0.36	0.17
query32	3.72	0.32	0.24
query33	1.38	1.38	1.51
query34	15.31	2.15	1.79
query35	1.74	1.72	1.74
query36	0.51	0.30	0.29
query37	0.35	0.04	0.04
query38	0.05	0.03	0.03
query39	0.21	0.02	0.03
query40	0.24	0.08	0.08
query41	0.40	0.03	0.02
query42	0.18	0.03	0.02
query43	0.04	0.02	0.02
Total cold run time: 117.6 s
Total hot run time: 14.82 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 2.36% (8/339) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants