Skip to content

[fix](virtual slot) Materialize constant virtual columns before caching - #67112

Merged
yiguolei merged 1 commit into
apache:branch-4.1from
foxtail463:fix/cse-branch-4.1
Aug 26, 2026
Merged

[fix](virtual slot) Materialize constant virtual columns before caching#67112
yiguolei merged 1 commit into
apache:branch-4.1from
foxtail463:fix/cse-branch-4.1

Conversation

@foxtail463

Copy link
Copy Markdown
Contributor

Problem Summary:
Virtual-column expressions may return ColumnConst, including all-NULL nullable constants. SegmentIterator previously stored these results directly in the
output block, while downstream consumers expect row-aligned concrete columns. This could cause type assertion failures during CSE virtual-column queries with
short-circuit evaluation.

Solution:
Materialize constant expression results with convert_to_full_column_if_const() before caching them in the virtual-column slot.

@foxtail463
foxtail463 requested a review from yiguolei as a code owner August 25, 2026 06:20
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@foxtail463

Copy link
Copy Markdown
Contributor Author

I20260825 11:27:38.316625 3244801 fragment_mgr.cpp:504] query_id: 3a4ef02e30ad4a4b-90c6f655611eb4d3, coord_addr: TNetworkAddress(hostname=172.16.0.1, port=29720), total fragment num on current host: 49, fe process uuid: 1787626825489, query type: SELECT, report audit fe:TNetworkAddress(hostname=172.16.0.1, port=29720), use wg:1787626825281,normal
I20260825 11:27:38.331720 3244803 fragment_mgr.cpp:415] Query 3a4ef02e30ad4a4b-90c6f655611eb4d3 start execution
F20260825 11:27:38.334636 3243993 status.h:467] Bad cast from type:doris::ColumnConst to doris::ColumnNullable
*** Check failure stack trace: ***
*** Query id: 3a4ef02e30ad4a4b-90c6f655611eb4d3 ***
*** tablet id: 0 ***
*** Aborted at 1787628458 (unix time) try "date -d @1787628458" if you are using GNU date ***
*** Current BE git commitID: 4c5f0e90576 ***
*** SIGABRT unknown detail explain (@0x3f300317aa1) received by PID 3242657 (TID 3243993 OR 0x7f46991d5700) from PID 3242657; stack trace: ***
I20260825 11:27:38.368360 3246079 workload_group_manager.cpp:239]
Process Memory Summary: process memory used 1.99 GB(= 1.99 GB[vm/rss] + 0[reserved] + 0B[waiting_refresh]), sys available memory 623.42 GB(= 623.42 GB[proc/available] - 0[reserved] - 0B[waiting_refresh]), all workload groups memory usage: 45.19 KB
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at ../src/common/signal_handler.h:418
1# 0x00007F4CFB1D3420 in /lib/x86_64-linux-gnu/libpthread.so.0
2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
3# abort at /build/glibc-B3wQXB/glibc-2.31/stdlib/abort.c:81
4# google::LogMessage::SendToLog() in /data7/yangtao555/code/doris/output/be/lib/doris_be
5# google::LogMessage::Flush() in /data7/yangtao555/code/doris/output/be/lib/doris_be
6# google::LogMessageFatal::~LogMessageFatal() in /data7/yangtao555/code/doris/output/be/lib/doris_be
7# doris::Status doris::Status::FatalError<true, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >(std::basic_string_view<char, std::char_traits >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >&&, std::__cxx11::basic_string<char, std::char_traits, std::allocator >&&) in /data7/yangtao555/code/doris/output/be/lib/doris_be
8# doris::ColumnNullable const& assert_cast<doris::ColumnNullable const&, (TypeCheckOnRelease)1, doris::IColumn const&>(doris::IColumn const&)::{lambda(auto:1&&)#1}::operator()<doris::IColumn const&>(doris::IColumn const&) const at ../src/core/assert_cast.h:82
9# doris::ColumnNullable const& assert_cast<doris::ColumnNullable const&, (TypeCheckOnRelease)1, doris::IColumn const&>(doris::IColumn const&) at ../src/core/assert_cast.h:89
10# doris::ColumnNullable::insert_range_from(doris::IColumn const&, unsigned long, unsigned long) at ./be/src/core/column/column_nullable.cpp:368
11# doris::MutableBlock::add_rows(doris::Block const*, unsigned long, unsigned long) at ./be/src/core/block/block.cpp:1158
12# doris::VCollectIterator::_topn_next(doris::Block*) at ./be/build_debug/../src/storage/iterator/vcollect_iterator.cpp:401
13# doris::VCollectIterator::next(doris::Block*) at ./be/build_debug/../src/storage/iterator/vcollect_iterator.cpp:266
14# doris::BlockReader::_direct_next_block(doris::Block*, bool*) at ./be/src/storage/iterator/block_reader.cpp:665
15# doris::BlockReader::next_block_with_aggregation(doris::Block*, bool*) at ./be/src/storage/iterator/block_reader.cpp:72
16# doris::OlapScanner::_get_block_impl(doris::RuntimeState*, doris::Block*, bool*) at ./be/src/exec/scan/olap_scanner.cpp:845
17# doris::Scanner::get_block(doris::RuntimeState*, doris::Block*, bool*) at ./be/src/exec/scan/scanner.cpp:167
18# doris::Scanner::get_block_after_projects(doris::RuntimeState*, doris::Block*, bool*) at ./be/src/exec/scan/scanner.cpp:104
19# doris::ScannerScheduler::_scanner_scan(std::shared_ptrdoris::ScannerContext, std::shared_ptrdoris::ScanTask) at ./be/src/exec/scan/scanner_scheduler.cpp:180
20# doris::ScannerScheduler::submit(std::shared_ptrdoris::ScannerContext, std::shared_ptrdoris::ScanTask)::$_0::operator()() const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const at ./be/src/exec/scan/scanner_scheduler.cpp:79
21# doris::ScannerScheduler::submit(std::shared_ptrdoris::ScannerContext, std::shared_ptrdoris::ScanTask)::$_0::operator()() const::{lambda()#1}::operator()() const at ./be/src/exec/scan/scanner_scheduler.cpp:83
22# bool std::__invoke_impl<bool, doris::ScannerScheduler::submit(std::shared_ptrdoris::ScannerContext, std::shared_ptrdoris::ScanTask)::$_0::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::ScannerScheduler::submit(std::shared_ptrdoris::ScannerContext, std::shared_ptrdoris::ScanTask)::$_0::operator()() const::{lambda()#1}&) at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63
23# std::enable_if<is_invocable_r_v<bool, doris::ScannerScheduler::submit(std::shared_ptrdoris::ScannerContext, std::shared_ptrdoris::ScanTask)::$_0::operator()() const::{lambda()#1}&>, bool>::type std::__invoke_r<bool, doris::ScannerScheduler::submit(std::shared_ptrdoris::ScannerContext, std::shared_ptrdoris::ScanTask)::$_0::operator()() const::{lambda()#1}&>(doris::ScannerScheduler::submit(std::shared_ptrdoris::ScannerContext, std::shared_ptrdoris::ScanTask)::$_0::operator()() const::{lambda()#1}&) at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116
24# std::_Function_handler<bool (), doris::ScannerScheduler::submit(std::shared_ptrdoris::ScannerContext, std::shared_ptrdoris::ScanTask)::$_0::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292
25# std::function<bool ()>::operator()() const at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593
26# doris::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) at ./be/src/exec/scan/scanner_scheduler.cpp:383
27# doris::PrioritizedSplitRunner::process() at ./be/src/exec/scan/task_executor/time_sharing/prioritized_split_runner.cpp:115
28# doris::TimeSharingTaskExecutor::_dispatch_thread() at ./be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:572
29# void std::__invoke_impl<void, void (doris::TimeSharingTaskExecutor::&)(), doris::TimeSharingTaskExecutor&>(std::__invoke_memfun_deref, void (doris::TimeSharingTaskExecutor::&)(), doris::TimeSharingTaskExecutor&) at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:76
30# std::__invoke_result<void (doris::TimeSharingTaskExecutor::&)(), doris::TimeSharingTaskExecutor&>::type std::__invoke<void (doris::TimeSharingTaskExecutor::&)(), doris::TimeSharingTaskExecutor&>(void (doris::TimeSharingTaskExecutor::&)(), doris::TimeSharingTaskExecutor&) at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:98
31# void std::_Bind<void (doris::TimeSharingTaskExecutor::(doris::TimeSharingTaskExecutor))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/functional:515
32# void std::_Bind<void (doris::TimeSharingTaskExecutor::(doris::TimeSharingTaskExecutor))()>::operator()<, void>() at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/functional:600
33# void std::__invoke_impl<void, std::_Bind<void (doris::TimeSharingTaskExecutor::(doris::TimeSharingTaskExecutor))()>&>(std::__invoke_other, std::_Bind<void (doris::TimeSharingTaskExecutor::(doris::TimeSharingTaskExecutor))()>&) at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63
34# std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::TimeSharingTaskExecutor::(doris::TimeSharingTaskExecutor))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::TimeSharingTaskExecutor::(doris::TimeSharingTaskExecutor))()>&>(std::_Bind<void (doris::TimeSharingTaskExecutor::(doris::TimeSharingTaskExecutor))()>&) at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:119
35# std::_Function_handler<void (), std::_Bind<void (doris::TimeSharingTaskExecutor::(doris::TimeSharingTaskExecutor))()> >::_M_invoke(std::_Any_data const&) at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292
36# std::function<void ()>::operator()() const at /home/yangtao555/local/opt/ldb-toolchain-0.28/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593
37# doris::Thread::supervise_thread(void*) at ./be/src/util/thread.cpp:460
38# start_thread at /build/glibc-B3wQXB/glibc-2.31/nptl/pthread_create.c:478
39# __clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

@foxtail463

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 74.10% (31149/42038)
Line Coverage 58.35% (345514/592180)
Region Coverage 55.21% (288244/522068)
Branch Coverage 56.02% (129579/231294)

@yiguolei
yiguolei merged commit facf9cc into apache:branch-4.1 Aug 26, 2026
32 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants