Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support vectorized schema scanner #1

Open
wants to merge 63 commits into
base: main
Choose a base branch
from
Open

support vectorized schema scanner #1

wants to merge 63 commits into from

Commits on Sep 9, 2021

  1. Bugfix: After copying the data from src to dst, the null flag should … 

    …be set to false. (StarRocks#113)

There is no corresponding setting for AggregateFuncTraits of VARCHAR type.

fixs StarRocks#123
    @lokax
    lokax authored Sep 9, 2021
    Configuration menu
    Copy the full SHA
    97daf1d View commit details
    Browse the repository at this point in the history

  2. Fix hyperscan thirdparty should use TP_INSTALL_DIR (StarRocks#125)

    @Astralidea
    Astralidea authored Sep 9, 2021
    Configuration menu
    Copy the full SHA
    e87eabb View commit details
    Browse the repository at this point in the history

  3. Remove the skipped tablet (StarRocks#135)

    @luyou86
    luyou86 authored Sep 9, 2021
    Configuration menu
    Copy the full SHA
    5a8ce3c View commit details
    Browse the repository at this point in the history

  4. Fix variables unused warning (StarRocks#130)

    @stdpain
    stdpain authored Sep 9, 2021
    Configuration menu
    Copy the full SHA
    92de223 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2021

  1. Change valueOf to parseInt, improve efficiency (StarRocks#140)

    @r-asou
    r-asou authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    f700624 View commit details
    Browse the repository at this point in the history

  2. Add some comments for deleting old query optimizer related codes (Sta… 

    …rRocks#133)
    @kangkaisen
    kangkaisen authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    078a2fb View commit details
    Browse the repository at this point in the history

  3. [SR-4347] support vectorized es_http_scan_node (StarRocks#28)

    @stdpain
    stdpain authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    0593c40 View commit details
    Browse the repository at this point in the history

  4. [BUG] fix const raw string add number (StarRocks#138)

    @stdpain
    stdpain authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    7b78f97 View commit details
    Browse the repository at this point in the history

  5. [SR-4453] Provide aggregate operators for pipeline engine (StarRocks#136 

    )
    @liuyehcf
    liuyehcf authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    ceff937 View commit details
    Browse the repository at this point in the history

  6. [SR-4416] Upgrade lib for enhanced security (StarRocks#82)

    @Astralidea
    Astralidea authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    c9f1cb0 View commit details
    Browse the repository at this point in the history

  7. Modify Parameter Naming (StarRocks#157)

    @Pslydhh
    Pslydhh authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    403854e View commit details
    Browse the repository at this point in the history

  8. [SR-3676] Export supports specified columns (StarRocks#118) 

    * [SR-3676] Export supports specified columns

* Support different column order
    @wyb
    wyb authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    c6dc140 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2021

  1. Rename class OlapMeta to KVStore (StarRocks#141)

    @BanBuDu0
    BanBuDu0 authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    ee08fe4 View commit details
    Browse the repository at this point in the history

  2. [Compile] Fix hyberscan not found Fix boost found problem (StarRocks#158 

    )
    @stdpain
    stdpain authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    c504ad4 View commit details
    Browse the repository at this point in the history

  3. [SR-4446] Update librdkafka to new version, fix the bug of memory lea… 

    …k in lz4 decompress (StarRocks#154)
    @trueeyu
    trueeyu authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    62ab162 View commit details
    Browse the repository at this point in the history

  4. Upgrade web frontend lib for enhanced security (StarRocks#152)

    @Astralidea
    Astralidea authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    4142f9d View commit details
    Browse the repository at this point in the history

  5. [SR-4473] allow negative score compaction (StarRocks#159)

    @decster
    decster authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    1f36b46 View commit details
    Browse the repository at this point in the history

  6. Add some comments for deleting old query executor related codes (Star… 

    …Rocks#145)
    @kangkaisen
    kangkaisen authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    a3e50c5 View commit details
    Browse the repository at this point in the history

  7. Update issue templates (StarRocks#150) 

    * Update issue template
    @sduzh
    sduzh authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    b573b34 View commit details
    Browse the repository at this point in the history

  8. [SR-4475] Vectorization engine reuses aggregator components (StarRock… 

    …s#151)
    @liuyehcf
    liuyehcf authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    b0cb803 View commit details
    Browse the repository at this point in the history

  9. [Bug][ES-Extern] fix type convert error while get value from es-exter… 

    …n table (StarRocks#164)
    @stdpain
    stdpain authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    7ed9b01 View commit details
    Browse the repository at this point in the history

  10. compatible with bitmap/hll/percentile append_value_multiple_times usa… 

    …ge by using append_strings instead in default value column iterator. (StarRocks#93)
    @Linkerist
    Linkerist authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    70edab8 View commit details
    Browse the repository at this point in the history

  11. [SR-3794] Olap external table support (StarRocks#66) 

    Now you can create a olap external table whose data will be stored in
another StarRocks cluster. When you insert values into such table,
data are written to remote table actually.
    @wuleistarrocks
    wuleistarrocks authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    ef45f29 View commit details
    Browse the repository at this point in the history

  12. Fix show tablet failed when drop be (StarRocks#163)

    gengjun-git authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    d00dea4 View commit details
    Browse the repository at this point in the history

  13. Just a tiny refactor, remove dead code for TupleDescriptor while code… 

    … walkthrough. (StarRocks#170)
    @Linkerist
    Linkerist authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    11eefaf View commit details
    Browse the repository at this point in the history

  14. update jprotobuf-rpc-core for enhanced security (StarRocks#155)

    @Astralidea
    Astralidea authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    85281c9 View commit details
    Browse the repository at this point in the history

  15. Fix explain verbose plan of HdfsNode (StarRocks#168)

    @Youngwb
    Youngwb authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    c76fa6a View commit details
    Browse the repository at this point in the history

  16. [SR-4370] Support use Min/Max column statistics to estimate && Calcul… 

    …ate predicate statistics for cross join (StarRocks#107)
    @Youngwb
    Youngwb authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    a143396 View commit details
    Browse the repository at this point in the history

  17. Support bucket shuffle on multiple partition join (StarRocks#68)

    @Seaven
    Seaven authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    383f64b View commit details
    Browse the repository at this point in the history

  18. update guava for enhanced security (StarRocks#169)

    @Astralidea
    Astralidea authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    935b8d2 View commit details
    Browse the repository at this point in the history

  19. [SR-3845] add defense for large binary column in right table of hash … 

    …join (StarRocks#95)
    @trueeyu
    trueeyu authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    f231877 View commit details
    Browse the repository at this point in the history

  20. Update junit for enhanced security (StarRocks#174)

    @Astralidea
    Astralidea authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    e7c06f3 View commit details
    Browse the repository at this point in the history

  21. [SR-4468] Add LocalExchange to make multiple streams pipe into one so… 

    …rt/aggregation operator (StarRocks#156)
    @satanson
    satanson authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    368fb14 View commit details
    Browse the repository at this point in the history

  22. Fix build starrocks_be faild (StarRocks#176) 

    I have cloned the project and try to build it with running build.sh in the root, but got below error.This pr solve the build error.
    @mchades
    mchades authored Sep 11, 2021
    Configuration menu
    Copy the full SHA
    7068a63 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2021

  1. Fix license header for agg files (StarRocks#180)

    @kangkaisen
    kangkaisen authored Sep 12, 2021
    Configuration menu
    Copy the full SHA
    68becaa View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2021

  1. [Compile] Fix build thirdparty error and remove boost for mysql (Star… 

    …Rocks#182)
    @stdpain
    stdpain authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    0456034 View commit details
    Browse the repository at this point in the history

  2. Add enhancement/general-question issue template, and adjust descripti… 

    …on in feature-request/bug-report. (StarRocks#188)
    @Linkerist
    Linkerist authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    9c9e0cf View commit details
    Browse the repository at this point in the history

  3. Fix FE ut does not work after upgrade jprotobuf (StarRocks#190) 

    this because new version jprotobuf use async.
it is not in jprotobuf-rpc-common 1.8.
    @Astralidea
    Astralidea authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    302c6ec View commit details
    Browse the repository at this point in the history

  4. Reduce the number of disk accesses while parsing segment footer (Star… 

    …Rocks#175)

* Enhancement: Reduce disk accesses when parsing segment footer

Issue Number StarRocks#129

Reduce the number of disk accesses by passing a hint value of the
segment footer length, when parsing the segment footer. The hint
value is based on the previous segment footer length of the same
rowset.
    @sduzh
    sduzh authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    8eae136 View commit details
    Browse the repository at this point in the history

  5. Update Readme.md (StarRocks#194)

    @wangsimo0
    wangsimo0 authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    aaa53b0 View commit details
    Browse the repository at this point in the history

  6. Open Rowset when it is committed (StarRocks#191) 

    * Open Rowset when it is committed

* clear segments in do_load
    @decster
    decster authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    d39b5e6 View commit details
    Browse the repository at this point in the history

  7. [StarRocks#161] Optimize equivalence predicate on outer/semi join (St… 

    …arRocks#162)
    @Seaven
    Seaven authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    2f48056 View commit details
    Browse the repository at this point in the history

  8. [SR-4416] remove mysql-connector-java (StarRocks#201) 

    this dependency is not use
    @Astralidea
    Astralidea authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    d3f47c3 View commit details
    Browse the repository at this point in the history

  9. [SR-4489] Fixup IN predicate overflow decimal string convertion failu… 

    …re (StarRocks#198)
    @satanson
    satanson authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    959e440 View commit details
    Browse the repository at this point in the history

  10. [SR-3811] optimize high card count distinct (StarRocks#139)

    @dirtysalt
    dirtysalt authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    f52ddac View commit details
    Browse the repository at this point in the history

  11. Avoid loading short key indexes when they are not needed (StarRocks#197) 

    Issue number StarRocks#196
    @sduzh
    sduzh authored Sep 13, 2021
    Configuration menu
    Copy the full SHA
    6687271 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2021

  1. [SR-3289] FE schedule algorithm for backend disk balance (StarRocks#119) 

    1. On the basis of the original cluster tablet balance, add backend disk balance schedule algorithm.
2. Implement backend local migration schedule process.
3. Improve the storage medium migration process.
4. Add storage medium, tablet num in BackendProcNode.
    @wyb
    wyb authored Sep 14, 2021
    Configuration menu
    Copy the full SHA
    cef91e8 View commit details
    Browse the repository at this point in the history

  2. [SR-3289] BE migration for backend disk balance (StarRocks#120) 

    1. Support tablet local migration for backend disk balance.
2. Improve backend tablet migration process.
3. Improve migration lock, and migration will not cause write fail.
4. Change path_hash type to int64, which is consistent with thrift api and fe.
    @wyb
    wyb authored Sep 14, 2021
    Configuration menu
    Copy the full SHA
    7383dd7 View commit details
    Browse the repository at this point in the history

  3. fix possible compiler optimization error (StarRocks#208)

    @dirtysalt
    dirtysalt authored Sep 14, 2021
    Configuration menu
    Copy the full SHA
    b7cb8ae View commit details
    Browse the repository at this point in the history

  4. distrubute system fs.hdfs.impl.disable.cache change to be configurable ( 

    StarRocks#210)
    @xuzifu666
    xuzifu666 authored Sep 14, 2021
    Configuration menu
    Copy the full SHA
    d5250c7 View commit details
    Browse the repository at this point in the history

  5. Refactor Coordinator for colocation and bucket shuffle join (StarRock… 

    …s#213)
    @kangkaisen
    kangkaisen authored Sep 14, 2021
    Configuration menu
    Copy the full SHA
    92d8297 View commit details
    Browse the repository at this point in the history

  6. [SR-4409] Provide aggregate operators for pipeline engine (StarRocks#166 

    )
    @liuyehcf
    liuyehcf authored Sep 14, 2021
    Configuration menu
    Copy the full SHA
    29d8610 View commit details
    Browse the repository at this point in the history

  7. padding hashset serialization data to ensure min size (StarRocks#220)

    @dirtysalt
    dirtysalt authored Sep 14, 2021
    Configuration menu
    Copy the full SHA
    7360603 View commit details
    Browse the repository at this point in the history

  8. [Feature] Use mariadb-connector api access mysql-external table inste… 

    …ad of mysql-client (StarRocks#215)
    @stdpain
    stdpain authored Sep 14, 2021
    Configuration menu
    Copy the full SHA
    6cfb84c View commit details
    Browse the repository at this point in the history

  9. [SR-4612] Rename driver_instance_count to degree_of_parallelism (Star… 

    …Rocks#223)
    @satanson
    satanson authored Sep 14, 2021
    Configuration menu
    Copy the full SHA
    608fe5e View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2021

  1. [StarRocks#203] Broadcast Join passes the Gather property down (StarR… 

    …ocks#204)
    @Youngwb
    Youngwb authored Sep 15, 2021
    Configuration menu
    Copy the full SHA
    94d47b3 View commit details
    Browse the repository at this point in the history

  2. [SR-3574] optimizing some object columnp agg-function performance and… 

    … b… (StarRocks#171)
    @stdpain
    stdpain authored Sep 15, 2021
    Configuration menu
    Copy the full SHA
    131febf View commit details
    Browse the repository at this point in the history

  3. Remove unused dependencies (StarRocks#236)

    @decster
    decster authored Sep 15, 2021
    Configuration menu
    Copy the full SHA
    2c0baa1 View commit details
    Browse the repository at this point in the history

  4. [SR-4078] Add exchange merge sort operator (StarRocks#205)

    @Pslydhh
    Pslydhh authored Sep 15, 2021
    Configuration menu
    Copy the full SHA
    5c43663 View commit details
    Browse the repository at this point in the history

  5. [StarRocks#237] Move tpch mock statistic to UT (StarRocks#238)

    @Seaven
    Seaven authored Sep 15, 2021
    Configuration menu
    Copy the full SHA
    8c9638b View commit details
    Browse the repository at this point in the history

  6. Upgrade zookeeper to more safe version (StarRocks#239)

    @Astralidea
    Astralidea authored Sep 15, 2021
    Configuration menu
    Copy the full SHA
    f3b6b43 View commit details
    Browse the repository at this point in the history

  7. [SR-4450] support vectorized schema scanner

    starrocks-xueli committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    f0eb713 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2021

  1. fix bug

    starrocks-xueli committed Sep 16, 2021
    Configuration menu
    Copy the full SHA
    115c50e View commit details
    Browse the repository at this point in the history