[pull] main from StarRocks:main #5
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
…ble binary file starrocks_be (#45042)" (#45355) Signed-off-by: trueeyu <[email protected]>
Signed-off-by: Albert T. Wong <[email protected]> Signed-off-by: evelyn.zhaojie <[email protected]> Signed-off-by: evelynzhaojie <[email protected] Co-authored-by: evelyn.zhaojie <[email protected]> Co-authored-by: evelyn.zhaojie <[email protected]>
Signed-off-by: simo <[email protected]> Signed-off-by: evelynzhaojie <[email protected] Co-authored-by: evelyn.zhaojie <[email protected]>
Signed-off-by: starrocks-xupeng <[email protected]> Signed-off-by: evelynzhaojie <[email protected] Co-authored-by: evelyn.zhaojie <[email protected]> Co-authored-by: evelyn.zhaojie <[email protected]>
Signed-off-by: Kevin Xiaohua Cai <[email protected]>
Signed-off-by: packy92 <[email protected]>
…45344) Signed-off-by: silverbullet233 <[email protected]>
Signed-off-by: zihe.liu <[email protected]>
…fix the CVE-2023-25194 vulnerability (#45234) Signed-off-by: Rohit Satardekar <[email protected]>
#45033) Signed-off-by: meegoo <[email protected]>
…hema.tables (#45351) Signed-off-by: HangyuanLiu <[email protected]>
Signed-off-by: stephen <[email protected]>
github-actions
bot
added
documentation
pull
bot
added
⤵️ pull
and removed
documentation
…um` (#43616) Why I'm doing: Rigjht now hdfs scanner optimization on count(1) is to output const column of expected count. And we can see in extreme case(large dataset), the chunk number flows in pipeline will be extremely huge, and operator time and overhead time is not neglectable. And here is a profile of select count(*) from hive.hive_ssb100g_parquet.lineorder. To reproduce this extreme case, I've changed code to scale morsels by 20x and repeat row groups by 10x. in concurrency=1 case , total time is 51s - OverheadTime: 25s37ms - __MAX_OF_OverheadTime: 25s111ms - __MIN_OF_OverheadTime: 24s962ms - PullTotalTime: 12s376ms - __MAX_OF_PullTotalTime: 13s147ms - __MIN_OF_PullTotalTime: 11s885ms What I'm doing: Rewrite the count(1) query to sum like. So each row group reader will only emit at one chunk(size = 1). And total time is 9s. Original plan is like +----------------------------------+ | Explain String | +----------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS:18: count | | PARTITION: UNPARTITIONED | | | | RESULT SINK | | | | 4:AGGREGATE (merge finalize) | | | output: count(18: count) | | | group by: | | | | | 3:EXCHANGE | | | | PLAN FRAGMENT 1 | | OUTPUT EXPRS: | | PARTITION: RANDOM | | | | STREAM DATA SINK | | EXCHANGE ID: 03 | | UNPARTITIONED | | | | 2:AGGREGATE (update serialize) | | | output: count(*) | | | group by: | | | | | 1:Project | | | <slot 20> : 1 | | | | | 0:HdfsScanNode | | TABLE: lineorder | | partitions=1/1 | | cardinality=600037902 | | avgRowSize=5.0 | +----------------------------------+ And rewritted plan is like +-----------------------------------+ | Explain String | +-----------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS:18: count | | PARTITION: UNPARTITIONED | | | | RESULT SINK | | | | 3:AGGREGATE (merge finalize) | | | output: sum(18: count) | | | group by: | | | | | 2:EXCHANGE | | | | PLAN FRAGMENT 1 | | OUTPUT EXPRS: | | PARTITION: RANDOM | | | | STREAM DATA SINK | | EXCHANGE ID: 02 | | UNPARTITIONED | | | | 1:AGGREGATE (update serialize) | | | output: sum(19: ___count___) | | | group by: | | | | | 0:HdfsScanNode | | TABLE: lineorder | | partitions=1/1 | | cardinality=1 | | avgRowSize=1.0 | +-----------------------------------+ Fixes #45242 Signed-off-by: yanz <[email protected]>
github-actions
bot
added
the
documentation
…45266) Signed-off-by: xiangguangyxg <[email protected]> Signed-off-by: 絵空事スピリット <[email protected]> Co-authored-by: 絵空事スピリット <[email protected]>
Signed-off-by: shuming.li <[email protected]>
Signed-off-by: trueeyu <[email protected]> Signed-off-by: evelynzhaojie <[email protected] Co-authored-by: evelyn.zhaojie <[email protected]>
Signed-off-by: yandongxiao <[email protected]>
Signed-off-by: packy92 <[email protected]>
…ler (#45241) Signed-off-by: shuming.li <[email protected]>
Signed-off-by: HangyuanLiu <[email protected]>
Signed-off-by: HangyuanLiu <[email protected]>
Signed-off-by: hellolilyliuyi <[email protected]> Signed-off-by: 絵空事スピリット <[email protected]> Signed-off-by: evelyn.zhaojie <[email protected]> Co-authored-by: 絵空事スピリット <[email protected]> Co-authored-by: evelyn.zhaojie <[email protected]>
Signed-off-by: zihe.liu <[email protected]>
Signed-off-by: MatthewH00 <[email protected]> Signed-off-by: hmx <[email protected]>
Signed-off-by: shuming.li <[email protected]>
Signed-off-by: luohaha <[email protected]>
#45674) Signed-off-by: stephen <[email protected]>
…ribution key is NULL (#45537) Signed-off-by: HangyuanLiu <[email protected]>
…e profile report (#45675) Signed-off-by: PengFei Li <[email protected]>
Signed-off-by: Smith Cruise <[email protected]>
Signed-off-by: stdpain <[email protected]>
Signed-off-by: wyb <[email protected]>
Signed-off-by: gengjun-git <[email protected]>
Why I'm doing: For the CVE problem, we need to upgrade Hadoop SDK from 3.3.6 -> 3.4.0 It will introduce aws java SDK v2, so we can delete SDK v1. Signed-off-by: Smith Cruise <[email protected]>
…20 (#45678) Signed-off-by: evelynzhaojie <[email protected]
Signed-off-by: meegoo <[email protected]>
…45552) Signed-off-by: wyb <[email protected]>
…metadata locks in FE (#45526) Signed-off-by: HangyuanLiu <[email protected]>
Signed-off-by: zombee0 <[email protected]>
…set is empty (#45715) Signed-off-by: luohaha <[email protected]>
…45741) Signed-off-by: luohaha <[email protected]>
Signed-off-by: Smith Cruise <[email protected]>
Signed-off-by: Murphy <[email protected]>
Add titles for the intro pages. In the future if we use auto-generated nav these are required. Signed-off-by: DanRoscigno <[email protected]>
Signed-off-by: Alex Zhu <[email protected]>
…ata mode (#45665) Signed-off-by: Alex Zhu <[email protected]>
Signed-off-by: srlch <[email protected]>
Signed-off-by: trueeyu <[email protected]>
Signed-off-by: HangyuanLiu <[email protected]>
…45753) Signed-off-by: Zijie Lu <[email protected]>
Signed-off-by: Dejun Xia <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )