Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC: Remove unnecessary null check for GroupColumn #12947

Closed
wants to merge 28 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
ca033e0
define `ByteGroupValueViewBuilder`.
Rachelint Oct 8, 2024
ffcc1a2
impl append.
Rachelint Oct 8, 2024
4842965
impl equal to.
Rachelint Oct 8, 2024
66bb7be
fix compile.
Rachelint Oct 9, 2024
ef1efce
fix comments.
Rachelint Oct 9, 2024
152a8b1
impl take_n.
Rachelint Oct 11, 2024
d61c3ec
impl build.
Rachelint Oct 11, 2024
151377e
impl rest functions in `GroupColumn`.
Rachelint Oct 11, 2024
63e11cb
fix output when panic.
Rachelint Oct 11, 2024
15d8349
add e2e sql tests.
Rachelint Oct 12, 2024
d9ee724
add unit tests.
Rachelint Oct 12, 2024
beffa35
switch to a really elegant style codes from alamb.
Rachelint Oct 12, 2024
46822f9
fix take_n.
Rachelint Oct 12, 2024
3a93584
improve comments.
Rachelint Oct 12, 2024
f99f55c
fix compile.
Rachelint Oct 12, 2024
37b4816
fix clippy.
Rachelint Oct 13, 2024
d78c68d
define more testcases in `test_byte_view_take_n`.
Rachelint Oct 13, 2024
7cb7dfc
connect up.
Rachelint Oct 13, 2024
e6c7e7e
fix doc.
Rachelint Oct 13, 2024
36d556e
Do not re-validate output is utf8
alamb Oct 13, 2024
f76c376
Merge pull request #1 from alamb/alamb/tweak-group
Rachelint Oct 13, 2024
1fd926f
switch to unchecked when building array.
Rachelint Oct 13, 2024
34918cb
improve naming.
Rachelint Oct 13, 2024
8348024
use let else to make the codes clearer.
Rachelint Oct 13, 2024
023ed64
fix typo.
Rachelint Oct 13, 2024
c4d45c7
improve unit test coverage for `ByteViewGroupValueBuilder`.
Rachelint Oct 13, 2024
a1f8d0c
Merge remote-tracking branch 'apache/main' into impl-byte-view-column
alamb Oct 15, 2024
6e63eec
tmp
Rachelint Oct 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions datafusion/physical-plan/src/aggregates/group_values/column.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,16 @@
// under the License.

use crate::aggregates::group_values::group_column::{
ByteGroupValueBuilder, GroupColumn, PrimitiveGroupValueBuilder,
ByteGroupValueBuilder, ByteViewGroupValueBuilder, GroupColumn,
PrimitiveGroupValueBuilder,
};
use crate::aggregates::group_values::GroupValues;
use ahash::RandomState;
use arrow::compute::cast;
use arrow::datatypes::{
Date32Type, Date64Type, Float32Type, Float64Type, Int16Type, Int32Type, Int64Type,
Int8Type, UInt16Type, UInt32Type, UInt64Type, UInt8Type,
BinaryViewType, Date32Type, Date64Type, Float32Type, Float64Type, Int16Type,
Int32Type, Int64Type, Int8Type, StringViewType, UInt16Type, UInt32Type, UInt64Type,
UInt8Type,
};
use arrow::record_batch::RecordBatch;
use arrow_array::{Array, ArrayRef};
Expand Down Expand Up @@ -119,6 +121,8 @@ impl GroupValuesColumn {
| DataType::LargeBinary
| DataType::Date32
| DataType::Date64
| DataType::Utf8View
| DataType::BinaryView
)
}
}
Expand Down Expand Up @@ -184,6 +188,14 @@ impl GroupValues for GroupValuesColumn {
let b = ByteGroupValueBuilder::<i64>::new(OutputType::Binary);
v.push(Box::new(b) as _)
}
&DataType::Utf8View => {
let b = ByteViewGroupValueBuilder::<StringViewType>::new();
v.push(Box::new(b) as _)
}
&DataType::BinaryView => {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
dt => {
return not_impl_err!("{dt} not supported in GroupValuesColumn")
}
Expand Down
Loading
Loading