-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade to Datafusion 43 #905
Conversation
6a36cfd
to
7255ec2
Compare
It was removed upstream. Ref: apache/datafusion#13076
7255ec2
to
d86520b
Compare
d86520b
to
f7f518c
Compare
For context on the On this branch, the > /home/mike/workspace/datafusion-python/dev/python/tests/test_udwf.py(143)evaluate_all()
-> results = []
(Pdb) values
[<pyarrow.lib.Int64Array object at 0x7fff8ec772e0>
[
0,
1,
2,
3,
4,
5,
6
]] On
|
Do you need help troubleshooting the test? |
@timsaucer - yes, if you wouldn't mind. Here's where I'm at.
RUST: evaluating partition_evaluator_args: PartitionEvaluatorArgs { input_exprs: [Column { name: "a", index: 0 }, Column { name: "b", index: 1 }], input_types: [Int64, Int64], is_reversed: false, ignore_nulls: false } But then RUST: evaluating evaluate_all with 1 value arrays: [PrimitiveArray<Int64>
[
0,
1,
2,
3,
4,
5,
6,
]] My next step would be to dig deeper into the upstream machinery, so I'd appreciate if you could take a quick sanity check before that. |
… for the switch from string to string_view as default
Ok, I think this is ready to merge. @Michael-J-Ward do you want to look over the changes I added in to account for the new string views? |
@timsaucer - thanks for finishing this up! I think it's good to go. Especially since this 1st upgrade PR is primarily for the dependencies. |
Which issue does this PR close?
Closes #.
Rationale for this change
What changes are included in this PR?
Are there any user-facing changes?
Python Changes:
PyLimit
methods forskip
andfetch
because upstreamLimit
now returns expressions instead of constantsCrossjoin
expression was removed upstreamTODO
SmoothTwoColumn
test (theevaluate_all
python method is only receiving thea
column).