Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

take_into(array, indices, builder) compute function #2330

Open
joseph-isaacs opened this issue Feb 12, 2025 · 1 comment
Open

take_into(array, indices, builder) compute function #2330

joseph-isaacs opened this issue Feb 12, 2025 · 1 comment

Comments

@joseph-isaacs
Copy link
Member

joseph-isaacs commented Feb 12, 2025

We should create a take_into(array, indices, builder) compute function.

When decoding/canonicalizing a dictionary we use take(values, indices) We can do better if we have a way to store the taken values in a builder take_into(array, indices, builder).

If we have a dictionary where the codes are value are primitive and the bit_width(codes) <= bit_width(values) we could allocate bit_width(values) and widen each code into bit_width(values) and then on a second pass replace codes with value without a second allocation.

If we have a chunked of dictionary, we can unpack each chunk into the same builder, which remove the second copy when decoding a dict

@joseph-isaacs
Copy link
Member Author

joseph-isaacs commented Feb 12, 2025

Should we replace take with take_into?

When is take preferred?

This was referenced Feb 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant