Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add from_arrow_host functions for cudf interop with nanoarrow #15645

Merged
merged 32 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
cf3c818
Add `from_arrow_device_host` functions for cudf interop with nanoarrow
zeroshade May 3, 2024
5e06b19
rename functions and lint
zeroshade May 6, 2024
79414d1
from feedback
zeroshade May 6, 2024
a05e788
back to original impl
zeroshade May 8, 2024
05edc9a
Merge branch 'branch-24.06' into from-arrow-device-host
zeroshade May 9, 2024
d58909f
Merge branch 'branch-24.06' into from-arrow-device-host
zeroshade May 14, 2024
7bfc8d4
updates from feedback and including extra comments
zeroshade May 14, 2024
e2a3c6b
Update cpp/src/interop/from_arrow_host.cu
zeroshade May 15, 2024
e2065cd
updates from feedback
zeroshade May 15, 2024
ff90122
Merge branch 'branch-24.06' into from-arrow-device-host
zeroshade May 15, 2024
7e117a8
removing includes by suggestion
zeroshade May 20, 2024
32ae306
Merge branch 'branch-24.06' into from-arrow-device-host
zeroshade May 20, 2024
5ac8d6c
updates from feedback
zeroshade May 21, 2024
679993c
add `from_arrow` overload
zeroshade May 21, 2024
8610c91
remove excess comment
zeroshade May 21, 2024
fc02aed
Merge branch 'branch-24.06' into from-arrow-device-host
zeroshade May 22, 2024
2f49293
shift function from to_arrow_utilities to just arrow_utilities
zeroshade May 22, 2024
5b57bb3
fix style problems
zeroshade May 22, 2024
923c422
forward declare ArrowArray
zeroshade May 22, 2024
d429004
fix forgotten view usage
zeroshade May 22, 2024
c0a3c4e
add `from_arrow_column` overload
zeroshade May 22, 2024
6fa921e
Merge branch 'branch-24.06' into from-arrow-device-host
zeroshade May 23, 2024
4fa83d0
fix test and lint
zeroshade May 23, 2024
ad28303
refactor and shift tests
zeroshade May 23, 2024
14f39e4
Merge branch 'branch-24.06' into from-arrow-device-host
vyasr May 23, 2024
4b4c887
fix build issues
zeroshade May 23, 2024
f357ad7
style fixes
zeroshade May 24, 2024
b11d846
fix expected exception
zeroshade May 24, 2024
0cbcfaf
Merge branch 'branch-24.06' into from-arrow-device-host
vyasr May 24, 2024
7203197
Update cpp/src/interop/from_arrow_host.cu
zeroshade May 28, 2024
08a8b54
Merge branch 'branch-24.08' into from-arrow-device-host
vyasr May 28, 2024
a7b078b
Merge branch 'branch-24.08' into from-arrow-device-host
vyasr May 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions cpp/include/cudf/interop.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,49 @@ std::unique_ptr<cudf::scalar> from_arrow(
rmm::cuda_stream_view stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = rmm::mr::get_current_device_resource());

/**
* @brief Create `cudf::table` from given ArrowDeviceArray input
*
* @throws cudf::logic_error if either schema or input are NULL
*
* @throws cudf::logic_error if the device_type is not `ARROW_DEVICE_CPU`
zeroshade marked this conversation as resolved.
Show resolved Hide resolved
*
* @throws cudf::data_type_error if the input array is not a struct array,
* non-struct arrays should be passed to `from_arrow_host_column` instead.
*
* @param schema `ArrowSchema` pointer to describe the type of the data
* @param input `ArrowDeviceArray` pointer to object owning the Arrow data
* @param stream CUDA stream used for device memory operations and kernel launches
* @param mr Device memory resource used to perform cuda allocation
* @return cudf table generated from the given Arrow data
*/
std::unique_ptr<table> from_arrow_host(
ArrowSchema const* schema,
ArrowDeviceArray const* input,
rmm::cuda_stream_view stream = cudf::get_default_stream(),
rmm::mr::device_memory_resource* mr = rmm::mr::get_current_device_resource());

/**
zeroshade marked this conversation as resolved.
Show resolved Hide resolved
* @brief Create `cudf::column` from given ArrowDeviceArray input
*
* @throws cudf::logic_error if either schema or input are NULL
*
* @throws cudf::logic_error if the device_type is not `ARROW_DEVICE_CPU`
zeroshade marked this conversation as resolved.
Show resolved Hide resolved
*
* @throws cudf::data_type_error if input arrow data type is not supported in cudf.
*
* @param schema `ArrowSchema` pointer to describe the type of the data
* @param input `ArrowDeviceArray` pointer to object owning the Arrow data
* @param stream CUDA stream used for device memory operations and kernel launches
* @param mr Device memory resource used to perform cuda allocation
* @return cudf table generated from the given Arrow data
zeroshade marked this conversation as resolved.
Show resolved Hide resolved
*/
std::unique_ptr<column> from_arrow_host_column(
ArrowSchema const* schema,
ArrowDeviceArray const* input,
rmm::cuda_stream_view stream = cudf::get_default_stream(),
rmm::mr::device_memory_resource* mr = rmm::mr::get_current_device_resource());

/**
* @brief typedef for a vector of owning columns, used for conversion from ArrowDeviceArray
*
Expand Down
Loading
Loading