introduce libcuvs wheels

[jameslamb]

Contributes to rapidsai/build-planning#33.

Proposes packaging libcuvs as a wheel, which is then re-used by cuvs-cu{11,12}.

Similar changes were recently made in RAFT: rapidsai/raft#2531

As part of this, also proposes:

• introducing a new CMake option, CUVS_COMPILE_DYNAMIC_ONLY, to allow building/installing only the dynamic shared library (i.e. skipping the static library)
• enforcing rapids-cmake's preferred CMake style (similar introduce libraft wheels raft#2531 (comment))
• standardizing clang pins across the project, and pinning to clang 16 for Rust builds (introduce libcuvs wheels #594 (comment))

Notes for Reviewers

Benefits of these changes

• smaller wheels for projects that depend on cuVS (they can dynamically link against libcuvs instead of statically linking in the pieces they need)
• fewer CI resources used for cuVS wheels (no more re-compiling for every Python minor version)
• faster, cheaper cuML wheel builds (introduce libcuml wheels cuml#6199 (comment))
• other benefits mentioned in Support dynamic linking between RAPIDS wheels build-planning#33

Wheel contents

libcuvs:

• libcuvs.so and libcuvs_c.so (shared library)
• cuVS headers
• vendored dependencies (hnswlib)

cuvs:

• cuvs Python / Cython code and compiled Cython extensions

Size changes (CUDA 12, Python 3.12, x86_64)

wheel	num files (before)	num files (this PR)	size (before)	size (this PR)
libcuvs	---	67	---	843M
cuvs	88	84	845M	2M
TOTAL	88	131	845M	845M

NOTES: size = compressed, "before" = 2025-01-22 nightlies

how I calculated those (click me)

• nightly commit = rapidsai/cuml@7c715c4
• PR = this PR

docker run \
    --rm \
    --network host \
    --env RAPIDS_NIGHTLY_DATE=2025-01-22 \
    --env CUVS_NIGHTLY_SHA=f1de1b2 \
    --env CUVS_PR="pull-request/594" \
    --env CUVS_PR_SHA="97c56178cd0e07e4b6b138bb0904af78379f1bb3" \
    --env RAPIDS_PY_CUDA_SUFFIX=cu12 \
    --env WHEEL_DIR_BEFORE=/tmp/wheels-before \
    --env WHEEL_DIR_AFTER=/tmp/wheels-after \
    -it rapidsai/ci-wheel:cuda12.5.1-rockylinux8-py3.12 \
    bash

# --- nightly wheels --- #
mkdir -p ./wheels-before

export RAPIDS_BUILD_TYPE=branch
export RAPIDS_REF_NAME="branch-25.02"

# cuvs
RAPIDS_PY_WHEEL_NAME="cuvs_${RAPIDS_PY_CUDA_SUFFIX}" \
RAPIDS_REPOSITORY=rapidsai/cuvs \
RAPIDS_SHA=${CUVS_NIGHTLY_SHA} \
    rapids-download-wheels-from-s3 python ./wheels-before

# --- wheels from CI --- #
mkdir -p ./wheels-after

export RAPIDS_BUILD_TYPE="pull-request"

# libcuvs
RAPIDS_PY_WHEEL_NAME="libcuvs_${RAPIDS_PY_CUDA_SUFFIX}" \
RAPIDS_REPOSITORY=rapidsai/cuvs \
RAPIDS_REF_NAME="${CUVS_PR}" \
RAPIDS_SHA="${CUVS_PR_SHA}" \
    rapids-download-wheels-from-s3 cpp ./wheels-after

# cuvs
RAPIDS_PY_WHEEL_NAME="cuvs_${RAPIDS_PY_CUDA_SUFFIX}" \
RAPIDS_REPOSITORY=rapidsai/cuvs \
RAPIDS_REF_NAME="${CUVS_PR}" \
RAPIDS_SHA="${CUVS_PR_SHA}" \
    rapids-download-wheels-from-s3 python ./wheels-after

pip install pydistcheck
pydistcheck \
    --inspect \
    --select 'distro-too-large-compressed' \
    ./wheels-before/*.whl \
| grep -E '^checking|files: | compressed' \
> ./before.txt

# get more exact sizes
du -sh ./wheels-before/*

pydistcheck \
    --inspect \
    --select 'distro-too-large-compressed' \
    ./wheels-after/*.whl \
| grep -E '^checking|files: | compressed' \
> ./after.txt

# get more exact sizes
du -sh ./wheels-after/*

How I tested this

• add libcuvs Python builds devcontainers#440
• introduce libcuml wheels cuml#6199

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[jameslamb]

One of the wheel-test jobs (CUDA 11.8, arm64) is failing, with a few cases like this:

FAILED python/cuvs/cuvs/test/test_ivf_pq.py::test_ivf_pq_search_params[params2] - cuvs.common.exceptions.CuvsException: cuBLAS error encountered at: file=/pyenv/versions/3.10.16/lib/python3.10/site-packages/libraft/include/raft/linalg/detail/cublaslt_wrappers.hpp line=261: call='cublasLtMatmul(resource::get_cublaslt_handle(res), mm_desc->desc, alpha, a_ptr, mm_desc->a, b_ptr, mm_desc->b, beta, c_ptr, mm_desc->c, c_ptr, mm_desc->c, &(mm_desc->heuristics.algo), nullptr, 0, stream)', Reason=13:CUBLAS_STATUS_EXECUTION_FAILED
Obtained 49 stack frames
#1 in /pyenv/versions/3.12.8/lib/python3.12/site-packages/libcuvs/lib64/libcuvs.so: raft::cublas_error::cublas_error(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) +0xb0 [0xfffdca6a5360]
#2 in /pyenv/versions/3.12.8/lib/python3.12/site-packages/libcuvs/lib64/libcuvs.so: void raft::linalg::detail::legacy_matmul<false, float, float, float, float>(raft::resources const&, bool, bool, unsigned long, unsigned long, unsigned long, float const*, float const*, unsigned long, float const*, unsigned long, float const*, float*, unsigned long, CUstream_st*) +0x6c0 [0xfffdca6c4900]

(build link)

All the failures are in one set of test cases:

FAILED python/cuvs/cuvs/test/test_ivf_pq.py

I don't think those are related to this PR... I see them on other PRs like #593 (which only remove whitespace):
https://github.com/rapidsai/cuvs/actions/runs/12875298461/job/35896833763?pr=593

[jameslamb]

This should be fixed by rapidsai/raft#2548... will leave this thread open until I confirm that.

[jameslamb]

I want to see CUDA 11 wheel tests actually pass in CI on this PR before resolving this thread... but that requires merging #601 which requires merging #596.

[jameslamb]

Just picking a mostly-arbitrary place on the diff to start this thread... the rust build is failing here.

  warning: argument unused during compilation: '-L/opt/conda/envs/rust/targets/x86_64-linux/lib' [-Wunused-command-line-argument]
  warning: argument unused during compilation: '-L/opt/conda/envs/rust/targets/x86_64-linux/lib/stubs' [-Wunused-command-line-argument]
  /usr/include/limits.h:124:16: fatal error: 'limits.h' file not found
  clang diag: warning: argument unused during compilation: '-L/opt/conda/envs/rust/targets/x86_64-linux/lib' [-Wunused-command-line-argument]
  clang diag: warning: argument unused during compilation: '-L/opt/conda/envs/rust/targets/x86_64-linux/lib/stubs' [-Wunused-command-line-argument]
  thread 'main' panicked at cuvs-sys/build.rs:97:10:
  Unable to generate cagra_c bindings: ClangDiagnostic("/usr/include/limits.h:124:16: fatal error: 'limits.h' file not found\n")

(build link)

I strongly suspect it's related to this PR, because I've seen it on multiple runs and because that build is succeeding on branch builds and other PRs. For example, it just passed 6 minutes ago on #596: https://github.com/rapidsai/cuvs/actions/runs/12895689316/job/35960708413?pr=596

[jameslamb]

I'm able to reproduce this locally.

docker run \
    --gpus 1 \
    --rm \
    --env RAPIDS_BUILD_TYPE=pull-request \
    --env RAPIDS_REF_NAME=pull-request/594 \
    --env RAPIDS_REPOSITORY=rapidsai/cuvs \
    --env RAPIDS_SHA=97c56178cd0e07e4b6b138bb0904af78379f1bb3 \
    -v $(pwd):/opt/work \
    -w /opt/work \
    -it rapidsai/ci-conda \
    bash

ci/build_rust.sh

Working on a fix.

[jameslamb]

Found it! Looks like having cython in the rapids_build group in dependencies.yaml was load-bearing 😅

With it in that group, the build environment for rust builds gets clang-19. Without it, it gets clang==8.0.0.

I'll update the pin here.

[jameslamb]

Pushed 86648b9 updating this.

Normally I'd say that should be it's own PR, but since we're so CI-constrained right now and since this change only matters for the benefit of this PR, I think we should just include it in the scope here.

[jameslamb]

Looks like clang is needed to build docs here too

Unable to find libclang: "couldn't find any valid shared libraries matching: ['libclang.so', 'libclang-.so', 'libclang.so.', 'libclang-.so.'], set the LIBCLANG_PATH environment variable to a path where one of these files can be found (invalid: [])"

(failed docs build link)

, because the Rust stuff is rebuilt in docs builds

cuvs/ci/build_docs.sh

Lines 44 to 48 in 1c91e1f

	rapids-logger "Build Rust docs"
	pushd rust
	export LIBCLANG_PATH=$(dirname $(find /opt/conda -name libclang.so | head -n 1))
	cargo doc -p cuvs --no-deps
	popd

Added it in baae0a7

[bdice]

All looks fine to me. One nit (an extra blank line), please just wrap that into your next commit that reverts the commented C++ tests.

[bdice]

Final touches look good. Re-approving.

[bdice]

I will pre-emptively merge this. All CI has passed except for one C++ job which we expect to take a while (see #602 for a proposed improvement to runtime).

[bdice]

/merge