Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce libcuvs wheels #594

Merged
merged 19 commits into from
Jan 23, 2025
Merged

introduce libcuvs wheels #594

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
fbe777d
introduce libcuvs wheels
jameslamb Jan 21, 2025
941c711
enforce cmake-format checks
jameslamb Jan 21, 2025
1a07f09
more changes
jameslamb Jan 21, 2025
0f7c7d8
stop building and installing static library
jameslamb Jan 21, 2025
591bcff
more build deps
jameslamb Jan 21, 2025
0bc1f0a
fix copy-paste error
jameslamb Jan 21, 2025
7e04e6d
fix RPATHs and sizes
jameslamb Jan 21, 2025
270081e
Merge branch 'branch-25.02' into libcuvs-wheels
jameslamb Jan 21, 2025
895545f
more changes
jameslamb Jan 21, 2025
4724d80
misc changes
jameslamb Jan 22, 2025
929609f
use CUDA wheels on CUDA 11
jameslamb Jan 22, 2025
7c37b90
Merge branch 'libcuvs-wheels' of github.com:jameslamb/cuvs into libcu…
jameslamb Jan 22, 2025
97c5617
Merge branch 'branch-25.02' into libcuvs-wheels
jameslamb Jan 22, 2025
9740e51
more changes
jameslamb Jan 22, 2025
86648b9
stop relying on cython to transitively pin clang
jameslamb Jan 22, 2025
81b4162
Merge branch 'libcuvs-wheels' of github.com:jameslamb/cuvs into libcu…
jameslamb Jan 22, 2025
8640519
Merge branch 'branch-25.02' of github.com:rapidsai/cuvs into libcuvs-…
jameslamb Jan 22, 2025
a08c821
Merge branch 'branch-25.02' into libcuvs-wheels
jameslamb Jan 22, 2025
baae0a7
add clang to docs env, clean up
jameslamb Jan 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/workflows/build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,30 @@ jobs:
node_type: "gpu-v100-latest-1"
run_script: "ci/build_docs.sh"
sha: ${{ inputs.sha }}
wheel-build-libcuvs:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
sha: ${{ inputs.sha }}
date: ${{ inputs.date }}
script: ci/build_wheel_libcuvs.sh
# build for every combination of arch and CUDA version, but only for the latest Python
matrix_filter: group_by([.ARCH, (.CUDA_VER|split(".")|map(tonumber)|.[0])]) | map(max_by(.PY_VER|split(".")|map(tonumber)))
wheel-publish-libcuvs:
needs: wheel-build-libcuvs
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
sha: ${{ inputs.sha }}
date: ${{ inputs.date }}
package-name: libcuvs
package-type: cpp
wheel-build-cuvs:
needs: wheel-build-libcuvs
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
Expand All @@ -99,3 +122,4 @@ jobs:
sha: ${{ inputs.sha }}
date: ${{ inputs.date }}
package-name: cuvs
package-type: python
28 changes: 19 additions & 9 deletions .github/workflows/pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,13 @@ jobs:
- changed-files
- checks
- conda-cpp-build
- conda-cpp-tests
# - conda-cpp-tests
jameslamb marked this conversation as resolved.
Show resolved Hide resolved
- conda-cpp-checks
- conda-python-build
- conda-python-tests
- docs-build
- rust-build
- wheel-build-libcuvs
- wheel-build-cuvs
- wheel-tests-cuvs
- devcontainer
Expand Down Expand Up @@ -87,13 +88,13 @@ jobs:
with:
build_type: pull-request
node_type: cpu16
conda-cpp-tests:
needs: [conda-cpp-build, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_cpp
with:
build_type: pull-request
# conda-cpp-tests:
# needs: [conda-cpp-build, changed-files]
# secrets: inherit
# uses: rapidsai/shared-workflows/.github/workflows/[email protected]
# if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_cpp
# with:
# build_type: pull-request
conda-cpp-checks:
needs: conda-cpp-build
secrets: inherit
Expand Down Expand Up @@ -135,10 +136,19 @@ jobs:
arch: "amd64"
container_image: "rapidsai/ci-conda:latest"
run_script: "ci/build_rust.sh"
wheel-build-cuvs:
wheel-build-libcuvs:
needs: checks
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: pull-request
script: ci/build_wheel_libcuvs.sh
# build for every combination of arch and CUDA version, but only for the latest Python
matrix_filter: group_by([.ARCH, (.CUDA_VER|split(".")|map(tonumber)|.[0])]) | map(max_by(.PY_VER|split(".")|map(tonumber)))
wheel-build-cuvs:
needs: wheel-build-libcuvs
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: pull-request
script: ci/build_wheel_cuvs.sh
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ repos:
cpp/cmake/modules/FindAVX\.cmake|
- id: verify-alpha-spec
- repo: https://github.com/rapidsai/dependency-file-generator
rev: v1.16.0
rev: v1.17.0
hooks:
- id: rapids-dependency-file-generator
args: ["--clean"]
Expand Down
8 changes: 1 addition & 7 deletions build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -313,12 +313,6 @@ if [[ ${CMAKE_TARGET} == "" ]]; then
CMAKE_TARGET="all"
fi


SKBUILD_EXTRA_CMAKE_ARGS="${EXTRA_CMAKE_ARGS}"
if [[ "${EXTRA_CMAKE_ARGS}" != *"DFIND_CUVS_CPP"* ]]; then
SKBUILD_EXTRA_CMAKE_ARGS="${SKBUILD_EXTRA_CMAKE_ARGS};-DFIND_CUVS_CPP=ON"
fi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just picking a mostly-arbitrary place on the diff to start this thread... the rust build is failing here.

  warning: argument unused during compilation: '-L/opt/conda/envs/rust/targets/x86_64-linux/lib' [-Wunused-command-line-argument]
  warning: argument unused during compilation: '-L/opt/conda/envs/rust/targets/x86_64-linux/lib/stubs' [-Wunused-command-line-argument]
  /usr/include/limits.h:124:16: fatal error: 'limits.h' file not found
  clang diag: warning: argument unused during compilation: '-L/opt/conda/envs/rust/targets/x86_64-linux/lib' [-Wunused-command-line-argument]
  clang diag: warning: argument unused during compilation: '-L/opt/conda/envs/rust/targets/x86_64-linux/lib/stubs' [-Wunused-command-line-argument]
  thread 'main' panicked at cuvs-sys/build.rs:97:10:
  Unable to generate cagra_c bindings: ClangDiagnostic("/usr/include/limits.h:124:16: fatal error: 'limits.h' file not found\n")

(build link)

I strongly suspect it's related to this PR, because I've seen it on multiple runs and because that build is succeeding on branch builds and other PRs. For example, it just passed 6 minutes ago on #596: https://github.com/rapidsai/cuvs/actions/runs/12895689316/job/35960708413?pr=596

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm able to reproduce this locally.

docker run \
    --gpus 1 \
    --rm \
    --env RAPIDS_BUILD_TYPE=pull-request \
    --env RAPIDS_REF_NAME=pull-request/594 \
    --env RAPIDS_REPOSITORY=rapidsai/cuvs \
    --env RAPIDS_SHA=97c56178cd0e07e4b6b138bb0904af78379f1bb3 \
    -v $(pwd):/opt/work \
    -w /opt/work \
    -it rapidsai/ci-conda \
    bash

ci/build_rust.sh

Working on a fix.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found it! Looks like having cython in the rapids_build group in dependencies.yaml was load-bearing 😅

With it in that group, the build environment for rust builds gets clang-19. Without it, it gets clang==8.0.0.

image

I'll update the pin here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed 86648b9 updating this.

Normally I'd say that should be it's own PR, but since we're so CI-constrained right now and since this change only matters for the benefit of this PR, I think we should just include it in the scope here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like clang is needed to build docs here too

Unable to find libclang: "couldn't find any valid shared libraries matching: ['libclang.so', 'libclang-.so', 'libclang.so.', 'libclang-.so.'], set the LIBCLANG_PATH environment variable to a path where one of these files can be found (invalid: [])"

(failed docs build link)

, because the Rust stuff is rebuilt in docs builds

cuvs/ci/build_docs.sh

Lines 44 to 48 in 1c91e1f

rapids-logger "Build Rust docs"
pushd rust
export LIBCLANG_PATH=$(dirname $(find /opt/conda -name libclang.so | head -n 1))
cargo doc -p cuvs --no-deps
popd

Added it in baae0a7


# If clean given, run it prior to any other steps
if (( ${CLEAN} == 1 )); then
# If the dirs to clean are mounted dirs in a container, the
Expand Down Expand Up @@ -434,7 +428,7 @@ fi

# Build and (optionally) install the cuvs Python package
if (( ${NUMARGS} == 0 )) || hasArg python; then
SKBUILD_CMAKE_ARGS="${SKBUILD_EXTRA_CMAKE_ARGS}" \
SKBUILD_CMAKE_ARGS="${EXTRA_CMAKE_ARGS}" \
SKBUILD_BUILD_OPTIONS="-j${PARALLEL_LEVEL}" \
python -m pip install --no-build-isolation --no-deps --config-settings rapidsai.disable-cuda=true ${REPODIR}/python/cuvs
fi
Expand Down
34 changes: 18 additions & 16 deletions ci/build_wheel.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
#!/bin/bash
# Copyright (c) 2023-2024, NVIDIA CORPORATION.
# Copyright (c) 2023-2025, NVIDIA CORPORATION.

set -euo pipefail

package_name=$1
package_dir=$2
package_type=$3
underscore_package_name=$(echo "${package_name}" | tr "-" "_")

source rapids-configure-sccache
Expand All @@ -16,21 +17,22 @@ rapids-generate-version > ./VERSION

cd "${package_dir}"

case "${RAPIDS_CUDA_VERSION}" in
12.*)
EXCLUDE_ARGS=(
--exclude "libcublas.so.12"
--exclude "libcublasLt.so.12"
--exclude "libcurand.so.10"
--exclude "libcusolver.so.11"
--exclude "libcusparse.so.12"
--exclude "libnvJitLink.so.12"
EXCLUDE_ARGS=(
--exclude "libraft.so"
--exclude "libcublas.so.*"
--exclude "libcublasLt.so.*"
--exclude "libcurand.so.*"
--exclude "libcusolver.so.*"
--exclude "libcusparse.so.*"
--exclude "libnvJitLink.so.*"
)

if [[ "${package_dir}" != "python/libcuvs" ]]; then
EXCLUDE_ARGS+=(
--exclude "libcuvs_c.so"
--exclude "libcuvs.so"
)
;;
11.*)
EXCLUDE_ARGS=()
;;
esac
fi

rapids-logger "Building '${package_name}' wheel"

Expand All @@ -48,4 +50,4 @@ sccache --show-adv-stats
mkdir -p final_dist
python -m auditwheel repair -w final_dist "${EXCLUDE_ARGS[@]}" dist/*

RAPIDS_PY_WHEEL_NAME="${underscore_package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 python final_dist
RAPIDS_PY_WHEEL_NAME="${underscore_package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 ${package_type} final_dist
23 changes: 11 additions & 12 deletions ci/build_wheel_cuvs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,20 @@
#!/bin/bash
# Copyright (c) 2023-2024, NVIDIA CORPORATION.
# Copyright (c) 2023-2025, NVIDIA CORPORATION.

set -euo pipefail

package_dir="python/cuvs"

case "${RAPIDS_CUDA_VERSION}" in
12.*)
EXTRA_CMAKE_ARGS=";-DUSE_CUDA_MATH_WHEELS=ON"
;;
11.*)
EXTRA_CMAKE_ARGS=";-DUSE_CUDA_MATH_WHEELS=OFF"
;;
esac
RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"

# Set up skbuild options. Enable sccache in skbuild config options
export SKBUILD_CMAKE_ARGS="-DDETECT_CONDA_ENV=OFF;-DFIND_CUVS_CPP=OFF${EXTRA_CMAKE_ARGS}"
# Downloads libcuvs wheels from this current build,
# then ensures 'cuvs' wheel builds always use the 'libcuvs' just built in the same CI run.
#
# Using env variable PIP_CONSTRAINT is necessary to ensure the constraints
# are used when creating the isolated build environment.
RAPIDS_PY_WHEEL_NAME="libcuvs_${RAPIDS_PY_CUDA_SUFFIX}" rapids-download-wheels-from-s3 cpp /tmp/libcuvs_dist
echo "libcuvs-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo /tmp/libcuvs_dist/libcuvs_*.whl)" > /tmp/constraints.txt
export PIP_CONSTRAINT="/tmp/constraints.txt"

ci/build_wheel.sh cuvs ${package_dir}
ci/build_wheel.sh cuvs ${package_dir} python
ci/validate_wheel.sh ${package_dir} final_dist
32 changes: 32 additions & 0 deletions ci/build_wheel_libcuvs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash
# Copyright (c) 2025, NVIDIA CORPORATION.

set -euo pipefail

package_name="libcuvs"
package_dir="python/libcuvs"

rapids-logger "Generating build requirements"
matrix_selectors="cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};cuda_suffixed=true"

rapids-dependency-file-generator \
--output requirements \
--file-key "py_build_${package_name}" \
--file-key "py_rapids_build_${package_name}" \
--matrix "${matrix_selectors}" \
| tee /tmp/requirements-build.txt

rapids-logger "Installing build requirements"
python -m pip install \
-v \
--prefer-binary \
-r /tmp/requirements-build.txt

# build with '--no-build-isolation', for better sccache hit rate
# 0 really means "add --no-build-isolation" (ref: https://github.com/pypa/pip/issues/5735)
export PIP_NO_BUILD_ISOLATION=0

RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"

ci/build_wheel.sh libcuvs ${package_dir} cpp
ci/validate_wheel.sh ${package_dir} final_dist libcuvs
7 changes: 7 additions & 0 deletions ci/check_style.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,12 @@ rapids-dependency-file-generator \
rapids-mamba-retry env create --yes -f env.yaml -n checks
conda activate checks

# get config for cmake-format checks
jameslamb marked this conversation as resolved.
Show resolved Hide resolved
RAPIDS_VERSION_MAJOR_MINOR="$(rapids-version-major-minor)"
FORMAT_FILE_URL="https://raw.githubusercontent.com/rapidsai/rapids-cmake/branch-${RAPIDS_VERSION_MAJOR_MINOR}/cmake-format-rapids-cmake.json"
export RAPIDS_CMAKE_FORMAT_FILE=/tmp/rapids_cmake_ci/cmake-formats-rapids-cmake.json
mkdir -p $(dirname ${RAPIDS_CMAKE_FORMAT_FILE})
wget -O ${RAPIDS_CMAKE_FORMAT_FILE} ${FORMAT_FILE_URL}

# Run pre-commit checks
pre-commit run --all-files --show-diff-on-failure
4 changes: 3 additions & 1 deletion ci/release/update-version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,10 @@ echo "${NEXT_FULL_TAG}" > VERSION
DEPENDENCIES=(
dask-cuda
cuvs
pylibraft
libcuvs
libraft
jameslamb marked this conversation as resolved.
Show resolved Hide resolved
librmm
pylibraft
rmm
rapids-dask-dependency
)
Expand Down
9 changes: 6 additions & 3 deletions ci/test_wheel_cuvs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
#!/bin/bash
# Copyright (c) 2023-2024, NVIDIA CORPORATION.
# Copyright (c) 2023-2025, NVIDIA CORPORATION.

set -euo pipefail

mkdir -p ./dist
RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"
RAPIDS_PY_WHEEL_NAME="cuvs_${RAPIDS_PY_CUDA_SUFFIX}" rapids-download-wheels-from-s3 ./dist
RAPIDS_PY_WHEEL_NAME="libcuvs_${RAPIDS_PY_CUDA_SUFFIX}" rapids-download-wheels-from-s3 cpp ./local-libcuvs-dep
RAPIDS_PY_WHEEL_NAME="cuvs_${RAPIDS_PY_CUDA_SUFFIX}" rapids-download-wheels-from-s3 python ./dist

# echo to expand wildcard before adding `[extra]` requires for pip
python -m pip install $(echo ./dist/cuvs*.whl)[test]
python -m pip install \
./local-libcuvs-dep/libcuvs*.whl \
"$(echo ./dist/cuvs*.whl)[test]"

python -m pytest ./python/cuvs/cuvs/test
12 changes: 0 additions & 12 deletions ci/validate_wheel.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,12 @@ wheel_dir_relative_path=$2

RAPIDS_CUDA_MAJOR="${RAPIDS_CUDA_VERSION%%.*}"

# some packages are much larger on CUDA 11 than on CUDA 12
if [[ "${RAPIDS_CUDA_MAJOR}" == "11" ]]; then
PYDISTCHECK_ARGS=(
--max-allowed-size-compressed '1.4G'
)
else
PYDISTCHECK_ARGS=(
--max-allowed-size-compressed '950M'
)
fi

cd "${package_dir}"

rapids-logger "validate packages with 'pydistcheck'"

pydistcheck \
--inspect \
"${PYDISTCHECK_ARGS[@]}" \
"$(echo ${wheel_dir_relative_path}/*.whl)"

rapids-logger "validate packages with 'twine'"
Expand Down
5 changes: 2 additions & 3 deletions conda/environments/all_cuda-118_arch-aarch64.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ channels:
dependencies:
- breathe>=4.35.0
- c-compiler
- clang
- clang-tools=16.0.6
- clang-tools==16.0.6
- clang==16.0.6
- cmake>=3.26.4,!=3.30.0
- cuda-nvtx=11.8
Expand All @@ -26,7 +25,7 @@ dependencies:
- gcc_linux-aarch64=11.*
- graphviz
- ipython
- libclang
- libclang==16.0.6
- libcublas-dev=11.11.3.6
- libcublas=11.11.3.6
- libcurand-dev=10.3.0.86
Expand Down
5 changes: 2 additions & 3 deletions conda/environments/all_cuda-118_arch-x86_64.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ channels:
dependencies:
- breathe>=4.35.0
- c-compiler
- clang
- clang-tools=16.0.6
- clang-tools==16.0.6
- clang==16.0.6
- cmake>=3.26.4,!=3.30.0
- cuda-nvtx=11.8
Expand All @@ -26,7 +25,7 @@ dependencies:
- gcc_linux-64=11.*
- graphviz
- ipython
- libclang
- libclang==16.0.6
- libcublas-dev=11.11.3.6
- libcublas=11.11.3.6
- libcurand-dev=10.3.0.86
Expand Down
5 changes: 2 additions & 3 deletions conda/environments/all_cuda-125_arch-aarch64.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ channels:
dependencies:
- breathe>=4.35.0
- c-compiler
- clang
- clang-tools=16.0.6
- clang-tools==16.0.6
- clang==16.0.6
- cmake>=3.26.4,!=3.30.0
- cuda-cudart-dev
Expand All @@ -27,7 +26,7 @@ dependencies:
- gcc_linux-aarch64=13.*
- graphviz
- ipython
- libclang
- libclang==16.0.6
- libcublas-dev
- libcurand-dev
- libcusolver-dev
Expand Down
5 changes: 2 additions & 3 deletions conda/environments/all_cuda-125_arch-x86_64.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ channels:
dependencies:
- breathe>=4.35.0
- c-compiler
- clang
- clang-tools=16.0.6
- clang-tools==16.0.6
- clang==16.0.6
- cmake>=3.26.4,!=3.30.0
- cuda-cudart-dev
Expand All @@ -27,7 +26,7 @@ dependencies:
- gcc_linux-64=13.*
- graphviz
- ipython
- libclang
- libclang==16.0.6
- libcublas-dev
- libcurand-dev
- libcusolver-dev
Expand Down
Loading
Loading