Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RELEASE] rmm v24.12 #1736

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open

[RELEASE] rmm v24.12 #1736

wants to merge 31 commits into from

Conversation

raydouglass
Copy link
Member

❄️ Code freeze for branch-24.12 and v24.12 release

What does this mean?

Only critical/hotfix level issues should be merged into branch-24.12 until release (merging of this PR).

What is the purpose of this PR?

  • Update documentation
  • Allow testing for the new release
  • Enable a means to merge branch-24.12 into main for the release

raydouglass and others added 30 commits September 19, 2024 11:43
@raydouglass
DOC v24.12 Updates [skip ci]
f4ec463
@GPUtester
Merge pull request #1684 from rapidsai/branch-24.10
1823321 
Forward-merge branch-24.10 into branch-24.12
@GPUtester
Merge pull request #1686 from rapidsai/branch-24.10
163db70 
Forward-merge branch-24.10 into branch-24.12
@jameslamb
exclude 'gcovr' from list of development pip packages (#1688)
b514473 
This project currently lists `gcovr` (https://pypi.org/project/gcovr/) as a pip dependency for development. I strongly suspect that that was unintentional... it doesn't look like it has any reliance on getting that package via `pip` (just conda, in the C++ test jobs and for local C++ development).

This proposes removing `gcovr` from the list of pip dependencies, so it won't get installed in the DLFW images or other places where `rapids-make-pip-env` from https://github.com/rapidsai/devcontainers is called.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1688
@wence-
Use rmm::percent_of_free_device_memory in arena test (#1689)
9e410c0 
Rather than hand-coding a fraction of the device memory use the utility routine.

- Closes #1674

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Rong Ou (https://github.com/rongou)

URL: #1689
@Matt711
[Improvement] Reorganize Cython to separate C++ bindings and make Cyt…
6489bb7 
…hon classes public (#1676)

Closes #1280

Authors:
  - Matthew Murray (https://github.com/Matt711)
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1676
@Matt711
Fix rmm ._lib imports (#1693)
8150032 
This PR fixes a bug in #1676. It makes sure that rmm imports work correctly using both `from rmm._lib...` and `import rmm._lib...` syntax.

I'm adding DO NOT MERGE until I do some more testing.

Authors:
  - Matthew Murray (https://github.com/Matt711)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1693
@KyleFromNVIDIA
Prune workflows based on changed files (#1695)
c494395 
Contributes to rapidsai/build-planning#94

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #1695
@GPUtester
Merge pull request #1700 from rapidsai/branch-24.10
a381645 
Forward-merge branch-24.10 into branch-24.12
@wence-
Update cross-link to cuda-python object (#1699)
4e519bb 
NVIDIA/cuda-python#137 reorganised the low-level binding structure which broke our cross-linking, update to the new name to fix.

- Closes #1698

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Matthew Murray (https://github.com/Matt711)
  - Bradley Dice (https://github.com/bdice)

URL: #1699
@wence-
Add BUILD_SHARED_LIBS option defaulting to ON (#1702)
69a297d 
This means that downstream libraries that get their `fmt` dependency from RMM will use `-DFMT_SHARED` in the compile command: this matches what the rapids combined devcontainers do, so we get sccache hits.

- Closes #1701

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

URL: #1702
@jameslamb
make conda installs in CI stricter (#1696)
f715518 
Contributes to rapidsai/build-planning#106

Proposes specifying the RAPIDS version in `conda install` calls in CI that install CI artifacts, to reduce the risk of CI jobs picking up artifacts from other releases.

Authors:
  - James Lamb (https://github.com/jameslamb)
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)

URL: #1696
@charlesbluca
Fix typos in .gitignore (#1697)
90a5631 
Small fix to some typos that cropped up in the .gitignore with #1676

Authors:
  - Charles Blackmon-Luca (https://github.com/charlesbluca)
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Mark Harris (https://github.com/harrism)

URL: #1697
@jameslamb
make conda installs in CI stricter (part 2) (#1703)
1b70ffd 
Follow-up to #1696

Changes relative to that PR:

* switches to pinning CI conda installs to the output of `rapids-version` (`{major}.{minor}.{patch}`) instead of `rapids-version-major-minor` (`{major}.{minor}`), to get a bit more protection in the presence of hotfix releases
* restores some exporting of variables needed for docs builds

In #1696, I'd missed that this project's Doxygen setup is expecting to find `RAPIDS_VERSION_MAJOR_MINOR` defined in the environment.

https://github.com/rapidsai/rmm/blob/90a5631e1093ce44c4feceb88fcf557c3dfc043b/ci/build_docs.sh#L36

https://github.com/rapidsai/rmm/blob/90a5631e1093ce44c4feceb88fcf557c3dfc043b/doxygen/Doxyfile#L41

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Mike Sarahan (https://github.com/msarahan)

URL: #1703
@vyasr
Deprecate support for directly accessing logger (#1690)
de42f57 
Contributes to rapidsai/build-planning#104




This PR removes support for accessing rmm's underlying spdlog logger directly.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Lawrence Mitchell (https://github.com/wence-)
  - Mark Harris (https://github.com/harrism)

URL: #1690
@bdice
Fix docs warning (#1706)
50e60a8 
Closes #1705.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

URL: #1706
@robertmaynard
Update rmm tests to use rapids_cmake_support_conda_env (#1707)
1024a12 
Fixes issue brought up in rapidsai/rapids-cmake#634 (comment) where rmm wasn't using rapids_cmake_support_conda_env

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)
  - Mark Harris (https://github.com/harrism)

URL: #1707
@jjacobelli
devcontainer: replace VAULT_HOST with AWS_ROLE_ARN (#1708)
1ebfe0a 
This PR is replacing the `VAULT_HOST` variable with `AWS_ROLE_ARN`. This is required to use the new token service to get AWS credentials.

Authors:
  - Jordan Jacobelli (https://github.com/jjacobelli)

Approvers:
  - Paul Taylor (https://github.com/trxcllnt)
  - Bradley Dice (https://github.com/bdice)

URL: #1708
@jameslamb
print sccache stats in builds (#1712)
47dae24 
Contributes to rapidsai/build-planning#111

Proposes some small packaging/CI changes, matching similar changes being made across RAPIDS.

* printing `sccache` stats to CI logs
* updating to the latest `rapids-dependency-file-generator` (v1.16.0)
* reducing verbosity of `pip wheel` in wheel builds

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)

URL: #1712
@Matt711
Deprecate rmm._lib (#1713)
8d49fff 
Follows up #1676 to add deprecation warnings to the `rmm._lib` sub package.

Authors:
  - Matthew Murray (https://github.com/Matt711)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1713
@robertmaynard
Correct rmm tests for validity of device pointers (#1714)
9b76d36 
The `is_host_memory` function has been updated to understand that `cudaMemoryTypeUnregistered` is returned when provided pointers allocated by `malloc` and other host side allocation functions.

The `is_device_memory` function has been restricted to report only when device pointer that is usable by the calling cuda context. For that reason the tests now also set the active cuda device for all calling threads.

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Rong Ou (https://github.com/rongou)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1714
@trivialfis
[fea] Expose the arena mr to the Python interface. (#1711)
dbae8c0 
Close #830 .

- Add the arena allocator to the public Python interface.
- Small changes to the logger initialization to avoid exposing spdlog in the shared objects.

Authors:
  - Jiaming Yuan (https://github.com/trivialfis)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1711
@bdice
Disallow cuda-python 12.6.1 and 11.8.4 (#1720)
d4c0635 
Due to a bug in cuda-python we must disallow cuda-python 12.6.1 and 11.8.4. See rapidsai/build-planning#116 for more information.

This PR disallows those versions, and other changes following from that:

* specifying `python` in both `host:` and `run:` dependencies for the `rmm` conda package
* ignoring deprecation warnings raised by newer versions of `cuda-python`

Authors:
  - Bradley Dice (https://github.com/bdice)
  - James Lamb (https://github.com/jameslamb)
  - https://github.com/jakirkham

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #1720
@jameslamb
WIP: put a ceiling on cuda-python (#1723)
a98c22a 
Follow-up to #1720

Contributes to rapidsai/build-planning#116

That PR used `!=` requirements to skip a particular version of `cuda-python` that `rmm` was incompatible with. A newer version of `cuda-python` (12.6.2 for CUDA 12, 11.8.5 for CUDA 11) was just released, and it also causes some build issues for RAPIDS libraries: rapidsai/cuvs#445 (comment)

To unblock CI across RAPIDS, this proposes **temporarily** switching to ceilings on `rmm`'s `cuda-python` dependency.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1723
@jameslamb
use rapids-generate-pip-constraints to pin to oldest dependencies in …
771ccdc 
…CI (#1716)

Follow-up to #1613

Similar to rapidsai/cudf#17131

Proposes using the new `rapids-generate-pip-constraints` tool from `gha-tools` to generate a list of pip constraints pinning to the oldest supported verisons of dependencies here.

## Notes for Reviewers

### How I tested this

`wheel-tests`:

* oldest-deps: numpy 1.x ([build link](https://github.com/rapidsai/rmm/actions/runs/11620907528/job/32364032641?pr=1716#step:8:106))
* latest-deps: numpy 2.x ([build link](https://github.com/rapidsai/rmm/actions/runs/11620907528/job/32364032835?pr=1716#step:8:104))

And the testing of the general approach in rapidsai/gha-tools#114 (comment)

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #1716
@jameslamb
enforce wheel size limits, README formatting in CI (#1726)
56f3a46 
Contributes to rapidsai/build-planning#110

Proposes adding 2 types of validation on wheels in CI, to ensure we continue to produce wheels that are suitable for PyPI.

* checks on wheel size (compressed),
  - *to be sure they're under PyPI limits*
  - *and to prompt discussion on PRs that significantly increase wheel sizes*
* checks on README formatting
  - *to ensure they'll render properly as the PyPI project homepages*
  - *e.g. like how https://github.com/scikit-learn/scikit-learn/blob/main/README.rst becomes https://pypi.org/project/scikit-learn/*

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #1726
@harrism
Treat deprecation warnings as errors and fix deprecation warnings in …
220003e 
…replay benchmark (#1728)

Fixes #1727
Contributes to rapidsai/build-planning#26

 - Removes `-Wno-error=deprecated-declarations`
 - Replaces deprecated usage of `rmm::logger()` in reply benchmark with supported `RMM_LOG_INFO` macros.

Note the latter duplicates a change in #1724 which allows the two PRs to be merged independently.

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Rong Ou (https://github.com/rongou)
  - Bradley Dice (https://github.com/bdice)

URL: #1728
@harrism
Remove all explicit usage of fmtlib (#1724)
52d61c5 
Fixes #1717
Also fixes #1710 in 5330063

I have replaced fmt-style format string placeholders (`"... {} ..."`) with printf-style placeholders by adding a function `rmm::detail::formatted_log()`, which I modified from @vyasr 's #1722.

~The only remaining mention of fmt is in CMakeLists.txt. Do we still need to explicitly fetch fmt?~
Removed.

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1724
@msarahan
adding telemetry (#1692)
c7fc017 
close #1691

Authors:
  - Mike Sarahan (https://github.com/msarahan)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #1692
@robertmaynard
Make cudaMallocAsync logic non-optional as we require CUDA 11.2+ (#…
929a595 
…1667)

We can remove the optimizations around  `CUDA_STATIC_RUNTIME` and instead see if the function is already in the process space so that RMM doesn't need to have any build context to run properly

Fixes #1679

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1667
@harrism
Query total memory in failure_callback_resource_adaptor tests (#1734)
3b5f6af 
Fixes #1733 by querying total device memory and using twice as much in tests that are expected to fail allocation.

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #1734
@raydouglass raydouglass requested review from a team as code owners November 21, 2024 20:49
@github-actions github-actions bot added CMake Python Related to RMM Python API conda cpp Pertains to C++ code ci labels Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bdice bdice Awaiting requested review from bdice bdice is a code owner automatically assigned from rapidsai/packaging-codeowners

@jrhemstad jrhemstad Awaiting requested review from jrhemstad jrhemstad is a code owner automatically assigned from rapidsai/rmm-cpp-codeowners

At least 3 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
ci CMake conda cpp Pertains to C++ code Python Related to RMM Python API
Projects
RMM Project Board
Status: No status
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.