Update to latest CUDA, use NCCL_jll

[simonbyrne]

This updates NCCL.jl to CUDA.jl 5, and to use the new NCCL_jll build.

[simonbyrne]

Okay, this is now working locally for me: I've tried to keep the argument names/keywords args somewhat consistent with MPI.jl.

Remaining items

• Set up CI: if you can set up a pipeline, I'm happy to try to figure it out. Note that NCCL appears to require at least 3 devices, see Cannot initialize on 1 or 2 devices. NVIDIA/nccl#1125.
• Set up Docs

[simonbyrne]

@maleadt ?

[maleadt]

We don't have enough GPUs to take 3 devices and assign them to a single worker, only for testing of NCCL.jl...

[simonbyrne]

Fair enough. Okay, I think this is ready. It may need a DOCUMENTER_KEY if it doesn't already exist.

[ToucheSir]

FWIW I've had no issue using libraries which use NCCL on systems with only two devices. Maybe a local environment or config issue?

[maleadt]

I'll try setting up a 2 GPU worker.