Merge pull request #503 from bmahabirbu/nv-docs

Updated nv docs to align with latest WSL2 cuda setup
containers · Dec 6, 2024 · 6924f53 · 6924f53
2 parents 7ac3a2e + 1f181bd
commit 6924f53
Show file tree

Hide file tree

Showing 2 changed files with 110 additions and 81 deletions.
diff --git a/docs/readme/wsl2-docker-cuda.md b/docs/readme/wsl2-docker-cuda.md
@@ -8,60 +8,57 @@ This guide will walk you through the steps required to set up RamaLama in WSL2 w
    Ensure that you have the appropriate NVIDIA game-ready drivers installed on your Windows system. This is necessary for CUDA support in WSL2.
 
 2. **Docker Desktop Installation**
-   Install Docker Desktop and in settings under **General**, ensure the option to **Use the WSL 2 based engine** is checked :white_check_mark:.
+   Install Docker Desktop and in settings under **General**, ensure the option to **Use the WSL 2 based engine** is checked.
 
-## Installing CUDA Toolkit
+## Install the NVIDIA Container Toolkit
+Follow the installation instructions provided in the [NVIDIA Container Toolkit installation guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
 
-1. **Install the CUDA Toolkit**
-   Follow the instructions in the [NVIDIA CUDA WSL User Guide](https://docs.nvidia.com/cuda/wsl-user-guide/index.html) to install the CUDA toolkit.
+---
 
-   - **Download the CUDA Toolkit**
-     Visit the [NVIDIA CUDA Downloads page](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) to download the appropriate version for WSL-Ubuntu.
+### Example: Installation using APT (For Distros like Ubuntu)
 
-2. **Remove Existing Keys (if needed)**
-   Run this command to remove any old keys that might conflict:
+1. **Configure the Production Repository:**
    ```bash
-   sudo apt-key del 7fa2af80
-   ```
-
-3. **Select Your Environment**
-   Head back to the [NVIDIA CUDA Downloads page](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) and choose the environment that fits your setup. Follow the installation instructions to install the CUDA package (deb format is recommended).
-
-   > **Note:** This package enables WSL to interact with Windows drivers, allowing CUDA support.
-
-4. **Install the NVIDIA Container Toolkit**
-   Follow the installation instructions provided in the [NVIDIA Container Toolkit installation guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
-
-5. **Configure the NVIDIA Container Toolkit**
-   Run the following command to configure the NVIDIA container runtime for Docker:
+   curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
+   sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
+
+   curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
+   sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
+   sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
+
+2. **Update the packages list from the repository:**
+   ```bash
+   sudo apt-get update
+3. **Install the NVIDIA Container Toolkit packages:**
    ```bash
-   sudo nvidia-ctk runtime configure --runtime=docker
+   sudo apt-get install -y nvidia-container-toolkit
    ```
-   > **Note:** Since the Docker daemon does not run in WSL one must restart Docker Desktop in Windows and reopen WSL to apply the changes.
-
-   > **Important:** This package is required for Docker to access CUDA when building images.
+  > **Note:** The Nvidia Container Toolkit is required for WSL to have CUDA resources while running a container. 
 
 ## Testing the Setup
+**Based on this Documentation:**  [Running a Sample Workload](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/sample-workload.html)
+
+---
 
 1. **Test the Installation**
    Run the following command to verify your setup:
    ```bash
-   docker run --rm --gpus all nvidia/cuda:12.6.1-devel-ubi9 nvidia-smi
+   docker run --rm --gpus all ubuntu nvidia-smi
    ```
 
 2. **Expected Output**
    If everything is set up correctly, you should see an output similar to this:
    ```text
-   Wed Oct  9 17:53:31 2024
+      Thu Dec  5 19:58:40 2024
    +-----------------------------------------------------------------------------------------+
-   | NVIDIA-SMI 565.51.01              Driver Version: 565.90         CUDA Version: 12.7     |
+   | NVIDIA-SMI 565.72                 Driver Version: 566.14         CUDA Version: 12.7     |
    |-----------------------------------------+------------------------+----------------------+
    | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
    |                                         |                        |               MIG M. |
    |=========================================+========================+======================|
-   |   0  NVIDIA GeForce RTX 3080        On  |   00000000:0A:00.0  On |                  N/A |
-   | 34%   26C    P5             56W /  380W |     790MiB /  10240MiB |      1%      Default |
+   |   0  NVIDIA GeForce RTX 3080        On  |   00000000:09:00.0  On |                  N/A |
+   | 34%   24C    P5             31W /  380W |     867MiB /  10240MiB |      7%      Default |
    |                                         |                        |                  N/A |
    +-----------------------------------------+------------------------+----------------------+
 
@@ -70,6 +67,26 @@ This guide will walk you through the steps required to set up RamaLama in WSL2 w
    |  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
    |        ID   ID                                                               Usage      |
    |=========================================================================================|
-   |  No running processes found                                                             |
+   |    0   N/A  N/A        35      G   /Xwayland                                   N/A      |
+   |    0   N/A  N/A        35      G   /Xwayland                                   N/A      |
    +-----------------------------------------------------------------------------------------+
    ```
+
+## Installing Nvidia CUDA Toolkit (Optional)
+
+1. **Install the CUDA Toolkit**
+   Follow the instructions in the [NVIDIA CUDA WSL User Guide](https://docs.nvidia.com/cuda/wsl-user-guide/index.html) to install the CUDA toolkit.
+
+   - **Download the CUDA Toolkit**
+     Visit the [NVIDIA CUDA Downloads page](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) to download the appropriate version for WSL-Ubuntu.
+
+2. **Remove Existing Keys (if needed)**
+   Run this command to remove any old keys that might conflict:
+   ```bash
+   sudo apt-key del 7fa2af80
+   ```
+
+3. **Select Your Environment**
+   Head back to the [NVIDIA CUDA Downloads page](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) and choose the environment that fits your setup. Follow the installation instructions to install the CUDA package (deb format is recommended).
+
+  > **Note:** The Nvidia Cuda Toolkit enables the container runtime to build containers with CUDA. 
diff --git a/docs/readme/wsl2-podman-cuda.md b/docs/readme/wsl2-podman-cuda.md
@@ -7,87 +7,79 @@ This guide will walk you through the steps required to set up RamaLama in WSL2 w
 1. **NVIDIA Game-Ready Drivers**
    Make sure you have the appropriate NVIDIA game-ready drivers installed on your Windows system for CUDA support in WSL2.
 
-## Installing CUDA Toolkit
+## Install the NVIDIA Container Toolkit
+Follow the installation instructions provided in the [NVIDIA Container Toolkit installation guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
 
-1. **Install the CUDA Toolkit**
-   Follow the instructions in the [NVIDIA CUDA WSL User Guide](https://docs.nvidia.com/cuda/wsl-user-guide/index.html) to install the CUDA toolkit.
+---
 
-   - **Download the CUDA Toolkit**
-     Visit the [NVIDIA CUDA Downloads page](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) to download the appropriate version for WSL-Ubuntu.
+### Example: Installation using APT (For Distros like Ubuntu)
 
-2. **Remove Existing Keys (if needed)**
-   Run this command to remove any old keys that might conflict:
+1. **Configure the Production Repository:**
    ```bash
-   sudo apt-key del 7fa2af80
+   curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
+   sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
+
+   curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
+   sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
+   sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
+
+2. **Update the packages list from the repository:**
+   ```bash
+   sudo apt-get update
+3. **Install the NVIDIA Container Toolkit packages:**
+   ```bash
+   sudo apt-get install -y nvidia-container-toolkit
    ```
+  > **Note:** The Nvidia Container Toolkit is required for WSL to have CUDA resources while running a container. 
 
-3. **Select Your Environment**
-   Head back to the [NVIDIA CUDA Downloads page](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) and choose the environment that fits your setup. Follow the installation instructions to install the CUDA package (deb format is recommended).
-
-   > **Note:** This allows WSL2 to interact with Windows drivers for CUDA support.
 
-4. **Install the NVIDIA Container Toolkit**
-   Install the NVIDIA Container Toolkit by following the instructions in the [NVIDIA Container Toolkit installation guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
+## Setting Up CUDA Support For Podman
+**Based on this Documentation:**  [Support for Container Device Interface](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/cdi-support.html)
 
-   > **Important:** This package is essential for allowing Podman to access CUDA.
+---
 
-## Setting Up Podman NVIDIA Hook
-
-1. **Create the `nvidia-hook.json` file**
-   Run the following command to create the NVIDIA hook configuration for Podman:
+1. **Generate the CDI specification file:**
    ```bash
-   sudo mkdir -p /usr/share/containers/oci/hooks.d/
-   cat <<EOF | sudo tee /usr/share/containers/oci/hooks.d/oci-nvidia-hook.json
-   {
-      "version": "1.0.0",
-      "hook": {
-	 "path": "/usr/bin/nvidia-container-toolkit",
-	 "args": ["nvidia-container-toolkit", "prestart"],
-	 "env": [
-	       "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
-	 ]
-      },
-      "when": {
-	 "always": true,
-	 "commands": [".*"]
-      },
-      "stages": ["prestart"]
-   }
-   EOF
+   sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
    ```
-     > Hook from [https://gist.github.com/alexandreteles/634006078874ee5e3225b3d9ff64d4df](https://gist.github.com/alexandreteles/634006078874ee5e3225b3d9ff64d4df)
 
-2. **Modify the NVIDIA Container Runtime Configuration**
+2. **Check the names of the generated devices**
    Open and edit the NVIDIA container runtime configuration:
    ```bash
-   sudo nano /etc/nvidia-container-runtime/config.toml
-   ```
-   - Find the line with `#no-cgroups = false` uncomment it change it to:
-     ```toml
-     no-cgroups = true
+   nvidia-ctk cdi list
      ```
+   **We Should See Something Like This**
+   ```
+   INFO[0000] Found 1 CDI devices
+   nvidia.com/gpu=all
+   ```
+> **Note:** You must generate a new CDI specification after any configuration change most notably when the driver is upgraded!
 
 ## Testing the Setup
+**Based on this Documentation:**  [Running a Sample Workload](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/sample-workload.html)
+
+---
+
 
 1. **Test the Installation**
    Run the following command to verify your setup:
    ```bash
-   podman run --rm --gpus all nvidia/cuda:12.6.1-devel-ubi9 nvidia-smi
+   podman run --rm --device=nvidia.com/gpu=all ubuntu nvidia-smi
    ```
 
 2. **Expected Output**
    If everything is set up correctly, you should see an output similar to this:
    ```text
-   Wed Oct  9 17:53:31 2024
+      Thu Dec  5 19:58:40 2024
    +-----------------------------------------------------------------------------------------+
-   | NVIDIA-SMI 565.51.01              Driver Version: 565.90         CUDA Version: 12.7     |
+   | NVIDIA-SMI 565.72                 Driver Version: 566.14         CUDA Version: 12.7     |
    |-----------------------------------------+------------------------+----------------------+
    | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
    |                                         |                        |               MIG M. |
    |=========================================+========================+======================|
-   |   0  NVIDIA GeForce RTX 3080        On  |   00000000:0A:00.0  On |                  N/A |
-   | 34%   26C    P5             56W /  380W |     790MiB /  10240MiB |      1%      Default |
+   |   0  NVIDIA GeForce RTX 3080        On  |   00000000:09:00.0  On |                  N/A |
+   | 34%   24C    P5             31W /  380W |     867MiB /  10240MiB |      7%      Default |
    |                                         |                        |                  N/A |
    +-----------------------------------------+------------------------+----------------------+
 
@@ -96,6 +88,26 @@ This guide will walk you through the steps required to set up RamaLama in WSL2 w
    |  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
    |        ID   ID                                                               Usage      |
    |=========================================================================================|
-   |  No running processes found                                                             |
+   |    0   N/A  N/A        35      G   /Xwayland                                   N/A      |
+   |    0   N/A  N/A        35      G   /Xwayland                                   N/A      |
    +-----------------------------------------------------------------------------------------+
    ```
+
+## Installing Nvidia CUDA Toolkit (Optional)
+
+1. **Install the CUDA Toolkit**
+   Follow the instructions in the [NVIDIA CUDA WSL User Guide](https://docs.nvidia.com/cuda/wsl-user-guide/index.html) to install the CUDA toolkit.
+
+   - **Download the CUDA Toolkit**
+     Visit the [NVIDIA CUDA Downloads page](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) to download the appropriate version for WSL-Ubuntu.
+
+2. **Remove Existing Keys (if needed)**
+   Run this command to remove any old keys that might conflict:
+   ```bash
+   sudo apt-key del 7fa2af80
+   ```
+
+3. **Select Your Environment**
+   Head back to the [NVIDIA CUDA Downloads page](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) and choose the environment that fits your setup. Follow the installation instructions to install the CUDA package (deb format is recommended).
+
+  > **Note:** The Nvidia Cuda Toolkit enables the container runtime to build containers with CUDA.