Skip to content

Commit

Permalink
chore(docs): add documentation for l4t images
Browse files Browse the repository at this point in the history
Signed-off-by: Ettore Di Giacinto <[email protected]>
  • Loading branch information
mudler committed Jan 29, 2025
1 parent 1f4e66d commit 7f62b41
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 3 deletions.
14 changes: 13 additions & 1 deletion docs/content/docs/getting-started/container-images.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ Images are available with and without python dependencies. Note that images with

Images with `core` in the tag are smaller and do not contain any python dependencies.

{{< tabs tabTotal="7" >}}
{{< tabs tabTotal="8" >}}
{{% tab tabName="Vanilla / CPU Images" %}}

| Description | Quay | Docker Hub |
Expand Down Expand Up @@ -236,6 +236,18 @@ Images with `core` in the tag are smaller and do not contain any python dependen
| Versioned image including FFMpeg, no python | `quay.io/go-skynet/local-ai:{{< version >}}-vulkan-fmpeg-core` | `localai/localai:{{< version >}}-vulkan-fmpeg-core` |
{{% /tab %}}

{{% tab tabName="Nvidia Linux for tegra" %}}

These images are compatible with Nvidia ARM64 devices, such as the Jetson Nano, Jetson Xavier NX, and Jetson AGX Xavier. For more information, see the [Nvidia L4T guide]({{%relref "docs/reference/nvidia-l4t" %}}).

| Description | Quay | Docker Hub |
| --- | --- |-------------------------------------------------------------|
| Latest images from the branch (development) | `quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core` | `localai/localai:master-nvidia-l4t-arm64-core` |
| Latest tag | `quay.io/go-skynet/local-ai:latest-nvidia-l4t-arm64-core` | `localai/localai:latest-nvidia-l4t-arm64-core` |
| Versioned image | `quay.io/go-skynet/local-ai:{{< version >}}-nvidia-l4t-arm64-core` | `localai/localai:{{< version >}}-nvidia-l4t-arm64-core` |

{{% /tab %}}

{{< /tabs >}}

## See Also
Expand Down
10 changes: 8 additions & 2 deletions docs/content/docs/reference/nvidia-l4t.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,21 @@ git clone https://github.com/mudler/LocalAI

cd LocalAI

docker build --build-arg SKIP_DRIVERS=true --build-arg BUILD_TYPE=cublas --build-arg BASE_IMAGE=nvcr.io/nvidia/l4t-jetpack:r36.4.0 --build-arg IMAGE_TYPE=core -t localai-orin .
docker build --build-arg SKIP_DRIVERS=true --build-arg BUILD_TYPE=cublas --build-arg BASE_IMAGE=nvcr.io/nvidia/l4t-jetpack:r36.4.0 --build-arg IMAGE_TYPE=core -t quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core .
```

Otherwise images are available on quay.io and dockerhub:

```bash
docker pull quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core
```

## Usage

Run the LocalAI container on Nvidia ARM64 devices using the following command, where `/data/models` is the directory containing the models:

```bash
docker run -e DEBUG=true -p 8080:8080 -v /data/models:/build/models -ti --restart=always --name local-ai --runtime nvidia --gpus all localai-orin
docker run -e DEBUG=true -p 8080:8080 -v /data/models:/build/models -ti --restart=always --name local-ai --runtime nvidia --gpus all quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core
```

Note: `/data/models` is the directory containing the models. You can replace it with the directory containing your models.

0 comments on commit 7f62b41

Please sign in to comment.