Skip to content

Commit

Permalink
docs: add blog post of offline deployment
Browse files Browse the repository at this point in the history
  • Loading branch information
wwayne committed Mar 18, 2024
1 parent 6d3883e commit 6847fef
Show file tree
Hide file tree
Showing 3 changed files with 55 additions and 33 deletions.
52 changes: 52 additions & 0 deletions website/blog/2024-03-18-deploying-offline-with-docker/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
slug: deploying-offline-with-docker
title: "Deploying offline with Docker"
authors: [wwayne]
tags: [deployment offline]
---

<div align="center">

![No internet access](./no-internet.png)

</div>

Are you facing a situation where your work environment lacks internet access, and you're wondering if you can still deploy Tabby? Fear not, because the answer is YES! 🐱📣

## Prerequisite📋
* Docker installed on both the internet-connected computer and the offline computer.

## The Game Plan🚀
* Create a Docker image on a computer with internet access.
* Transfer the image to your offline computer.
* Run the Docker image and let Tabby work its magic! ✨

## Step-by-Step Guide🐾
1. Create a new **Dockerfile** on a computer with internet access.
```docker
FROM tabbyml/tabby
RUN /opt/tabby/bin/tabby-cpu download --model TabbyML/StarCoder-1B
```
2. Build the Docker image which containing the model
```bash
docker build -t tabby-offline .
```
3. Save the Docker image to a tar file:
```bash
docker save -o tabby-offline.tar tabby-offline
```
4. Copy the `tabby-offline.tar` file to the computer without internet access.
5. Load the Docker image from the tar file:
```bash
docker load -i tabby-offline.tar
```
6. Run the Tabby container
```bash
docker run -it \
--gpus all -p 8080:8080 -v $HOME/.tabby:/data \
tabby-offline \
serve --model TabbyML/StarCoder-1B --device cuda
```

If you encounter any further issues or have questions, feel free to explore our [community](https://slack.tabbyml.com/). Our friendly Tabby enthusiasts are always ready to lend a helping paw and guide you to the answers you seek! 😸💡
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
33 changes: 0 additions & 33 deletions website/docs/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,37 +48,4 @@ For details on the registry format, please refer to [models.json](https://github

Tabby also supports loading models from a local directory that follow our specifications as outlined in [MODEL_SPEC.md](https://github.com/TabbyML/tabby/blob/main/MODEL_SPEC.md).

</Collapse>

<Collapse title="The computer doesn't have internet access.">
The plan is: we'll create a Docker image on a computer that's connected to the internet. Then, we'll copy the image to the computer that doesn't have internet access and run the Docker image there. Here are the steps:

1. On a computer with internet access, create a new **Dockerfile**.
```
FROM tabbyml/tabby
RUN /opt/tabby/bin/tabby-cpu download --model TabbyML/StarCoder-1B
```
2. Build the Docker image which containing the model
```bash
docker build -t tabby-offline .
```
3. Save the Docker image to a tar file:
```bash
docker save -o tabby-offline.tar tabby-offline
```
4. Copy the `tabby-offline.tar` file to the computer without internet access.
5. On the offline computer, load the Docker image from the tar file:
```bash
docker load -i tabby-offline.tar
```
6. Run the Tabby container
```bash
docker run -it \
--gpus all -p 8080:8080 -v $HOME/.tabby:/data \
tabby-offline \
serve --model TabbyML/StarCoder-1B --device cuda
```


</Collapse>

0 comments on commit 6847fef

Please sign in to comment.