-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: add blog post of offline deployment
- Loading branch information
Showing
3 changed files
with
55 additions
and
33 deletions.
There are no files selected for viewing
52 changes: 52 additions & 0 deletions
52
website/blog/2024-03-18-deploying-offline-with-docker/index.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
--- | ||
slug: deploying-offline-with-docker | ||
title: "Deploying offline with Docker" | ||
authors: [wwayne] | ||
tags: [deployment offline] | ||
--- | ||
|
||
<div align="center"> | ||
|
||
![No internet access](./no-internet.png) | ||
|
||
</div> | ||
|
||
Are you facing a situation where your work environment lacks internet access, and you're wondering if you can still deploy Tabby? Fear not, because the answer is YES! 🐱📣 | ||
|
||
## Prerequisite📋 | ||
* Docker installed on both the internet-connected computer and the offline computer. | ||
|
||
## The Game Plan🚀 | ||
* Create a Docker image on a computer with internet access. | ||
* Transfer the image to your offline computer. | ||
* Run the Docker image and let Tabby work its magic! ✨ | ||
|
||
## Step-by-Step Guide🐾 | ||
1. Create a new **Dockerfile** on a computer with internet access. | ||
```docker | ||
FROM tabbyml/tabby | ||
RUN /opt/tabby/bin/tabby-cpu download --model TabbyML/StarCoder-1B | ||
``` | ||
2. Build the Docker image which containing the model | ||
```bash | ||
docker build -t tabby-offline . | ||
``` | ||
3. Save the Docker image to a tar file: | ||
```bash | ||
docker save -o tabby-offline.tar tabby-offline | ||
``` | ||
4. Copy the `tabby-offline.tar` file to the computer without internet access. | ||
5. Load the Docker image from the tar file: | ||
```bash | ||
docker load -i tabby-offline.tar | ||
``` | ||
6. Run the Tabby container | ||
```bash | ||
docker run -it \ | ||
--gpus all -p 8080:8080 -v $HOME/.tabby:/data \ | ||
tabby-offline \ | ||
serve --model TabbyML/StarCoder-1B --device cuda | ||
``` | ||
|
||
If you encounter any further issues or have questions, feel free to explore our [community](https://slack.tabbyml.com/). Our friendly Tabby enthusiasts are always ready to lend a helping paw and guide you to the answers you seek! 😸💡 |
3 changes: 3 additions & 0 deletions
3
website/blog/2024-03-18-deploying-offline-with-docker/no-internet.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters