Skip to content

Commit

Permalink
update README.md and docs
Browse files Browse the repository at this point in the history
  • Loading branch information
megatron6000 committed Mar 4, 2024
1 parent 2b0e5f2 commit 3e1b4fa
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 6 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ ___
* Do you already have a subscription to a Cloud Provider for any the models above? Configure
the model using your credentials and start querying!
* Are you interested in knowing how to self-deploy open-source models in Azure and GCP?
Consult our [guide](#TODO megatron6000 PR)
Consult our [guide](https://allms.allegro.tech/usage/deploy_open_source_models/)

___

Expand Down
8 changes: 4 additions & 4 deletions docs/usage/deploy_open_source_models.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# How to Deploy Open-source LLMs?
# How to Deploy Open-source LLMs on Azure and GCP?

## How to Deploy Azure Open-source LLMs?
## Azure

To use Open-source models like Llama or Mistral with allms, first you have to deploy it on your own on Azure as a ML Online Endpoint.
Here's how to do it:
Expand Down Expand Up @@ -35,5 +35,5 @@ llama_response = llama_model.generate("2+2 is?")
In case of any problems with deployment, you can review this guide on the Azure blog:
[Introducing Llama 2 on Azure](https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/introducing-llama-2-on-azure/ba-p/3881233)

## How to Deploy GCP Open-source LLMs?
To deploy a model on the GCP VertexAI Model Garden, [follow the following guide](https://cloud.google.com/vertex-ai/docs/start/explore-models#deploy-a-model).
## GCP
[Follow the following guide](https://cloud.google.com/vertex-ai/docs/start/explore-models#deploy-a-model) to deploy a model on the GCP VertexAI Model Garden.
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ nav:
- Basic: usage/basic.md
- Advanced: usage/advanced.md
- Forcing Structured Output Format: usage/forcing_response_format.md
- Azure Open-source Models Deployment: usage/deploy_open_source_models.md
- How to Deploy Open-source Models on Azure and GCP?: usage/deploy_open_source_models.md
- API:
- Domain: api/input_output_dataclasses.md
- Models:
Expand Down

0 comments on commit 3e1b4fa

Please sign in to comment.