diff --git a/README.md b/README.md index a966415..0495b59 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ ___ * Do you already have a subscription to a Cloud Provider for any the models above? Configure the model using your credentials and start querying! * Are you interested in knowing how to self-deploy open-source models in Azure and GCP? -Consult our [guide](#TODO megatron6000 PR) +Consult our [guide](https://allms.allegro.tech/usage/deploy_open_source_models/) ___ diff --git a/docs/usage/deploy_open_source_models.md b/docs/usage/deploy_open_source_models.md index ab19f1b..8a790b6 100644 --- a/docs/usage/deploy_open_source_models.md +++ b/docs/usage/deploy_open_source_models.md @@ -1,6 +1,6 @@ -# How to Deploy Open-source LLMs? +# How to Deploy Open-source LLMs on Azure and GCP? -## How to Deploy Azure Open-source LLMs? +## Azure To use Open-source models like Llama or Mistral with allms, first you have to deploy it on your own on Azure as a ML Online Endpoint. Here's how to do it: @@ -35,5 +35,5 @@ llama_response = llama_model.generate("2+2 is?") In case of any problems with deployment, you can review this guide on the Azure blog: [Introducing Llama 2 on Azure](https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/introducing-llama-2-on-azure/ba-p/3881233) -## How to Deploy GCP Open-source LLMs? -To deploy a model on the GCP VertexAI Model Garden, [follow the following guide](https://cloud.google.com/vertex-ai/docs/start/explore-models#deploy-a-model). \ No newline at end of file +## GCP +[Follow the following guide](https://cloud.google.com/vertex-ai/docs/start/explore-models#deploy-a-model) to deploy a model on the GCP VertexAI Model Garden. \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index e492835..f24845c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -8,7 +8,7 @@ nav: - Basic: usage/basic.md - Advanced: usage/advanced.md - Forcing Structured Output Format: usage/forcing_response_format.md - - Azure Open-source Models Deployment: usage/deploy_open_source_models.md + - How to Deploy Open-source Models on Azure and GCP?: usage/deploy_open_source_models.md - API: - Domain: api/input_output_dataclasses.md - Models: