diff --git a/Instructions/02-Explore-model-catalog.md b/Instructions/02-Explore-model-catalog.md index 24a956f..933adcc 100644 --- a/Instructions/02-Explore-model-catalog.md +++ b/Instructions/02-Explore-model-catalog.md @@ -15,6 +15,8 @@ This exercise will take approximately **25** minutes. An Azure AI hub provides a collaborative workspace within which you can define one or more *projects*. Let's create a project and Azure AI hub. +1. In a web browser, open [Azure AI Foundry portal](https://ai.azure.com) at `https://ai.azure.com` and sign in using your Azure credentials. + 1. In the home page, select **+ Create project**. In the **Create a project** wizard you can see all the Azure resources that will be automatically created with your project, or you can customize the following settings by selecting **Customize** before selecting **Create**: - **Hub name**: *A unique name* @@ -27,7 +29,7 @@ An Azure AI hub provides a collaborative workspace within which you can define o > \* Azure OpenAI resources are constrained at the tenant level by regional quotas. The listed regions in the location helper include default quota for the model type(s) used in this exercise. Randomly choosing a region reduces the risk of a single region reaching its quota limit. In the event of a quota limit being reached later in the exercise, there's a possibility you may need to create another resource in a different region. Learn more about [model availability per region](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#gpt-35-turbo-model-availability) 1. If you selected **Customize**, select **Next** and review your configuration. -1. 1. Select **Create** and wait for the process to complete. +1. Select **Create** and wait for the process to complete. After the Azure AI hub and project have been created, it should look similar to the following image: @@ -50,16 +52,16 @@ Imagine you want to create a custom copilot that serves as a travel assistant. S Your copilot will need to provide factually accurate information, so groundedness is important. Next to that, you want the copilot's answers to be easy to read and understand. Therefore, you also want to pick a model that rates high on fluency and coherence. -1. In the Azure AI Foundry project portal, navigate to **Model benchmarks** using the menu on the left. - In the **Quality benchmarks** tab, you can find some charts already visualized for you, comparing different models. -1. Filter the shown models: - - **Tasks**: Question answering - - **Collections**: Azure OpenAI - - **Metrics**: Coherence, Fluency, Groundedness -1. Explore the resulting charts and the comparison table. When exploring, you can try and answer the following questions: +1. In the Azure AI Foundry project portal, navigate to **Model catalog** using the menu on the left. + In the catalog page, select **Compare with benchmarks**. In the Model benchmarks page, you will find a chart already plotted for you, comparing different models. +1. Select **+ Model to compare** and add **gpt-4-32k** and **gpt-4** to the metrics chart. In the **X-axis** dropdown menu, under **Quality**, select the following metrics and observe each resulting chart before switching to the next: + - Coherence + - Fluency + - Groundedness +1. When exploring the results, you can try and answer the following questions: - Do you notice a difference in performance between GPT-3.5 and GPT-4 models? - Is there a difference between versions of the same model? - - How do the 32k variants differ from the base models? + - How does the 32k variant of GPT-4 differ from the base model? From the Azure OpenAI collection, you can choose between GPT-3.5 and GPT-4 models. Let's deploy these two models and explore how they compare for your use case. @@ -82,6 +84,8 @@ Let's start by deploying a model from the Model catalog. You may prefer this opt - **Content filter**: DefaultV2 - **Enable dynamic quota**: Disabled + > **Note**: If your current AI resource location doesn't have quota available for the model you want to deploy, you will be asked to choose a different location where a new AI resource will be created and connected to your project. + ### Deploy a model through Models + endpoints If you already know exactly which model you want to deploy, you may prefer to do it through **Models + endpoints**. @@ -97,8 +101,6 @@ If you already know exactly which model you want to deploy, you may prefer to do - **Content filter**: DefaultV2 - **Enable dynamic quota**: Disabled - > **Note**: If your current AI resource location doesn't have quota available for the model you want to deploy, you will be asked to choose a different location where a new AI resource will be created and connected to your project. - ## Test your models in the chat playground Now that we have two models to compare, let's see how the models behave in a conversational interaction. diff --git a/Instructions/04-Use-own-data.md b/Instructions/04-Use-own-data.md index 3787b05..b49e020 100644 --- a/Instructions/04-Use-own-data.md +++ b/Instructions/04-Use-own-data.md @@ -72,7 +72,9 @@ You need two models to implement your solution: - **Tokens per Minute Rate Limit (thousands)**: 5K - **Content filter**: DefaultV2 - **Enable dynamic quota**: Disabled - + + > **Note**: If your current AI resource location doesn't have quota available for the model you want to deploy, you will be asked to choose a different location where a new AI resource will be created and connected to your project. + 1. Repeat the previous steps to deploy a **gpt-35-turbo-16k** model with the deployment name `gpt-35-turbo-16k`. > **Note**: Reducing the Tokens Per Minute (TPM) helps avoid over-using the quota available in the subscription you are using. 5,000 TPM is sufficient for the data used in this exercise. diff --git a/Instructions/05-Finetune-model.md b/Instructions/05-Finetune-model.md index 2f964c8..6d991fd 100644 --- a/Instructions/05-Finetune-model.md +++ b/Instructions/05-Finetune-model.md @@ -55,9 +55,9 @@ As fine-tuning a model takes some time to complete, you'll start the fine-tuning

If you receive a permissions error, try the following to troubleshoot:

diff --git a/Instructions/07-Evaluate-prompt-flow.md b/Instructions/07-Evaluate-prompt-flow.md index 021da12..273fb16 100644 --- a/Instructions/07-Evaluate-prompt-flow.md +++ b/Instructions/07-Evaluate-prompt-flow.md @@ -42,6 +42,9 @@ To use a language model in prompt flow, you need to deploy a model first. The Az - **Tokens per Minute Rate Limit (thousands)**: 5K - **Content filter**: DefaultV2 - **Enable dynamic quota**: Disabled + + > **Note**: If your current AI resource location doesn't have quota available for the model you want to deploy, you will be asked to choose a different location where a new AI resource will be created and connected to your project. + 1. Wait for the model to be deployed. When the deployment is ready, select **Open in playground**. 1. In the **Give the model instructions and context** text box, change the content to the following: