Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UI updates - Labs 2, 4, 5 and 7 #50

Merged
merged 7 commits into from
Jan 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 13 additions & 11 deletions Instructions/02-Explore-model-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ This exercise will take approximately **25** minutes.

An Azure AI hub provides a collaborative workspace within which you can define one or more *projects*. Let's create a project and Azure AI hub.

1. In a web browser, open [Azure AI Foundry portal](https://ai.azure.com) at `https://ai.azure.com` and sign in using your Azure credentials.

1. In the home page, select **+ Create project**. In the **Create a project** wizard you can see all the Azure resources that will be automatically created with your project, or you can customize the following settings by selecting **Customize** before selecting **Create**:

- **Hub name**: *A unique name*
Expand All @@ -27,7 +29,7 @@ An Azure AI hub provides a collaborative workspace within which you can define o
> \* Azure OpenAI resources are constrained at the tenant level by regional quotas. The listed regions in the location helper include default quota for the model type(s) used in this exercise. Randomly choosing a region reduces the risk of a single region reaching its quota limit. In the event of a quota limit being reached later in the exercise, there's a possibility you may need to create another resource in a different region. Learn more about [model availability per region](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#gpt-35-turbo-model-availability)

1. If you selected **Customize**, select **Next** and review your configuration.
1. 1. Select **Create** and wait for the process to complete.
1. Select **Create** and wait for the process to complete.

After the Azure AI hub and project have been created, it should look similar to the following image:

Expand All @@ -50,16 +52,16 @@ Imagine you want to create a custom copilot that serves as a travel assistant. S

Your copilot will need to provide factually accurate information, so groundedness is important. Next to that, you want the copilot's answers to be easy to read and understand. Therefore, you also want to pick a model that rates high on fluency and coherence.

1. In the Azure AI Foundry project portal, navigate to **Model benchmarks** using the menu on the left.
In the **Quality benchmarks** tab, you can find some charts already visualized for you, comparing different models.
1. Filter the shown models:
- **Tasks**: Question answering
- **Collections**: Azure OpenAI
- **Metrics**: Coherence, Fluency, Groundedness
1. Explore the resulting charts and the comparison table. When exploring, you can try and answer the following questions:
1. In the Azure AI Foundry project portal, navigate to **Model catalog** using the menu on the left.
In the catalog page, select **Compare with benchmarks**. In the Model benchmarks page, you will find a chart already plotted for you, comparing different models.
1. Select **+ Model to compare** and add **gpt-4-32k** and **gpt-4** to the metrics chart. In the **X-axis** dropdown menu, under **Quality**, select the following metrics and observe each resulting chart before switching to the next:
- Coherence
- Fluency
- Groundedness
1. When exploring the results, you can try and answer the following questions:
- Do you notice a difference in performance between GPT-3.5 and GPT-4 models?
- Is there a difference between versions of the same model?
- How do the 32k variants differ from the base models?
- How does the 32k variant of GPT-4 differ from the base model?

From the Azure OpenAI collection, you can choose between GPT-3.5 and GPT-4 models. Let's deploy these two models and explore how they compare for your use case.

Expand All @@ -82,6 +84,8 @@ Let's start by deploying a model from the Model catalog. You may prefer this opt
- **Content filter**: DefaultV2
- **Enable dynamic quota**: Disabled

> **Note**: If your current AI resource location doesn't have quota available for the model you want to deploy, you will be asked to choose a different location where a new AI resource will be created and connected to your project.

### Deploy a model through Models + endpoints

If you already know exactly which model you want to deploy, you may prefer to do it through **Models + endpoints**.
Expand All @@ -97,8 +101,6 @@ If you already know exactly which model you want to deploy, you may prefer to do
- **Content filter**: DefaultV2
- **Enable dynamic quota**: Disabled

> **Note**: If your current AI resource location doesn't have quota available for the model you want to deploy, you will be asked to choose a different location where a new AI resource will be created and connected to your project.

## Test your models in the chat playground

Now that we have two models to compare, let's see how the models behave in a conversational interaction.
Expand Down
4 changes: 3 additions & 1 deletion Instructions/04-Use-own-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,9 @@ You need two models to implement your solution:
- **Tokens per Minute Rate Limit (thousands)**: 5K
- **Content filter**: DefaultV2
- **Enable dynamic quota**: Disabled


> **Note**: If your current AI resource location doesn't have quota available for the model you want to deploy, you will be asked to choose a different location where a new AI resource will be created and connected to your project.

1. Repeat the previous steps to deploy a **gpt-35-turbo-16k** model with the deployment name `gpt-35-turbo-16k`.

> **Note**: Reducing the Tokens Per Minute (TPM) helps avoid over-using the quota available in the subscription you are using. 5,000 TPM is sufficient for the data used in this exercise.
Expand Down
6 changes: 3 additions & 3 deletions Instructions/05-Finetune-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,9 @@ As fine-tuning a model takes some time to complete, you'll start the fine-tuning
<p>If you receive a permissions error, try the following to troubleshoot:</p>
<ul>
<li>In the Azure portal, select the AI Services resource.</li>
<li>On the IAM page, in the Identity tab, confirm that it is system assigned managed identity.</li>
<li>Navigate to the associated Storage Account. On the IAM page, add role assignment <em>Storage blob data reader</em>.</li>
<li>Under <strong>Assign access to</strong>, choose <strong>Managed Identity</strong>, <strong>+ Select members</strong>, and select the <strong>All system-assigned managed identities</strong>.</li>
<li>Under Resource Management, in the Identity tab, confirm that it is system assigned managed identity.</li>
<li>Navigate to the associated Storage Account. On the IAM page, add role assignment <em>Storage Blob Data Owner</em>.</li>
<li>Under <strong>Assign access to</strong>, choose <strong>Managed Identity</strong>, <strong>+ Select members</strong>, select the <strong>All system-assigned managed identities</strong>, and select your Azure AI services resource.</li>
<li>Review and assign to save the new settings and retry the previous step.</li>
</ul>
</details>
Expand Down
3 changes: 3 additions & 0 deletions Instructions/07-Evaluate-prompt-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ To use a language model in prompt flow, you need to deploy a model first. The Az
- **Tokens per Minute Rate Limit (thousands)**: 5K
- **Content filter**: DefaultV2
- **Enable dynamic quota**: Disabled

> **Note**: If your current AI resource location doesn't have quota available for the model you want to deploy, you will be asked to choose a different location where a new AI resource will be created and connected to your project.

1. Wait for the model to be deployed. When the deployment is ready, select **Open in playground**.
1. In the **Give the model instructions and context** text box, change the content to the following:

Expand Down