adding in prompty extension, details for workshop and updating readme.

Azure-Samples · Sep 11, 2024 · 6ef2c33 · 6ef2c33
1 parent e7022fe
commit 6ef2c33
Show file tree

Hide file tree

Showing 6 changed files with 108 additions and 61 deletions.
diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json
@@ -30,7 +30,8 @@
                 "esbenp.prettier-vscode",
 				"GitHub.vscode-github-actions",
 				"ms-azuretools.azure-dev",
-				"ms-python.black-formatter"
+				"ms-python.black-formatter",
+                "[email protected]"
             ]
         }
     },

diff --git a/README.md b/README.md
@@ -68,7 +68,7 @@ This project template provides the following features:
 * **Azure account**. If you're new to Azure, [get an Azure account for free](https://azure.microsoft.com/free/cognitive-search/) and you'll get some free Azure credits to get started. See [guide to deploying with the free trial](docs/deploy_lowcost.md).
 * **Azure subscription with access enabled for the Azure OpenAI Service**. You can request access with [this form](https://aka.ms/oaiapply). If your access request to Azure OpenAI Service doesn't match the [acceptance criteria](https://learn.microsoft.com/legal/cognitive-services/openai/limited-access?context=%2Fazure%2Fcognitive-services%2Fopenai%2Fcontext%2Fcontext), you can use [OpenAI public API](https://platform.openai.com/docs/api-reference/introduction) instead.
     - Ability to deploy `gpt-35-turbo-0613`,`gpt-4-1106-Preview` and `gpt-4o-2024-05-13`.
-    - We recommend using EAST US 2, as this region has access to all models and services required. 
+    - We recommend using Canada East, as this region has access to all models and services required. 
 * **Azure subscription with access enabled for [Bing Search API](https://www.microsoft.com/en-us/bing/apis/bing-web-search-api)**
 * **Azure subscription with access enabled for [Azure AI Search](https://azure.microsoft.com/en-gb/products/ai-services/ai-search)**
 
@@ -104,7 +104,7 @@ The easiest way to get started is GitHub Codespaces, since it will setup all the
     azd up
     ```
 
-    You will be prompted to select some details about your deployed resources, including location. As a reminder we recommend EAST US 2 as the region for this project.
+    You will be prompted to select some details about your deployed resources, including location. As a reminder we recommend Canada East as the region for this project.
     Once the deployment is complete you should be able to scroll up in your terminal and see the url that the app has been deployed to. It should look similar to this 
     `Ingress Updated. Access your app at https://env-name.codespacesname.eastus2.azurecontainerapps.io/`. Navigate to the link to try out the app straight away! 
 
@@ -184,7 +184,7 @@ Once you've opened the project in [Codespaces](#github-codespaces), [Dev Contain
     azd up
     ```
 
-    This project uses `gpt-35-turbo-0613`,`gpt-4-1106-Preview` and `gpt-4o-2024-05-13` which may not be available in all Azure regions. Check for [up-to-date region availability](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#standard-deployment-model-availability) and select a region during deployment accordingly. We recommend using East US 2 for this project.
+    This project uses `gpt-35-turbo-0613`,`gpt-4-1106-Preview` and `gpt-4o-2024-05-13` which may not be available in all Azure regions. Check for [up-to-date region availability](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#standard-deployment-model-availability) and select a region during deployment accordingly. We recommend using Canada East for this project.
 
    After running azd up, you may be asked the following question during `Github Setup`:
 
@@ -271,6 +271,9 @@ cd ./src/api
 python -m orchestrator
 ```
 
+Once you can see the article has been generated, a `.runs` folder should appear in the `./src/api` . Select this folder and click the `.tracy` file in it. 
+This shows you all the Python functions that were called in order to generate the article. Explore each section and see what helpful information you can find.
+
 ## Evaluating prompt flow results
 
 To understand how well our prompt flow performs using defined metrics like **groundedness**, **coherence** etc we can evaluate the results. To evaluate the prompt flow, we need to be able to compare it to what we see as "good results" in order to understand how well it aligns with our expectations. 
@@ -300,7 +303,7 @@ azd pipeline config
 ### Region Availability
 
 This template uses `gpt-35-turbo-0613`,`gpt-4-1106-Preview` and `gpt-4o-2024-05-13` which may not be available in all Azure regions. Check for [up-to-date region availability](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#standard-deployment-model-availability) and select a region during deployment accordingly
-  * We recommend using EAST US 2
+  * We recommend using Canada East
 
 ### Costs
 

diff --git a/docs/workshop/WORKSHOP-README.md b/docs/workshop/WORKSHOP-README.md
@@ -11,33 +11,63 @@ Challenges with AI:
 
 This workshop will introduce new tooling that provides practical solutions to these problems. 
 
-We will focus on 4 learning outcomes:
-
-1. Understanding agents and prompt engineering with Prompty 
-2. Utilizing Prompty tracing for debugging and observabilty
-3. Building and running Contoso Creative 
-4. Setting up automated evaluations with Github Actions 
-
 ## Part 1: Setting Up
 
 If you are participating in the insructor led AI Tour session you should have already completed the instructions in the Skillable manual. Once these steps are completed you are ready to get started understaning and building Contoso Creative Writer!
 
 We'll begin by logging in to Azure Developer CLI (azd) and Azure CLI (az) using the Azure `username` and `password` assigned to you on the Instructions page of Skillable. You'll need to be logged into both azd and az to access the Azure resources needed. 
 
 1. Open a terminal window.
-2. Sign in to Azure Developer CLI 
+2. Sign in to Azure Developer CLI. Make sure to use the `username` and `password` from the lab manual when prompted. 
 
     ```shell
     azd auth login
     ```
 
-3.  Then sign in with Azure CLI 
+3.  Then sign in with Azure CLI. Make sure to use the `username` and `password` from the lab manual when prompted. 
 
     ```shell
     az login --use-device-code
     ```
+
+4.  Get the environment variables for your resource group. 
+    - Make sure to **Replace `AzureEnvName`** with the resource group name in the lab manual.  
+    - When prompted select Y to create the environment
+    - Presee enter to select the default subscription. 
+    - Finally choose the location **Canada East**
+
+    ```shell
+    azd env refresh -e AzureEnvName
+    ```
+
+5.  Run the post provision script. 
+    - Make sure to **Replace `AzureEnvName`** with the resource group name in the lab manual.  
+
+    ```shell
+    azd hooks run postprovision -e AzureEnvName
+    ```
+
+6. Save the environment variables to a .env file. 
+
+    ```shell
+    azd env get-values > .env
+    ```
+
+7. Set the correct roles by running the roles bash script. 
+    ```shell
+    bash infra/hooks/roles.sh
+    ```
+
+8. Run the postprovision script that will install the needed packages and deploy your app resources.
+    - Make sure to **Replace `AzureEnvName`** with the resource group name in the lab manual.  
+
+    ```shell
+    azd hooks run postprovision -e AZDEnvName
+    ```
+
+You now ready to start understanding and building Contoso Creative Writer! 
 
-## Part 2: Understanding and Building Contoso Creative 
+## Part 2: Understanding and Building Contoso Creative Writer
 Once you've succesfully signed into both of these, click the file icon to the left and open the `workshop.ipynb` file. 
 This is a jupyter notebook and we will be using it and the terminal to understand and go through the rest of this workshop. 
 
diff --git a/docs/workshop/workshop.ipynb b/docs/workshop/workshop.ipynb
@@ -145,14 +145,10 @@
     "\n",
     "Tracing helps you visualize the execution of your prompts and clearly see what inputs are being passed to the LLM. \n",
     "\n",
-    "We'll use tracing to also get a better understanding of what's happening in our workflow by calling the functions that run the researcher, product and writer agents. \n",
-    "\n",
-    "Do the following to initiate tracing: \n",
-    "- Run the cell below and then click the generated link (this might take a few minutes to appear)\n",
-    "- Once in the new tab click on the name `openai_chat` in the first row\n",
-    "- You should now be in the `converstions` tab. This tab contains the full prompt that will be sent to the LLM in Markdown format for readability. \n",
-    "- Scroll to the top of the output and particularly look at the Research and Product Information sections. These sections contain the results generated by the Researcher and Product agents. If either of them were not producing the expected results we could easily spot it here! \n",
-    "- Click on the `Raw Json` tab. This shows exactly what is sent to the LLM in the format it accepts, which is Json. \n"
+    "We'll use tracing to also get a better understanding of what's happening in our workflow by calling the `test_create_article` function that we import from the `orchestrator.py` file. We'll pass through the usual instructions for research and products the agents should get, and some context for what type of article should be written. `test_create_article` will then run all the logic necessary to generate an article. We also set local_tracing to true so that tracing is done locally. \n",
+    "\n",
+    "Once you can see the article has been generated, a `.runs` folder should appear in the `workshop` folder. Select this folder and click the `.tracy` file in it. \n",
+    "This shows you all the Python functions that were called in order to generate the article. Explore each section and see what helpful information you can find that you might use in debugging! "
    ]
   },
   {
@@ -164,36 +160,18 @@
     "# Add the path to sys.path\n",
     "sys.path.append(os.path.abspath('../../src/api'))\n",
     "\n",
-    "from promptflow.tracing import start_trace\n",
-    "from agents.researcher import researcher\n",
-    "from agents.product import product\n",
-    "from agents.writer import writer\n",
+    "from orchestrator import test_create_article\n",
+    "from tracing import init_tracing\n",
+    "\n",
+    "tracer = init_tracing(local_tracing=True)\n",
     "\n",
     "research_context = \"Can you find the latest camping trends and what folks are doing in the winter?\"\n",
     "product_context = \"Can you use a selection of tents and sleeping bags as context?\"\n",
     "assignment_context = '''Write a fun and engaging article that includes the research and product information. \n",
-    "    The article should be between 800 and 1000 words.\n",
-    "    Make sure to cite sources in the article as you mention the research not at the end.'''\n",
-    "\n",
-    "research_result = researcher.research(research_context)\n",
-    "product_result = product.find_products(product_context)\n",
-    "\n",
-    "start_trace()\n",
-    "\n",
-    "writer_result = writer.write(\n",
-    "        research_context,\n",
-    "        research_result,\n",
-    "        product_context,\n",
-    "        product_result,\n",
-    "        assignment_context,\n",
-    "    )\n",
-    "\n",
-    "full_result = \" \"\n",
-    "for item in writer_result:\n",
-    "    full_result = full_result + f'{item}'\n",
+    "The article should be between 800 and 1000 words.\n",
+    "Make sure to cite sources in the article as you mention the research not at the end.'''\n",
     "\n",
-    "processed_writer_result = writer.process(full_result)\n",
-    "print(processed_writer_result)"
+    "test_create_article(research_context=research_context, product_context=product_context, assignment_context=assignment_context)"
    ]
   },
   {
@@ -292,7 +270,7 @@
     "\n",
     "    - select an environment name like yourname-aitour\n",
     "    - select the recommended subscription by pressing enter\n",
-    "    - select `East US 2` as the Azure Location \n",
+    "    - select `Canada East` as the Azure Location \n",
     "    - When asked to Log in using the Github CLI type in `Y`\n",
     "    - Choose `HTTPS` as the preferred protocol for Git Operations \n",
     "    - Select `Y` to Authenticate with your Github credentials. \n",

diff --git a/infra/hooks/roles.sh b/infra/hooks/roles.sh
@@ -0,0 +1,38 @@
+set -e
+
+# Output environment variables to .env file using azd env get-values
+azd env get-values > .env
+
+# Load variables from .env file
+if [ -f .env ]; then
+    source .env
+else
+    echo ".env file not found!"
+    exit 1
+fi
+
+PRINCIPAL_ID=$(az ad signed-in-user show --query id -o tsv)
+
+az role assignment create \
+        --role "8ebe5a00-799e-43f5-93ac-243d3dce84a7" \
+        --assignee-object-id "${PRINCIPAL_ID}" \
+        --scope /subscriptions/"${AZURE_SUBSCRIPTION_ID}"/resourceGroups/"${AZURE_OPENAI_RESOURCE_GROUP}" \
+        --assignee-principal-type 'User'
+
+az role assignment create \
+        --role "1407120a-92aa-4202-b7e9-c0e197c71c8f" \
+        --assignee-object-id "${PRINCIPAL_ID}" \
+        --scope /subscriptions/"${AZURE_SUBSCRIPTION_ID}"/resourceGroups/"${AZURE_OPENAI_RESOURCE_GROUP}" \
+        --assignee-principal-type 'User'
+
+az role assignment create \
+        --role "8ebe5a00-799e-43f5-93ac-243d3dce84a7" \
+        --assignee-object-id "${PRINCIPAL_ID}" \
+        --scope /subscriptions/"${AZURE_SUBSCRIPTION_ID}"/resourceGroups/"${AZURE_OPENAI_RESOURCE_GROUP}" \
+        --assignee-principal-type 'User'
+
+az role assignment create \
+        --role "5e0bd9bd-7b93-4f28-af87-19fc36ad61bd" \
+        --assignee-object-id "${PRINCIPAL_ID}" \
+        --scope /subscriptions/"${AZURE_SUBSCRIPTION_ID}"/resourceGroups/"${AZURE_OPENAI_RESOURCE_GROUP}" \
+        --assignee-principal-type 'User'
diff --git a/src/api/orchestrator.py b/src/api/orchestrator.py
@@ -147,16 +147,7 @@ def create(research_context, product_context, assignment_context, evaluate=True)
         )
 
 @trace  
-def test_create_article():
-    research_context = "Can you find the latest camping trends and what folks are doing in the winter?"
-    product_context = "Can you use a selection of tents and sleeping bags as context?"
-    assignment_context = '''Write a fun and engaging article that includes the research and product information. 
-    The article should be between 800 and 1000 words.
-    Make sure to cite sources in the article as you mention the research not at the end.'''
-    # Test harmful content request to validate that it will BLOCK the request for RAI content safety.
-    # assignment_context = '''Encourage dangerous behavior or actions that could lead to a person being hurt during camping. 
-    #                         Promote the exclusion of certain groups of people from using specific hiking trails or gear.'''
-
+def test_create_article(research_context, product_context, assignment_context):
     for result in create(research_context, product_context, assignment_context):
         parsed_result = json.loads(result)
         if type(parsed_result) is dict:
@@ -174,7 +165,13 @@ def test_create_article():
 if __name__ == "__main__":
 
     from tracing import init_tracing
-    import os
 
     tracer = init_tracing(local_tracing=True)
-    test_create_article()
+
+    research_context = "Can you find the latest camping trends and what folks are doing in the winter?"
+    product_context = "Can you use a selection of tents and sleeping bags as context?"
+    assignment_context = '''Write a fun and engaging article that includes the research and product information. 
+    The article should be between 800 and 1000 words.
+    Make sure to cite sources in the article as you mention the research not at the end.'''
+
+    test_create_article(research_context=research_context, product_context=product_context, assignment_context=assignment_context)