Skip to content

Commit

Permalink
added meta lesson
Browse files Browse the repository at this point in the history
  • Loading branch information
@koreyspace
koreyspace committed Oct 3, 2024
1 parent 078a44d commit 8e37099
Show file tree
Hide file tree
Showing 3 changed files with 396 additions and 0 deletions.
155 changes: 155 additions & 0 deletions 21-meta/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# Building With the Meta Family Models

## Introduction

This lesson will cover:

- Exploring the two main Meta family models - Llama 3.1 and Llama 3.2
- Understanding the use-cases and scenarios for each model
- Code sample to show the unique features of each model


## The Meta Family of Models

In this lesson, we will explore 2 models from the Meta family or "Llama Herd" - Llama 3.1 and Llama 3.2

These models come in different variants and are available on the Github Model marketplace. Here are more details on using Github Models to [prototype with AI models](https://docs.github.com/en/github-models/prototyping-with-ai-models?WT.mc_id=academic-105485-koreyst).

Model Variants:
Llama 3.1 - 70B Instruct
Llama 3.1 - 405B Instruct
Llama 3.2 - 11B Vision Instruct
Llama 3.2 - 90B Vision Instruct

*Note: Llama 3 is also available on Github Models but won't be covered in this lesson*

## Llama 3.1

At 405 Billion Parameters, Llama 3.1 fits into the open source LLM category.

The mode is an upgrade to the earlier release Llama 3 by offering:

- Larger context window - 128k tokens vs 8k tokens
- Larger Max Output Tokens - 4096 vs 2048
- Better Multilingual Support - due to increase in training tokens

These enables Llama 3.1 to handle more complex use cases when building GenAI applications including:
- Native Function Calling - the ability to call external tools and functions outside of the LLM workflow
- Better RAG Performance - due to the higher context window
- Synthetic Data Generation - the ability to create effective data for tasks such as fine-tuning

### Native Function Calling

Llama 3.1 has been fine-tuned to be more effective at making function or tool calls. It also has two built-in tools that the model can identify as needing to be used based on the prompt from the user. These tools are:

- **Brave Search** - Can be used to get up-to-date information like the weather by performing a web search
- **Wolfram Alpha** - Can be used for more complex mathematical calculations so writing your own functions is not required.

You can also create your own custom tools that LLM can call.

In the code example below:

- We define the available tools (brave_search, wolfram_alpha) in the system prompt.
- Send a user prompt that asks about the weather in a certain city.
- The LLM will respond with a tool call to the Brave Search tool which will look like this `<|python_tag|>brave_search.call(query="Stockholm weather")`

*Note: This example only make the tool call, if you would like to get the results, you will need to create a free account on the Brave API page and define the function itself`

```python
import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import AssistantMessage, SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential

token = os.environ["GITHUB_TOKEN"]
endpoint = "https://models.inference.ai.azure.com"
model_name = "meta-llama-3.1-405b-instruct"

client = ChatCompletionsClient(
endpoint=endpoint,
credential=AzureKeyCredential(token),
)


tool_prompt=f"""
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
Environment: ipython
Tools: brave_search, wolfram_alpha
Cutting Knowledge Date: December 2023
Today Date: 23 July 2024
You are a helpful assistant<|eot_id|>
"""

messages = [
SystemMessage(content=tool_prompt),
UserMessage(content="What is the weather in Stockholm?"),

]

response = client.complete(messages=messages, model=model_name)

print(response.choices[0].message.content)
```

## Llama 3.2

Despite being a LLM, one limitation that Llama 3.1 has is multimodality. That is, being able to use different types of input such as images as prompts and providing responses. This ability is one of the main features of Llama 3.2. These features also include:

- Multimodality - has the ability to evaluate both text and image prompts
- Small to Medium size variations (11B and 90B) - this provides flexible deployment options,
- Text-only variations (1B and 3B) - this allows the model to be deployed on edge / mobile devices and provides low latency

The multimodal support represents a big step in the world of open source models. The code example below takes both an image and text prompt to get an analysis of the image from Llama 3.2 90B.


### Multimodal Support with Llama 3.2

```python
import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import (
SystemMessage,
UserMessage,
TextContentItem,
ImageContentItem,
ImageUrl,
ImageDetailLevel,
)
from azure.core.credentials import AzureKeyCredential

token = os.environ["GITHUB_TOKEN"]
endpoint = "https://models.inference.ai.azure.com"
model_name = "Llama-3.2-90B-Vision-Instruct"

client = ChatCompletionsClient(
endpoint=endpoint,
credential=AzureKeyCredential(token),
)

response = client.complete(
messages=[
SystemMessage(
content="You are a helpful assistant that describes images in details."
),
UserMessage(
content=[
TextContentItem(text="What's in this image?"),
ImageContentItem(
image_url=ImageUrl.load(
image_file="sample.jpg",
image_format="jpg",
detail=ImageDetailLevel.LOW)
),
],
),
],
model=model_name,
)

print(response.choices[0].message.content)
```



241 changes: 241 additions & 0 deletions 21-meta/python/githubmodels-assignment.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Building With the Meta Family Models \n",
"\n",
"## Introduction \n",
"\n",
"This lesson will cover: \n",
"\n",
"- Exploring the two main Meta family models - Llama 3.1 and Llama 3.2 \n",
"- Understanding the use-cases and scenarios for each model \n",
"- Code sample to show the unique features of each model \n",
"\n",
"\n",
"## The Meta Family of Models \n",
"\n",
"In this lesson, we will explore 2 models from the Meta family or \"Llama Herd\" - Llama 3.1 and Llama 3.2 \n",
"\n",
"These models come in different variants and are available on the Github Model marketplace. Here are more details on using Github Models to [prototype with AI models](https://docs.github.com/en/github-models/prototyping-with-ai-models?WT.mc_id=academic-105485-koreyst).\n",
"\n",
"Model Variants: \n",
"Llama 3.1 - 70B Instruct \n",
"Llama 3.1 - 405B Instruct \n",
"Llama 3.2 - 11B Vision Instruct \n",
"Llama 3.2 - 90B Vision Instruct \n",
"\n",
"*Note: Llama 3 is also available on Github Models but won't be covered in this lesson*\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Llama 3.1 \n",
"\n",
"At 405 Billion Parameters, Llama 3.1 fits into the open source LLM category. \n",
"\n",
"The mode is an upgrade to the earlier release Llama 3 by offering: \n",
"\n",
"- Larger context window - 128k tokens vs 8k tokens \n",
"- Larger Max Output Tokens - 4096 vs 2048 \n",
"- Better Multilingual Support - due to increase in training tokens \n",
"\n",
"These enables Llama 3.1 to handle more complex use cases when building GenAI applications including: \n",
"- Native Function Calling - the ability to call external tools and functions outside of the LLM workflow\n",
"- Better RAG Performance - due to the higher context window \n",
"- Synthetic Data Generation - the ability to create effective data for tasks such as fine-tuning \n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Native Function Calling \n",
"\n",
"Llama 3.1 has been fine-tuned to be more effective at making function or tool calls. It also has two built-in tools that the model can identify as needing to be used based on the prompt from the user. These tools are: \n",
"\n",
"- **Brave Search** - Can be used to get up-to-date information like the weather by performing a web search \n",
"- **Wolfram Alpha** - Can be used for more complex mathematical calculations so writing your own functions is not required. \n",
"\n",
"You can also create your own custom tools that LLM can call. \n",
"\n",
"In the code example below: \n",
"\n",
"- We define the available tools (brave_search, wolfram_alpha) in the system prompt. \n",
"- Send a user prompt that asks about the weather in a certain city. \n",
"- The LLM will respond with a tool call to the Brave Search tool which will look like this `<|python_tag|>brave_search.call(query=\"Stockholm weather\")` \n",
"\n",
"*Note: This example only make the tool call, if you would like to get the results, you will need to create a free account on the Brave API page and define the function itself` "
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<|python_tag|>brave_search.call(query=\"Stockholm weather\")\n"
]
}
],
"source": [
"import os\n",
"from azure.ai.inference import ChatCompletionsClient\n",
"from azure.ai.inference.models import AssistantMessage, SystemMessage, UserMessage\n",
"from azure.core.credentials import AzureKeyCredential\n",
"\n",
"token = os.environ[\"GITHUB_TOKEN\"]\n",
"endpoint = \"https://models.inference.ai.azure.com\"\n",
"model_name = \"meta-llama-3.1-405b-instruct\"\n",
"\n",
"client = ChatCompletionsClient(\n",
" endpoint=endpoint,\n",
" credential=AzureKeyCredential(token),\n",
")\n",
"\n",
"\n",
"tool_prompt=f\"\"\"\n",
"<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n",
"\n",
"Environment: ipython\n",
"Tools: brave_search, wolfram_alpha\n",
"Cutting Knowledge Date: December 2023\n",
"Today Date: 23 July 2024\n",
"\n",
"You are a helpful assistant<|eot_id|>\n",
"\"\"\"\n",
"\n",
"messages = [\n",
" SystemMessage(content=tool_prompt),\n",
" UserMessage(content=\"What is the weather in Stockholm?\"),\n",
"\n",
"]\n",
"\n",
"response = client.complete(messages=messages, model=model_name)\n",
"\n",
"print(response.choices[0].message.content)\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Llama 3.2 \n",
"\n",
"Despite being a LLM, one limitation that Llama 3.1 has is multimodality. That is, being able to use different types of input such as images as prompts and providing responses. This ability is one of the main features of Llama 3.2. These features also include: \n",
"\n",
"- Multimodality - has the ability to evaluate both text and image prompts \n",
"- Small to Medium size variations (11B and 90B) - this provides flexible deployment options, \n",
"- Text-only variations (1B and 3B) - this allows the model to be deployed on edge / mobile devices and provides low latency \n",
"\n",
"The multimodal support represents a big step in the world of open source models. The code example below takes both an image and text prompt to get an analysis of the image from Llama 3.2 90B. \n",
"\n",
"### Multimodal Support with Llama 3.2"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The image shows a screenshot from a video or presentation about AI, with a woman on the left side and a catalog of models on the right side. The woman is wearing a gray hoodie and has blonde hair. She is visible from the chest up and appears to be speaking or presenting.\n",
"\n",
"The model catalog is titled \"MODEL CATALOG\" in large text at the top, with the subtitle \"Find the right model to build your custom AI solution\" below it. The catalog lists various models, including \"gpt-4,\" \"gpt-35-turbo-instruct,\" and \"mistralai-Mistral-Rx228-v0-1.\" Each model has a brief description and a button to view more information.\n",
"\n",
"At the bottom of the image, there is a blue bar with white text that reads \"Try it yourself\" and a URL that says \"https://ai.azure.com.\" The background of the image is white, with some colorful graphics and logos in the corners. Overall, the image appears to be a screenshot from a presentation or video about AI and machine learning, showcasing a catalog of models that can be used to build custom AI solutions.\n"
]
}
],
"source": [
"import os\n",
"from azure.ai.inference import ChatCompletionsClient\n",
"from azure.ai.inference.models import (\n",
" SystemMessage,\n",
" UserMessage,\n",
" TextContentItem,\n",
" ImageContentItem,\n",
" ImageUrl,\n",
" ImageDetailLevel,\n",
")\n",
"from azure.core.credentials import AzureKeyCredential\n",
"\n",
"token = os.environ[\"GITHUB_TOKEN\"]\n",
"endpoint = \"https://models.inference.ai.azure.com\"\n",
"model_name = \"Llama-3.2-90B-Vision-Instruct\"\n",
"\n",
"client = ChatCompletionsClient(\n",
" endpoint=endpoint,\n",
" credential=AzureKeyCredential(token),\n",
")\n",
"\n",
"response = client.complete(\n",
" messages=[\n",
" SystemMessage(\n",
" content=\"You are a helpful assistant that describes images in details.\"\n",
" ),\n",
" UserMessage(\n",
" content=[\n",
" TextContentItem(text=\"What's in this image?\"),\n",
" ImageContentItem(\n",
" image_url=ImageUrl.load(\n",
" image_file=\"sample.jpg\",\n",
" image_format=\"jpg\",\n",
" detail=ImageDetailLevel.LOW)\n",
" ),\n",
" ],\n",
" ),\n",
" ],\n",
" model=model_name,\n",
")\n",
"\n",
"print(response.choices[0].message.content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Learning does not stop here, continue the Journey\n",
"\n",
"After completing this lesson, check out our [Generative AI Learning collection](https://aka.ms/genai-collection?WT.mc_id=academic-105485-koreyst) to continue leveling up your Generative AI knowledge!"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Binary file added 21-meta/python/sample.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 8e37099

Please sign in to comment.