AI Agent Span Semantic Convention

docs/gen-ai/gen-ai-agent-spans.md

@@ -0,0 +1,95 @@
+<!--- Hugo front matter used to generate the website version of this page:
+linkTitle: Generative AI traces
+--->
+
+# Semantic Conventions for GenAI agent and framework spans
+
+**Status**: [Experimental][DocumentStatus]
+
+<!-- Re-generate TOC with `markdown-toc --no-first-h1 -i` -->
+
+<!-- toc -->
+
+- [SPAN KIND](#span-kind)
+- [Name](#name)
+- [AI Agent attributes](#ai-agent-attributes)
+  - [Workflow Attributes](#workflow-attributes)
+  - [Agent Attributes](#agent-attributes)
+  - [Tools Attributes](#tools-attributes)
+  - [Task Attributes](#task-attributes)
+  - [Interaction Attributes](#interaction-attributes)
+- [Examples](#examples)
+  - [Workflow: Market Analysis Pipeline](#workflow-market-analysis-pipeline)
+  - [Agents Involved](#agents-involved)
+    - [DataCollectorAgent](#datacollectoragent)
+    - [DataAnalystAgent](#dataanalystagent)
+  - [Tasks and Tools](#tasks-and-tools)
+    - [Task: Data Collection](#task-data-collection)
+    - [Tool Utilized: WebScraperTool](#tool-utilized-webscrapertool)
+    - [Task: Data Analysis](#task-data-analysis)
+    - [Tool Utilized: DataAnalyzerTool](#tool-utilized-dataanalyzertool)
+
+<!-- tocstop -->
+
+Generative AI scenarios may involve multiple steps. For example RAG pattern implementation usually involves
+the following steps:
+
+- initial request to the GenAI model to rewrite user query
+- getting embeddings for the rewritten query
+- searching for similar documents in the vector store using embeddings
+- getting completion from the GenAI model based on the similar documents.
+
+These steps may be done by the application itself using GenAI models and vector databases.
+It's also common to leverage client framework such as [LangGraph](https://github.com/langchain-ai/langgraph)
+or service offerings such as [OpenAI Assistants](https://platform.openai.com/docs/assistants),
+[Azure AI Agents](TODO link), or [Amazon Bedrock Agents](https://aws.amazon.com/bedrock/agents/).
+
+This document defines semantic conventions for GenAI agent calls that are
+handled by the remote agent service.
+It MAY be applicable to agent operations that are performed by the GenAI
+framework locally.
+
+The semantic conventions for GenAI agents extend and override the semantic conventions
+for [Gen AI Spans](gen-ai-spans.md).
+
+## Spans
+
+### Create Workflow Span
+
+### Create Agent Span
+
+Describes GenAI agent creation and is usually applicable when working with remote agent
+services.
+
+The `gen_ai.operation.name` MUST be `create_agent`.
+
+The **span name** SHOULD be `create_agent {gen_ai.agent.name}`.
+Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format.
+
+<!-- semconv trace.gen_ai.client.create_agent -->
+<!-- endsemconv -->
+
+### Create Task Span
+
+<!-- semconv trace.gen_ai.client.create_task -->
+<!-- endsemconv -->
+
+### Create Tool Span
+
+<!-- semconv trace.gen_ai.client.create_tool -->
+<!-- endsemconv -->
+
+### Run Tool Span
+
+<!-- semconv trace.gen_ai.client.run_tool -->
+<!-- endsemconv -->
+
+### Run Task Span
+
+<!-- semconv trace.gen_ai.client.run_task -->
+<!-- endsemconv -->
+
+### Run Workflow Span
+
+<!-- semconv trace.gen_ai.client.run_workflow -->
+<!-- endsemconv -->

model/gen-ai/registry.yaml

@@ -141,6 +141,96 @@ groups:
               brief: 'Output tokens (completion, response, etc.)'
         brief: The type of token being counted.
         examples: ['input', 'output']
+      - id: gen_ai.agent.workflow.name
+        stability: experimental
+        type: string
+        brief: The name of the workflow that the agent is part of.
+        examples: ["Data Processing Pipeline"]
+      - id: gen_ai.agent.workflow.tasks
+        stability: experimental
+        type: string[]
+        brief: List of tasks included in the workflow.
+        examples: ["Data Collection", "Data Analysis"]
+      - id: gen_ai.agent.workflow.agents
+        stability: experimental
+        type: string[]
+        brief: System or environment where the workflow is executed..
+        examples: ["CrewAI", "LangGraph"]
+      - id: gen_ai.agent.workflow.system
+        stability: experimental
+        type: string
+        brief: The GenAI system used to run the workflow.
+        examples: ["OpenAI"]
+      - id: gen_ai.agent.id   # alternatives: assistant (openai)
+        stability: experimental
+        type: string
+        brief: The unique identifier of the GenAI agent.
+        examples: ['asst_5j66UpCpwteGg4YSxUnt7lPY']
+      - id: gen_ai.agent.name
+        stability: experimental
+        type: string
+        brief: Human-readable name of the GenAI agent provided by the application.
+        examples: ["Math Tutor", "Fiction Writer"]
+      - id: gen_ai.agent.description
+        stability: experimental
+        type: string
+        brief: Free-form description of the GenAI agent provided by the application.
+        examples: ["Helps with math problems", "Generates fiction stories"]
+      - id: gen_ai.agent.workflow_name
+        stability: experimental
+        type: string
+        brief: The name of the workflow that the agent is part of.
+        examples: ["Math Tutoring", "Fiction Writing"]
+      - id: gen_ai.agent.tools
+        stability: experimental
+        type: string[]
+        brief: List of tools available to the agent.
+        examples: ["Web Scraper", "Analyzer"]
+      - id: gen_ai.agent.tool.name
+        stability: experimental
+        type: string
+        brief: 	Name of the tool utilized by the agent.
+        examples: ["Web Scraper"]
+      - id: gen_ai.agent.tool.function
+        stability: experimental
+        type: string
+        brief: Description of the function of the tool utilized by the agent.
+        examples: ["Scrapes the web for data"]
+      - id: gen_ai.agent.tool.output_format
+        stability: experimental
+        type: string
+        brief: The format of the output generated by the tool.
+        examples: ["JSON", "CSV"]
+      - id: gen_ai.agent.tool.output
+        stability: experimental
+        type: string
+        brief: The format of the output generated by the tool.
+        examples: {"data": [...]}
+      - id: gen_ai.agent.task.name
+        stability: experimental
+        type: string
+        brief: The name of the task that the agent is performing.
+        examples: ["Data Collection"]
+      - id: gen_ai.agent.task.agent_name
+        stability: experimental
+        type: string
+        brief: The name of the agent that is performing the task.
+        examples: ["Web Scraper"]
+      - id: gen_ai.agent.task.description
+        stability: experimental
+        type: string
+        brief: A description of the task that the agent is performing.
+        examples: ["Collecting data from various sources"]
+      - id: gen_ai.agent.task.input
+        stability: experimental
+        type: string
+        brief: The input required for the task.
+        examples: ["Identify the next big trend for AI Agent"]
+      - id: gen_ai.agent.task.output
+        stability: experimental
+        type: string
+        brief: The output generated by the task.
+        examples: {"data": [...]}
       - id: gen_ai.operation.name
         stability: experimental
         type:
@@ -157,6 +247,38 @@ groups:
               value: "embeddings"
               brief: 'Embeddings operation such as [OpenAI Create embeddings API](https://platform.openai.com/docs/api-reference/embeddings/create)'
               stability: experimental
+            - id: create_agent
+              value: "create_agent"
+              brief: 'Create GenAI agent'
+              stability: experimental
+            - id: create_task
+              value: "create_task"
+              brief: 'Create GenAI task'
+              stability: experimental
+            - id: create_tool
+              value: "create_tool"
+              brief: 'Create GenAI tool'
+              stability: experimental
+            - id: create_workflow
+              value: "create_workflow"
+              brief: 'Create GenAI Workflow'
+              stability: experimental
+            - id: run_workflow
+              value: "run_workflow"
+              brief: 'Run GenAI Workflow'
+              stability: experimental
+            - id: run_agent
+              value: "run_agent"
+              brief: 'Run GenAI Agent'
+              stability: experimental
+            - id: run_task
+              value: "run_task"
+              brief: 'Run GenAI Task'
+              stability: experimental
+            - id: run_tool
+              value: "run_tool"
+              brief: 'Run GenAI Tool'
+              stability: experimental
         brief: The name of the operation being performed.
         note: >
           If one of the predefined values applies, but specific system uses a different name it's RECOMMENDED to document it in the semantic

model/gen-ai/spans.yaml

@@ -122,3 +122,135 @@ groups:
       - ref: server.port
         requirement_level:
           conditionally_required: If not default (443).
+  - id: trace.gen_ai.client.create_workflow
+    type: span
+    stability: experimental
+    brief: >
+      Describes GenAI external tool creation and is usually applicable when working
+      with remote external services.
+    note: |
+      The `gen_ai.operation.name` MUST be `create_tool`.
+      The **span name** SHOULD be `create_tool {gen_ai.agent.tool.name}`.
+      Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format.
+    extends: trace.gen_ai.client.create_task
+    attributes:
+      - ref: gen_ai.agent.tool.name
+        requirement_level: required
+  - id: trace.gen_ai.client.create_agent
+    type: span
+    stability: experimental
+    brief: >
+      Describes GenAI agent creation and is usually applicable when working
+      with remote agent services.
+    note: |
+      The `gen_ai.operation.name` MUST be `create_agent`.
+      The **span name** SHOULD be `create_agent {gen_ai.agent.name}`.
+      Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format.
+    extends: trace.gen_ai.client.common_attributes
+    attributes:
+      - ref: gen_ai.system
+        requirement_level: required
+      - ref: gen_ai.agent.id
+        requirement_level:
+          conditionally_required: if applicable.
+      - ref: gen_ai.agent.name
+        requirement_level:
+          conditionally_required: If provided by the application.
+      - ref: gen_ai.agent.description
+        requirement_level:
+          conditionally_required: If provided by the application.
+      - ref: gen_ai.request.model
+        requirement_level:
+          conditionally_required: If provided by the application.
+        note: >
+            The name of the GenAI model a request is being made to. If the model is supplied by a vendor,
+            then the value must be the exact name of the model requested. If the model is a fine-tuned
+            custom model, the value should have a more specific name than the base model that's been fine-tuned.
+      - ref: gen_ai.request.temperature
+        requirement_level:
+          conditionally_required: If provided by the application.
+      - ref: gen_ai.request.top_p
+        requirement_level:
+          conditionally_required: If provided by the application.
+  - id: trace.gen_ai.client.create_task
+    type: span
+    stability: experimental
+    brief: >
+      Describes GenAI task creation and is usually applicable when working
+      with remote task services.
+    note: |
+      The `gen_ai.operation.name` MUST be `create_task`.
+      The **span name** SHOULD be `create_task {gen_ai.agent.task.name}`.
+      Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format.
+    attributes:
+      - ref: gen_ai.agent.task.name
+        requirement_level: required
+  - id: trace.gen_ai.client.create_tool
+    type: span
+    stability: experimental
+    brief: >
+      Describes GenAI external tool creation and is usually applicable when working
+      with remote external services.
+    note: |
+      The `gen_ai.operation.name` MUST be `create_tool`.
+      The **span name** SHOULD be `create_tool {gen_ai.agent.tool.name}`.
+      Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format.
+    attributes:
+      - ref: gen_ai.agent.tool.name
+        requirement_level: required
+  - id: trace.gen_ai.client.run_tool
+    type: span
+    stability: experimental
+    brief: >
+      Describes GenAI external tool creation and is usually applicable when working
+      with remote external services.
+    note: |
+      The `gen_ai.operation.name` MUST be `create_tool`.
+      The **span name** SHOULD be `create_tool {gen_ai.agent.tool.name}`.
+      Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format.
+    extends: trace.gen_ai.client.create_tool
+    attributes:
+      - ref: gen_ai.agent.tool.name
+        requirement_level: required
+  - id: trace.gen_ai.client.run_task
+    type: span
+    stability: experimental
+    brief: >
+      Describes GenAI external tool creation and is usually applicable when working
+      with remote external services.
+    note: |
+      The `gen_ai.operation.name` MUST be `create_tool`.
+      The **span name** SHOULD be `create_tool {gen_ai.agent.tool.name}`.
+      Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format.
+    extends: trace.gen_ai.client.create_task
+    attributes:
+      - ref: gen_ai.agent.tool.name
+        requirement_level: required
+  - id: trace.gen_ai.client.create_workflow
+    type: span
+    stability: experimental
+    brief: >
+      Describes GenAI external tool creation and is usually applicable when working
+      with remote external services.
+    note: |
+      The `gen_ai.operation.name` MUST be `create_tool`.
+      The **span name** SHOULD be `create_tool {gen_ai.agent.tool.name}`.
+      Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format.
+    extends: trace.gen_ai.client.create_task
+    attributes:
+      - ref: gen_ai.agent.tool.name
+        requirement_level: required
+  - id: trace.gen_ai.client.run_workflow
+    type: span
+    stability: experimental
+    brief: >
+      Describes GenAI external tool creation and is usually applicable when working
+      with remote external services.
+    note: |
+      The `gen_ai.operation.name` MUST be `create_tool`.
+      The **span name** SHOULD be `create_tool {gen_ai.agent.tool.name}`.
+      Semantic conventions for individual GenAI systems and frameworks MAY specify different span name format.
+    extends: trace.gen_ai.client.create_task
+    attributes:
+      - ref: gen_ai.agent.tool.name
+        requirement_level: required