From 24b7a60e67f5129abcc3eac691ff7dbc87ec4aa9 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Sun, 8 Dec 2024 17:36:39 -0800 Subject: [PATCH] fix todos --- docs/gen-ai/gen-ai-events.md | 44 ++++++++++++++----- model/gen-ai/events.yaml | 9 +++- .../registry/markdown/body_field_table.j2 | 5 ++- 3 files changed, 43 insertions(+), 15 deletions(-) diff --git a/docs/gen-ai/gen-ai-events.md b/docs/gen-ai/gen-ai-events.md index 5f6a4948e9..20fb582088 100644 --- a/docs/gen-ai/gen-ai-events.md +++ b/docs/gen-ai/gen-ai-events.md @@ -48,7 +48,7 @@ Instrumentations MAY offer configuration options allowing to disable events or a ## Event: `gen_ai.system.message` - + @@ -65,7 +65,7 @@ This event describes the instructions passed to the GenAI system inside the prom |---|---|---|---|---|---| | [`gen_ai.system`](/docs/attributes-registry/gen-ai.md) | string | The Generative AI product as identified by the client or server instrumentation. [1] | `openai` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -**[1]:** The `gen_ai.system` describes a family of GenAI models with specific model identified +**[1] `gen_ai.system`:** The `gen_ai.system` describes a family of GenAI models with specific model identified by `gen_ai.request.model` and `gen_ai.response.model` attributes. The actual GenAI product may differ from the one identified by the client. @@ -75,13 +75,17 @@ is set to `openai` based on the instrumentation's best knowledge. For custom model, a custom friendly name SHOULD be used. If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. +--- + `gen_ai.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | Value | Description | Stability | |---|---|---| | `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `aws.bedrock` | AWS Bedrock | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `az.ai.inference` | Azure AI Inference | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `ibm.watsonx.ai` | IBM Watsonx AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `vertex_ai` | Vertex AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | @@ -99,7 +103,7 @@ If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. ## Event: `gen_ai.user.message` - + @@ -116,7 +120,7 @@ This event describes the prompt message specified by the user. |---|---|---|---|---|---| | [`gen_ai.system`](/docs/attributes-registry/gen-ai.md) | string | The Generative AI product as identified by the client or server instrumentation. [1] | `openai` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -**[1]:** The `gen_ai.system` describes a family of GenAI models with specific model identified +**[1] `gen_ai.system`:** The `gen_ai.system` describes a family of GenAI models with specific model identified by `gen_ai.request.model` and `gen_ai.response.model` attributes. The actual GenAI product may differ from the one identified by the client. @@ -126,13 +130,17 @@ is set to `openai` based on the instrumentation's best knowledge. For custom model, a custom friendly name SHOULD be used. If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. +--- + `gen_ai.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | Value | Description | Stability | |---|---|---| | `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `aws.bedrock` | AWS Bedrock | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `az.ai.inference` | Azure AI Inference | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `ibm.watsonx.ai` | IBM Watsonx AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `vertex_ai` | Vertex AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | @@ -150,7 +158,7 @@ If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. ## Event: `gen_ai.assistant.message` - + @@ -167,7 +175,7 @@ This event describes the assistant message passed to GenAI system or received fr |---|---|---|---|---|---| | [`gen_ai.system`](/docs/attributes-registry/gen-ai.md) | string | The Generative AI product as identified by the client or server instrumentation. [1] | `openai` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -**[1]:** The `gen_ai.system` describes a family of GenAI models with specific model identified +**[1] `gen_ai.system`:** The `gen_ai.system` describes a family of GenAI models with specific model identified by `gen_ai.request.model` and `gen_ai.response.model` attributes. The actual GenAI product may differ from the one identified by the client. @@ -177,13 +185,17 @@ is set to `openai` based on the instrumentation's best knowledge. For custom model, a custom friendly name SHOULD be used. If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. +--- + `gen_ai.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | Value | Description | Stability | |---|---|---| | `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `aws.bedrock` | AWS Bedrock | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `az.ai.inference` | Azure AI Inference | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `ibm.watsonx.ai` | IBM Watsonx AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `vertex_ai` | Vertex AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | @@ -193,7 +205,7 @@ If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. |---|---|---|---|---|---| | `content` | undefined | The contents of the tool message. | `The weather in Paris is rainy and overcast, with temperatures around 57°F` | `Opt-In` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `role` | string | The actual role of the message author as passed in the message. | `assistant`; `bot` | `Conditionally Required` if available and not equal to `assistant`. | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `tool_calls`: | map | The tool calls generated by the model, such as function calls. | | `Conditionally Required` if available | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `tool_calls`: | map[] | The tool calls generated by the model, such as function calls. | | `Conditionally Required` if available | ![Experimental](https://img.shields.io/badge/-experimental-blue) | |   `function`: | map | The function call. | | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | |     `arguments` | undefined | The arguments of the function as provided in the LLM response. [1] | `{\"location\": \"Paris\"}` | `Opt-In` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | |     `name` | string | The name of the function. | `get_weather` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | @@ -216,7 +228,7 @@ Semantic conventions for individual systems MAY specify a different type for arg ## Event: `gen_ai.tool.message` - + @@ -233,7 +245,7 @@ This event describes the tool or function response message. |---|---|---|---|---|---| | [`gen_ai.system`](/docs/attributes-registry/gen-ai.md) | string | The Generative AI product as identified by the client or server instrumentation. [1] | `openai` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -**[1]:** The `gen_ai.system` describes a family of GenAI models with specific model identified +**[1] `gen_ai.system`:** The `gen_ai.system` describes a family of GenAI models with specific model identified by `gen_ai.request.model` and `gen_ai.response.model` attributes. The actual GenAI product may differ from the one identified by the client. @@ -243,13 +255,17 @@ is set to `openai` based on the instrumentation's best knowledge. For custom model, a custom friendly name SHOULD be used. If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. +--- + `gen_ai.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | Value | Description | Stability | |---|---|---| | `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `aws.bedrock` | AWS Bedrock | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `az.ai.inference` | Azure AI Inference | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `ibm.watsonx.ai` | IBM Watsonx AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `vertex_ai` | Vertex AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | @@ -268,7 +284,7 @@ If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. ## Event: `gen_ai.choice` - + @@ -285,7 +301,7 @@ This event describes the Gen AI response message. |---|---|---|---|---|---| | [`gen_ai.system`](/docs/attributes-registry/gen-ai.md) | string | The Generative AI product as identified by the client or server instrumentation. [1] | `openai` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -**[1]:** The `gen_ai.system` describes a family of GenAI models with specific model identified +**[1] `gen_ai.system`:** The `gen_ai.system` describes a family of GenAI models with specific model identified by `gen_ai.request.model` and `gen_ai.response.model` attributes. The actual GenAI product may differ from the one identified by the client. @@ -295,13 +311,17 @@ is set to `openai` based on the instrumentation's best knowledge. For custom model, a custom friendly name SHOULD be used. If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. +--- + `gen_ai.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | Value | Description | Stability | |---|---|---| | `anthropic` | Anthropic | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `aws.bedrock` | AWS Bedrock | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `az.ai.inference` | Azure AI Inference | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `cohere` | Cohere | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `ibm.watsonx.ai` | IBM Watsonx AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `openai` | OpenAI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | | `vertex_ai` | Vertex AI | ![Experimental](https://img.shields.io/badge/-experimental-blue) | @@ -314,7 +334,7 @@ If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. | `message`: | map | GenAI response message. | | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | |   `content` | undefined | The contents of the assistant message. | `The weather in Paris is rainy and overcast, with temperatures around 57°F` | `Opt-In` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | |   `role` | string | The actual role of the message author as passed in the message. | `assistant`; `bot` | `Conditionally Required` if available and not equal to `assistant`. | ![Experimental](https://img.shields.io/badge/-experimental-blue) | -| `tool_calls`: | map | The tool calls generated by the model, such as function calls. | | `Conditionally Required` if available | ![Experimental](https://img.shields.io/badge/-experimental-blue) | +| `tool_calls`: | map[] | The tool calls generated by the model, such as function calls. | | `Conditionally Required` if available | ![Experimental](https://img.shields.io/badge/-experimental-blue) | |   `function`: | map | The function that the model called. | | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | |     `arguments` | undefined | The arguments of the function as provided in the LLM response. [1] | `{\"location\": \"Paris\"}` | `Opt-In` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | |     `name` | string | The name of the function. | `get_weather` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) | diff --git a/model/gen-ai/events.yaml b/model/gen-ai/events.yaml index 97fc5f31d6..48605f9ee5 100644 --- a/model/gen-ai/events.yaml +++ b/model/gen-ai/events.yaml @@ -91,7 +91,7 @@ groups: requirement_level: conditionally_required: if available and not equal to `assistant`. - id: tool_calls - type: map # TODO: it's an array + type: map[] stability: experimental brief: > The tool calls generated by the model, such as function calls. @@ -210,18 +210,23 @@ groups: members: - id: stop value: 'stop' + stability: experimental brief: Stop - id: tool_calls value: 'tool_calls' + stability: experimental brief: Tool Calls - id: content_filter value: 'content_filter' + stability: experimental brief: Content Filter - id: length value: 'length' + stability: experimental brief: Length - id: error value: 'error' + stability: experimental brief: Error stability: experimental brief: > @@ -250,7 +255,7 @@ groups: requirement_level: conditionally_required: if available and not equal to `assistant`. - id: tool_calls - type: map # TODO: it's an array + type: map[] stability: experimental brief: > The tool calls generated by the model, such as function calls. diff --git a/templates/registry/markdown/body_field_table.j2 b/templates/registry/markdown/body_field_table.j2 index ca97a077f0..b600f6dd7f 100644 --- a/templates/registry/markdown/body_field_table.j2 +++ b/templates/registry/markdown/body_field_table.j2 @@ -6,7 +6,10 @@ {% macro flatten(fields, ns, depth) %}{% if fields %}{% for f in fields | sort(attribute="id") %} {% set ns.flat = [ns.flat, [{'field':f,'depth':depth}]] | flatten %}{% if f.fields %}{% set _= flatten(f.fields, ns, depth + 1) %}{% endif %} {% endfor %}{% endif %}{% endmacro %} -{% macro field_name(field, depth) %}{% set name= " " * 2 * depth ~ '`' ~ field.id ~ '`' %}{% if field.type == "map" %}{{ name ~ ":"}}{% else %}{{ name }}{% endif %}{% endmacro %} +{% macro field_name(field, depth) -%} +{%- set name= " " * 2 * depth ~ '`' ~ field.id ~ '`' -%} +{%- if (field.type == "map") or (field.type == "map[]") %}{{ name ~ ":"}}{% else -%} +{{ name }}{% endif %}{% endmacro %} {#- Macro for creating body table -#} {% macro generate(fields) %}{% if (fields | length > 0) %}{% set ns = namespace(flat=[])%}{% set _ = flatten(fields, ns, 0) %}| Body Field | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability | |---|---|---|---|---|---|