Skip to content

Commit

Permalink
Upgrade OTel from v0.87.0 to v0.96.0
Browse files Browse the repository at this point in the history
  • Loading branch information
@ptodev
ptodev committed Mar 19, 2024
1 parent d550490 commit 8f97b37
Show file tree
Hide file tree
Showing 79 changed files with 1,304 additions and 1,177 deletions.
117 changes: 117 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,123 @@ Main (unreleased)

- Upgrade to Go 1.22.1 (@thampiotr)

- Upgrade from OpenTelemetry Collector v0.87.0 to v0.96.0:
* [ottl]: Fix bug where named parameters needed a space after the equal sign (`=`)
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/28511
* [exporters] Additional enqueue_failed metrics
https://github.com/open-telemetry/opentelemetry-collector/issues/8673
* [otelcol.receiver.kafka]: Fix issue where counting number of logs emitted could cause panic
* [otelcol.processor.k8sattributes]: The time format of k8s.pod.start_time attribute value migrated to RFC3339:
Before: 2023-07-10 12:34:39.740638 -0700 PDT m=+0.020184946
After: 2023-07-10T12:39:53.112485-07:00
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/28817
* [otelcol.processor.tail_sampling] A new `upper_threshold_ms` argument for the `latency` policy.
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/26115
* [otelcol.connector.spanmetrics] Add a new `events` metric.
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27451
* [otelcol.connector.spanmetrics] A new `max_per_data_point` argument for exemplar generation.
* https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/22620
* [ottl] Add IsBool Converter
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27897
* [otelcol.processor.tail_sampling] Optimize memory performance of tailsamplingprocessor
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/27889
* [otelcol.connector.servicegraph] Add a `metrics_flush_interval` argument.
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27679
* [ottl] Add IsDouble Converter
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27895
* [ottl] Add new `silent` ErrorMode
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/29710
* [otelcol.connector.spanmetrics] A new `resource_metrics_cache_size` argument.
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27654
* [ottl] Add IsInt Converter
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27894
* [ottl] Validate that all path elements are used
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30042
* [ottl] Validate Keys are used
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30162
* [otelcol.receiver.vcenter] Add statement of support for version 8 of ESXi and vCenter
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30274
* [ottl] Add Hour converter
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/29468
* [otelcol.connector.spanmetrics] A new `resource_metrics_key_attributes` argument to fix broken spanmetrics counters
after a span producing service restart, when resource attributes contain dynamic/ephemeral values (e.g. process id).
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/29711
* [ottl] Issue with the hash value of a match group in the replace_pattern editors
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/29409
* [ottl] Fix bug where IsBool wasn't usable
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30151
* [ottl] Add flatten function
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30455
* [ottl] Fix bugs with parsing of string escapes in OTTL
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23238
* [ottl]: Add functions for parsing CSV
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/30921
* [ottl] Allow users to specify the format of the hashed replacement string in the `replace_pattern` editors
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27820
* [ottl] Add ParseKeyValue function
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/30998
* [otelcol.receiver.opencensus] Fix memory leak on shutdown
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/31152
* [otelcol.processor.memory_limiter] Fix leaking goroutine
https://github.com/open-telemetry/opentelemetry-collector/issues/9099
* Additional `http2_read_idle_timeout` and `http2_ping_timeout` arguments for HTTP clients
https://github.com/open-telemetry/opentelemetry-collector/pull/9022
* [otelcol.auth.bearer] Fix for "401 Unauthorized" on HTTP connections
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/24656
* Update to OTLP 1.1
https://github.com/open-telemetry/opentelemetry-collector/pull/9588
* [otelcol.auth.basic] Accept empty usernames.
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30470
* [exporters] Do not re-enqueue failed batches, rely on the `retry_on_failure` strategy instead.
https://github.com/open-telemetry/opentelemetry-collector/issues/8382
* [otelcol.exporter.otlphttp] A `Host` header is added automatically.
https://github.com/open-telemetry/opentelemetry-collector/issues/9395
* [exporters] PartialSuccess is treated as success, logged as warning.
https://github.com/open-telemetry/opentelemetry-collector/issues/9243
* [otelcol.exporter.otlphttp] Supports JSON encoding through an additional `encoding` argument.
https://github.com/open-telemetry/opentelemetry-collector/issues/6945
* [exporters] A new `include_system_ca_certs_pool` argument for TLS config.
https://github.com/open-telemetry/opentelemetry-collector/issues/7774
* [otelcol.receiver.vcenter] The receiver emits vCenter performance metrics with object metric label dimension.
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/30615
* [otelcol.processor.transform] Add copy_metric function
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30846
* [otelcol.exporter.loadbalancing] Optimized CPU performance
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30141
* [otelcol.processor.k8sattributes] Set attributes from namespace/node labels or annotations even if node/namespaces attribute are not set.
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/28837
* [otelcol.receiver.kafka] An additional `resolve_canonical_bootstrap_servers_only` argument
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/26022
* [otelcol.receiver.kafka] Add Azure Resource Log Support
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/18210
* [otelcol.processor.resourcedetection] Add a `k8s.cluster.name` resource attribute for AKS and EKS.
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/26794
* [otelcol.processor.resourcedetection] Add detection of `host.ip` to system detector.
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/24450
* [otelcol.processor.resourcedetection] Add detection of `host.mac` to system detector.
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/29587
* [otelcol.processor.resourcedetection] Change type of `host.cpu.model.id` and `host.cpu.model.family` to string.
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/29025
* [otelcol.processor.resourcedetection] Add a `aws.ecs.task.id` attribute
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/8274
* [otelcol.exporter.otlp] Additional RPC debug metrics such as `rpc_client_duration_milliseconds`.
* [otelcol.receiver.otlp] Additional RPC debug metrics such as `rpc_server_duration_milliseconds`.

* [otelcol.connector.servicegraph] Additional debug metrics
<!-- TODO: Document those metrics -->
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/29917
* [otelcol.processor.filter] Additional debug metrics
<!-- TODO: Document those metrics -->
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30736
* Scope name for all generated Meter/Tracer funcs now includes full package name
<!-- TODO: Does this affect the Agent's metrics and logs? -->
https://github.com/open-telemetry/opentelemetry-collector/issues/9494
* [otelcol.connector.servicegraph] Measure latency in seconds instead of milliseconds
<!-- TODO: This is a breaking change for Static mode? -->
<!-- TODO: Is this a breaking change for Flow? -->
https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27488


v0.40.2 (2024-03-05)
--------------------

Expand Down
3 changes: 3 additions & 0 deletions docs/sources/flow/reference/components/otelcol.connector.servicegraph.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ Name | Type | Description | Default | Required
`dimensions` | `list(string)` | A list of dimensions to add with the default dimensions. | `[]` | no
`cache_loop` | `duration` | Configures how often to delete series which have not been updated. | `"1m"` | no
`store_expiration_loop` | `duration` | The time to expire old entries from the store periodically. | `"2s"` | no
`metrics_flush_interval` | `duration` | The interval at which metrics are flushed to downstream components. | `"0s"` | no

Service graphs work by inspecting traces and looking for spans with
parent-children relationship that represent a request.
Expand Down Expand Up @@ -113,6 +114,8 @@ Additional labels can be included using the `dimensions` configuration option:
* Firstly the resource attributes will be searched. If the attribute is not found,
the span attributes will be searched.

When `metrics_flush_interval` is set to `0s`, metrics will be flushed on every received batch of traces.

[Span Kind]: https://opentelemetry.io/docs/concepts/signals/traces/#span-kind

## Blocks
Expand Down
69 changes: 49 additions & 20 deletions docs/sources/flow/reference/components/otelcol.connector.spanmetrics.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,13 +67,15 @@ otelcol.connector.spanmetrics "LABEL" {

`otelcol.connector.spanmetrics` supports the following arguments:

| Name | Type | Description | Default | Required |
| ------------------------- | -------------- | --------------------------------------------------------------------- | -------------- | -------- |
| `dimensions_cache_size` | `number` | How many dimensions to cache. | `1000` | no |
| `aggregation_temporality` | `string` | Configures whether to reset the metrics after flushing. | `"CUMULATIVE"` | no |
| `metrics_flush_interval` | `duration` | How often to flush generated metrics. | `"15s"` | no |
| `namespace` | `string` | Metric namespace. | `""` | no |
| `exclude_dimensions` | `list(string)` | List of dimensions to be excluded from the default set of dimensions. | `false` | no |
| Name | Type | Description | Default | Required |
| --------------------------------- | -------------- | ------------------------------------------------------------------------------------------ | -------------- | -------- |
| `aggregation_temporality` | `string` | Configures whether to reset the metrics after flushing. | `"CUMULATIVE"` | no |
| `dimensions_cache_size` | `number` | How many dimensions to cache. | `1000` | no |
| `exclude_dimensions` | `list(string)` | List of dimensions to be excluded from the default set of dimensions. | `[]` | no |
| `metrics_flush_interval` | `duration` | How often to flush generated metrics. | `"15s"` | no |
| `namespace` | `string` | Metric namespace. | `""` | no |
| `resource_metrics_cache_size` | `number` | The size of the cache holding metrics for a service. | `1000` | no |
| `resource_metrics_key_attributes` | `list(string)` | Span resources with the same values for those resource attributes are aggregated together. | `[]` | no |

Adjusting `dimensions_cache_size` can improve the Agent process' memory usage.

Expand All @@ -84,19 +86,28 @@ The supported values for `aggregation_temporality` are:

If `namespace` is set, the generated metric name will be added a `namespace.` prefix.

`resource_metrics_cache_size` is mostly relevant for cumulative temporality. It helps avoid issues with increasing memory and with incorrect metric timestamp resets.

`resource_metrics_key_attributes` can be used to avoid situations where resource attributes may change across service restarts,
causing metric counters to break (and duplicate). A resource does not need to have all of the attributes.
The list must include enough attributes to properly identify unique resources or risk aggregating data from more than one service and span.
For example, `["service.name", "telemetry.sdk.language", "telemetry.sdk.name"]`.

## Blocks

The following blocks are supported inside the definition of
`otelcol.connector.spanmetrics`:

| Hierarchy | Block | Description | Required |
| ----------------------- | --------------- | ------------------------------------------------------- | -------- |
| dimension | [dimension][] | Dimensions to be added in addition to the default ones. | no |
| histogram | [histogram][] | Configures the histogram derived from spans durations. | yes |
| histogram > exponential | [exponential][] | Configuration for a histogram with exponential buckets. | no |
| histogram > explicit | [explicit][] | Configuration for a histogram with explicit buckets. | no |
| exemplars | [exemplars][] | Configures how to attach exemplars to histograms. | no |
| output | [output][] | Configures where to send telemetry data. | yes |
| Hierarchy | Block | Description | Required |
| ----------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
| dimension | [dimension][] | Dimensions to be added in addition to the default ones. | no |
| events | [events][] | Configures the events metric. | no |
| events > dimension | [dimension][] | Span event attributes to add as dimensions to the events metric, _on top of_ the default ones and the ones configured in the top-level `dimensions` block. | no |
| exemplars | [exemplars][] | Configures how to attach exemplars to histograms. | no |
| histogram | [histogram][] | Configures the histogram derived from spans durations. | yes |
| histogram > explicit | [explicit][] | Configuration for a histogram with explicit buckets. | no |
| histogram > exponential | [exponential][] | Configuration for a histogram with exponential buckets. | no |
| output | [output][] | Configures where to send telemetry data. | yes |

It is necessary to specify either a "[exponential][]" or an "[explicit][]" block:

Expand All @@ -108,6 +119,7 @@ It is necessary to specify either a "[exponential][]" or an "[explicit][]" block
[exponential]: #exponential-block
[explicit]: #explicit-block
[exemplars]: #exemplars-block
[events]: #events-block
[output]: #output-block

### dimension block
Expand All @@ -128,8 +140,8 @@ The following attributes are supported:

| Name | Type | Description | Default | Required |
| --------- | -------- | ------------------------------------------------ | ------- | -------- |
| `name` | `string` | Span attribute or resource attribute to look up. | | yes |
| `default` | `string` | Value to use if the attribute is missing. | null | no |
| `name` | `string` | Span attribute or resource attribute to look up. | | yes |

`otelcol.connector.spanmetrics` will look for the `name` attribute in the span's
collection of attributes. If it is not found, the resource attributes will be checked.
Expand All @@ -139,6 +151,20 @@ If the attribute is missing in both the span and resource attributes:
- If `default` is not set, the dimension will be omitted.
- If `default` is set, the dimension will be added and its value will be set to the value of `default`.

### events block

The `events` block configures the `events` metric, which tracks [span events][span-events].

The following attributes are supported:

| Name | Type | Description | Default | Required |
| --------- | ------ | -------------------------- | ------- | -------- |
| `enabled` | `bool` | Enables all events metric. | `false` | no |

`dimensions` is required if `enabled` is set to `true`.

[span-events]: https://opentelemetry.io/docs/concepts/signals/traces/#span-events

### histogram block

The `histogram` block configures the histogram derived from spans' durations.
Expand All @@ -147,8 +173,8 @@ The following attributes are supported:

| Name | Type | Description | Default | Required |
| --------- | -------- | ------------------------------- | ------- | -------- |
| `unit` | `string` | Configures the histogram units. | `"ms"` | no |
| `disable` | `bool` | Disable all histogram metrics. | `false` | no |
| `unit` | `string` | Configures the histogram units. | `"ms"` | no |

The supported values for `unit` are:

Expand Down Expand Up @@ -181,9 +207,12 @@ The `exemplars` block configures how to attach exemplars to histograms.

The following attributes are supported:

| Name | Type | Description | Default | Required |
| --------- | ------ | -------------------------------------------------- | ------- | -------- |
| `enabled` | `bool` | Configures whether to add exemplars to histograms. | `false` | no |
| Name | Type | Description | Default | Required |
| -------------------- | -------- | --------------------------------------------------------------------------- | ------- | -------- |
| `enabled` | `bool` | Configures whether to add exemplars to histograms. | `false` | no |
| `max_per_data_point` | `number` | Limits the number of exemplars that can be added to a unique dimension set. | `null` | no |

`max_per_data_point` can help with reducing memory consumption.

### output block

Expand Down
Loading

0 comments on commit 8f97b37

Please sign in to comment.