v0.40.0-rc.0

This is release v0.40.0-rc.0 of Grafana Agent.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

Breaking changes

• Prohibit the configuration of services within modules. (@wildum)

• For otelcol.exporter components, change the default value of disable_high_cardinality_metrics to true. (@ptodev)

• Rename component prometheus.exporter.agent to prometheus.exporter.self to clear up ambiguity. (@hainenber)

Deprecations

• Module components have been deprecated in favor of import and declare configuration blocks. These deprecated components will be removed in the next release. (@wildum)

Features

• Modules have been redesigned to split the import logic from the instantiation.
  You can now define custom components via the declare config block and import modules via import.git, import.http, import.string, import.file. (@wildum)

• A new discovery.process component for discovering Linux OS processes on the current host. (@korniltsev)

• A new pyroscope.java component for profiling Java processes using async-profiler. (@korniltsev)

• A new otelcol.processor.resourcedetection component which inserts resource attributes
  to OTLP telemetry based on the host on which Grafana Agent is running. (@ptodev)

• Expose track_timestamps_staleness on Prometheus scraping, to fix the issue where container metrics live for 5 minutes after the container disappears. (@ptodev)

• Introduce the remotecfg service that enables loading configuration from a
  remote endpoint. (@tpaschalis)

• Add otelcol.connector.host_info component to gather usage metrics for cloud users. (@rlankfo, @jcreixell)

For a full list of changes, please refer to the CHANGELOG!

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode

v0.39.2

This is release v0.39.2 of Grafana Agent.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

Bugfixes

• Fix error introduced in v0.39.0 preventing remote write to Amazon Managed Prometheus. (@captncraig)

• An error will be returned in the converter from Static to Flow when scrape_integration is set
  to true but no remote_write is defined. (@erikbaranowski)

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode

v0.39.1

This is release v0.39.1 of Grafana Agent.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

Security fixes

• Fixes following vulnerabilities (@hainenber)
  • GO-2023-2409
  • GO-2023-2412
  • CVE-2023-49568

Bugfixes

• Fix issue where installing the Windows Agent Flow installer would hang then crash. (@mattdurham)

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode

v0.39.0

This is release v0.39.0 of Grafana Agent.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

Breaking changes

• otelcol.receiver.prometheus will drop all otel_scope_info metrics when converting them to OTLP. (@wildum)

  • If the otel_scope_info metric has labels otel_scope_name and otel_scope_version,
    their values will be used to set OTLP Instrumentation Scope name and version respectively.
  • Labels of otel_scope_info metrics other than otel_scope_name and otel_scope_version
    are added as scope attributes with the matching name and version.

• The target block in prometheus.exporter.blackbox requires a mandatory name
  argument instead of a block label. (@hainenber)

• In the azure exporter, dimension options will no longer be validated by the Azure API. (@kgeckhart)

  • This change will not break any existing configurations and you can opt in to validation via the validate_dimensions configuration option.
  • Before this change, pulling metrics for azure resources with variable dimensions required one configuration per metric + dimension combination to avoid an error.
  • After this change, you can include all metrics and dimensions in a single configuration and the Azure APIs will only return dimensions which are valid for the various metrics.

Features

• A new discovery.ovhcloud component for discovering scrape targets on OVHcloud. (@ptodev)
• Allow specifying additional containers to run. (@juangom)

Enhancements

• Flow Windows service: Support environment variables. (@jkroepke)

• Allow disabling collection of root Cgroup stats in
  prometheus.exporter.cadvisor (flow mode) and the cadvisor integration
  (static mode). (@hainenber)

• Grafana Agent on Windows now automatically restarts on failure. (@hainenber)

• Added metrics, alerts and dashboard visualisations to help diagnose issues
  with unhealthy components and components that take too long to evaluate. (@thampiotr)

• The http config block may now reference exports from any component.
  Previously, only remote.* and local.* components could be referenced
  without a circular dependency. (@rfratto)

• Add support for Basic Auth-secured connection with Elasticsearch cluster using prometheus.exporter.elasticsearch. (@hainenber)

• Add a resource_to_telemetry_conversion argument to otelcol.exporter.prometheus
  for converting resource attributes to Prometheus labels. (@hainenber)

• pyroscope.ebpf support python on arm64 platforms. (@korniltsev)

• otelcol.receiver.prometheus does not drop histograms without buckets anymore. (@wildum)

• Added exemplars support to otelcol.receiver.prometheus. (@wildum)

• mimir.rules.kubernetes may now retry its startup on failure. (@hainenber)

• Added links between compatible components in the documentation to make it
  easier to discover them. (@thampiotr)

• Allow defining HTTPClientConfig for discovery.ec2. (@cmbrad)

• The remote.http component can optionally define a request body. (@tpaschalis)

• Added support for loki.write to flush WAL on agent shutdown. (@thepalbi)

• Add support for integrations-next static to flow config conversion. (@erikbaranowski)

• Add support for passing extra arguments to the static converter such as -config.expand-env. (@erikbaranowski)

• Added 'country' mmdb-type to log pipeline-stage geoip. (@superstes)

• Azure exporter enhancements for flow and static mode, (@kgeckhart)

  • Allows for pulling metrics at the Azure subscription level instead of resource by resource
  • Disable dimension validation by default to reduce the number of exporter instances needed for full dimension coverage

• Add max_cache_size to prometheus.relabel to allow configurability instead of hard coded 100,000. (@mattdurham)

• Add support for http_sd_config within a scrape_config for prometheus to flow config conversion. (@erikbaranowski)

• discovery.lightsail now supports additional parameters for configuring HTTP client settings. (@ptodev)

• Add sample_age_limit to remote_write config to drop samples older than a specified duration. (@marctc)

For a full list of changes, please refer to the CHANGELOG!

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode

v0.39.0-rc.0

This is release v0.39.0-rc.0 of Grafana Agent.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

Breaking changes

• otelcol.receiver.prometheus will drop all otel_scope_info metrics when converting them to OTLP. (@wildum)

  • If the otel_scope_info metric has labels otel_scope_name and otel_scope_version,
    their values will be used to set OTLP Instrumentation Scope name and version respectively.
  • Labels of otel_scope_info metrics other than otel_scope_name and otel_scope_version
    are added as scope attributes with the matching name and version.

• The target block in prometheus.exporter.blackbox requires a mandatory name
  argument instead of a block label. (@hainenber)

• In the azure exporter, dimension options will no longer be validated by the Azure API. (@kgeckhart)

  • This change will not break any existing configurations and you can opt in to validation via the validate_dimensions configuration option.
  • Before this change, pulling metrics for azure resources with variable dimensions required one configuration per metric + dimension combination to avoid an error.
  • After this change, you can include all metrics and dimensions in a single configuration and the Azure APIs will only return dimensions which are valid for the various metrics.

Features

• A new discovery.ovhcloud component for discovering scrape targets on OVHcloud. (@ptodev)

Enhancements

• Flow Windows service: Support environment variables. (@jkroepke)

• Allow disabling collection of root Cgroup stats in
  prometheus.exporter.cadvisor (flow mode) and the cadvisor integration
  (static mode). (@hainenber)

• Grafana Agent on Windows now automatically restarts on failure. (@hainenber)

• Added metrics, alerts and dashboard visualisations to help diagnose issues
  with unhealthy components and components that take too long to evaluate. (@thampiotr)

• The http config block may now reference exports from any component.
  Previously, only remote.* and local.* components could be referenced
  without a circular dependency. (@rfratto)

• Add support for Basic Auth-secured connection with Elasticsearch cluster using prometheus.exporter.elasticsearch. (@hainenber)

• Add a resource_to_telemetry_conversion argument to otelcol.exporter.prometheus
  for converting resource attributes to Prometheus labels. (@hainenber)

• pyroscope.ebpf support python on arm64 platforms. (@korniltsev)

• otelcol.receiver.prometheus does not drop histograms without buckets anymore. (@wildum)

• Added exemplars support to otelcol.receiver.prometheus. (@wildum)

• mimir.rules.kubernetes may now retry its startup on failure. (@hainenber)

• Added links between compatible components in the documentation to make it
  easier to discover them. (@thampiotr)

• Allow defining HTTPClientConfig for discovery.ec2. (@cmbrad)

• The remote.http component can optionally define a request body. (@tpaschalis)

• Added support for loki.write to flush WAL on agent shutdown. (@thepalbi)

• Add support for integrations-next static to flow config conversion. (@erikbaranowski)

• Add support for passing extra arguments to the static converter such as -config.expand-env. (@erikbaranowski)

• Added 'country' mmdb-type to log pipeline-stage geoip. (@superstes)

• Azure exporter enhancements for flow and static mode, (@kgeckhart)

  • Allows for pulling metrics at the Azure subscription level instead of resource by resource
  • Disable dimension validation by default to reduce the number of exporter instances needed for full dimension coverage

• Add max_cache_size to prometheus.relabel to allow configurability instead of hard coded 100,000. (@mattdurham)

• Add support for http_sd_config within a scrape_config for prometheus to flow config conversion. (@erikbaranowski)

• discovery.lightsail now supports additional parameters for configuring HTTP client settings. (@ptodev)

• Add sample_age_limit to remote_write config to drop samples older than a specified duration. (@marctc)

For a full list of changes, please refer to the CHANGELOG!

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode

v0.38.1

This is release v0.38.1 of Grafana Agent.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

Security fixes

• Fix CVE-2023-47108 by updating otelgrpc from v0.45.0 to v0.46.0. (@hainenber)

Features

• Agent Management: Introduce support for templated configuration. (@jcreixell)

Bugfixes

• Permit X-Faro-Session-ID header in CORS requests for the faro.receiver
  component (flow mode) and the app_agent_receiver integration (static mode).
  (@cedricziel)

• Fix issue with windows_exporter defaults not being set correctly. (@mattdurham)

• Fix agent crash when process null OTel's fan out consumers. (@hainenber)

• Fix issue in prometheus.operator.* where targets would be dropped if two crds share a common prefix in their names. (@Paul424, @captncraig)

• Fix issue where convert command would generate incorrect Flow Mode config
  when provided promtail configuration that uses docker_sd_configs (@thampiotr)

• Fix converter issue with loki.relabel and max_cache_size being set to 0 instead of default (10_000). (@mattdurham)

Other changes

• Add Agent Deploy Mode to usage report. (@captncraig)

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode

v0.38.0

This is release v0.38.0 of Grafana Agent.

v0.38.0 is a huge release, with many new features, improvements, and bugfixes. This page only covers a very small portion of those changes. For a full list of changes, refer to our CHANGELOG.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

Breaking changes

• The deprecated otelcol.exporter.jaeger component has been removed as scheduled in v0.36.
• Some metrics for the mysqld exporter integration ahve been removed or renamed.

Features

• New flow mode components:
  • otelcol.processor.filter filters OTLP telemetry data using OpenTelemetry Transformation Language (OTTL).
  • otelcol.receiver.vcenter receives metrics telemetry data from vCenter.
• Changes to River:
  • River now supports "raw strings" to avoid needing to escape special characters.
  • River now permits using [] to access non-existent keys in an object, rather than generating an error.

Enhancements

• Improvements to log collection in flow mode:
  • Improve detection of rolled log files in loki.source.kubernetes and loki.source.podlogs.
  • Clustering is now supported in loki.source.kubernetes and loki.source.podlogs.
  • Added support for replaying unsent data for loki.write when WAL is enabled.
• Improvements to the flow controller:
  • Improved resilience of graph evaluation in presence of slow components.

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode

v0.38.0-rc.1

This is release v0.38.0-rc.1 of Grafana Agent.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

The following changes are included on top of v0.38.0-rc.0:

Enhancements

• Allow agent to start with module.git config if cached before. (@hainenber)

Bugfixes

• Fix default configuration file grafana-agent-flow.river used in downstream packages. (@bricewge)

• Fix converter output for prometheus.exporter.windows to not unnecessarily add empty blocks. (@erikbaranowski)

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode

v0.38.0-rc.0

This is release v0.38.0-rc.0 of Grafana Agent.

v0.38.0-rc.0 is a huge release, with many new features, improvements, and bugfixes. This page only covers a very small portion of those changes. For a full list of changes, refer to our CHANGELOG.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes

Breaking changes

• The deprecated otelcol.exporter.jaeger component has been removed as scheduled in v0.36.
• Some metrics for the mysqld exporter integration ahve been removed or renamed.

Features

• New flow mode components:
  • otelcol.processor.filter filters OTLP telemetry data using OpenTelemetry Transformation Language (OTTL).
  • otelcol.receiver.vcenter receives metrics telemetry data from vCenter.
• Changes to River:
  • River now supports "raw strings" to avoid needing to escape special characters.
  • River now permits using [] to access non-existent keys in an object, rather than generating an error.

Enhancements

• Improvements to log collection in flow mode:
  • Improve detection of rolled log files in loki.source.kubernetes and loki.source.podlogs.
  • Clustering is now supported in loki.source.kubernetes and loki.source.podlogs.
  • Added support for replaying unsent data for loki.write when WAL is enabled.
• Improvements to the flow controller:
  • Improved resilience of graph evaluation in presence of slow components.

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode

v0.37.4

This is release v0.37.4 of Grafana Agent.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

This patch release has two small changes

• Added an add_metric_suffixes option to otelcol.exporter.prometheus in flow mode, which configures whether to add type and unit suffixes to metrics names. (@mar4uk)

• Fixes a bug where reloading the configuration of a loki.write component lead to a panic. (@tpaschalis)

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode