v7.60.1

Compare Source

v7.60.0

Compare Source

======

.. _Release Notes_7.60.0_Prelude:

Prelude

Release on: 2024-12-16

• Please refer to the 7.60.0 tag on integrations-core <https://github.com/DataDog/integrations-core/blob/master/AGENT_CHANGELOG.md#datadog-agent-version-7600>_ for the list of changes on the Core Checks

.. _Release Notes_7.60.0_Upgrade Notes:

Upgrade Notes

• • Parameter peer_tags_aggregation (a.k.a. environment variable DD_APM_PEER_TAGS_AGGREGATION) is now enabled by default. This means that aggregation of peer related tags (e.g., peer.service, db.instance, etc.) now happens in the Agent, which enables statistics for Inferred Entities. If you want to disable this feature, set peer_tags_aggregation to false in your Agent configuration.

  • Parameter compute_stats_by_span_kind (a.k.a. environment variable DD_APM_COMPUTE_STATS_BY_SPAN_KIND) is now enabled by default. This means spans with an eligible span.kind will have stats computed. If disabled, only top-level and measured spans will have stats computed. If you want to disable this feature, set compute_stats_by_span_kind to false in your Agent configuration.

    Note: When using peer_tags_aggregation and compute_stats_by_span_kind, a high cardinality of peer tags or APM resources can contribute to higher CPU and memory consumption. If enabling both causes the Agent to consume too many resources, try disabling compute_stats_by_span_kind first.
    It is recommended that you update your tracing libraries according to the instructions here <https://docs.datadoghq.com/tracing/guide/inferred-service-opt-in/?tab=java#apm-tracing-library-configuration>_ and set DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLED (or dd.trace.remove.integration-service-names.enabled) to true.

• Upgraded JMXFetch to 0.49.5 <https://github.com/DataDog/jmxfetch/releases/0.49.5>_ which adds support for UnloadedClassCount metric
  and IBM J9 gc metrics. See 0.49.5 <https://github.com/DataDog/jmxfetch/releases/tag/0.49.5>_ for more details.

.. _Release Notes_7.60.0_New Features:

New Features

• Inferred Service dependencies <https://docs.datadoghq.com/tracing/guide/inferred-service-opt-in/>_ are now Generally Available (exiting Beta) and enabled by default. Inferred Services of all kinds now have trace metrics and are available in dependency maps. apm_config.peer_tags_aggregation and apm_config.compute_stats_by_span_kind both now default to true unless explicitly set to false.

• Add check_tag_cardinality parameter config check.

  By default check_tag_cardinality is not set which doesn't change the behavior of the checks.
  Once it is set in pod annotaions, it overrides the cardinality value provided in the base agent configuration.
  Example of usage:

  ad.datadoghq.com/redis.checks: |
    {
      "redisdb": {
        "check_tag_cardinality": "high", 
        "instances": [
          {
            "host": "%%host%%",
            "port": "6379"
          }
        ]
      }
    }

• Added a new feature flag enable_receive_resource_spans_v2 in DD_APM_FEATURES that gates a refactored implementation of ReceiveResourceSpans for OTLP.

.. _Release Notes_7.60.0_Enhancement Notes:

Enhancement Notes

• Added information about where the Agent sourced BTF data for eBPF to the Agent flare. When applicable, this will appear in system-probe/ebpf_btf_loader.log.

• The Agent flare now returns NAT debug information from conntrack in the system-probe directory.

• The flare subcommand includes a --provider-timeout option to set a timeout for each file collection (default is 10s), useful for unblocking slow flare creation.

• This change reduces the number of DNS queries made by Network Traffic
  based paths in Network Path.
  A cache of reverse DNS lookups is used to reduce the number of DNS
  queries. Additionally, reverse DNS lookups are now performed only
  for private IPs and not for public IPs.

• Agent flare now includes system-probe telemetry data via system-probe/system_probe_telemetry.log.

• The MSI installer uses 7zr.exe to decompress the embedded Python.

• On Windows, the endpoint /windows_crash_detection/check has been modified to report crashes in
  an asynchronous manner, to allow processing of large crash dumps without blocking or timing out.
  The first check will return a busy status and continue to do so until the processing is completed.

.. _Release Notes_7.60.0_Deprecation Notes:

Deprecation Notes

• Prebuilt eBPF for the network tracer system-probe module has been
  deprecated in favor of CO-RE and runtime compilation variants on Linux
  kernel versions 6+ and RHEL kernel versions 5.14+. To continue to use
  the prebuilt eBPF network tracer, set
  system_probe_config.allow_prebuilt_fallback in the
  system-probe config file, or set the environment variable
  DD_ALLOW_PREBUILT_FALLBACK, to true on these platforms.

• The feature flag service_monitoring_config.enable_http_stats_by_status_code was deprecated and removed.
  No impact on USM's behavior.

.. _Release Notes_7.60.0_Bug Fixes:

Bug Fixes

• Fixes an issue added in 7.50 that causes the Windows event log tailer to drop
  events if it cannot open their publisher metadata.

• Fix a bug in the config parser that broke ignored_ip_addresses from working in NDM Autodiscovery.

• Fixes host tags with a configurable duration so the metric's context hash doesn't change, preventing the aggregator from mistaking it as a new metric.

• Fix could not parse voltage fields error in Nvidia Jetson integration when tegrastats output contains mW units.

• Fix building of Python extension containing native code.

• [oracle] Fix broken activity sampling with an external Oracle client.

• Fix nil pointer error on Oracle DBM query when the check's connection is lost before SELECT statement executes.

• Fix a regression that caused the Agent to not be able to run if its
  capabilities had been modified with the setcap command.

• Fix bug wherein single line truncated logs ended with whitespace characters were not being tagged as truncated.
  Fix issue with the truncation message occasionally causing subsequent logs to think they were truncated when they were not (single line logs only).

.. _Release Notes_7.59.1:

v7.59.1

Compare Source

======

.. _Release Notes_7.59.1_Prelude:

Prelude

Release on: 2024-12-02

.. _Release Notes_7.59.1_Enhancement Notes:

Enhancement Notes

• Setting up a temporary directory for JMXFetch <https://github.com/DataDog/jmxfetch>_ to use
  when it runs. Using the same one the Agent uses when running as this guarantees a directory where
  JMXFetch can write to. This helps when JMXFetch sends metrics over
  Unix Domain Socket <https://docs.datadoghq.com/developers/dogstatsd/unix_socket/?tab=host> as it needs
  access to a temp directory which it can write to.

.. _Release Notes_7.59.0:

v7.59.0

Compare Source

======

.. _Release Notes_7.59.0_Prelude:

Prelude

Release on: 2024-11-07

• Please refer to the 7.59.0 tag on integrations-core <https://github.com/DataDog/integrations-core/blob/master/AGENT_CHANGELOG.md#datadog-agent-version-7590>_ for the list of changes on the Core Checks

.. _Release Notes_7.59.0_Upgrade Notes:

Upgrade Notes

• Removed the deprecated config option otlp_config.debug.loglevel in favor of otlp_config.debug.verbosity:
  • loglevel: debug maps to verbosity: detailed
  • loglevel: info maps to verbosity: normal
  • loglevel: warn/error maps to verbosity: basic
  • loglevel: disabled maps to verbosity: none

.. _Release Notes_7.59.0_New Features:

New Features

• Add ability to run process/container collection on the core Agent (Linux only). This is controlled
  by the process_config.run_in_core_agent.enabled option in datadog.yaml.

• DBM: Add configuration options to SQL obfuscator to customize the obfuscation of SQL statements:

  • KeepJSONPath - option to control whether JSON paths following JSON operators in SQL statements should be obfuscated. This option is only valid when ObfuscationMode is obfuscate_and_normalize.

• APM: Add new 'sqllexer' feature flag for the Trace Agent, which enables
  the sqllexer imprementation of the SQL Obfuscator.

• Introduce new Kubernetes tag gpu_vendor for the GPU resource requested by a container.

.. _Release Notes_7.59.0_Enhancement Notes:

Enhancement Notes

• Added additional Agent telemetry metrics for the log tailer code flow: logs.bytes_sent, logs.encoded_bytes_sent, and logs.bytes_missed

• Datadog may collect environmental, performance, and feature usage
  information about the Datadog Agent. This may include diagnostic
  logs and crash dumps of the Datadog Agent with obfuscated stack
  traces to support and further improve the Datadog Agent.

  More details could be found in the
  docs <https://docs.datadoghq.com/data_security/agent/#telemetry-collection>_

• APM: Updates peer tags for peer.db.system.

• Agents are now built with Go 1.22.7.

• Agents are now built with Go 1.22.8.

• Agents are now built with Go 1.22.8.

• While using the AWS Lambda Extension, when a Lambda Function is invoked by
  a [properly instrumented][1] Step Function, the Lambda Function will create
  its Trace and Parent IDs deterministically based on the Step Function's
  execution context.
  [1]: https://docs.datadoghq.com/serverless/step_functions/installation/?tab=custom "Install Serverless Monitoring for AWS Step Functions"

• Updates default .NET library used for auto-instrumentation from v2 to v3

• The system-probe selinux policy is now installed on Oracle Linux

• Increases the default input channel, processing channel, and context store sizes
  for network traffic paths.

• Adds support for file log collection from Podman rootless containers when logs_config.use_podman_logs is set to true and podman_db_path is configured.

• Allow Python integrations to emit Agent telemetry data.

.. _Release Notes_7.59.0_Security Notes:

Security Notes

• Update OpenSSL to 3.3.2 (on Linux & macOS) in order to mitigate CVE-2024-6119.

.. _Release Notes_7.59.0_Bug Fixes:

Bug Fixes

• Fixes the default configuration template to include the Cloud Security Management configuration options.

• Fixing a bug introduced in 7.55 where in some specific scenarios, checks associated with a deleted container or POD would keep running until the Agent is restarted.

• Fix the forwarder health check so that it reports unhealthy when the API key is invalid.

• Fix the removal of 'non-core' integrations during Agent upgrades.

• Fix Process Agent argument scrubbing to allow scrubbing of quoted arguments.

• Fix Orchestrator argument scrubbing to allow scrubbing of quoted arguments.

• Fixes an issue where TCP traceroute latency was not being calculated correctly.

• Fixes the telemetry type for Oracle metrics.

• APM: Fix obfuscation of SQL queries containing non-numeric prepared statement variables.

.. _Release Notes_7.59.0_Other Notes:

Other Notes

• Adds Postgres integration metrics to cross-org telemetry whitelist.

• The Agent is now built with a custom toolchain that targets our minimally
  supported glibc version (2.17 on x86_64 and 2.23 on aarch64)

• On Windows, the TCP socket transport mechanism for system probe
  communications has been replaced with a named pipe.
  This deprecates the system_probe_config.sysprobe_socket configuration
  entry for Windows.
  The new fixed named pipe path is \.\pipe\dd_system_probe.

.. _Release Notes_7.58.2:

v7.58.2

Compare Source

======

.. _Release Notes_7.58.2_Prelude:

Prelude

Release on: 2024-11-04

.. _Release Notes_7.58.2_Bug Fixes:

Bug Fixes

• Use of cloud-provided hostname as default when running the Agent
  in AKS introduced in 7.56.0 is reverted due to cases where the
  hostname returned is non-unique. This feature will be fixed and
  added again in a future release.

.. _Release Notes_7.58.1:

v7.58.1

Compare Source

======

.. _Release Notes_7.58.1_Prelude:

Prelude

Release on: 2024-10-24

.. _Release Notes_7.58.1_Enhancement Notes:

Enhancement Notes

• Removes a log statement which was causing a lot of noise in the Network Path logs.

.. _Release Notes_7.58.1_Bug Fixes:

Bug Fixes

• [CWS] Fixes an issue where the cws-instrumentation trace command could panic
  before launching the traced executable when running on AWS Fargate.

• [CWS] Fixes an issue where ECS Fargate tags would not be resolved correctly
  on CWS events.

• Fixes an error in system-probe triggered by packet capture in environments with multiple VLANs.

• Fix USM's GO-TLS support for Golang 1.23

.. _Release Notes_7.58.0:

v7.58.0

Compare Source

======

.. _Release Notes_7.58.0_Prelude:

Prelude

Release on: 2024-10-21

• Please refer to the 7.58.0 tag on integrations-core <https://github.com/DataDog/integrations-core/blob/master/AGENT_CHANGELOG.md#datadog-agent-version-7580>_ for the list of changes on the Core Checks

.. _Release Notes_7.58.0_Upgrade Notes:

Upgrade Notes

• Changes behavior of the timeout for Network Path. Previously, the timeout
  signified the total time to wait for a full traceroute to complete. Now,
  the timeout signifies the time to wait for each hop in the traceroute.
  Additionally, the default timeout has been changed to 1000ms.

.. _Release Notes_7.58.0_New Features:

New Features

• Added capability to tag any Kubernetes resource based on labels and annotations.
  This feature can be configured with kubernetes_resources_annotations_as_tags and kubernetes_resources_labels_as_tags.
  These feature configurations are associate group resources with annotations-to-tags (or labels-to-tags) map
  For example, pods can be associated with an annotations-to-tags map to configure annotations as tags for pods.
  Example:
  {pods: {annotationKey1: tag1, annotationKey2: tag2}}

• The Kubernetes State Metrics (KSM) check can now be configured to collect
  pods from the Kubelet in node agents instead of collecting them from the API
  Server in the Cluster Agent or the Cluster check runners. This is useful in
  clusters with a large number of pods where emitting pod metrics from a
  single check instance can cause performance issues due to the large number
  of metrics emitted.

• NPM - adds UDP "Packets Sent" and "Packets Received" to the network telemetry in Linux.

• [oracle] Add the active_session_history configuration parameter to optionally ingest Oracle active session history samples instead of query sampling.

• Added config option logs_config.tag_truncated_logs. When
  enabled, file logs will come with a tag truncated:true if
  they were truncated by the Agent.

.. _Release Notes_7.58.0_Enhancement Notes:

Enhancement Notes

• [DBM] Bump go-sqllexer to 0.0.14 to skip collecting CTE tables as SQL metadata.

• Agents are now built with Go 1.22.7.

• Add the ability to tag cisco-sdwan device and interface metrics with user-defined tags.

• Add support for setting a custom log source from resource attribute or log attribute datadog.log.source.

• The default UDP port for traceroute (port 33434) is now used for Network Traffic based paths, instead of the port detected by NPM.

• [oracle] Add oracle_client_lib_dir config parameter.

• [oracle] Increase tablespace check interval from 1 to 10 minutes.

• [oracle] Don't try to fetch execution plans where plan_hash_value is 0

• The OTLP ingest endpoint now maps the new OTel semantic convention deployment.environment.name to env

• Prevents the use of the process_config.run_in_core_agent.enabled configuration option in unsupported environments.

• APM: Trace payloads are now compressed with zstd by default.

.. _Release Notes_7.58.0_Security Notes:

Security Notes

• Bump embedded Python version to 3.12.6 to address CVE-2024-4030 and CVE-2024-4741.

• Update cURL to 8.9.1.

• Update OpenSSL to 3.3.2 (on Linux & macOS) in order to mitigate CVE-2024-6119.

.. _Release Notes_7.58.0_Bug Fixes:

Bug Fixes

• Adds missing support for the logs config key to work with AD annotations V2.

• Fix agent jmx [command] subcommands for container environments with annotations-based configs.

• Fixed issue with openSUSE 15 RC 6 where the eBPF tracer wouldn't start due to a failed validation of the tcp_sendpage probe.

• Fixed a rare issue where short-lived containers could cause
  logs to be sent with the wrong container ID.

• Fix Windows Process Agent argument stripping to account for spaces in the executable path.

• Fixes issue with the kubelet corecheck where kubernetes.kubelet.volume.* metrics
  were not properly being reported if any matching namespace exclusion filter was present.

• OOM Kill Check now reports the cgroup name of the victim process rather than the triggering process.

• The process agent will no longer exit prematurely when language detection is enabled or
  when there is a misconfiguration stemming from process_config.run_in_core_agent.enabled's
  default enablement in Kubernetes.

• Change the datadog-security-agent Windows service display name from Datadog Security Service to
  Datadog Security Agent for consistency with other Agent services.

• Fix a bug preventing SNMP V3 reconnection.

.. _Release Notes_7.58.0_Other Notes:

Other Notes

• Add metric origins for the Kubeflow integration.

• Add functional tests to Oracle using a Docker service to host the database instance.

• Adds Agent telemetry for Oracle collector.

.. _Release Notes_7.57.2:

v7.57.2

Compare Source

======

.. _Release Notes_7.57.2_Prelude:

Prelude

Release on: 2024-09-24

.. _Release Notes_7.57.2_Enhancement Notes:

Enhancement Notes

• Agents are now built with Go 1.22.7.

.. _Release Notes_7.57.2_Bug Fixes:

Bug Fixes

• Fix OOM error with cluster agent auto instrumentation by increasing default memory request from 20Mi to 100Mi.

• Fixes a panic caused by running the Agent on readonly filesystems. The
  Agent returns integration launchers and handles memory gracefully.

.. _Release Notes_7.57.1: