From 0413c5079a0a33f3da813d14a505f36d76f9ee1a Mon Sep 17 00:00:00 2001 From: Tim Pansino Date: Thu, 11 Jan 2024 14:56:43 -0800 Subject: [PATCH 1/5] Add new config settings for python agent logging context data --- .../python-agent-configuration.mdx | 169 +++++++++++++++++- 1 file changed, 168 insertions(+), 1 deletion(-) diff --git a/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx b/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx index 10fb45be2c1..f2c819d080a 100644 --- a/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx +++ b/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx @@ -3895,7 +3895,7 @@ For some tips on configuring logs for the Python agent, see [Configure Python lo If `true`, enables log decoration and the collection of log events and logging metrics if these sub-feature configurations are also enabled. If `false`, no logging instrumentation features are enabled. - @@ -3950,6 +3950,173 @@ For some tips on configuring logs for the Python agent, see [Configure Python lo + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Type + + Boolean +
+ Default + + `false` +
+ [Set in](#options) + + Config file, environment variable +
+ [Environ variable](#environment-variables) + + `NEW_RELIC_APPLICATION_LOGGING_FORWARDING_CONTEXT_DATA_ENABLED` +
+ + + If `true`, the agent will capture available context data (extras, dictionary message attributes, attributes provided by logging frameworks) and add its contents as attributes on the logs forwarded to New Relic. You can control this behavior through the settings under the `application_logging.forwarding.context_data` stanza. + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Type + + List of strings +
+ Default + + (none) +
+ [Set in](#options) + + Config file, environment variable +
+ [Environ variable](#environment-variables) + + `NEW_RELIC_APPLICATION_LOGGING_FORWARDING_CONTEXT_DATA_INCLUDE` +
+ + If attributes are enabled for context_data, all attribute keys found in this list will be sent to us in transaction traces. For more information, see the [agent attribute rules](/docs/apm/other-features/attributes/agent-attributes). + + + When adding context attributes, the agent prefixes the keys with `context.` for attributes from the logging framework context, and `message.` for attributes from a dictionary message context. + + These prefixes are NOT included when matching against include/exclude filtering rules. + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Type + + List of Strings +
+ Default + + (none) +
+ [Set in](#options) + + Config file, environment variable +
+ [Environ variable](#environment-variables) + + `NEW_RELIC_APPLICATION_LOGGING_FORWARDING_CONTEXT_DATA_EXCLUDE` +
+ + All attribute keys found in this list will not be sent in context_data. For more information, see the [agent attribute rules](/docs/apm/other-features/attributes/agent-attributes). + + + When adding context attributes, the agent prefixes the keys with `context.` for attributes from the logging framework context, and `message.` for attributes from a dictionary message context. + + These prefixes are NOT included when matching against include/exclude filtering rules. + + +
+ Date: Thu, 11 Jan 2024 15:19:30 -0800 Subject: [PATCH 2/5] Update record_log_event API for python-agent --- .../recordlogevent-python-agent-api.mdx | 30 +++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/src/content/docs/apm/agents/python-agent/python-agent-api/recordlogevent-python-agent-api.mdx b/src/content/docs/apm/agents/python-agent/python-agent-api/recordlogevent-python-agent-api.mdx index c5510a7320a..6404272287a 100644 --- a/src/content/docs/apm/agents/python-agent/python-agent-api/recordlogevent-python-agent-api.mdx +++ b/src/content/docs/apm/agents/python-agent/python-agent-api/recordlogevent-python-agent-api.mdx @@ -46,11 +46,13 @@ This records a [log event](/docs/logs/logs-context/configure-logs-context-python `message` - _string_ + _string_, _dictionary_ - Required. The `message` defines the log message and must be a string. + Required. The `message` that defines the log message. For dictionary values, the key `message` will be extracted if available, and any other items will be considered context data attributes under the prefix `message.`. + + To report these attributes, [enable context data forwarding](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.enabled) and optionally configure [include](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.include) and [exclude](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.exclude) rules. @@ -78,6 +80,20 @@ This records a [log event](/docs/logs/logs-context/configure-logs-context-python + + + `attributes` + + _dictionary_ + + + + Optional. Items included in this dictionary will be considered context data attributes under the prefix `context.`. + + To report these attributes, [enable context data forwarding](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.enabled) and optionally configure [include](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.include) and [exclude](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.exclude) rules. + + + `application` @@ -131,3 +147,13 @@ def fetch(): newrelic.agent.record_log_event('Fetching data.') # do some type of work in this transaction... ``` + +### Record log event with context data attributes [#context-data-example] + +An example of recording a log event using message attributes and context attributes + +```py +def fetch(product_id): + newrelic.agent.record_log_event({"message": "Fetching data", "product_id": product_id}, attributes={"thread_id": threading.get_ident()}) + # do some type of work in this transaction... +``` From 0a177d5385b31c48127e2c6736ee9cbd107c63b0 Mon Sep 17 00:00:00 2001 From: Tim Pansino Date: Thu, 11 Jan 2024 16:46:13 -0800 Subject: [PATCH 3/5] Add python agent v9.5.0 release notes --- .../python-agent-90500.mdx | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-90500.mdx diff --git a/src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-90500.mdx b/src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-90500.mdx new file mode 100644 index 00000000000..eee3dd14818 --- /dev/null +++ b/src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-90500.mdx @@ -0,0 +1,40 @@ +--- +subject: Python agent +releaseDate: '2024-01-11' +version: 9.5.0 +downloadLink: 'https://pypi.python.org/pypi/newrelic' +features: ['Add context data attributes to logs','Add support for dictionary type log messages','Obfuscate license keys and user API keys in logs','Add attribute support to `record_log_event` API'] +bugs: [] +security: [] +--- + +## Notes + +This release of the Python agent adds context data attributes to logs, support for dictionary type log messages, adds attribute support to `record_log_event` API, and obfuscation of logged license keys and user API keys. + +Install the agent using `easy_install/pip/distribute` via the [Python Package Index](https://pypi.python.org/pypi/newrelic) or download it directly from the [New Relic download site](https://download.newrelic.com/python_agent/release). + + +## New features + +* **Add context data attributes to logs** +Context data available from logging frameworks can now be recorded as attributes on logs. These attributes are prefixed with `context.`, which is not considered when filtering attributes from logs. + +Context data may include extras, dictionary message attributes, and attributes provided by logging frameworks. + +To report these attributes, [enable context data forwarding](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.enabled) and optionally configure [include](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.include) and [exclude](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.exclude) rules to filter the allowed attributes. + +* **Add support for dictionary type log messages** +Dictionary type log messages are now supported for logging frameworks which support dictionary type log messages (including the builtin `logging` module and `structlog`). The key `message` will be extracted if available and treated as the log message, and any other items will be considered context data attributes and prefixed with `message.`. These attributes are subject to the same filtering rules as other context data. + +* **Add attribute support to `record_log_event` API** +The `record_log_event` API now allows passing context data in using the optional `attributes` keyword argument, and allows the message argument to be a dictionary as described above. + +* **Obfuscate license keys and user API keys in logs** +The agent now obfuscates all instances of license keys or user API keys in agent logs and audit logs. + +## Support statement + +We recommend updating to the latest agent version as soon as it's available. If you can't upgrade to the latest version, update your agents to a version no more than 90 days old. Read [more](/docs/new-relic-solutions/new-relic-one/install-configure/update-new-relic-agent/) about keeping agents up to date. + +See the New Relic Python agent [EOL policy](/docs/apm/agents/python-agent/getting-started/python-agent-eol-policy/) for information about agent releases and support dates. From fe1886b1de8871a7535af88f095341f9e747730c Mon Sep 17 00:00:00 2001 From: Tim Pansino Date: Fri, 12 Jan 2024 12:50:47 -0800 Subject: [PATCH 4/5] Add missing HTML tags --- .../python-agent/configuration/python-agent-configuration.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx b/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx index f2c819d080a..4ced130637e 100644 --- a/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx +++ b/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx @@ -4037,6 +4037,7 @@ For some tips on configuring logs for the Python agent, see [Configure Python lo Config file, environment variable + @@ -4094,6 +4095,7 @@ For some tips on configuring logs for the Python agent, see [Configure Python lo Config file, environment variable + From 64fe13d0b56e497353ca943752707f19f0c8b5c8 Mon Sep 17 00:00:00 2001 From: ally sassman <42753584+ally-sassman@users.noreply.github.com> Date: Fri, 12 Jan 2024 16:28:13 -0800 Subject: [PATCH 5/5] fix(releaseNotes): style edits --- .../python-agent-configuration.mdx | 2 +- .../recordlogevent-python-agent-api.mdx | 2 +- .../python-agent-90500.mdx | 19 +++++++++++-------- 3 files changed, 13 insertions(+), 10 deletions(-) diff --git a/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx b/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx index 4ced130637e..9234daa428f 100644 --- a/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx +++ b/src/content/docs/apm/agents/python-agent/configuration/python-agent-configuration.mdx @@ -4051,7 +4051,7 @@ For some tips on configuring logs for the Python agent, see [Configure Python lo - If attributes are enabled for context_data, all attribute keys found in this list will be sent to us in transaction traces. For more information, see the [agent attribute rules](/docs/apm/other-features/attributes/agent-attributes). + If attributes are enabled for `context_data`, all attribute keys found in this list will be sent to us in transaction traces. For more information, see the [agent attribute rules](/docs/apm/other-features/attributes/agent-attributes). When adding context attributes, the agent prefixes the keys with `context.` for attributes from the logging framework context, and `message.` for attributes from a dictionary message context. diff --git a/src/content/docs/apm/agents/python-agent/python-agent-api/recordlogevent-python-agent-api.mdx b/src/content/docs/apm/agents/python-agent/python-agent-api/recordlogevent-python-agent-api.mdx index 6404272287a..9001ffed68b 100644 --- a/src/content/docs/apm/agents/python-agent/python-agent-api/recordlogevent-python-agent-api.mdx +++ b/src/content/docs/apm/agents/python-agent/python-agent-api/recordlogevent-python-agent-api.mdx @@ -150,7 +150,7 @@ def fetch(): ### Record log event with context data attributes [#context-data-example] -An example of recording a log event using message attributes and context attributes +Here's an example of recording a log event using message attributes and context attributes: ```py def fetch(product_id): diff --git a/src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-90500.mdx b/src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-90500.mdx index eee3dd14818..7236a450efe 100644 --- a/src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-90500.mdx +++ b/src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-90500.mdx @@ -10,7 +10,7 @@ security: [] ## Notes -This release of the Python agent adds context data attributes to logs, support for dictionary type log messages, adds attribute support to `record_log_event` API, and obfuscation of logged license keys and user API keys. +This release of the Python agent adds context data attributes to logs, support for dictionary type log messages, adds attribute support to `record_log_event` API, and obfuscates logged license keys and user API keys. Install the agent using `easy_install/pip/distribute` via the [Python Package Index](https://pypi.python.org/pypi/newrelic) or download it directly from the [New Relic download site](https://download.newrelic.com/python_agent/release). @@ -18,20 +18,23 @@ Install the agent using `easy_install/pip/distribute` via the [Python Package In ## New features * **Add context data attributes to logs** -Context data available from logging frameworks can now be recorded as attributes on logs. These attributes are prefixed with `context.`, which is not considered when filtering attributes from logs. -Context data may include extras, dictionary message attributes, and attributes provided by logging frameworks. - -To report these attributes, [enable context data forwarding](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.enabled) and optionally configure [include](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.include) and [exclude](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.exclude) rules to filter the allowed attributes. + * Context data available from logging frameworks can now be recorded as attributes on logs. These attributes are prefixed with `context.`, which is not considered when filtering attributes from logs. + * Context data may include extras, dictionary message attributes, and attributes provided by logging frameworks. + * To report these attributes, [enable context data forwarding](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.enabled) and optionally configure [include](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.include) and [exclude](/docs/agents/python-agent/configuration/python-agent-configuration#application_logging.forwarding.context_data.exclude) rules to filter the allowed attributes. * **Add support for dictionary type log messages** -Dictionary type log messages are now supported for logging frameworks which support dictionary type log messages (including the builtin `logging` module and `structlog`). The key `message` will be extracted if available and treated as the log message, and any other items will be considered context data attributes and prefixed with `message.`. These attributes are subject to the same filtering rules as other context data. + + * Dictionary type log messages are now supported for logging frameworks which support dictionary type log messages (including the builtin `logging` module and `structlog`). + * The key `message` will be extracted if available and treated as the log message, and any other items will be considered context data attributes and prefixed with `message.`. These attributes are subject to the same filtering rules as other context data. * **Add attribute support to `record_log_event` API** -The `record_log_event` API now allows passing context data in using the optional `attributes` keyword argument, and allows the message argument to be a dictionary as described above. + + * The `record_log_event` API now allows passing context data in using the optional `attributes` keyword argument, and allows the message argument to be a dictionary as described above. * **Obfuscate license keys and user API keys in logs** -The agent now obfuscates all instances of license keys or user API keys in agent logs and audit logs. + + * The agent now obfuscates all instances of license keys or user API keys in agent logs and audit logs. ## Support statement