Skip to content

Latest commit

 

History

History
328 lines (234 loc) · 17.6 KB

CHANGELOG.next.asciidoc

File metadata and controls

328 lines (234 loc) · 17.6 KB
Raw

Beats version HEAD

Breaking changes

Affecting all Beats - Fix status reporting to Elastic-Agent when output configuration is invalid running under Elastic-Agent 35719 - Upgrade Go to 1.20.7 36241

Auditbeat

Filebeat

Heartbeat

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

  • Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 35193

  • Fix the ability to use filtering features (e.g. ignore_older, event_id, provider, level) while reading .evtx files. 16826 36173

Functionbeat

Elastic Logging Plugin

Bugfixes

Affecting all Beats - Support for multiline zookeeper logs 2496 - Add checks to ensure reloading of units if the configuration actually changed. 34346 - Fix namespacing on self-monitoring 32336 - Fix namespacing on self-monitoring 32336 - Fix Beats started by agent do not respect the allow_older_versions: true configuration flag 34227 34964 - Fix performance issues when we have a lot of inputs starting and stopping by allowing to disable global processors under fleet. 35000 35031 - In cases where the matcher detects a non-string type in a match statement, report the error as a debug statement, and not a warning statement. 35119 - 'add_cloud_metadata' processor - add cloud.region field for GCE cloud provider - 'add_cloud_metadata' processor - update azure metadata api version to get missing cloud.account.id field - Make sure k8s watchers are closed when closing k8s meta processor. 35630 - Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues 35640 - Fix panic when MaxRetryInterval is specified, but RetryInterval is not 35820 - Do not print context cancelled error message when running under agent 36006 - Fix recovering from invalid output configuration when running under Elastic-Agent 36016 - Improve StreamBuf append to improve performance when reading long lines from files. 35928 - Eliminate cloning of event in deepUpdate 35945 - Fix ndjson parser to store JSON fields correctly under target 29395 - Support build of projects outside of beats directory 36126

Auditbeat

  • auditd: Expanded the bitmask applied to ECS file.mode so that the SUID, SGID, and sticky bits can be represented. 36294

Filebeat

  • [Gcs Input] - Added missing locks for safe concurrency 34914

  • Fix the ignore_inactive option being ignored in Filebeat’s filestream input 34770

  • Fix TestMultiEventForEOFRetryHandlerInput unit test of CometD input 34903

  • Add input instance id to request trace filename for httpjson and cel inputs 35024

  • Fix panic in TCP and UDP inputs on Linux when collecting socket metrics from OS. 35064

  • Correctly collect TCP and UDP metrics for unspecified address values. 35111

  • Fix base for UDP and TCP queue metrics and UDP drops metric. 35123

  • Sanitize filenames for request tracer in httpjson input. 35143

  • decode_cef processor: Fix ECS output by making observer.ip into an array of strings instead of string. 35140 35149

  • Fix handling of MySQL audit logs with strict JSON parser. 35158 35160

  • Sanitize filenames for request tracer in cel input. 35154

  • Fix accidental error overwrite in defer statement in entityanalytics Azure AD input. 35153 35169

  • Fixing the grok expression outputs of log files 35221

  • Fixes "Can only start an input when all related states are finished" error when running under Elastic-Agent 35250 33653

  • Move repeated Windows event channel not found errors in winlog input to debug level. 35314 35317

  • Fix crash when processing forwarded logs missing a message. 34705 34865

  • Fix crash when loading azurewebstorage cursor with no partially processed data. 35433

  • Add support in s3 input for JSON with array of objects. 35475

  • RFC5424 syslog timestamps with offset 'Z' will be treated as UTC rather than using the default timezone. 35360

  • Fix syslog message parsing for fortinet.firewall to take into account quoted values. 35522

  • [system] sync system/auth dataset with system integration 1.29.0. 35581

  • [GCS Input] - Fixed an issue where bucket_timeout was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. 35605

  • Fix filestream false positive log error "filestream input with ID 'xyz' already exists" 31767

  • Fix error message formatting from filestream input. 35658

  • Fix error when trying to use include_message parser 35440

  • Fix handling of IPv6 unspecified addresses in TCP input. 35064 35637

  • Fixed a minor code error in the GCS input scheduler where a config value was being used directly instead of the source struct. 35729

  • Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. 35772

  • Fix CEL input JSON marshalling of nested objects. 35763 35774

  • Fix metric collection in GCPPubSub input. 35773

  • Fix end point deregistration in http_endpoint input. 35899 35903

  • Fix duplicate ID panic in filestream metrics. 35964 35972

  • Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. 35996

  • Fix handling of NUL-terminated log lines in Fortinet Firewall module. 36026 36027

  • Make redact field configuration recommended in CEL input and log warning if missing. 36008

  • Fix handling of region name configuration in awss3 input 36034

  • Fixed concurrency and flakey tests issue in azure blob storage input. 35983 36124

  • Fix panic when sqs input metrics getter is invoked 36101 36077

  • Make CEL input’s now global variable static for evaluation lifetime. 36107

  • Update mito CEL extension library to v1.5.0. 36146

  • Filter out duplicate paths resolved from matching globs. 36253 36256

  • Fix handling of TCP/UDP address resolution during metric initialization. 35064 36287

  • Fix handling of Juniper SRX structured data when there is no leading junos element. 36270 36308

  • Remove erroneous error log in GCPPubSub input. 36296

Heartbeat

  • Fix panics when parsing dereferencing invalid parsed url. 34702

  • Fix broken zip URL monitors. NOTE: Zip URL Monitors will be removed in version 8.7 and replaced with project monitors. 33723

  • Fix integration hashing to prevent reloading all when updated. 34697

  • Fix release of job limit semaphore when context is cancelled. 34697

  • Fix bug where states.duration_ms was incorrect type. 33563

  • Fix handling of long UDP messages in UDP input. 33836 33837

  • Fix browser monitor summary reporting as up when monitor is down. 33374 33819

  • Fix beat capabilities on Docker image. 33584

  • Fix serialization of state duration to avoid scientific notation. 34280

  • Enable nodejs engine strict validation when bundling synthetics. 34470 with the ecs field name container. 34403 automatic splitting at root level, if root level element is an array. 34155

  • Fix broken mapping for state.ends field. 34891

  • Fix issue using projects in airgapped environments by disabling npm audit. 34936

  • Fix broken state ID location naming. 35336

  • Fix project monitor temp directories permission to include group access. 35398

  • Fix output pipeline exit on run_once. 35376

  • Fix formatting issue with socket trace timeout. 35434

  • Update gval version. 35636

  • Fix serialization of processors when running diagnostics. 35698

  • Filter dev flags for ui monitors inside synthetics_args. 35788

  • Fix temp dir running out of space with project monitors. 35843

  • Fixing the grok expression outputs of log files 35221

  • Enable heartbeat-wide publish timeout setting with run_once. 35721

  • Added default timezone UTC to heartbeat docker images to fix synthetics journeys navigation errors. 36193

Metricbeat

  • in module/windows/perfmon, changed collection method of the second counter value required to create a displayable value 32305

  • Fix and improve AWS metric period calculation to avoid zero-length intervals 32724

  • Add missing cluster metadata to k8s module metricsets 32979 33032

  • Add GCP CloudSQL region filter 32943

  • Fix logstash cgroup mappings 33131

  • Remove unused elasticsearch.node_stats.indices.bulk.avg_time.bytes mapping 33263

  • Make generic SQL GA 34637

  • Collect missing remote_cluster in elasticsearch ccr metricset 34957

  • Add context with timeout in AWS API calls 35425

  • Fix no error logs displayed in CloudWatch EC2, RDS and SQS metadata 34985 35035

  • Remove Beta warning from IIS application_pool metricset 35480

  • Improve documentation for ActiveMQ module 35113 35558

  • Fix EC2 host.cpu.usage 35717

  • Resolve statsd module’s prematurely halting of metrics parsing upon encountering an invalid packet. 35075

  • Fix the gap in fetching forecast API metrics at the end of each month for Azure billing module 36142

  • Add option in SQL module to execute queries for all dbs. 35688

  • Add support for api_key authentication in elasticsearch module 36274

Osquerybeat

Packetbeat

  • Fix handling of Npcap installation options from Fleet. 35541 35935

Winlogbeat

  • Fix powershell details regexp to prevent excessive backtracking when processing command invocations. 36178

Elastic Logging Plugin

Added

Affecting all Beats

  • Added append Processor which will append concrete values or values from a field to target. 29934 33364

  • When running under Elastic-Agent the status is now reported per Unit instead of the whole Beat 35874 36183

  • Add warning message to SysV init scripts for RPM-based systems that lack /etc/rc.d/init.d/functions. 35708 36188

  • Mark translate_sid processor is GA. 36279 36280

Auditbeat

Filebeat

  • add documentation for decode_xml_wineventlog processor field mappings. 32456

  • httpjson input: Add request tracing logger. 32402 32412

  • Add cloudflare R2 to provider list in AWS S3 input. 32620

  • Add support for single string containing multiple relation-types in getRFC5988Link. 32811

  • Added separation of transform context object inside httpjson. Introduced new clause .parent_last_response.* 33499

  • Adding filename details from zip to response for httpjson 33952 34044

  • Added metric sqs_messages_waiting_gauge for aws-s3 input. 34488

  • Add nginx.ingress_controller.upstream.ip to related.ip 34645 34672

  • Add unix socket log parsing for nginx ingress_controller 34732

  • Added metric sqs_worker_utilization for aws-s3 input. 34793

  • Add MySQL authentication message parsing and related.ip and related.user fields 34810

  • Add nginx ingress_controller parsing if one of upstreams fails to return response 34787

  • Add oracle authentication messages parsing 35127

  • Add sanitization capabilities to azure-eventhub input 34874

  • Add support for CRC validation in Filebeat’s HTTP endpoint input. 35204

  • Add support for CRC validation in Zoom module. 35604

  • Add execution budget to CEL input. 35409

  • Add XML decoding support to HTTPJSON. 34438 35235

  • Add delegated account support when using Google ADC in httpjson input. 35507

  • Allow specifying since when to read journald entries. 35408

  • Add metrics for filestream input. 35529

  • Add support for collecting httpjson metrics. 35392

  • Add XML decoding support to CEL. 34438 35372

  • Mark CEL input as GA. 35559

  • Add metrics for gcp-pubsub input. 35614

  • [GCS] Added scheduler debug logs and improved the context passing mechanism by removing them from struct params and passing them as function arguments. 35674

  • Allow non-AWS endpoints for awss3 input. 35496 35520

  • Under elastic-agent the input metrics will now be included in agent diagnostics dumps. 35798

  • Add Okta input package for entity analytics. 35611

  • Expose harvester metrics from filestream input 35835 33771

  • Add device support for Azure AD entity analytics. 35807

  • Improve CEL input performance. 35915

  • Adding filename details from zip to response for httpjson 33952 34044

  • Added support for min/max template functions in httpjson input. 36094 36036

  • Add clean_session configuration setting for MQTT input. 16204

  • Add fingerprint mode for the filestream scanner and new file identity based on it 34419 35734

  • Add file system metadata to events ingested via filestream 35801 36065

  • Add support for localstack based input integration testing 35727

  • Allow parsing bytes in and bytes out as long integer in CEF processor. 36100 36108

  • Add support for registered owners and users to AzureAD entity analytics provider. 36092

  • Add support for endpoint resolver in AWS config 36208

  • Added support for Okta OAuth2 provider in the httpjson input. 36273

  • Add support of the interval parameter in Salesforce setupaudittrail-rest fileset. 35917 35938

  • Add device handling to Okta input package for entity analytics. 36049

  • Add setup option --force-enable-module-filesets, that will act as if all filesets have been enabled in a module during setup. 30915 99999

  • Add setup option --force-enable-module-filesets, that will act as if all filesets have been enabled in a module during setup. 30915 36286

Auditbeat

Libbeat

Heartbeat - Added status to monitor run log report.

Metricbeat

  • Add per-thread metrics to system_summary 33614

  • Add GCP CloudSQL metadata 33066

  • Add support for multiple regions in GCP 32964

  • Add GCP Carbon Footprint metricbeat data 34820

  • Add event loop utilization metric to Kibana module 35020

  • Support collecting metrics from both the monitoring account and linked accounts from AWS CloudWatch. 35540

  • Add new parameter include_linked_accounts to enable/disable metrics collection from multiple linked AWS Accounts 35648

  • Migrate Azure Billing, Monitor, and Storage metricsets to the newer SDK. 33585

  • Add support for float64 values parsing for statsd metrics of counter type. 35099

  • Add kubernetes.deployment.status.* fields for Kubernetes module 35999

Osquerybeat

Packetbeat

  • Added packetbeat.interfaces.fanout_group to allow a Packetbeat sniffer to join an AF_PACKET fanout group. 35451 35453

  • Add AF_PACKET metrics. 35428 35489

  • Under elastic-agent the input metrics will now be included in agent diagnostics dumps. 35798

  • Add support for multiple regions in GCP 32964

Packetbeat

Winlogbeat

Functionbeat

Winlogbeat

  • Set host.os.type and host.os.family to "windows" if not already set. 35435

  • Handle empty DNS answer data in QueryResults for the Sysmon Pipeline 35207

  • Under elastic-agent the input metrics will now be included in agent diagnostics dumps. 35798

Elastic Log Driver Elastic Logging Plugin

Deprecated

Auditbeat

Filebeat

Heartbeat

  • Deprecate aws_elb autodiscover provider. 36191

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

Functionbeat

Elastic Logging Plugin

Known Issues