Skip to content

Commit

Permalink
[exporter/splunkhecexporter] Add OtelAttrsToHec struct and related …
Browse files Browse the repository at this point in the history
…config fields (open-telemetry#35476)

**Description:** Adding a new config fields
`otel_attrs_to_hec_metadata/*` to replace the
`hec_metadata_to_otel_attrs/*` config fields.

**Link to tracking Issue:** fixes open-telemetry#35092

**Testing:** No new tests added, modified existing tests.

**Documentation:** Adding field-level description to exporter README.md.
  • Loading branch information
bderrly authored and sbylica-splunk committed Dec 17, 2024
1 parent be6d9f1 commit 57af894
Show file tree
Hide file tree
Showing 9 changed files with 208 additions and 15 deletions.
28 changes: 28 additions & 0 deletions .chloggen/iss-35092.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: deprecation

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: splunkhecexporter

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add `otel_attrs_to_hec_metadata/*` config fields to replace `hec_metadata_to_otel_attrs/*` fields.

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [35092]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext: |
`otel_attrs_to_hec_metadata/*` config fields will replace the `hec_metadata_to_otel_attrs/*` fields in a later release.
# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: [user]
8 changes: 8 additions & 0 deletions exporter/splunkhecexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,18 @@ The following configuration options can also be configured:
- `health_path` (default = '/services/collector/health'): The path reporting [health checks](https://docs.splunk.com/Documentation/Splunk/9.0.1/RESTREF/RESTinput#services.2Fcollector.2Fhealth).
- `health_check_enabled` (default = false): Whether to perform Splunk HEC Health Check during the exporter's startup.
- `export_raw` (default = false): send only the log's body, targeting a Splunk HEC raw endpoint.
- `otel_attrs_to_hec_metadata/source` (default = 'com.splunk.source'): Specifies the mapping of a specific unified model attribute value to the standard source field of a HEC event.
- `otel_attrs_to_hec_metadata/sourcetype` (default = 'com.splunk.sourcetype'): Specifies the mapping of a specific unified model attribute value to the standard sourcetype field of a HEC event.
- `otel_attrs_to_hec_metadata/index` (default = 'com.splunk.index'): Specifies the mapping of a specific unified model attribute value to the standard index field of a HEC event.
- `otel_attrs_to_hec_metadata/host` (default = 'host.name'): Specifies the mapping of a specific unified model attribute value to the standard host field and the `host.name` field of a HEC event.
- `hec_metadata_to_otel_attrs/source` (default = 'com.splunk.source'): Specifies the mapping of a specific unified model attribute value to the standard source field of a HEC event.
**Deprecated** (v0.116.0): prefer `otel_attrs_to_hec_metadata/source`.
- `hec_metadata_to_otel_attrs/sourcetype` (default = 'com.splunk.sourcetype'): Specifies the mapping of a specific unified model attribute value to the standard sourcetype field of a HEC event.
**Deprecated** (v0.116.0): prefer `otel_attrs_to_hec_metadata/sourcetype`.
- `hec_metadata_to_otel_attrs/index` (default = 'com.splunk.index'): Specifies the mapping of a specific unified model attribute value to the standard index field of a HEC event.
**Deprecated** (v0.116.0): prefer `otel_attrs_to_hec_metadata/index`.
- `hec_metadata_to_otel_attrs/host` (default = 'host.name'): Specifies the mapping of a specific unified model attribute value to the standard host field and the `host.name` field of a HEC event.
**Deprecated** (v0.116.0): prefer `otel_attrs_to_hec_metadata/host`.
- `otel_to_hec_fields/severity_text` (default = `otel.log.severity.text`): Specifies the name of the field to map the severity text field of log events.
- `otel_to_hec_fields/severity_number` (default = `otel.log.severity.number`): Specifies the name of the field to map the severity number field of log events.
- `otel_to_hec_fields/name` (default = `"otel.log.name`): Specifies the name of the field to map the name field of log events.
Expand Down
5 changes: 5 additions & 0 deletions exporter/splunkhecexporter/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,12 @@ type Config struct {

// App version is used to track telemetry information for Splunk App's using HEC by App version. Defaults to the current OpenTelemetry Collector Contrib build version.
SplunkAppVersion string `mapstructure:"splunk_app_version"`

// OtelAttrsToHec creates a mapping from attributes to HEC specific metadata: source, sourcetype, index and host.
OtelAttrsToHec splunk.HecToOtelAttrs `mapstructure:"otel_attrs_to_hec_metadata"`

// HecToOtelAttrs creates a mapping from attributes to HEC specific metadata: source, sourcetype, index and host.
// Deprecated: [v0.113.0] Use OtelAttrsToHec instead.
HecToOtelAttrs splunk.HecToOtelAttrs `mapstructure:"hec_metadata_to_otel_attrs"`
// HecFields creates a mapping from attributes to HEC fields.
HecFields OtelToHecFields `mapstructure:"otel_to_hec_fields"`
Expand Down
6 changes: 6 additions & 0 deletions exporter/splunkhecexporter/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,12 @@ func TestLoadConfig(t *testing.T) {
MaxSizeItems: 10,
},
},
OtelAttrsToHec: splunk.HecToOtelAttrs{
Source: "mysource",
SourceType: "mysourcetype",
Index: "myindex",
Host: "myhost",
},
HecToOtelAttrs: splunk.HecToOtelAttrs{
Source: "mysource",
SourceType: "mysourcetype",
Expand Down
6 changes: 6 additions & 0 deletions exporter/splunkhecexporter/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,12 @@ func createDefaultConfig() component.Config {
MaxContentLengthMetrics: defaultContentLengthMetricsLimit,
MaxContentLengthTraces: defaultContentLengthTracesLimit,
MaxEventSize: defaultMaxEventSize,
OtelAttrsToHec: splunk.HecToOtelAttrs{
Source: splunk.DefaultSourceLabel,
SourceType: splunk.DefaultSourceTypeLabel,
Index: splunk.DefaultIndexLabel,
Host: conventions.AttributeHostName,
},
HecToOtelAttrs: splunk.HecToOtelAttrs{
Source: splunk.DefaultSourceLabel,
SourceType: splunk.DefaultSourceTypeLabel,
Expand Down
39 changes: 35 additions & 4 deletions exporter/splunkhecexporter/logdata_to_splunk.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,22 +23,53 @@ const (
traceIDFieldKey = "trace_id"
)

// copyOtelAttrs copies values from HecToOtelAttrs to OtelAttrsToHec struct.
func copyOtelAttrs(config *Config) {
defaultCfg := createDefaultConfig().(*Config)
if config.OtelAttrsToHec.Equal(defaultCfg.OtelAttrsToHec) {
if !config.HecToOtelAttrs.Equal(defaultCfg.HecToOtelAttrs) {
// Copy settings to ease deprecation of HecToOtelAttrs.
config.OtelAttrsToHec = config.HecToOtelAttrs
}
} else {
if !config.HecToOtelAttrs.Equal(defaultCfg.HecToOtelAttrs) {
// Replace all default fields in OtelAttrsToHec.
if config.OtelAttrsToHec.Source == defaultCfg.OtelAttrsToHec.Source {
config.OtelAttrsToHec.Source = config.HecToOtelAttrs.Source
}
if config.OtelAttrsToHec.SourceType == defaultCfg.OtelAttrsToHec.SourceType {
config.OtelAttrsToHec.SourceType = config.HecToOtelAttrs.SourceType
}
if config.OtelAttrsToHec.Index == defaultCfg.OtelAttrsToHec.Index {
config.OtelAttrsToHec.Index = config.HecToOtelAttrs.Index
}
if config.OtelAttrsToHec.Host == defaultCfg.OtelAttrsToHec.Host {
config.OtelAttrsToHec.Host = config.HecToOtelAttrs.Host
}
}
}
}

func mapLogRecordToSplunkEvent(res pcommon.Resource, lr plog.LogRecord, config *Config) *splunk.Event {
body := lr.Body().AsRaw()
if body == nil || body == "" {
// events with no body are rejected by Splunk.
return nil
}

// Manage the deprecation of HecToOtelAttrs config parameters.
// TODO: remove this once HecToOtelAttrs is removed from Config.
copyOtelAttrs(config)

host := unknownHostName
source := config.Source
sourcetype := config.SourceType
index := config.Index
fields := map[string]any{}
sourceKey := config.HecToOtelAttrs.Source
sourceTypeKey := config.HecToOtelAttrs.SourceType
indexKey := config.HecToOtelAttrs.Index
hostKey := config.HecToOtelAttrs.Host
sourceKey := config.OtelAttrsToHec.Source
sourceTypeKey := config.OtelAttrsToHec.SourceType
indexKey := config.OtelAttrsToHec.Index
hostKey := config.OtelAttrsToHec.Host
severityTextKey := config.HecFields.SeverityText
severityNumberKey := config.HecFields.SeverityNumber
if spanID := lr.SpanID(); !spanID.IsEmpty() {
Expand Down
116 changes: 105 additions & 11 deletions exporter/splunkhecexporter/logdata_to_splunk_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,100 @@ import (
"github.com/open-telemetry/opentelemetry-collector-contrib/internal/splunk"
)

func Test_copyOtelAttrs(t *testing.T) {
tests := []struct {
name string
configDataFn func() *Config
wantConfigDataFn func() *Config
}{
{
name: "defaults",
configDataFn: func() *Config {
return createDefaultConfig().(*Config)
},
wantConfigDataFn: func() *Config {
return createDefaultConfig().(*Config)
},
},
{
name: "override hec_metadata_to_otel_attrs",
configDataFn: func() *Config {
cfg := createDefaultConfig().(*Config)

cfg.HecToOtelAttrs.Index = "testIndex"
cfg.HecToOtelAttrs.Source = "testSource"
cfg.HecToOtelAttrs.SourceType = "testSourceType"
cfg.HecToOtelAttrs.Host = "testHost"

return cfg
},
wantConfigDataFn: func() *Config {
cfg := createDefaultConfig().(*Config)

cfg.HecToOtelAttrs.Index = "testIndex"
cfg.HecToOtelAttrs.Source = "testSource"
cfg.HecToOtelAttrs.SourceType = "testSourceType"
cfg.HecToOtelAttrs.Host = "testHost"

cfg.OtelAttrsToHec.Index = "testIndex"
cfg.OtelAttrsToHec.Source = "testSource"
cfg.OtelAttrsToHec.SourceType = "testSourceType"
cfg.OtelAttrsToHec.Host = "testHost"

return cfg
},
},
{
name: "partial otel_attrs_to_hec_metadata",
configDataFn: func() *Config {
cfg := createDefaultConfig().(*Config)

cfg.OtelAttrsToHec.Source = "testSource"
cfg.OtelAttrsToHec.Index = "testIndex"

return cfg
},
wantConfigDataFn: func() *Config {
cfg := createDefaultConfig().(*Config)

cfg.OtelAttrsToHec.Source = "testSource"
cfg.OtelAttrsToHec.Index = "testIndex"

return cfg
},
},
{
name: "prefer otel_attrs_to_hec_metadata",
configDataFn: func() *Config {
cfg := createDefaultConfig().(*Config)

cfg.HecToOtelAttrs.Index = "hecIndex"

cfg.OtelAttrsToHec.Index = "otelIndex"

return cfg
},
wantConfigDataFn: func() *Config {
cfg := createDefaultConfig().(*Config)

cfg.HecToOtelAttrs.Index = "hecIndex"

cfg.OtelAttrsToHec.Index = "otelIndex"

return cfg
},
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := tt.configDataFn()
copyOtelAttrs(cfg)
assert.Equal(t, tt.wantConfigDataFn(), cfg)
})
}
}

func Test_mapLogRecordToSplunkEvent(t *testing.T) {
ts := pcommon.Timestamp(123)

Expand Down Expand Up @@ -153,18 +247,18 @@ func Test_mapLogRecordToSplunkEvent(t *testing.T) {
},
logResourceFn: pcommon.NewResource,
configDataFn: func() *Config {
return &Config{
HecToOtelAttrs: splunk.HecToOtelAttrs{
Source: "mysource",
SourceType: "mysourcetype",
Index: "myindex",
Host: "myhost",
},
HecFields: OtelToHecFields{
SeverityNumber: "myseveritynum",
SeverityText: "myseverity",
},
config := createDefaultConfig().(*Config)
config.HecToOtelAttrs = splunk.HecToOtelAttrs{
Source: "mysource",
SourceType: "mysourcetype",
Index: "myindex",
Host: "myhost",
}
config.HecFields = OtelToHecFields{
SeverityNumber: "myseveritynum",
SeverityText: "myseverity",
}
return config
},
wantSplunkEvents: []*splunk.Event{
func() *splunk.Event {
Expand Down
5 changes: 5 additions & 0 deletions exporter/splunkhecexporter/testdata/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ splunk_hec/allsettings:
max_size_items: 10
splunk_app_name: "OpenTelemetry-Collector Splunk Exporter"
splunk_app_version: "v0.0.1"
otel_attrs_to_hec_metadata:
source: "mysource"
sourcetype: "mysourcetype"
index: "myindex"
host: "myhost"
hec_metadata_to_otel_attrs:
source: "mysource"
sourcetype: "mysourcetype"
Expand Down
10 changes: 10 additions & 0 deletions internal/splunk/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,16 @@ type HecToOtelAttrs struct {
Host string `mapstructure:"host"`
}

func (h HecToOtelAttrs) Equal(o HecToOtelAttrs) bool {
if h.Host != o.Host ||
h.Source != o.Source ||
h.SourceType != o.SourceType ||
h.Index != o.Index {
return false
}
return true
}

type AckRequest struct {
Acks []uint64 `json:"acks"`
}

0 comments on commit 57af894

Please sign in to comment.