Skip to content

Commit

Permalink
[pkg/ottl] Add ToKeyValueString Converter (#35409)
Browse files Browse the repository at this point in the history
**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->

Implements ToKeyValueString OTTL Converter. 

**Link to tracking Issue:** #35334

**Testing:** Added unit tests and e2e

**Documentation:** Added
  • Loading branch information
kuiperda authored Oct 1, 2024
1 parent 5f96c1a commit a9a44c3
Show file tree
Hide file tree
Showing 6 changed files with 462 additions and 0 deletions.
27 changes: 27 additions & 0 deletions .chloggen/ottl-tokeyvaluestring.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: pkg/ottl

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add ToKeyValueString Converter

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [35334]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
24 changes: 24 additions & 0 deletions pkg/ottl/e2e/e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -621,6 +621,30 @@ func Test_e2e_converters(t *testing.T) {
m.PutStr("k2", "v2__!__v2")
},
},
{
statement: `set(attributes["test"], ToKeyValueString(ParseKeyValue("k1=v1 k2=v2"), "=", " ", true))`,
want: func(tCtx ottllog.TransformContext) {
tCtx.GetLogRecord().Attributes().PutStr("test", "k1=v1 k2=v2")
},
},
{
statement: `set(attributes["test"], ToKeyValueString(ParseKeyValue("k1:v1,k2:v2", ":" , ","), ":", ",", true))`,
want: func(tCtx ottllog.TransformContext) {
tCtx.GetLogRecord().Attributes().PutStr("test", "k1:v1,k2:v2")
},
},
{
statement: `set(attributes["test"], ToKeyValueString(ParseKeyValue("k1=v1 k2=v2"), "!", "+", true))`,
want: func(tCtx ottllog.TransformContext) {
tCtx.GetLogRecord().Attributes().PutStr("test", "k1!v1+k2!v2")
},
},
{
statement: `set(attributes["test"], ToKeyValueString(ParseKeyValue("k1=v1 k2=v2=v3"), "=", " ", true))`,
want: func(tCtx ottllog.TransformContext) {
tCtx.GetLogRecord().Attributes().PutStr("test", "k1=v1 k2=\"v2=v3\"")
},
},
{
statement: `set(attributes["test"], ParseXML("<Log id=\"1\"><Message>This is a log message!</Message></Log>"))`,
want: func(tCtx ottllog.TransformContext) {
Expand Down
39 changes: 39 additions & 0 deletions pkg/ottl/ottlfuncs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -459,6 +459,7 @@ Available Converters:
- [String](#string)
- [Substring](#substring)
- [Time](#time)
- [ToKeyValueString](#tokeyvaluestring)
- [TraceID](#traceid)
- [TruncateTime](#truncatetime)
- [Unix](#unix)
Expand Down Expand Up @@ -1660,6 +1661,44 @@ Examples:
- `Time("mercoledì set 4 2024", "%A %h %e %Y", "", "it")`
- `Time("Febrero 25 lunes, 2002, 02:03:04 p.m.", "%B %d %A, %Y, %r", "America/New_York", "es-ES")`

### ToKeyValueString

`ToKeyValueString(target, Optional[delimiter], Optional[pair_delimiter], Optional[sort_output])`

The `ToKeyValueString` Converter takes a `pcommon.Map` and converts it to a `string` of key value pairs.

- `target` is a Getter that returns a `pcommon.Map`.
- `delimiter` is an optional string that is used to join keys and values, the default is `=`.
- `pair_delimiter` is an optional string that is used to join key value pairs, the default is a single space (` `).
- `sort_output` is an optional bool that is used to deterministically sort the keys of the output string. It should only be used if the output is required to be in the same order each time, as it introduces some performance overhead.

For example, the following map `{"k1":"v1","k2":"v2","k3":"v3"}` will use default delimiters and be converted into the following string:

```
`k1=v1 k2=v2 k3=v3`
```

**Note:** Any nested arrays or maps will be represented as a JSON string. It is recommended to [flatten](#flatten) `target` before using this function.

For example, `{"k1":"v1","k2":{"k3":"v3","k4":["v4","v5"]}}` will be converted to:

```
`k1=v1 k2={\"k3\":\"v3\",\"k4\":[\"v4\",\"v5\"]}`
```

**Note:** If any keys or values contain either delimiter, they will be double quoted. If any double quotes are present in the quoted value, they will be escaped.

For example, `{"k1":"v1","k2":"v=2","k3"="\"v=3\""}` will be converted to:

```
`k1=v1 k2="v=2" k3="\"v=3\""`
```

Examples:

- `ToKeyValueString(body)`
- `ToKeyValueString(body, ":", ",", true)`

### TraceID

`TraceID(bytes)`
Expand Down
122 changes: 122 additions & 0 deletions pkg/ottl/ottlfuncs/func_to_key_value_string.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package ottlfuncs // import "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/ottlfuncs"

import (
"context"
"fmt"
gosort "sort"
"strings"

"go.opentelemetry.io/collector/pdata/pcommon"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
)

type ToKeyValueStringArguments[K any] struct {
Target ottl.PMapGetter[K]
Delimiter ottl.Optional[string]
PairDelimiter ottl.Optional[string]
SortOutput ottl.Optional[bool]
}

func NewToKeyValueStringFactory[K any]() ottl.Factory[K] {
return ottl.NewFactory("ToKeyValueString", &ToKeyValueStringArguments[K]{}, createToKeyValueStringFunction[K])
}

func createToKeyValueStringFunction[K any](_ ottl.FunctionContext, oArgs ottl.Arguments) (ottl.ExprFunc[K], error) {
args, ok := oArgs.(*ToKeyValueStringArguments[K])

if !ok {
return nil, fmt.Errorf("ToKeyValueStringFactory args must be of type *ToKeyValueStringArguments[K]")
}

return toKeyValueString[K](args.Target, args.Delimiter, args.PairDelimiter, args.SortOutput)
}

func toKeyValueString[K any](target ottl.PMapGetter[K], d ottl.Optional[string], p ottl.Optional[string], s ottl.Optional[bool]) (ottl.ExprFunc[K], error) {
delimiter := "="
if !d.IsEmpty() {
if d.Get() == "" {
return nil, fmt.Errorf("delimiter cannot be set to an empty string")
}
delimiter = d.Get()
}

pairDelimiter := " "
if !p.IsEmpty() {
if p.Get() == "" {
return nil, fmt.Errorf("pair delimiter cannot be set to an empty string")
}
pairDelimiter = p.Get()
}

if pairDelimiter == delimiter {
return nil, fmt.Errorf("pair delimiter %q cannot be equal to delimiter %q", pairDelimiter, delimiter)
}

sortOutput := false
if !s.IsEmpty() {
sortOutput = s.Get()
}

return func(ctx context.Context, tCtx K) (any, error) {
source, err := target.Get(ctx, tCtx)
if err != nil {
return nil, err
}

return convertMapToKV(source, delimiter, pairDelimiter, sortOutput), nil
}, nil
}

// convertMapToKV converts a pcommon.Map to a key value string
func convertMapToKV(target pcommon.Map, delimiter string, pairDelimiter string, sortOutput bool) string {

var kvStrings []string
if sortOutput {
var keyValues []struct {
key string
val pcommon.Value
}

// Sort by keys
target.Range(func(k string, v pcommon.Value) bool {
keyValues = append(keyValues, struct {
key string
val pcommon.Value
}{key: k, val: v})
return true
})
gosort.Slice(keyValues, func(i, j int) bool {
return keyValues[i].key < keyValues[j].key
})

// Convert KV pairs
for _, kv := range keyValues {
kvStrings = append(kvStrings, buildKVString(kv.key, kv.val, delimiter, pairDelimiter))
}
} else {
target.Range(func(k string, v pcommon.Value) bool {
kvStrings = append(kvStrings, buildKVString(k, v, delimiter, pairDelimiter))
return true
})
}

return strings.Join(kvStrings, pairDelimiter)
}

func buildKVString(k string, v pcommon.Value, delimiter string, pairDelimiter string) string {
key := escapeAndQuoteKV(k, delimiter, pairDelimiter)
value := escapeAndQuoteKV(v.AsString(), delimiter, pairDelimiter)
return key + delimiter + value
}

func escapeAndQuoteKV(s string, delimiter string, pairDelimiter string) string {
s = strings.ReplaceAll(s, `"`, `\"`)
if strings.Contains(s, pairDelimiter) || strings.Contains(s, delimiter) {
s = `"` + s + `"`
}
return s
}
Loading

0 comments on commit a9a44c3

Please sign in to comment.