Support include_brackets feature just like Logstash KV filter

[tiantangkuohai]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

We have some log field like this:

{"detail": "query=[select 1 from table1] component=[aggregator] msg=[begin to send query to \[mysql.foo.com\]]"}

In logstash, it can parse the fields in detail field value using KV filter. But it seems that Vector doesn't have similar feature.

Attempted Solutions

I tries with parse_key_value and parse_regex_all. Both of them can't working perfectly.

Proposal

No response

References

No response

Version

0.42.0

[pront]

Hi @tiantangkuohai,

• Does this log come from the Logstash agent?
• Are you suggesting to implement this parsing logic in the logstash source?

[jszwedko]

I think this could be supported by enabling configuration of the expected "quote" characters in parse_key_value. I believe that parse_key_value currently expects keys/values to be quoted with "s (if quoted at all). We could allow configuration of quotes as brackets instead. Something like: parse_key_value(.message, start_quote: "[", end_quote: "]").