Metrics filters are not working properly

[uni-pooja-laad]

Hi team
Please see the last support chat:
https://discord.com/channels/742820443487993987/1285180794381275157/1285180794381275157
By latest suggrestion I got that I can use OR condition to get multiple prefix metrics.
My ask here is I just need metrics starting with either cpu.* or kong.* or mem.*
I am trying this remap:

filter_metrics:
    type: filter
    inputs: [datadog_agent.metrics]
    condition: 
       type: vrl
       source: match!(to_string(.name) ?? "default", r'^(cpu.*|kong.*|mem.*)')

Error is :

2024-11-26T07:20:11.641948Z ERROR vector::topology::builder: Configuration error. error=Transform "filter_metrics":
error[E620]: can't abort infallible function
  ┌─ :1:1
  │
1 │ match!(to_string(.name) ?? "default", r'^(cpu.*|kong.*|mem.*)')
  │ ^^^^^- remove this abort-instruction
  │ │
  │ this function can't fail
  │
  = see documentation about error handling at https://errors.vrl.dev/#handling
  = see language documentation at https://vrl.dev
  = try your code in the VRL REPL, learn more at https://vrl.dev/examples

Also tried this (tostring() function correction):

filter_metrics:
    type: filter
    inputs: [datadog_agent.metrics]
    condition: 
       type: vrl
       source: match!(tostring(.name) ?? "default", r'^(cpu.*|kong.*|mem.*)')

Error is:

2024-11-26T07:35:26.637294Z ERROR vector::topology::builder: Configuration error. error=Transform "filter_metrics":
error[E105]: call to undefined function
  ┌─ :1:8
  │
1 │ match!(tostring(.name) ?? "default", r'^(cpu.*|kong.*|mem.*)')
  │        ^^^^^^^^
  │        │
  │        undefined function
  │        did you mean "to_string"?
  │
  = learn more about error code 105 at https://errors.vrl.dev/105
  = see language documentation at https://vrl.dev
  = try your code in the VRL REPL, learn more at https://vrl.dev/examples

[pront]

There's no tostring. There's a to_string coercion function.

---

1 │ match!(to_string(.name) ?? "default", r'^(cpu.*|kong.*|mem.*)')
  │ ^^^^^- remove this abort-instruction

^ This means that you can remove the ! like so so match(to_string(.name) ....

But you also need to handle the to_string error so the final version is:

match(to_string!(.name), r'^(cpu.*|kong.*|mem.*)')

(you can drop the "default" since it will never match)

I can see how this should be a warning instead of an error, worth requesting this here: https://github.com/vectordotdev/vrl/issues/new.

[pront]

Do you mean that you want your VRL script to be in a separate file? Not for filter, but Remap supports this. In the next version, it will also support multiple files.

[uni-pooja-laad]

I need something like:
metrics_lixt.txt
kong_*
cpu_*
io.*

file format we can decide. and then in vector configuration:
filter_metrics:
type: filter
inputs: [datadog_agent.metrics]
condition:
type: vrl
source: match!(to_string(.name) ?? "default", r'^(fetch file and put logical or ( | ) condition after each line to make it work)

Or is there any other way to do it? I have 50+ metrics regex to include via filter transform.

[uni-pooja-laad]

@pront is there any way to process above requirement?

[pront]

The above message is a bit hard to understand due to lack of formatting. But my guess is you want your regexes to be defined in a file, and then to use that file to generate a regex in VRL. That is not supported.

[pront]

Feel free to create a feature request. It's not obvious what the best approach to this. As a workaround, I would generate the generate the regex with a script.

[pront]

FYI, I already created an issue: vectordotdev/vrl#1143