Curly braces in file source exclude patterns not supported

[pablosichert]

Vector Version

vector 0.12.0

Vector Configuration File

[sources.files]
  type = "file"
  include = ["*.log"]
  exclude = ["{foo,bar}.log"]

[sinks.console]
  inputs = ["files"]
  type = "console"
  encoding.codec = "text"

Expected Behavior

The vector file source ignores foo.log and bar.log.

Actual Behavior

Files foo.log and bar.log are ingested.

Additional Context

The exclude_patterns currently relies on glob::Pattern which does not support curly-brace patterns. The globwalk crate used for include_patterns doesn't provide any means to just pattern match without walking the FS.

Proposed approach:

GlobWalker internally uses ignore::Override, and supports adding patterns to the ignore::overrides::OverrideBuilder. The OverrideBuilder supports negating matches by prepending the glob with a !.

References

• Curly braces in the include directive, file pattern #5003
• fix(file source): Allow file pattern with curly braces in include directive #5927