Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Behaviour when pattern writes to same input field (without "overwrite" option) #180

Open
hackery opened this issue Jul 13, 2022 · 1 comment

Comments

@hackery
Copy link

hackery commented Jul 13, 2022

The behaviour of a grok where a semantic is named the same as the input field is not described in the documentation, and is counter-intuitive. e.g.:

input {
  generator {
    "message" => "hello world"
    "count" => 1
  }
}

filter {
  grok {
    match => { "message" => "hello %{GREEDYDATA:message}" }
  }
}

output { stdout{} }

If you didn't read the docs carefully, you might assume that message gets overwritten with world (perhaps the most obvious behaviour). Reading the overwrite section, you'd probably think OK, if I want that behaviour, I need to set overwrite but otherwise it looks like it's discarded.

What you actually get is an array, which is, I feel, an unwelcome surprise. The code explicitly promotes an existing string to an array and appends to it. If it was already an array, it appends to it - that's fair enough.

{
    ...
    "message" => [
        [0] "hello world",
        [1] "world"
    ],
}

Please note behaviour in documentation, or if it's not actually intended, consider a bug.
It looks like there in fact isn't a way to specify retaining (only) the original string field.

@wallrik
Copy link
Contributor

wallrik commented Oct 4, 2022

This behavior is consistent with add_field as well. If you try to add a field that already exists it will be added as an array. That particular issue is being tracked in elastic/logstash#11751

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants