Using "converter" to preprocess JSON -> JSON logs

[ostrolucky]

I often interact with JSON logs having this structure

{"context": "body_bytes_sent":0}

But, when these logs are shipped to New Relic, all the nesting is removed by changing it to flat structure like so

{"context.body_bytes_sent":0}

Instead of having to write and maintain 2 different format definitions, I was thinking to use conversion feature. However, not sure lnav can do such thing. Whole thing is a bit intimidating to me, because it seems it's designed for a binary conversion. Could you confirm automatic JSON->JSON conversion here is possible and shine a bit more light here how would I go about it?

I've tried something like

"converter": {
      "header": {
        "expr": {
          "newrelic": "decode(:header, 'hex') LIKE '%context.%'",
          "size": 128
        }
      },
      "command": "test.sh"
    },

But test.sh is never executed (I know because its contents are echo $@ > /tmp/test) and I can't see the result of sqlite query either, so I'm kinda stuck on this because I don't know how I can further debug this (I don't work with c++ either)

[tstack]

First, the "size" property should be in the "header" and not the "expr" object. You can pass -W to lnav to get a warning.

I changed the converter SQL expression so it would match the dev.log.bak2.json file you sent earlier and it seemed to "work". As in, lnav complains that it couldn't find the test.sh converter script. The expression looked like the following after my change:

"newrelic": "decode(:header, 'hex') LIKE '%\"context\"%'"

[ostrolucky]

Thanks. Yeah my problem was both expr.size + its value. Found a different problem, but I'll create an issue for it