Problems to use otel-collector as log pipeline

[gillg]

Use case sample
Syslogreceiver > Lokiexporter

Is your feature request related to a problem? Please describe.
Hello, as discussed in previous issues (I will search and attach them later) otel logs can have a structured body, and/or attributes. In different case we have very different results.
Ex: fluent forward receiver don't fill any body... We only have attributes.
In opposite, syslogreceiver don't fill attributes and we only have (a structured) body.

Both receivers are correct in term of opentelemtry.
But at the exporter level it can be complex to handle (here impossible for syslog) depending on the technology.
Loki need a body and labels (selected set on metadata), so we need a way to extract them from otel attributes, and/or structured body, and/or resources.

An additional processor to parse an unstructured body could be useful in some cases (even if I'm not fan for perf reasons), to convert it as structured or map extracted parts to attributes.

[punya]

Hi @gillg! Would you mind linking the previous issues relevant to this topic? Also, it would be helpful to call out specific enhancements that you're looking for, with as much precision and concrete use cases as possible.

[gillg]

Hi @punya thanks for the reminder ! 😅
A similar issue / brainstorming was on #14718
It's initially related to #2529

A PR is in progress on Loki exporter to allow a better mapping between otel / Loki but it was not plan to be able to map body attributes to Loki labels (specific log metadata similar to attributes but to chose carefully, at least one is mandatory)

I also started some rewrite (#5173) on fluent receiver to allow a manual mapping from attributes to body and severity.

[gillg]

Maybe, an alternative to these pr could be to extend the attributesprocessor (or a new thing) to manipulate a structured body as AttributeMap and be able to move data from attributes to body and vice versa.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.