Fluentbit to OTelcollector

[RitikaLaddha]

Hi,
I want to upload logs from fluent bit to open telemetry collector on AKS cluster

So, I have installed otel collector as deployment from here
and fluentbit from here

This is my fluentbit configration for output to OTel

[OUTPUT]
     Name                 opentelemetry
     Match                *
     Host                 http://my-opentelemetry-collector.namespace.svc.cluster.local
     Port                 4318
     Metrics_uri          /v1/metrics
     Logs_uri             /v1/logs
     Traces_uri           /v1/traces
     Log_response_payload True
     Tls                  On
     Tls.verify           Off

I have also added environment variable in fluentbit Daemonset
`env:

• name: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
  value: http://my-opentelemetry-collector.namespace.svc.cluster.local:4318`

But I am getting below error, in my fluentbit logs

[error] [tls] error: unexpected EOF
[2024/07/17 13:51:23] [error] [output:opentelemetry:opentelemetry.1] no upstream connections available to my-opentelemetry-collector.namespace.svc.cluster.local:4317
[2024/07/17 13:51:23] [ warn] [engine] failed to flush chunk '1-1721224282.439304987.flb', retry in 7 seconds: task_id=8, input=tail.0 > output=opentelemetry.1 (out_id=1)
[2024/07/17 13:51:23] [ warn] [engine] failed to flush chunk '1-1721224282.536490801.flb', retry in 9 seconds: task_id=9, input=tail.0 > output=opentelemetry.1 (out_id=1)

Do I need to make any configuration change in Otel configmap or fluentbit to make this work?

[RitikaLaddha]

Hi @edsiper
Sorry I didn't know whom to tag, could you please align anyone on this question?

I want to achieve below flow, would be very grateful for any guidance

Logs -> Fluentbit -> OTel Collector -> Uptrace
                                    -> Jaeger
                                    -> etc..

[agup006]

A couple notes, does your OTEL collector support TLS / have it configured?

[RitikaLaddha]

Thanks for replying @agup006
I tried by turning TLS off and I get below error

[2024/07/21 17:13:15] [ warn] [net] getaddrinfo(host='http://my-opentelemetry-collector.dis.svc.cluster.local', err=15): Out of memory
[2024/07/21 17:13:15] [error] [output:opentelemetry:opentelemetry.1] no upstream connections available to http://my-opentelemetry-collector.dis.svc.cluster.local:4318
[2024/07/21 17:13:15] [ warn] [engine] failed to flush chunk '1-1721581994.699105145.flb', retry in 7 seconds: task_id=4, input=tail.0 > output=opentelemetry.1 (out_id=1)