New Blog Post: Collecting file-based Java logs with OpenTelemetry #5600
Conversation
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Show resolved
Hide resolved
|
@cyrille-leclerc https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/tag/v2.10.0 was just released - I'll update the executable example in grafana/docker-otel-lgtm#145 now |
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
|
@cyrille-leclerc can you raise an issue for this PR following our guidelines? Thanks |
Thanks @svrnm. Please find the blog post issue: |
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
|
|
||
|  | ||
|
|
||
| ## Configure Java Application to Output OTLP/JSON Logs |
There was a problem hiding this comment.
the sample application you are using, is this available somewhere? If so, can you point to it? It's always good if people can reproduce your tutorial 1:1 without the need for them to find an application first
There was a problem hiding this comment.
@zeitlinger do we have a simplified demo or we use https://github.com/cyrille-leclerc/my-shopping-cart/? Note that I reverted my-shopping-cart to the otlp exporter so I would have to spend a few cycles to setup a demo.
|  | ||
|
|
||
| Source: [https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a) |
There was a problem hiding this comment.
I like that you provide the link to the otelbin source of that image, but let's figure out a way to visualize this better.
There was a problem hiding this comment.
|  | |
| Source: [https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a) | |
| {{< figure class="figure" src="otel-collector-otlpjson-pipeline.png" attr="View OTel Collector pipeline with OTelBin" attrlink="https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a" >}} |
What I meant is to have something that makes it clearer that the Source: (or View ...) belongs to the image, since the spacing is not right, similar to what bootstrap provides for figures. We have a shortcode for that via docsy, it's not exactly what I want, but we can take a look into that separately, cc @chalin
There was a problem hiding this comment.
Has this been addressed to your satisfaction @svrnm? See, https://deploy-preview-5600--opentelemetry.netlify.app/blog/2024/collecting-file-based-java-logs-with-opentelemetry/#configure-the-opentelemetry-collector-to-ingest-the-otlpjson-logs.
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
|
|
||
|  | ||
|
|
||
| ## Configure Java Application to Output OTLP/JSON Logs |
There was a problem hiding this comment.
@zeitlinger do we have a simplified demo or we use https://github.com/cyrille-leclerc/my-shopping-cart/? Note that I reverted my-shopping-cart to the otlp exporter so I would have to spend a few cycles to setup a demo.
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Show resolved
Hide resolved
chalin
force-pushed
the
collect-java-otlpjson-logs
branch
from
72614b9
to
e764e3b
Compare
| ## Reference architecture | ||
|
|
||
| The deployment architecture looks like the following: |
There was a problem hiding this comment.
I don't think that we need a section heading here, you're only introducing the diagram.
Also, this isn't a "reference architecture", nor even a "deployment architecture". Prefer the use of the simpler term "deployment diagram". Actually, if you just present the diagram, you don't really need to name it, but if you do (e.g., via the "alt" text in the image element below, then avoid the use of "architecture".
| ## Reference architecture | |
| The deployment architecture looks like the following: |
|
|
||
| The deployment architecture looks like the following: | ||
|
|
||
|  |
There was a problem hiding this comment.
This diagram is really busy, and the content too small. I'd suggest the following improvements:
- Remove the following text from the diagram and add it instead as prose to this section (before or after the image):
- "Logs contextualized with OTel metadata including: ..."
- "experimental-otlp/stout" exporter Available in Java Agent v2.10"
- Move the "Observability Platform" block to be below the rest so that the image becomes less wide -- that is, favor vertical image real estate rather than horizontal. Or, if you prefer, put the app at the top of the diagram with the OTel components below. Something (very roughly) like what I show below.
There was a problem hiding this comment.
What do you think of this style?
| <!-- prettier-ignore-start --> | ||
|
|
||
| ```json | ||
| {"resourceLogs":[{"resource":{"attributes":[{"key":"deployment.environment.name","value":{"stringValue":"staging"}},{"key":"service.instance.id","value":{"stringValue":"6ad88e10-238c-4fb7-bf97-38df19053366"}},{"key":"service.name","value":{"stringValue":"checkout"}},{"key":"service.namespace","value":{"stringValue":"shop"}},{"key":"service.version","value":{"stringValue":"1.1"}}]},"scopeLogs":[{"scope":{"name":"com.mycompany.checkout.CheckoutServiceServer$CheckoutServiceImpl","attributes":[]},"logRecords":[{"timeUnixNano":"1730435085776869000","observedTimeUnixNano":"1730435085776944000","severityNumber":9,"severityText":"INFO","body":{"stringValue":"Order order-12035 successfully placed"}, "attributes":[{"key":"customerId","value":{"stringValue":"customer-49"}},{"key":"thread.id","value":{"intValue":"44"}},{"key":"thread.name","value":{"stringValue":"grpc-default-executor-1"}}],"flags":1,"traceId":"42de1f0dd124e27619a9f3c10bccac1c","spanId":"270984d03e94bb8b"}]}],"schemaUrl":"https://opentelemetry.io/schemas/1.24.0"}]} | ||
| ``` | ||
|
|
||
| <!-- prettier-ignore-end --> |
There was a problem hiding this comment.
This JSON output is unreadable. Let Prettier format it, or format it manually. If it is too big, then wrap it in a <details> element.
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
| traces and metrics. | ||
|
|
||
| Any feedback or questions? Reach out on | ||
| [GitHub](https://github.com/open-telemetry/opentelemetry-specification/pull/4183). |
There was a problem hiding this comment.
Why is this linking to a spec PR?
Is this PR really the best place for folks to comment?
If so, that's ok, but then the link text should explain more clearly what the link target is.
|
/fix:refcache |
|
You triggered fix:refcache action run at https://github.com/open-telemetry/opentelemetry.io/actions/runs/11931927870 |
|
IMPORTANT: (RE-)RUN
|
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
…telemetry/index.md
…telemetry/index.md
…telemetry/index.md
Co-authored-by: Severin Neumann <[email protected]> Co-authored-by: Tiffany Hrabusa <[email protected]>
…telemetry/index.md Co-authored-by: Severin Neumann <[email protected]>
Co-authored-by: Patrice Chalin <[email protected]>
Co-authored-by: Tiffany Hrabusa <[email protected]>
chalin
force-pushed
the
collect-java-otlpjson-logs
branch
from
bd35fc6
to
0f04576
Compare
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
|
/fix:format |
|
You triggered fix:format action run at https://github.com/open-telemetry/opentelemetry.io/actions/runs/11956865359 |
|
IMPORTANT: (RE-)RUN
|
Co-authored-by: Patrice Chalin <[email protected]>
add link to reference setup
content/en/blog/2024/collecting-file-based-java-logs-with-opentelemetry/index.md
Outdated
Show resolved
Hide resolved
…telemetry/index.md Co-authored-by: Gregor Zeitlinger <[email protected]>
This post presents how to use the new
experimental-otlp/stdoutexport format introduced by OpenTelemetry Java Instrumentation (open-telemetry/opentelemetry-java#6675) to export application logs to files with full OpenTelemetry contextualization and ingest them with the OTel Collector File Log receiver and OTLP/JSON connector.In this post, we explain how to configure the OTel Java instrumentation agent and the OTel Collector, including when deploying on K8s, to implement the end to end file based logs ingestion pipeline.
Blog Post Issue: #5606
SIG: java and specifications
Sponsor:
Preview: https://deploy-preview-5600--opentelemetry.netlify.app/blog/2024/collecting-file-based-java-logs-with-opentelemetry/