Modelling HTTP client response body capture

[pavolloffay]

What are you trying to achieve?

I would like to discuss the correct way of capturing HTTP client response body. When a response body is captured the HTTP client span is usually finished - the body can be read from a stream after a response object is returned.

The ideal situation would be to add the body to the client span, but the span is usually finished as mentioned above. If the client span is finished our instrumentation creates a new span with a link to the HTTP client span with a name e.g. additional-attributes and attaches the body and immediately finishes the span. The "additional-attributes" span could be started when the body reading starts to also capture the timing information for the body read operation.

An alternative approach could be to allow flushing of data after the span is ended. The OTEL design does not seem to pursue this.

Has anybody come across this use case?

The instrumentation is available in
https://github.com/hypertrace/javaagent. e.g. the jaxrs-client instrumentation.

Additional context.

Related to #857

[kenfinnigan]

Another use case I heard of just the other day is wanting to capture as an attribute of a span the uncompressed size of the response, but the span is closed at response receipt.

[Oberon00]

The ideal situation would be to add the body to the client span, but the span is usually finished as mentioned above.

Indeed. If you want to instrument the streaming of a HTTP response, you probably want to consider that in the instrumentation of your HTTP client. There is no reason to close the span (too) early. What might make sense, spec-wise, is to add an event semantic convention response_headers_received to the client span.

Alternatively, I would recommend starting a new span already as parent of the HTTP client span that would contain the HTTP client span as child, and the additional attributes. It makes more sense to me semantically than a child span (linked span makes no sense at all IMHO), as the "core client span" then becomes part of a multi-part operation that ends when the body is received.

[pavolloffay]

Indeed. If you want to instrument the streaming of a HTTP response, you probably want to consider that in the instrumentation of your HTTP client. There is no reason to close the span (too) early.

Almost all client instrumentations in https://github.com/open-telemetry/opentelemetry-java-instrumentation close the client span before the response body is read.

Alternatively, I would recommend starting a new span already as parent of the HTTP client span that would contain the HTTP client span as child, and the additional attributes.

This looks weirder to me and more importantly harder to achieve with the current OTEL agent. We are extending https://github.com/open-telemetry/opentelemetry-java-instrumentation by adding instrumentations that capture bodies. The span lifecycle is managed by OTEL and the additional instrumentation just attaches more data to the respective span.