Update open telemetry dependencies and switch to using the built in J…

…DK http client for reporting traces (#867) * Update otel versions and switch to JDK http sender Switch to using otel provided boms (Bill Of Materials) where possible as that is their supported model for version alignment see: https://arc.net/l/quote/uuvuaxoz Signed-off-by: Sam Barker <[email protected]> * Force the use of http/protobuf instead of GRPC Signed-off-by: Sam Barker <[email protected]> * Enable both grpc-upstream and JDK senders. With both available we are back to feature parity with the OkHttp sender Signed-off-by: Sam Barker <[email protected]> * Extract grpc version as a variable Signed-off-by: Sam Barker <[email protected]> * Add replacement semantic attributes where appropriate. Suppresses deprecation warnings as we need to mark them as deprecated to users before deleting them. Signed-off-by: Sam Barker <[email protected]> * Add changelog entry Signed-off-by: Sam Barker <[email protected]> * Document the attributes supported on spans Signed-off-by: Sam Barker <[email protected]> * Switch from `url.full` to `url.scheme`, `url.path` & `url.query`. So that the bridge complies with the HTTP server semantic conventions https://opentelemetry.io/docs/specs/semconv/http/http-spans/#http-server-semantic-conventions Signed-off-by: Sam Barker <[email protected]> * PR Feedback ensure otel logs are disabled. Signed-off-by: Sam Barker <[email protected]> * PR Feedback update span attribute documentation Signed-off-by: Sam Barker <[email protected]> * Update OpenTelemetry naming Co-authored-by: Paolo Patierno <[email protected]> Signed-off-by: Sam Barker <[email protected]> * move version properties into the group. Signed-off-by: Sam Barker <[email protected]> * PR Feedback: Drop exclusion of opentelemetry-extension-incubator from opentelemetry-exporter-otlp. Signed-off-by: Sam Barker <[email protected]> * Include the dependency changes in the changelog. Signed-off-by: Sam Barker <[email protected]> * PR Feedback update changelog. Signed-off-by: Sam Barker <[email protected]> * PR Feedback: add further comments covering deprecated properties Signed-off-by: Sam Barker <[email protected]> * PR feedback fix typo in CHANGELOG.md Co-authored-by: Paolo Patierno <[email protected]> Signed-off-by: Sam Barker <[email protected]> * PR Feedback: update metrics comment Signed-off-by: Sam Barker <[email protected]> * Whitespace change to re-trigger travis. Signed-off-by: Sam Barker <[email protected]> * yet another whitespace change to re-trigger travis-ci Signed-off-by: Sam Barker <[email protected]> --------- Signed-off-by: Sam Barker <[email protected]> Signed-off-by: Sam Barker <[email protected]> Co-authored-by: Paolo Patierno <[email protected]>
strimzi · Mar 6, 2024 · 825537b · 825537b
1 parent 30a932c
commit 825537b
Show file tree

Hide file tree

Showing 4 changed files with 85 additions and 17 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,9 +2,16 @@
 
 ## 0.28.0
 
-* Dependency updates (Kafka 3.7.0, Kubernetes configuration provider 1.1.2, Vert.x 4.5.4, Netty 4.1.107.Final, Jackson FasterXML 2.16.1, Micrometer 1.12.3, OAuth 0.15.0)
+* Dependency updates (Kafka 3.7.0, Kubernetes configuration provider 1.1.2, Vert.x 4.5.4, Netty 4.1.107.Final, Jackson FasterXML 2.16.1, Micrometer 1.12.3, OAuth 0.15.0, OpenTelemetry 1.34.1, OpenTelemetry Semconv 1.21.0-alpha, OpenTelemetry instrumentation 1.32.0-alpha)
 * Fixed missing messaging semantic attributes to the Kafka consumer spans
 * Introduced a new text embedded format to send and receive plain strings for record key and value.
+* Removed the dependency on OkHttp and thus Kotlin.
+* This release deprecates several attributes (inline with changes from OpenTelemetry Semconv) which it attaches to spans. Both the deprecated attribute and its replacement will be added to spans in the current release. The deprecated attributes will be removed in a future version.
+  1. `http.method` is being replaced with `http.request.method`
+  2. `http.url` is being replaced with `url.scheme`, `url.path` & `url.query`
+  3. `messaging.destination` is being replaced with `messaging.destination.name`
+  4. `http.status_code` is being replaced with `http.response.status_code`
+* The span attribute `messaging.destination.kind=topic` is deprecated, by OpenTelemetry Semconv, without a replacement. It will be removed in a future release of the Strimzi Kafka Bridge.
 
 ## 0.27.0
 

diff --git a/documentation/modules/proc-configuring-kafka-bridge-tracing.adoc b/documentation/modules/proc-configuring-kafka-bridge-tracing.adoc
@@ -95,6 +95,22 @@ OTEL_EXPORTER_ZIPKIN_ENDPOINT=http://localhost:9411/api/v2/spans <2>
 <1> The name of the tracing system. In this example, Zipkin is specified.
 <2> The endpoint of the specific selected exporter that listens for spans. In this example, a Zipkin endpoint is specified.
 
+== Supported Span attributes
+
+The Kafka Bridge adds, in addition to the standard OpenTelemetry attributes, the following attributes from the https://opentelemetry.io/docs/specs/semconv/http/http-spans/#http-server-semantic-conventions[OpenTelemetry standard conventions for HTTP] to its spans.
+[cols="1,1"]
+|===
+| Attribute key | Attribute value
+| `peer.service` | Hardcoded to `kafka`
+| `http.request.method` | The http method used to make the request
+| `url.scheme` | The https://www.rfc-editor.org/rfc/rfc3986#section-3.1[URI scheme] component
+| `url.path` | The https://www.rfc-editor.org/rfc/rfc3986#section-3.3[URI path] component
+| `url.query` | The https://www.rfc-editor.org/rfc/rfc3986#section-3.4[URI query] component
+| `messaging.destination.name` | The name of the Kafka topic being produced to or read from
+| `messaging.system` | Hardcoded to `kafka`
+| `http.response.status_code` | `ok` for http responses between 200 and 300. `error` for all other status codes
+|===
+
 [role="_additional-resources"]
 .Additional resources
 

diff --git a/pom.xml b/pom.xml
@@ -127,8 +127,11 @@
 		<spotbugs.version>4.7.3</spotbugs.version>
 		<maven.spotbugs.version>4.7.3.0</maven.spotbugs.version>
 		<strimzi-oauth.version>0.15.0</strimzi-oauth.version>
-		<opentelemetry.alpha-version>1.19.0-alpha</opentelemetry.alpha-version>
-		<opentelemetry.version>1.19.0</opentelemetry.version>
+		<opentelemetry.version>1.34.1</opentelemetry.version>
+		<opentelemetry-alpha.version>${opentelemetry.version}-alpha</opentelemetry-alpha.version>
+		<opentelemetry.instrumentation.version>1.32.0-alpha</opentelemetry.instrumentation.version>
+		<opentelemetry-semconv.version>1.21.0-alpha</opentelemetry-semconv.version>
+		<grpc-netty-shaded.version>1.61.0</grpc-netty-shaded.version>
 		<micrometer.version>1.12.3</micrometer.version>
 		<jmx-prometheus-collector.version>0.18.0</jmx-prometheus-collector.version>
 		<prometheus-simpleclient.version>0.16.0</prometheus-simpleclient.version>
@@ -221,7 +224,7 @@
 		<dependency>
 			<groupId>io.opentelemetry</groupId>
 			<artifactId>opentelemetry-sdk-extension-autoconfigure</artifactId>
-			<version>${opentelemetry.alpha-version}</version>
+			<version>${opentelemetry.version}</version>
 		</dependency>
 		<dependency>
 			<groupId>io.opentelemetry</groupId>
@@ -234,24 +237,48 @@
 			<version>${opentelemetry.version}</version>
 		</dependency>
 		<dependency>
-			<groupId>io.opentelemetry</groupId>
-			<artifactId>opentelemetry-semconv</artifactId>
-			<version>${opentelemetry.alpha-version}</version>
+			<groupId>io.opentelemetry.semconv</groupId>
+			<artifactId>opentelemetry-semconv</artifactId> <!-- semconv -> io.opentelemetry.semconv https://github.com/open-telemetry/opentelemetry-java/pull/5807/files -->
+			<version>${opentelemetry-semconv.version}</version>
 		</dependency>
 		<dependency>
 			<groupId>io.opentelemetry.instrumentation</groupId>
 			<artifactId>opentelemetry-kafka-clients-2.6</artifactId>
-			<version>${opentelemetry.alpha-version}</version>
+			<version>${opentelemetry.instrumentation.version}</version>
 		</dependency>
 		<dependency>
 			<groupId>io.opentelemetry.instrumentation</groupId>
 			<artifactId>opentelemetry-instrumentation-api-semconv</artifactId>
-			<version>${opentelemetry.alpha-version}</version>
+			<version>${opentelemetry.instrumentation.version}</version>
 		</dependency>
 		<dependency>
 			<groupId>io.opentelemetry</groupId>
 			<artifactId>opentelemetry-exporter-otlp</artifactId>
 			<version>${opentelemetry.version}</version>
+			<exclusions>
+				<exclusion>
+					<groupId>io.opentelemetry</groupId>
+					<artifactId>opentelemetry-exporter-sender-okhttp</artifactId>
+				</exclusion>
+			</exclusions>
+		</dependency>
+		<dependency>
+			<groupId>io.opentelemetry</groupId>
+			<artifactId>opentelemetry-exporter-sender-jdk</artifactId>
+			<version>${opentelemetry-alpha.version}</version>
+			<scope>runtime</scope>
+		</dependency>
+		<dependency>
+			<groupId>io.opentelemetry</groupId>
+			<artifactId>opentelemetry-exporter-sender-grpc-managed-channel</artifactId>
+			<version>${opentelemetry.version}</version>
+			<scope>runtime</scope>
+		</dependency>
+		<dependency>
+			<groupId>io.grpc</groupId>
+			<artifactId>grpc-netty-shaded</artifactId>
+			<version>${grpc-netty-shaded.version}</version>
+			<scope>runtime</scope>
 		</dependency>
 		<dependency>
 			<groupId>io.micrometer</groupId>
@@ -534,6 +561,12 @@
 									<ignoredUnusedDeclaredDependency>io.opentelemetry:opentelemetry-exporter-otlp</ignoredUnusedDeclaredDependency>
 									<!-- OpenTelemetry - Vert.x using old version and OpenTelemetry breaking API compatibility. See dependency declaration for details. -->
 									<ignoredUnusedDeclaredDependency>io.opentelemetry:opentelemetry-sdk-trace</ignoredUnusedDeclaredDependency>
+									<!-- OpenTelemetry - used via classpath configuration opentelemetry-exporter-sender-jdk is required at runtime to replace OKHTTP -->
+									<ignoredUnusedDeclaredDependency>io.opentelemetry:opentelemetry-exporter-sender-jdk</ignoredUnusedDeclaredDependency>
+									<!-- OpenTelemetry - used via classpath configuration opentelemetry-exporter-sender-grpc is required at runtime to replace OKHTTP -->
+									<ignoredUnusedDeclaredDependency>io.opentelemetry:opentelemetry-exporter-sender-grpc-managed-channel</ignoredUnusedDeclaredDependency>
+									<!-- OpenTelemetry - used via classpath configuration opentelemetry-exporter-sender-grpc is required at runtime to replace OKHTTP -->
+									<ignoredUnusedDeclaredDependency>io.grpc:grpc-netty-shaded:jar</ignoredUnusedDeclaredDependency>
 								</ignoredUnusedDeclaredDependencies>
 						</configuration>
 					</execution>

diff --git a/src/main/java/io/strimzi/kafka/bridge/tracing/OpenTelemetryHandle.java b/src/main/java/io/strimzi/kafka/bridge/tracing/OpenTelemetryHandle.java
@@ -18,9 +18,9 @@
 import io.opentelemetry.context.propagation.TextMapGetter;
 import io.opentelemetry.context.propagation.TextMapPropagator;
 import io.opentelemetry.instrumentation.api.instrumenter.messaging.MessageOperation;
-import io.opentelemetry.instrumentation.kafkaclients.TracingProducerInterceptor;
+import io.opentelemetry.instrumentation.kafkaclients.v2_6.TracingProducerInterceptor;
 import io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk;
-import io.opentelemetry.semconv.trace.attributes.SemanticAttributes;
+import io.opentelemetry.semconv.SemanticAttributes;
 import io.strimzi.kafka.bridge.config.BridgeConfig;
 import io.vertx.ext.web.RoutingContext;
 import org.apache.kafka.clients.consumer.ConsumerRecord;
@@ -46,10 +46,15 @@ class OpenTelemetryHandle implements TracingHandle {
 
     private Tracer tracer;
 
+    @SuppressWarnings("deprecation") //SemanticAttributes.{HTTP_METHOD, HTTP_URL} are deprecated we continue to use it here for backwards compatibility with existing deployments.
     static void setCommonAttributes(SpanBuilder builder, RoutingContext routingContext) {
         builder.setAttribute(SemanticAttributes.PEER_SERVICE, KAFKA_SERVICE);
-        builder.setAttribute(SemanticAttributes.HTTP_METHOD, routingContext.request().method().name());
-        builder.setAttribute(SemanticAttributes.HTTP_URL, routingContext.request().uri());
+        builder.setAttribute(SemanticAttributes.HTTP_METHOD, routingContext.request().method().name()); // TODO remove as part of https://github.com/strimzi/strimzi-kafka-bridge/issues/875
+        builder.setAttribute(SemanticAttributes.HTTP_REQUEST_METHOD, routingContext.request().method().name());
+        builder.setAttribute(SemanticAttributes.HTTP_URL, routingContext.request().uri()); // TODO remove as part of https://github.com/strimzi/strimzi-kafka-bridge/issues/875
+        builder.setAttribute(SemanticAttributes.URL_SCHEME, routingContext.request().scheme());
+        builder.setAttribute(SemanticAttributes.URL_PATH, routingContext.request().path());
+        builder.setAttribute(SemanticAttributes.URL_QUERY, routingContext.request().query());
     }
 
     @Override
@@ -64,7 +69,8 @@ public String serviceName(BridgeConfig config) {
 
     @Override
     public void initialize() {
-        System.setProperty("otel.metrics.exporter", "none"); // disable metrics
+        System.setProperty("otel.metrics.exporter", "none"); // disable otel metrics exporter. This has no effect on metrics in the bridge.
+        System.setProperty("otel.logs.exporter", "none"); // disable otel logs exporter. This has no effect on logging in the bridge.
         // TODO: to remove when Vert.x won't be used anymore and the ThreadLocalContextStorage could be used again
         System.setProperty("io.opentelemetry.context.contextStorageProvider", "io.strimzi.kafka.bridge.tracing.BridgeContextStorageProvider");
         AutoConfiguredOpenTelemetrySdk.initialize();
@@ -89,12 +95,14 @@ private SpanBuilder getSpanBuilder(RoutingContext routingContext, String operati
         return spanBuilder;
     }
 
+    @SuppressWarnings("deprecation") //SemanticAttributes.{MESSAGING_DESTINATION_NAME,MESSAGING_DESTINATION_KIND} are deprecated we continue to use it here for backwards compatibility with existing deployments.
     @Override
     public <K, V> void handleRecordSpan(ConsumerRecord<K, V> record) {
         String operationName = record.topic() + " " + MessageOperation.RECEIVE.name().toLowerCase(Locale.ROOT);
         SpanBuilder spanBuilder = get().spanBuilder(operationName);
         spanBuilder.setAttribute(SemanticAttributes.MESSAGING_DESTINATION, record.topic());
-        spanBuilder.setAttribute(SemanticAttributes.MESSAGING_DESTINATION_KIND, SemanticAttributes.MessagingDestinationKindValues.TOPIC);
+        spanBuilder.setAttribute(SemanticAttributes.MESSAGING_DESTINATION_NAME, record.topic()); // TODO remove as part of https://github.com/strimzi/strimzi-kafka-bridge/issues/875
+        spanBuilder.setAttribute(SemanticAttributes.MESSAGING_DESTINATION_KIND, SemanticAttributes.MessagingDestinationKindValues.TOPIC); // TODO remove as part of https://github.com/strimzi/strimzi-kafka-bridge/issues/875
         spanBuilder.setAttribute(SemanticAttributes.MESSAGING_SYSTEM, "kafka");
         Context parentContext = propagator().extract(Context.current(), TracingUtil.toHeaders(record), MG);
         if (parentContext != null) {
@@ -177,10 +185,12 @@ public void inject(RoutingContext routingContext) {
             propagator().inject(Context.current(), routingContext, (rc, key, value) -> rc.response().headers().add(key, value));
         }
 
+        @SuppressWarnings("deprecation") //SemanticAttributes.HTTP_STATUS_CODE is deprecated we continue to use it here for backwards compatibility with existing deployments.
         @Override
         public void finish(int code) {
             try {
-                span.setAttribute(SemanticAttributes.HTTP_STATUS_CODE, code);
+                span.setAttribute(SemanticAttributes.HTTP_STATUS_CODE, code); // TODO remove as part of https://github.com/strimzi/strimzi-kafka-bridge/issues/875
+                span.setAttribute(SemanticAttributes.HTTP_RESPONSE_STATUS_CODE, code);
                 // OK status is fine for all 2xx HTTP status codes
                 span.setStatus(code >= 200 && code < 300 ? StatusCode.OK : StatusCode.ERROR);
                 scope.close();
@@ -189,10 +199,12 @@ public void finish(int code) {
             }
         }
 
+        @SuppressWarnings("deprecation") //SemanticAttributes.HTTP_STATUS_CODE is deprecated we continue to use it here for backwards compatibility with existing deployments.
         @Override
         public void finish(int code, Throwable cause) {
             try {
-                span.setAttribute(SemanticAttributes.HTTP_STATUS_CODE, code);
+                span.setAttribute(SemanticAttributes.HTTP_STATUS_CODE, code); // TODO remove as part of https://github.com/strimzi/strimzi-kafka-bridge/issues/875
+                span.setAttribute(SemanticAttributes.HTTP_RESPONSE_STATUS_CODE, code);
                 span.setStatus(code == HttpResponseStatus.OK.code() ? StatusCode.OK : StatusCode.ERROR);
                 span.recordException(cause);
                 scope.close();