Otel implementation approach for Themis.

[utsavbatra5]

We have added otel tracing in Themis as per the approach in xmidt-org/tr1d1um#202 .

We have added the same in themis/xhttp/xhttpserver/unmarshal.go as per the code below. We have also added the application name as configuration in talaria.yaml for the same.
( # name of the application
name: "themis")

Kindly review the code logic below and let us know if the approach seems fine. We have tested the same and the traceID and spanID are being generated as expected. The same code in xhttp/xhttpserver/unmarshal.go would work for argus as well.

package xhttpserver

import (
"fmt"
"net/http"
"os"
....................................................................
....................................................................

func (u Unmarshal) name() string {
if len(u.Name) > 0 {
return u.Name
}

return u.Key

}

**type firstNodeTraceEchoer struct {
propagator propagation.TextMapPropagator
}

type application struct {
Name string
TracingConfigKey string
}

// EchoStartNodeTrace should be run after the middleware that starts the main span of this application.
// Assuming this is the first node of the entire trace, the trace information is echoed through the
// HTTP response.
func (f firstNodeTraceEchoer) EchoStartNodeTrace(delegate http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r http.Request) {
ctx := f.propagator.Extract(r.Context(), propagation.HeaderCarrier(r.Header))
rsc := trace.RemoteSpanContextFromContext(ctx)
if !rsc.IsValid() {
sc := trace.SpanContextFromContext(ctx)
w.Header().Set("X-Midt-Span-ID", sc.SpanID().String())
w.Header().Set("X-Midt-Trace-ID", sc.TraceID().String())
}
delegate.ServeHTTP(w, r.WithContext(ctx))
})
}*

// Provide unmarshals a server using the Key field and creates a *mux.Router which is the root handler for
// that server's requests. This *mux.Router will be decorated with the constructors from NewServerChain as well
// as any ChainFactory's constructors.
func (u Unmarshal) Provide(in ServerIn) (*mux.Router, error) {
if !in.Unmarshaller.IsSet(u.Key) {
if !u.Optional {
return nil, ServerNotConfiguredError{Key: u.Key}
}

	return nil, nil
}

....................................................................
....................................................................

if in.ChainFactory != nil {
	more, err := in.ChainFactory.New(serverName, o)
	if err != nil {
		return nil, err
	}

	serverChain = serverChain.Extend(more)
}

**var application application

_ = in.Unmarshaller.UnmarshalKey("application", &application)

config := &candlelight.Config{
	ApplicationName: application.Name,
}

_ = in.Unmarshaller.UnmarshalKey(application.TracingConfigKey, config)

traceProvider, err := candlelight.ConfigureTracerProvider(*config)
if err != nil {
	fmt.Fprintf(os.Stderr, "Unable to build traceProvider: %s\n", err.Error())
}

propagator := propagation.TraceContext{}
traceMiddleware := firstNodeTraceEchoer{propagator: propagator}
rootRouter := mux.NewRouter()
otelMuxOptions := []otelmux.Option{
	otelmux.WithPropagators(propagator),
	otelmux.WithTracerProvider(traceProvider),
}
rootRouter.Use(otelmux.Middleware("mainSpan", otelMuxOptions...), traceMiddleware.EchoStartNodeTrace)**

var (
	router = rootRouter
	server = New(
		o,
		serverLogger,
		serverChain.Extend(u.Chain).Then(router),
	)
)

in.Lifecycle.Append(fx.Hook{
	OnStart: OnStart(o, server, serverLogger, func() { in.Shutdowner.Shutdown() }),
	OnStop:  OnStop(server, serverLogger),
})

return router, nil

}

// Annotated is like Unmarshal, save that it emits a named *mux.Router. This method is appropriate
// for applications with multiple servers. The name of the returned *mux.Router is either the Name field (if set)
// or the Key field (if Name is empty).
func (u Unmarshal) Annotated() fx.Annotated {
return fx.Annotated{
Name: u.name(),
Target: u.Provide,
}
}

[joe94]

@utsavbatra5, how critical is it for you guys to add tracing to Themis as part of the 7 services you mentioned on #43?

We will soon be making some large refactor to Themis and adding tracing to it might add complexity we're trying to avoid. I will try to post the GitHub issues here with the refactor details.

Also, I'm assuming that tracing will be most helpful to you for the path Tr1d1um => Scytale => Petasos => Talaria for API requests and Talaria => Caduceus for events

[utsavbatra5]

Hi Joe, Is it possible to add tracing capabilities in the current version as it is critical to our deployed architecture. We have enabled remote server in themis and hence also its quite important. You can incorporate the refractor in the new version of themis without tracing capabilities and we could then later on check to incorporate tracing in it as well.

Kindly advise on the above suggested themis design approach for otel.

[joe94]

Hey @utsavbatra5, I spoke with the team today and they gave the green light to adding tracing to Themis. I'm wrapping up a few changes for the OpenTelemetry integration in Scytale and I will look at Themis next and get back to you.

[joe94]

Hey @utsavbatra5, I finished up releasing the new versions for Tr1d1um and Scytale yesterday. I'm taking a look at Themis today. Since you're working on Petasos and Caduceus already, I'm gonna assign myself Themis and focus on that.