Estimate project
← All posts

Passing the OpenTelemetry context along with Virtual Threads & Structured Concurrency

Learn how to pass the OpenTelemetry context along when working with Virtual Threads & Structured Concurrency.

7 min read
JavaOpenTelemetryObservabilityVirtual ThreadsStructured Concurrency

In OpenTelemetry, the Context is a key element: it holds the metadata of the current invocation chain, for example the trace that is currently running. Thanks to it, several spans can be tied together into one logical unit.

By default, the Java OpenTelemetry SDK passes the Context around via ThreadLocals. Does that still behave as expected with the Virtual Threads and Structured Concurrency of Java 21+? And if it doesn’t, what goes wrong?

Sequential invocations

We’ll start with a simple scenario of sequential invocations, each of which starts a span. There are no threads in play yet, but it will serve as a useful baseline.

In the example below, a parent method starts a parent span and calls two nested methods, each starting a nested span of its own. This is the code:

import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk;

import java.util.concurrent.ExecutionException;

public class Demo {
  private static Void nested(int id, Tracer tracer) throws InterruptedException {
    var span = tracer.spanBuilder("nested-test-span-" + id).startSpan();
    try (var scope = span.makeCurrent()) {
      Thread.sleep(500);
      // pretending to do some work—e.g. sending an HTTP request
    } finally {
      span.end();
    }

    return null;
  }

  private static void parent(Tracer tracer) throws InterruptedException {
    var span = tracer.spanBuilder("parent-test-span").startSpan();
    try (var scope = span.makeCurrent()) {
      nested(1, tracer);
      nested(2, tracer);
    } finally {
      span.end();
    }
  }

  public static void main(String[] args) throws InterruptedException, ExecutionException {
    var otel = AutoConfiguredOpenTelemetrySdk.initialize();

    // span
    var tracer = otel.getOpenTelemetrySdk().getTracer("otel-demo");
    parent(tracer);

    // make sure telemetry is sent
    Thread.sleep(1000);
  }
}

The main method initializes OpenTelemetry from environment variables. These are handy for testing:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
OTEL_METRIC_EXPORT_INTERVAL=500
OTEL_SERVICE_NAME=demo

The exporter endpoint needs to point at a running instance – for example Grafana’s LGTM, a simple single-container Docker setup. Here is the corresponding docker-compose.yml:

services:
  # OpenTelemetry Collector, Prometheus, Loki, Tempo, Grafana
  observability:
    image: 'grafana/otel-lgtm'
    ports:
      - '3000:3000' # Grafana's UI
      - '4317:4317' # Exporter

This gives us an exporter as well as a UI that makes it easy to visualize the test results.

The parent method starts a span and calls nested twice, each call starting a child span. Once you run the example, the UI should show something like this:

So far, everything works. Now let’s introduce some parallelism.

Parallel invocations

When the nested invocations don’t depend on each other and take a significant amount of time (for instance, because they do some I/O), we may want to run them in parallel. This can be done in several ways, but we’ll follow the path laid out by the Java structured concurrency JEP (available in preview since Java 21).

Here, we’ll simply use the StructuredTaskScope API. The essential property of structured concurrency is that the lifetime of the threads executing the subtasks is determined by the syntactic structure of the code.

In other words, because the StructuredTaskScope sits inside the try (var scope = span.makeCurrent()), every thread started in the task scope is guaranteed to have finished by the time the observability scope & span are closed:

private static Void nested(int id, Tracer tracer) throws InterruptedException {
  var span = tracer.spanBuilder("nested-test-span-" + id).startSpan();

  try (var scope = span.makeCurrent()) {
    Thread.sleep(500);
  } finally {
    span.end();
  }

  return null;
}

private static void parent(Tracer tracer) throws InterruptedException, ExecutionException {
  var span = tracer.spanBuilder("parent-test-span").startSpan();
  try (var scope = span.makeCurrent()) {
    try (var taskScope = new StructuredTaskScope.ShutdownOnFailure()) {
      Supplier<Void> n1 = taskScope.fork(() -> nested(1, tracer));
      Supplier<Void> n2 = taskScope.fork(() -> nested(2, tracer));

      taskScope.join().throwIfFailed();

      n1.get();
      n2.get();
    }
  } finally {
    span.end();
  }
}

The code above launches two tasks in the background and then waits for both to finish with .join(). It resembles working with futures, but adds a guarantee: if one task fails, the other isn’t left “dangling” but gets interrupted.

When we run this example, though, the observability context is no longer passed along. Rather than one correlated trace, we end up with three separate traces:

Thanks to parallelization, the parent’s span now takes 500ms in total, but we’ve lost observability into the process as a whole.

Parallel invocations, with observability

What’s going wrong? Recall that the OpenTelemetry Context travels via ThreadLocals. That’s perfect for correlating invocations and spans inside one thread, but it falls apart once several threads are involved – whether “normal” ones (i.e. “platform” threads) or virtual ones. The reason is that ThreadLocals are not inherited.

OpenTelemetry attempts to address this in a few ways. You can, for instance, wrap an Executor with Context.taskWrapping(), so that the context is passed to every task launched through that executor. For structured concurrency, however, no such utilities exist (yet) – and with good reason, since the API is still in preview.

So we need to implement the propagation ourselves. One option is to supply a ThreadFactory when constructing the StructuredTaskScope. A custom factory like that can take care of passing the context along manually – in effect, we hand-implement inheritance of the thread-local value that holds the context:

static class PropagatingVirtualThreadFactory implements ThreadFactory {
  private ThreadFactory delegate = Thread.ofVirtual().factory();
  @Override
  public Thread newThread(Runnable r) {
    var parentContext = Context.current();
    return delegate.newThread(() -> {
      parentContext.makeCurrent();
      r.run();
    });
  }
}

Our custom factory first grabs the current context with Context.current() at the moment a new thread is created. It then creates a new virtual thread (delegating to the JDK-provided Thread.ofVirtual().factory()) and, before executing the code block, makes the captured context “current” – this time inside the new thread.

Only minimal changes to the parent method are needed:

private static void parent(Tracer tracer) throws InterruptedException, ExecutionException {
  var span = tracer.spanBuilder("parent-test-span-1").startSpan();
  try (var scope = span.makeCurrent()) {
    try (var taskScope = new StructuredTaskScope.ShutdownOnFailure(null, 
          new PropagatingVirtualThreadFactory())) {
      Supplier<Void> n1 = taskScope.fork(() -> nested(1, tracer));
      Supplier<Void> n2 = taskScope.fork(() -> nested(2, tracer));

      taskScope.join().throwIfFailed();

      n1.get();
      n2.get();
    }
  } finally {
    span.end();
  }
}

Running the example now shows the spans correlated within one trace again:

What about ScopedValues?

Together with the structured concurrency JEP, Java also brought in the ScopedValue API (see the JEP) – a next-generation ThreadLocal that is inherited by child virtual threads when they are created.

Scoped values are meant to be used in a structured manner, together with StructuredTaskScopes. Here’s an example taken from the JEP:

private static final ScopedValue<String> X = ScopedValue.newInstance();

void foo() {
  where(X, "hello").run(() -> bar());
}

void bar() {
  System.out.println(X.get()); // prints hello
  where(X, "goodbye").run(() -> baz());
  System.out.println(X.get()); // prints hello
}

void baz() {
  System.out.println(X.get()); // prints goodbye
}

Keep in mind that a ScopedValue can be given a new value only while the code block passed via where(...).run(...) is being evaluated. Again, the lifetime is dictated by the syntactic structure of the code – this time the lifetime of the value binding, whereas earlier it was that of the thread.

This design has some appealing properties, but it doesn’t lend itself to implementing an alternative OpenTelemetry ContextStorage. Implementations of that interface store and propagate the OpenTelemetry context, and, as noted earlier, ThreadLocalContextStorage is the default.

Writing a ScopedValueContextStorage implementation, so that the context gets passed along automatically to newly created child virtual threads, might seem attractive; the context storage API, however, is unstructured:

public interface ContextStorage {
  Scope attach(Context toAttach);
  Context current();
}

A context attached to a storage doesn’t have to live in line with the code’s structure (even though that’s typically what happens with the try-with-resources construct). The attach method returns a Scope instead, and the code may close it at any point it likes.

Because of this design, scoped values and context storage are fundamentally at odds.

In summary

“Out of the box”, the OpenTelemetry Java SDK offers no integration with Virtual Threads beyond passing the context along within one thread.

Virtual Threads, however, are designed to encourage freely creating new forks that parallelize code execution and boost the performance / throughput of the services involved. As a consequence, the telemetry context gets lost, and spans that aren’t correlated are reported as separate traces rather than one combined trace.

Propagating the context manually prevents this. In its simplest form, you capture the parent context before forking a computation and set it again in the child thread. A custom ThreadFactory, which also works with the StructuredTaskScope API, lets you automate this propagation to some degree.

Lastly, unless the design of ContextStorage changes, it looks impossible to build an implementation based on the upcoming ScopedValues. They do solve the inheritance problem, but they also require structured usage that has to match the syntactic structure of the code – which clashes with the way ContextStorage works.