Traces and spans
Nine seconds to join
Some players wait nine seconds after tapping "Join garden". The tap travels through four services: the gateway that receives it, the matchmaker, the garden service and the save database.
Each writes its own logs. But which of millions of lines belong to this one tap? Priya, who runs observability, needs the whole journey of a single request in one picture.
Traces and spans
In OpenTelemetry, a trace records the path taken by a single request. It is made of spans, each one unit of work, like "matchmaker finds a server".
A span holds a name, a start and end time, its parent span's ID, attributes, events and a status. The first span, covering the whole operation, has no parent: it is the root span. Every other span hangs under the one that started it, forming a tree.
Quick check
In a trace, the root span is the one with no parent span ID.
- True
- False
Show the answer
True
The root span covers the whole operation, like the full "Join garden" request, so nothing started it. Every other span names its parent.
Passing the trace along
Each span carries a span context: the trace ID shared by every span of the request, its own span ID, trace flags and trace state.
When the gateway calls the matchmaker, it must pass the context along, or the trace breaks in two. This is propagation. OpenTelemetry's default uses the W3C Trace Context standard, a header on every call:
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
Reading traceparent
- 00The version.
- 4bf92f35...0e4736The trace ID: 16 bytes, written as 32 hex characters.
- 00f067aa0ba902b7The parent ID, the span that made this call: 8 bytes, as 16 hex characters.
- 01The flags. The last bit set means sampled: the caller may have recorded this trace.
The W3C, the web's standards body, published it as a Recommendation in 2020 and updated it in 2021.
Read the header
1002 (version)30af7651916cd43dd48448eb211c80319c5 (trace ID, 32 chars)6b7ad6b71692033317 (parent ID)8019 (flags)
Which part must the garden service copy into every span it creates for this request?
- The trace ID, 0af7...319c
- The parent ID, b7ad...3331
- The flags, 01
Show the answer
The trace ID, 0af7...319c
Every span of one request shares the trace ID, so the 32-character part is copied. b7ad6b7169203331 is the caller's span, which becomes the parent of the garden service's new span.
Read the trace
1join-garden 9.1 s2 gateway 0.1 s3 matchmaker 8.6 s4 find-region 0.2 s5 wait-for-server 8.3 s6 garden-service 0.3 s
Where should the team look first?
- wait-for-server, inside the matchmaker
- The gateway
- garden-service
Show the answer
wait-for-server, inside the matchmaker
wait-for-server takes 8.3 of the 9.1 seconds. The matchmaker's total is big only because it contains that wait. Maybe no match server had room, which a gauge of free slots would show.
Kinds, status and events
Every span has a kind: Client, Server, Internal, Producer or Consumer, so tools know who called whom. Its status is Unset by default, or Error, or Ok.
Extra details go in two places. OpenTelemetry's rule: if the moment it happened matters, add a span event, like "save retried" at 2.4 s. If not, add an attribute, like garden.players=4.
Quick check
During a save span, the first try fails and the save is retried. How should Priya record the retry?
- As a span event, because the moment it happened matters
- As an attribute, because every detail is an attribute
- As a separate trace
Show the answer
As a span event, because the moment it happened matters
OpenTelemetry says to use a span event when the timestamp is meaningful. Seeing exactly when the retry happened shows how much of the span it cost.
Every request, every span?
One join makes six spans. Thousands of joins a second make far too many to keep them all.
Bolt says: Next: sampling, keeping the traces that matter.
Fact-checked 11 October 2026
- OpenTelemetry: Traces opentelemetry.io
- OpenTelemetry: Context propagation opentelemetry.io
- W3C: Trace Context w3.org