Exporting and keeping data

Play this lesson

Where does it all go?

Shard Garden's telemetry now comes from match servers, the save service, phones and browsers. Priya, who runs observability, has two new questions.

First: if the studio switches to a different storage tool next year, does every service need new code? Second: logs hold player IDs. How long may the studio keep them? One is about plumbing, the other about the law.

One protocol: OTLP

The OpenTelemetry Protocol (OTLP) is a general-purpose way to deliver telemetry. It runs over gRPC, on default port 4317, or over HTTP, on default port 4318.

Over HTTP, data is sent with POST to /v1/traces, /v1/metrics or /v1/logs, as Protobuf, either binary or JSON. Shard Garden's code speaks this one protocol, whatever tool stores the data in the end.

Send it right

The studio's collector accepts OTLP over HTTP at collector:4318. Where does Remy's match server send its spans?

  1. collector:4318/v1/traces
  2. collector:4317/v1/spans
  3. collector:4318/v1/logs
Show the answer

collector:4318/v1/traces

Spans belong to traces, and OTLP over HTTP sends traces to /v1/traces. 4317 is the gRPC port, and /v1/logs is for logs.

The Collector

The OpenTelemetry Collector is a vendor-agnostic program that receives, processes and exports telemetry. Its pipeline has receivers (data comes in), processors (data is changed, filtered or sampled) and exporters (data goes out to storage tools).

Each server hands its data quickly to a nearby collector, which handles retries, batching and encryption. To switch storage tools, Priya changes an exporter in the collector's config. The game code stays the same.

When the receiver is overwhelmed

During an outage, telemetry spikes just when the storage side is busiest. OTLP has rules for this. A server that can't keep up should answer 429 Too Many Requests or 503 Service Unavailable.

The client must then slow itself down: wait as long as the Retry-After header says, or, if there is none, use exponential backoff, waiting longer after each failed try.

Quick check

The collector answers 503 with Retry-After: 30. The match server should retry at once, again and again, so no data is lost.

Show the answer

False

OTLP says the client must throttle itself and honour Retry-After, here 30 seconds. Retrying at once would add load to a system that is already struggling.

How long to keep it

Prometheus keeps its local data for 15 days by default. Its docs warn that local storage is not clustered or replicated, so it won't survive a failed drive or server. For the long term it can write to a remote store. A sample takes only 1 to 2 bytes on average.

Priya's plan: metrics are cheap and hold no personal data, so keep them for months. Traces and logs are bigger and hold player IDs, so keep them for days or weeks.

The law: storage limitation

Data protection laws, like the UK GDPR, say personal data must be kept no longer than necessary: the storage limitation principle. The UK regulator, the ICO, says to justify each period, write down standard retention periods, review them, and erase or anonymise data that is no longer needed.

OWASP adds the other side: logs must not be destroyed before their required retention period, and must not be kept beyond it. Rules differ by country, so the studio checks with a lawyer.

Policy pick

Which retention plan fits Shard Garden best?

  1. Metrics for 13 months, traces for 14 days, logs with player IDs for 30 days, all deleted automatically
  2. Everything forever, in case it is useful one day
  3. Nothing at all, so there is no data to protect
Show the answer

Metrics for 13 months, traces for 14 days, logs with player IDs for 30 days, all deleted automatically

Each kind is kept as long as it is useful and no longer, and deletion is automatic, so nobody has to remember. Forever breaks storage limitation, and keeping nothing leaves the team blind.

Seeing inside the game

Shard Garden now has structured logs, metrics, sampled traces, dashboards built on SLOs, budget alerts, on-call runbooks, readable crashes and RUM, all flowing through one protocol and kept only as long as needed.

Bolt says: Boss time: one quiz across the whole Observe stage.

Fact-checked 11 October 2026