Response time and streaming

Play this lesson

Waiting for words

Testers tell Kai the Garden Guide feels slow. A model writes its answer a piece at a time. Without streaming, the player watches a spinner until the whole answer is ready. With streaming, words appear as they are written.

How it works

Anthropic's docs: set "stream": true "to incrementally stream the response using server-sent events (SSE)". The connection stays open and each new piece arrives as soon as it is written.

Quick check

With streaming, when does the player see the first words?

  1. As soon as the model writes them
  2. After the whole answer is finished
  3. Only on fast phones
Show the answer

As soon as the model writes them

Each piece is sent the moment it exists.

Two numbers to watch

Shorter is faster

Every word takes time to write. Shard Garden's prompt asks for answers under 60 words, and a limit on output tokens stops long ones. Short answers arrive sooner and cost less.

Read the dashboard

Garden Guide, p90
time to first word  0.6 stotal time          9.0 s

Priya's dashboard shows this, and players say answers feel slow. What should Remy change first?

  1. Shorter answers
  2. Turning off streaming
  3. A bigger spinner
Show the answer

Shorter answers

The first words come fast. The long wait is the length of the answer.

A Stop button

Players can tap Stop. The studio cancels the request too, so it doesn't pay for words nobody will read.

Quick check

Streaming makes the model write its whole answer faster.

Show the answer

False

The total time is the same. The player just starts reading sooner.

The same question, a thousand times

Many players ask the same things.

Bolt says: Next: caching AI answers.

Fact-checked 12 October 2026