Open models and APIs

Play this lesson

Rent or run

Theo wants a model to sort every bug report, all day. A studio can call a model through an API, paying per token, or run an open model on its own servers, paying for the machines.

The trade-offs

Quick check

Theo compares the bills. Which costs money even when nothing is being asked?

  1. Servers running an open model
  2. An API paid per token
  3. Neither
Show the answer

Servers running an open model

Servers cost money every hour they are on. An API charges only when it is used.

Read the licence

"Open" doesn't mean "no rules". Meta's Llama 3 licence says that a company whose products had more than 700 million monthly active users "must request a license from Meta". Anyone shipping it must show "Built with Meta Llama 3".

Read the licence

Shard Garden uses Llama 3 in a feature. What does the licence ask it to show?

  1. "Built with Meta Llama 3"
  2. Nothing
  3. A Meta logo on every screen
Show the answer

"Built with Meta Llama 3"

The licence asks for that line to be shown prominently, for example on a website or about page.

Use both

Theo does the sums for sorting bug reports. By API: about US$300 a month, rising with every report. A small open model on one rented server: about US$400 a month however many reports, plus Remy's time to look after it. Below the break-even the API is cheaper; well above it, the server wins. So Shard Garden uses both: an API for the Garden Guide, where quality matters most, and its own server once bug reports pass the break-even.

Quick check

A year later, bug reports have tripled: the API would cost about US$900 a month. The server is still about US$400. What should Theo do?

  1. Move bug sorting to the server, counting Remy's time
  2. Stay on the API: it is always cheaper
  3. Stop sorting bug reports
Show the answer

Move bug sorting to the server, counting Remy's time

Past the break-even, a fixed monthly cost beats paying per token. The server still needs looking after, so that time goes in the sum.

Keep a way out

Shard Garden's code calls every model through one small module. Switching provider means changing one file and running the eval, not rewriting the game.

Quick check

Why route every model call through one module?

  1. So the studio can switch models by changing one place
  2. To make calls slower
  3. Because APIs require it
Show the answer

So the studio can switch models by changing one place

One place to change, one eval to run. A retired model is an afternoon's work, not a rewrite.

AI on the team

AI doesn't only help players. It helps the studio's own people.

Bolt says: Next: AI agents across the team.

Fact-checked 12 October 2026