AI features in production

Play this lesson

The Garden Guide

Kai wants an AI helper in the next update: the Garden Guide, answering players' questions about the game, like how to grow a seed or where the greenhouse is. Remy builds a first version in a week. Then Kai asks the real question: with a million players, it answers thousands of questions a minute, and every answer speaks for the studio. What has to be true before it goes live?

What production means

Quick check

In testing, Remy's Garden Guide tells a player the wrong greenhouse price. If it shipped like that, who would be responsible?

  1. The studio: the guide speaks for Shard Garden
  2. The AI model
  3. The player, for asking
Show the answer

The studio: the guide speaks for Shard Garden

Players can't tell which answers came from an AI. The studio answers for all of them.

True story: the chatbot's promise

In November 2022, after their grandmother died, Jake Moffatt asked Air Canada's website chatbot about bereavement fares. It said they could apply after travelling, "within 90 days of the date your ticket was issued". Air Canada's own policy page said the opposite.

Air Canada argued it could not be held liable for the chatbot. British Columbia's Civil Resolution Tribunal disagreed on 14 February 2024: "It makes no difference whether the information comes from a static page or a chatbot." Air Canada had to pay C$812.02.

Source: Civil Resolution Tribunal: Moffatt v. Air Canada, 2024 BCCRT 149 (14 Feb 2024), decisions.civilresolutionbc.ca

Quick check

What did the tribunal decide?

  1. Air Canada is responsible for what its chatbot says
  2. The chatbot is responsible for itself
  3. Chatbot answers never count
Show the answer

Air Canada is responsible for what its chatbot says

The tribunal: "it is responsible for all the information on its website."

Answer from the game's own facts

The Garden Guide answers from the game's own data: prices, recipes and rules come from the same files the game uses. If the answer isn't there, it says so and links the help page.

Pick the answer

A player asks about a feature that doesn't exist. What should the Garden Guide say?

  1. "I don't know that one. Here's the help page."
  2. A likely-sounding answer it makes up
  3. Nothing at all
Show the answer

"I don't know that one. Here's the help page."

A made-up answer is a promise the studio can't keep. "I don't know" is honest and safe.

Start small

Kai launches the Garden Guide to 1% of players first, with a button to report a bad answer. Priya reads every report each morning, and the guide reaches more players only while reports stay under the studio's line: 3 for every 1,000 answers.

Quick check

An AI feature should launch to every player on its first day.

Show the answer

False

A small launch shows problems while few players see them.

Testing an AI

How does the team know the guide gives good answers before players see them?

Bolt says: Next: evals.

Fact-checked 12 October 2026