Predicting churn

Play this lesson

No goodbye button

Churn is players leaving: Course 1 measured it as the share who stop playing in a period. A subscription has a cancel button. A free game like Shard Garden doesn't: players simply stop coming.

Theo, the studio's data analyst, wants to spot it early: which new players are likely to leave this week, and, more useful, why?

Defining "left"

Churn has to be defined before it can be predicted. A Google Cloud example on the mobile game Flood It! left out players with under 10 minutes in the app as bounced: about 41%. It counted a player as churned with no activity after their first 24 hours: about 23% of the rest.

Google Analytics' churn probability asks: will a player who was active in the last 7 days not be active in the next 7?

Pick the label

Many Shard Garden players only play at weekends. Which definition of "churned" should Theo use?

  1. Active before, then no garden defended for 14 days
  2. No event in the next 24 hours
  3. Uninstalled the app
Show the answer

Active before, then no garden defended for 14 days

With 24 hours, every weekend player would look churned each Monday. Uninstalls can't be seen on the web or Steam. Two quiet weeks fits how people actually play.

Signals from the first days

The Flood It! model used the events in each player's first 24 hours: levels started, completed and reset, scores posted, rewarded ads watched, friends challenged, plus country, device and language.

Theo's first-three-days signals for Shard Garden: gardens defended, gardens lost in a row, co-op joins and editor sessions. All are counts of game events. No chat, no names.

A simple model first

The Google Cloud example starts with logistic regression, because it is the fastest to train, then tries others like XGBoost and neural networks. A model gives each player a probability between 0 and 1.

The data is split into a training set, which the model learns from, and a test set, which it never sees while learning. Scoring on the test set shows how it does on new players.

Quick check

A churn model should be judged on the same players it learned from.

Show the answer

False

A model can memorise the players it learned from and look far better than it is. Judging it on a test set it never saw shows how it will do on next week's players.

Precision and recall

Google's machine learning course defines two measures:

Improving one often makes the other worse. And plain accuracy misleads when one outcome is rare: if 1% of cases are positive, always guessing "no" scores 99%.

Read the numbers

churn model, last month's new players
Flagged by the model  1,000Flagged, and churned    600Churned in all        2,400

What are the model's precision and recall?

  1. Precision 60%, recall 25%
  2. Precision 25%, recall 60%
  3. Precision 60%, recall 60%
Show the answer

Precision 60%, recall 25%

Precision: 600 of the 1,000 flagged were right, 60%. Recall: it caught 600 of the 2,400 who churned, 25%. Most churners were missed.

Fix the game, don't chase the child

Google's example suggests reaching likely churners with targeted campaigns or reminder emails. For a game played by 10 to 12 year olds, Shard Garden doesn't do that: no messages to pull one child back, no guilt, no "your garden misses you".

Theo uses the model to learn why. Players who lose their first three gardens in a row churn twice as often. Remy rebalances the first gardens, for everyone.

Asking the data, carefully

Shard Garden now has events with a tracking plan, a pipeline with no duplicates, a warehouse, SQL, funnels, cohorts, honest A/B tests, LTV that follows fun, dashboards with one definition each, tested data, and analytics that protect children.

Bolt says: Boss time: one quiz across the whole Data stage.

Fact-checked 11 October 2026