Dashboards for the team

4.1 million, so what?

Priya's dashboards from the Observe stage tell the on-call teammate whether players are OK right now. The Monday dashboard has another job: helping the team decide what to build.

At Monday's meeting Kai shows: "4.1 million gardens defended this month, up 12%!" Remy asks: "Is that good?" More players? Each player playing more? A build sending events twice? Nobody can tell.

PULSE and HEART

The 2010 paper by Google's Kerry Rodden, Hilary Hutchinson and Xin Fu names the most common large-scale metrics PULSE: Page views, Uptime, Latency, Seven-day active users and Earnings.

They matter, but they are indirect: a rise in page views could mean a feature is popular, or that a confusing screen makes people click around, lost.

The paper's user-centred framework is HEART. A team picks the categories that fit, on purpose:

• HappinessAttitudes, like satisfaction, measured with surveys.
• EngagementHow often, how much and how deeply people use it.
• AdoptionNew users of the product or a feature in a period.
• RetentionUsers from one period still there in a later one.
• Task successHow quickly and how often people succeed, and their error rate.

Goals, signals, metrics

The paper's process has three steps:

• Goal: what should the editor achieve? Players build and share a level they are proud of.
• Signals: what would players do if it worked? Publish. Failure can be easier to see: closing the editor after 20 minutes without publishing.
• Metric: the share of editor sessions that end in a publish.

Only then does it go on the dashboard.

Per player, not in total

The paper's advice: raw counts go up as the user base grows, so ratios, percentages or averages per user are often more useful.

Its Gmail example: the share of active users who visited on five or more days in the last week. That number also predicted who would stay longer.

Kai's 4.1 million becomes: gardens defended per weekly player, 6.2, up from 6.0 last month.

One definition, one owner

If Kai's dashboard counts anyone who opened the app as "active" and Theo's counts only players who defended a garden, the team gets two answers to one question.

Theo's rule: every metric on the Monday dashboard has a written definition, its SQL in one shared place, an owner, and the question it answers. "Active" means defended a garden or built in the editor that week.

Are the numbers even right?

A well-designed dashboard is only as good as the data under it. One broken event can make a great chart tell a story that isn't true.

Bolt says: Next: data quality.