Boss: Ask the data

Play this quiz

Bug hunt

share.js
// weblog('level_shared',  { level_id: id });// phoneslog('Level_Shared',  { level_id: id });

Theo counts level_shared, and every share from phones is missing. Why?

  1. Event names are case-sensitive, so Level_Shared is a different event
  2. Phones can't send parameters
  3. level_id is a reserved name
Show the answer

Event names are case-sensitive, so Level_Shared is a different event

Google Analytics treats level_shared and Level_Shared as two separate events. One naming convention, written in the tracking plan, prevents this.

Quick check

If a phone retries and sends an event twice, an event ID created on the device lets the pipeline keep just one copy.

Show the answer

True

The retry carries the same ID, so the pipeline keeps one row per ID. An ID made on the server would give each copy a different ID.

Quick check

Adding LIMIT 10 to SELECT * makes a BigQuery query read less data.

Show the answer

False

LIMIT shortens the answer, not the scan. Choosing only the columns you need, and a date range on a partitioned table, is what reduces the data read.

Pick the query

Which part of a query keeps only levels shared more than 100 times?

  1. GROUP BY level_id HAVING COUNT(*) > 100
  2. WHERE COUNT(*) > 100
  3. ORDER BY COUNT(*) DESC LIMIT 100
Show the answer

GROUP BY level_id HAVING COUNT(*) > 100

WHERE runs before groups are counted, so it can't use COUNT. HAVING filters the groups after counting. ORDER BY with LIMIT gives the top 100, not those over 100.

Quick check

A closed first-week funnel shows 7,400 players defended a first garden and 3,100 then joined co-op within 7 days. What is that step's conversion?

  1. About 42%
  2. About 58%
  3. About 31%
Show the answer

About 42%

3,100 ÷ 7,400 is about 0.42. 58% is the share who dropped out at that step.

Read the result

An A/B test shows B ahead by +1.0 point, with a 95% confidence interval of −0.4 to +2.4. Kai has checked every morning and wants to stop now. What should Theo say?

  1. No significant difference yet, and stopping when it looks good inflates false positives
  2. Ship B: it is ahead
  3. Ship A: the interval goes below zero
Show the answer

No significant difference yet, and stopping when it looks good inflates false positives

The interval includes 0, so no difference was detected. Peeking and stopping early makes a false winner much more likely, so the test runs to its planned size.

Quick check

Moving a Buy button to where players are used to tapping Play is a fair way to raise lifetime value, if an A/B test shows LTV goes up.

Show the answer

False

It earns money from mistakes, a dark pattern. No A/B test result makes tricking players fair.

Quick check

Replacing players' names with a pseudonymous ID makes analytics data anonymous, so privacy law no longer applies.

Show the answer

False

The ICO says pseudonymised data is still personal data for anyone who can link it back, and COPPA counts persistent identifiers as personal information.