QA passes and bug tracking
Tests, playtests and QA
Garden Defense 1.1 is nearly ready. Every automated test is green and playtesters loved the new pest. Then Remy opens the shop on a German tablet: the Buy button is cut in half.
QA (quality assurance) means people checking the whole game against a written plan before release, hunting for anything broken. It fills a gap the other two leave:
- Automated testsCode checking code on every push (Stage 2). Fast and exact, but they only check what someone thought to write.
- PlaytestsWatching new players to learn if the game is fun and clear (Stage 3).
- A QA passTeammates working through a checklist on real devices to find what is broken: every screen, every setting, every odd case.
Quick check
Which question is a QA pass built to answer?
- Is the new pest fun to fight?
- Does every screen work on every device, browser and language we support?
- Does scoreFor(1, 2) return 20?
Show the answer
Does every screen work on every device, browser and language we support?
Fun is a playtest question, and one function's answer is a unit test's job. QA checks the finished game, everywhere it will run, before players see it.
The test plan
- Devices and browsersAn old Android phone, a new iPhone, a tablet, and Chrome, Safari, Firefox and Edge for Hyper Shards.
- Store buildsThe exact signed release build from each store's test track, never a debug build from a laptop.
- LanguagesEvery language we ship: text cut off, overlapping, or still in English.
- AccessibilityEvery option from the Accessibility lessons in Stage 3, switched on and played.
- Payments in test modeStripe test mode with card 4242 4242 4242 4242, Google Play license testers and Apple sandbox accounts: real purchase screens, no real money.
A test plan is the checklist for a QA pass, written before release so nothing depends on memory. It also lists odd cases: no internet, a phone call mid-wave, a full phone.
True story: pulled from the PlayStation Store
Cyberpunk 2077 came out on 10 December 2020, after more than 8 million pre-orders. Early feedback was mostly positive, but players on the base PlayStation 4 and Xbox One, the older consoles, reported glitches and bugs.
On 14 December, its maker CD PROJEKT promised patches and offered refunds. On 18 December, Sony removed the game from the PlayStation Store. It came back on 21 June 2021, six months later.
Every device we support goes on the test plan, the oldest included. A game that runs well on a new machine can still fail on an old one.
Source: CD PROJEKT current report 66/2020 (removal from the PlayStation Store), cdprojekt.com
A bug report a stranger could follow
Players report bugs roughly (the Updates and player feedback lesson in Stage 9). A QA report is precise enough for someone who has never seen the bug:
- Title: where and what. "Shop: Buy button cut off in German on tablets".
- Steps to reproduce: numbered actions that make it happen again.
- Expected vs actual: what should happen, and what did.
- Build and device: version 1.1.0 (code 12), Galaxy Tab A, Android 14.
- Evidence: a screenshot or short screen recording.
Bug report
Title: Shop takes coins, no hat. Steps: 1) open shop 2) buy Sprig hat. Expected: hat on Sprig. Actual: 50 coins gone, no hat. Screenshot attached.
What is the most important thing missing from Kai's report?
- The build number and device, because the bug may only happen on one version or phone
- How annoyed Kai felt when it happened
- Who wrote the shop code, so they can be blamed
Show the answer
The build number and device, because the bug may only happen on one version or phone
Without the build and device, Remy might test the wrong version on the wrong phone and say "can't reproduce". Feelings and blame don't help anyone fix it.
Severity vs priority
Severity is how bad a bug is for the player. Many studios use levels: critical (crash, lost saves, broken payments), major (a feature is broken), minor (annoying, with a workaround) and cosmetic (looks wrong).
Priority is how soon we fix it, decided by the team. It also weighs how many players hit it and where.
They usually match, but not always. A typo in the game's name on the title screen is cosmetic, yet every player sees it: high priority. A crash that only happens on one ten-year-old phone is critical, yet may wait.
Our bug tracker: GitHub Issues
- LabelsColoured tags. New repositories start with ten, including bug, duplicate and wontfix. We add our own, like severity: critical and platform: android.
- MilestonesA group of issues with a due date, like "1.1 release". GitHub shows the percentage done and how many are still open.
- AssigneesWho owns the fix. GitHub allows up to 10 per issue, but one clear owner works best.
- Issue formsTemplates in .github/ISSUE_TEMPLATE that ask for steps, expected, actual and device, so every report arrives complete.
Every bug becomes an issue, GitHub's task card (the Planning lesson in Stage 2). A pull request that says Fixes #87 closes it once merged into main.
Bolt says: Triage means sorting by how serious things are. In a short triage meeting a few times a week, the team goes through new issues: can we reproduce it? Is it a duplicate? Then it gets a severity, a priority, an owner and a milestone.
Triage call
Two bugs left before 1.1. A: the title screen says "Garden Defnese". B: the game crashes if you rotate one old tablet model 20 times on the pause screen.
Which triage decision fits best?
- A is cosmetic but high priority, because every player sees it. B is critical but so rare it may wait for 1.1.1
- A is critical, because spelling matters more than anything
- Close both, because neither loses saves
Show the answer
A is cosmetic but high priority, because every player sees it. B is critical but so rare it may wait for 1.1.1
Severity and priority are different questions. The typo is harmless but seen by everyone on launch day, while the crash is serious but almost nobody will ever trigger it. Both stay on the list with an owner.
Regression checks and release candidates
A fix can break something nearby. So when a bug is fixed, QA first verifies it on the same build and device, then runs a short regression check: the features around the fix, to catch anything that used to work and broke. The bug also gets an automated regression test (Stage 2), so it can never sneak back.
Near the end comes a release candidate (RC): a build that ships unless something serious turns up, like 1.1.0-rc.1. From then on, no new features, only fixes. Every fix makes a new candidate, rc.2, which gets checked again.
Go or no-go
Before submitting, the team holds a go/no-go meeting with one question: is this release candidate good enough for players? The rules are agreed in advance: any open critical bug, or a failed payment test, means no-go.
"Go" means the exact RC build goes to the stores' testers and review (the Testers and review lesson). "No-go" is not failure. It means QA did its job, and players never met that bug.
Bolt says: Our build is checked and ready. But what if someone copies it and shares it without paying? Next: piracy and DRM.
Fact-checked 11 October 2026
- GitHub: managing labels docs.github.com
- GitHub: configuring issue templates docs.github.com
- Stripe: testing docs.stripe.com
- Google Play Billing: test developer.android.com
- App Store Connect: testing in sandbox developer.apple.com
- CD PROJEKT current report 64/2020 (Cyberpunk 2077 pre-orders) cdprojekt.com
- CD PROJEKT current report 66/2020 (removal from the PlayStation Store) cdprojekt.com
- CD PROJEKT current report 34/2021 (return to the PlayStation Store) cdprojekt.com
- CD PROJEKT Management Board report for 2020 cdprojekt.com