Case study · Design QA
Education Center: five QA rounds, 132 findings, before launch
How I designed the QA of a new learning module to break it the way a first-time learner, a plain member and a team admin would, and turned a 132-item report into tickets engineering could close one at a time.
Senior Product Designer · Design QA lead · 21–24 Sep 2026, follow-up through Oct · Happly, white-label platform
Role
Senior Product Designer, design QA lead
Team
Front-end and back-end developers; product decisions and retests on the company side
Timeframe
Five rounds, 21–24 Sep 2026; follow-up through Oct 2026
Tools
Figma, Jira, and AI agents driving the browser and the API
Hero image: an Education Center course page on demo data (to come)
What the Education Center is
The Education Center is the learning module of the white-label platform: organizations publish courses made of video, audio, reading, quiz, survey and assignment lessons, learners enroll, and a certificate is issued at the end.
Courses are either Structured, where lessons unlock in order, or Self-paced.
Where a review by eye stops
When the build reached the dev environment, the main path worked: create, publish, enroll, finish, get a certificate. That is exactly when a review by eye says “looks good”.
But a learning module carries money, access and credentials. A quiz counted as passed when nobody took it, or a certificate issued without the work, is not a visual bug.
The problem was not to find ugly things. It was to find the places where the screen says one thing and the system does another.
Then write each one so a developer can fix it without a meeting, and prove it’s fixed. I designed and led the QA; AI agents ran the test passes.
What the QA tested against
The brief for this body of work, condensed. Every finding had to trace back to one of these lines.
Goal
Release the Education Center with behavior that matches its PRD, on both sides, in both languages.
Rules (PRD v4)
In Structured courses only surveys can be skipped, and quizzes must be passed.
In Self-paced courses, completion and the certificate need at least 70% of the non-survey lessons actually completed.
Draft content stays hidden from members until it is published.
Success criteria
Every finding is reproducible, tied to a PRD rule or a measured value, and has an expected state. Nothing is filed twice. Nothing is called fixed without a retest.
Non-goals
Redesigning the module, or changing the PRD inside a ticket. Where the PRD was silent, the finding became a decision question with a recommendation, not a bug.
Constraints
A dev environment that changed mid-session, four days, and one designer leading the whole loop.
By the numbers
The report, counted
Counted 13 Oct 2026
132
findings logged: 62 bugs, 50 UI, 20 newcomer
QA report, ID ranges
129
stand after 3 were withdrawn
Withdrawal noted in the report
21
bug tickets with acceptance criteria
Jira, 24 Sep 2026
Findings by stage
129 standing, in order
45
7
47
9
8
13
01
First pass
45
02
Extra tests
7
03
Rounds 3 and 4, three reviewer roles
47
04
Plain member, roles and phone
9
05
“Skip for now” focus
8
06
Round 5
13
5
QA rounds in 4 days, 21–24 Sep 2026
Report updates
3
Test accounts: admin, plain member, team role
The report
22 · 25 · 12
High, Medium and Low among the 59 rated bugs; one bug added later is unrated
Priority headings in the QA subtasks
82
Front-end and UI items scoped into implementation
Jira comment, 7 Oct 2026
Most findings came from rounds 3 and 4, when three reviewer roles ran at once; the last round still added 13.
Challenge 1 of 6
Rounds built to break it, run with three accounts
Situation
The first pass confirmed the core flow worked. The worst bugs only appeared in states, not screens: “Save draft” right after publishing took a live course offline.
Decision
Rounds, each with a different job: I designed them, AI agents ran them. In rounds 3 and 4, three reviewer roles ran at once, and a finding stayed only if it was seen twice. Round 5 tried to break it on purpose.
Which account saw it
Marked where the report records it
Admin
Plain member
Team role
8 bugs found with the admin
A course limited to a membership plan opened to members without it
“Admin review required” was ignored
A view-only team member edited other admins’ courses by link
Read down the last column: one admin account would have shipped all three access bugs.
Outcome
Eight bugs found with the admin reproduced identically for a plain member, so developers knew they weren’t admin-only edge cases.
What made it hard
None of these show up in a screenshot. They show up with two editor tabs open, an old tab, a direct link, a double click. And a newcomer’s problems (can’t find the certificate again) never show up for someone who already knows the product.
Options
One careful pass against the PRD; the PRD pass plus passes designed to break it; or wait for users to report.
More detail
The passes used three accounts, because different roles show different bugs: an admin, a plain member and a custom team role.
Another state bug: saving a quiz lesson deleted and recreated every answer option, so a learner who had already opened the quiz scored 0% for correct answers.
The three reviewer roles: a product review that checked every PRD rule, a senior design review that measured the UI (spacing, type, contrast, states), and a first-time user who knows nothing. The second of the two rounds re-checked every finding of the first.
Round 5 was two passes on purpose: strictly against the PRD, then “try to break it” with two editors at once, old tabs, direct links, double clicks and odd input.
Challenge 2 of 6
Three kinds of finding, three shapes, so nobody has to ask
Situation
Developers had a spec and a Figma file, but no way to tell “a bug” from “polish” from “the PRD being silent”.
Decision
Three kinds, three shapes. A bug (B) has Current, Steps and Expected. A UI finding (U) has Where, Current and Change, with exact copy and measured values, like a status chip’s proposed 5.6:1 text contrast. A newcomer finding (N) has what a newcomer expects and what the screen says.
Three shapes
Blank templates
Bug
B1–B62
Current
Steps
Expected
Dev notes
UI finding
U1–U50
Where
Current
Change: current → new
Measured values
Newcomer
N1–N20
Where
What a newcomer expects
What the screen says
Change
The shape says what kind of work it is: a state to fix, a value to change, or a sentence to rewrite.
Outcome
The front-end work could be cut into lanes that don’t share files: 82 front-end and UI items went into implementation in that shape.
What made it hard
A long report is only as useful as its weakest entry. If a developer has to message me to reproduce something, the finding costs me and them.
Options
A free-form report; a tracker row per finding; or a fixed shape per kind of finding.
More detail
Bugs carried dev notes where there was a concrete lead: for the “Save draft” bug, the editor was sending the course status from its own memory.
UI copy is written as “current → new” so it can be searched in code, with pixel sizes, hex colors and contrast ratios where they apply.
Every finding carries a screenshot where it helps and names its source: PRD section, product decision, admin UI copy or general quality. The PRD-over-Figma rule sat at the top of the report, so nobody had to re-argue it item by item.
Where the PRD and Figma disagreed, the PRD won and I updated Figma.
Challenge 3 of 6
Withdraw false findings in the open: 132 logged, 129 stand
Situation
On the second day I withdrew four findings: B13, B14, U5 and the sidebar part of U17. They came from the test browser the agents drove, which slows down timers, not from the app.
Decision
Remove them in the open, in the report, with the cause. A false finding costs a developer an afternoon and costs the next real finding some trust.
The count
From the report’s ID ranges
Bugs
B1–B62
62
UI findings
U1–U50
50
Newcomer findings
N1–N20
20
Logged
132
Withdrawn, test-browser artifacts
B13 · B14 · U5
−3
Stand
129
The number on the page is the one left after the withdrawals, not the bigger one that came first.
Outcome
Three IDs were withdrawn (the fourth removal was part of an item), so 129 stand. I’d rather publish “132 logged, 3 withdrawn” than a round number.
What made it hard
Removing them made the report smaller, and a smaller number looks worse.
Options
Leave them in as “needs investigation”; quietly delete them; or remove them in writing with the reason.
More detail
The same discipline ran the other way: the report also lists what works (the certificate opens right after the last lesson, quiz retries keep the highest score, comments are escaped), so developers know what not to touch.
Challenge 4 of 6
Split the report by impact: 21 bugs got tickets of their own
Situation
After rounds 1–4 the work lived in four large checklist tickets. A developer can’t “close 30% of a ticket”, and a manager can’t see which bug is blocking a release.
Decision
On 24 Sep I split by impact. Each high-impact logic or access finding got its own ticket: 17 from the first rounds plus 4 from “Skip for now”, 21 in all. Small UI and copy fixes and newcomer items stayed as grouped checklists.
The split
24 Sep 2026
The QA report
Four checklist tickets after rounds 1–4
21 tickets of their own
Full report, screenshot, acceptance criteria and a retest label each.
17 from the first rounds
4 for “Skip for now”
Grouped checklists
Small UI and copy fixes and the newcomer items stayed together.
2 duplicates closed
Each points to the ticket that kept the full repro.
Each ring is a bug someone can close on its own.
Outcome
Each serious bug could be fixed, retested and closed on its own. As of October 2026, the tickets were still being fixed.
What made it hard
The worst bugs (a quiz completes without being taken, a live course goes offline) were buried between small copy fixes in the same list.
Options
Keep one checklist; one ticket per finding, over a hundred tickets and mostly noise; or split by impact.
More detail
The round-5 items each got acceptance criteria the next day. Two duplicates were closed with a pointer to the ticket that kept the full repro.
Challenge 5 of 6
A decision question, not a guess, for “Skip for now”
Situation
In a Self-paced course, “Skip for now” on a quiz completed it: a course with a certificate could be finished without taking its quiz.
Decision
I set up a focused pass on “Skip for now” with all three accounts and wrote four bugs, each with screenshots, a video and acceptance criteria.
Product then locked the intent: a skip is not a completion, a skipped survey stays retakeable, and some surveys must be mandatory. That became a separate story for an admin control.
What a skip does
Self-paced course
Before: what the build did
Skip for now
→
Completed
→
Course complete
→
Certificate issued
Now: the locked rule
Skip for now
→
Skipped
Not a completion · a skipped survey stays retakeable
The button was never the bug; the missing rule was. Once “Skipped” existed as its own state, every screen could be checked against it.
Outcome
The skip rule is now written in the PRD, and “Skipped” is its own status everywhere.
What made it hard
This was not a plain bug: the PRD didn’t say what a skip should do to progress. If I’d just logged “wrong”, someone would have guessed.
Options
Pick a behavior in the ticket; leave it as a PRD gap; or turn it into a decision question with a recommendation.
More detail
The four bugs: skip completes the course and issues the certificate; “Allow skip = No” isn’t enforced; admin views disagree about skipped lessons; the skipped state has no way back.
The same defect existed on surveys, and the survey had no way back.
Challenge 6 of 6
Write the loop down, so the next feature starts at step one
Situation
A retest on 25 Sep showed an Education Center fix working, and found the same class of problem in other modules.
Decision
A cross-module rule followed: draft content stays hidden from members until published, in every module. Those findings became their own tickets.
And I wrote the loop down, so it can run on any feature.
The loop, written down
Reused on the next feature
01
Read the PRD and list every rule it contains
02
Read the known items, so nothing is filed twice
03
Set up the three accounts
04
Build the test matrix
05
Collect annotated evidence
06
Triage
07
Report
08
Write tickets
09
Retest
None of the steps is new. Writing them down meant the next feature’s QA started at step one instead of from scratch.
Outcome
The loop became the template for the next feature’s QA. A design-system sweep a few days later found that two of six Education Center fixes had defects of their own after a merge: the argument for retesting every fix.
What I learned
Test states, not screens.
The bugs that mattered needed a second tab, a role, or a double click. A screenshot review would have shipped them.
Fewer, sharper tickets win.
Splitting by impact made the serious bugs visible and closable.
Withdraw what you got wrong.
Taking four findings back in the open made the other 129 easier to believe.
A decision question beats a guess.
Where the PRD was silent, asking with a recommendation got a locked rule in a day.
Next time, measure it.
Tie each finding to a release metric so design QA shows up as a number, and run the loop before a feature reaches dev instead of on it.
Bugs found by design QA before release versus reported by users after; time from “reported” to “Final check ready”; how many findings reopen after a retest; and learner completion where a skip or a bypass used to count as done.