Usability Testing: How to Test a Website or Digital Product
How to plan, script, run and analyse usability tests, and how to benchmark your current site before a redesign so the re-test is a fair comparison.
Quick answer
Usability testing means watching representative users try to complete realistic tasks with your website, app or prototype, to find where the design confuses or slows them. Decide what you need to learn and how you'll test, recruit participants who match your users, and write and pilot a script with scenario-based tasks. Run sessions while participants think aloud, capture observations and a few simple metrics, and analyze issues by severity and frequency. Report the findings, fix them and test again. Small, frequent rounds of about five users per user group, as Nielsen Norman Group recommends, find most problems efficiently.
Before a redesign, test the current site first. A small qualitative round shows what to fix and what to keep. If you need to prove the new version is better, record a baseline now with a frozen set of tasks, at least 20 participants and 2 to 4 metrics, then repeat exactly the same test after launch. The section on testing before a redesign, after Step 7, gives a test plan, a script and a freeze sheet.
What Usability Testing Is and Why It Matters
Usability testing evaluates a design by observing people use it. It doesn't ask whether people like a design; it shows whether they can use it to do what they came to do. Teams know their product too well to see it the way new users do, and watching a few people struggle with a flow is often more persuasive than any debate.
It works at every stage: on sketches and prototypes before development, on live products before a redesign, and as part of a UX audit. The earlier you test, the cheaper the problems are to fix.
Step 1: Plan the Study
Write down what you need to learn, which flows to test (such as signup, checkout or search) and which decisions the results will inform. Define the participant profile, the devices, the method and the success criteria for each task. A one-page test plan keeps the team aligned and stops the study growing into everything at once.
Moderated vs Unmoderated
| Moderated | Unmoderated | |
|---|---|---|
| Facilitator | Present live | None; tasks completed alone |
| Depth | Can probe and follow up | Limited to what participants say |
| Speed and scale | Slower, fewer sessions | Faster, easier to scale |
| Best for | Complex flows, early prototypes | Specific tasks, quick checks |
Remote vs In-Person
| Remote | In-person | |
|---|---|---|
| Environment | Participant's own device and setting | Lab, office or on location |
| Reach | Participants anywhere | Limited to one location |
| Observation | Screen, voice and often face | Full view of device handling and body language |
| Best for | Websites and apps, distributed users | Physical or specialist products, complex setups |
Step 2: Recruit Participants
Recruit people who match your real users' behaviours and context, not just demographics: people who have bought this kind of product online, or who do this kind of work. Use a short screener questionnaire, and recruit from your customer list, a recruiting panel or a testing platform. Nielsen Norman Group's guidance is that about five participants per round, per distinct user group, reveals most usability problems, and that several small rounds beat one large one. Avoid testing with colleagues, and offer fair incentives.
Step 3: Write Tasks and the Test Script
Write realistic scenarios with a goal, not instructions. “You need a gift for a friend under a set budget; find something suitable and add it to your cart” works; “Click the Gifts menu” doesn't. Avoid words that appear in the interface, which turn tasks into word-matching exercises.
Put the tasks into a script so every session is consistent:
- Welcome, purpose and consent to record
- Reassurance: “We're testing the design, not you”
- Warm-up questions about the participant's context
- Task scenarios in a natural order, each with a clear end point
- A post-task question, such as how easy the task was
- Follow-up questions and a wrap-up
Pilot the Test
Run one pilot session with a colleague or friendly participant before the real ones. Pilots catch confusing task wording, broken prototype links, tasks that take far longer than planned and recording problems, and they cost far less than a wasted session.
Want your key flows tested with real users?
ZSpace Labs plans, runs and analyzes usability tests, and turns findings into concrete design changes.
Step 4: Run Sessions and Observe
Ask participants to think aloud. Stay neutral, resist helping, and answer questions with questions (“What would you expect to happen?”). Let them struggle long enough to understand the problem, then move on. Record sessions with consent, and invite teammates to observe live; watching together builds shared understanding faster than any report.
Observers should note what participants do, not what they would have done: where they hesitate, what they misread, what they try first, errors and recoveries, and direct quotes.
Qualitative Findings
Most of the value of small-sample testing is qualitative: the reasons behind behaviour. Capture each observation with the task, the participant, what happened and, where possible, why. Look for patterns across participants rather than acting on one person's opinion, and note what works well too, so it isn't lost in the redesign.
Common Usability Metrics
Simple measures help compare tasks and track change over time. For choosing which experience metrics to track over time, see UX metrics that matter beyond conversion rate.
| Metric | What it measures |
|---|---|
| Task success | Whether participants completed the task, fully, partly or not at all |
| Time on task | How long successful attempts took |
| Errors | Wrong paths, mistakes and recoveries |
| Single Ease Question (SEQ) | Perceived difficulty of each task, asked after the task |
| System Usability Scale (SUS) | Overall perceived usability, a 10-item questionnaire after the test |
Quantitative Measurements and Sample Size
Numbers from five participants are not statistics. Report them as observations (“3 of 5 participants didn't find the delivery options”) rather than percentages, and don't use them to claim one design is better than another. When you need reliable benchmarks, Nielsen Norman Group recommends at least 20 users for quantitative usability studies, and more if you need tighter confidence intervals. NN/g's article on measuring perceived usability explains when to use SEQ and SUS.
Step 5: Analyze the Results
Soon after the sessions, list every issue with where it happened, how many participants hit it and its severity: blocking, slowing or minor. Cluster related observations, separate symptoms from causes and rate each issue on a shared scale, such as the 0 to 4 severity scale. Then prioritize by severity, frequency and the importance of the task.
Step 6: Report and Share
Keep reports short and decision-focused: the goals, who took part, the top findings with evidence and severity, short video clips of key moments, what worked well and recommended changes. A 20-minute readout with clips often gets more action than a long document. Keep the full issue list in the team's backlog, not only in a slide deck.
Step 7: Iterate and Retest
Fix the most important problems, update the design and test again, ideally with new participants on the same tasks. Iteration is where testing pays off: each round confirms fixes and uncovers the next layer of problems. Build small, regular rounds into the UX design process rather than one big study. A before-and-after comparison of a whole redesign is different: it needs the stricter protocol in the next section.
Usability Testing Before a Redesign
A redesign is usually a large spend, and it is often decided on opinion: the site looks dated, a competitor has relaunched, or a new stakeholder dislikes the navigation. Testing the current site first turns that opinion into evidence. Hoa Loranger at Nielsen Norman Group puts it plainly: “Your old site is the best prototype of your new site”. It already has real content, real flows and real users, so watching people use it shows what the redesign must fix and, just as importantly, what it must not break.
Before you test, decide which of three questions you are answering. They need different methods and different sample sizes, and mixing them is the most common reason pre-redesign research produces numbers nobody trusts.
| Learn | Compare | Prove | |
|---|---|---|---|
| Question it answers | What is broken on the current site, and why? | How do we perform against one or two competitors on the same tasks? | Is the new design measurably better than the old one? |
| Decision it supports | What the redesign brief must fix and keep | Where to match, differ or ignore the competition | Whether the redesign worked, and whether to keep iterating |
| Method | Qualitative, moderated, think-aloud | Qualitative, same tasks on your site and 1 to 2 competitor sites, order rotated | Quantitative benchmark, often unmoderated, repeated after launch |
| Participants | About 5 per distinct user group | Small qualitative samples, as in Learn; at most 3 sites per participant | At least 20 per round, the same profile both times |
| Main outputs | Issues by severity, quotes, what works well | Patterns across sites, ideas worth borrowing | 2 to 4 metrics, such as task success, time on task, SEQ and SUS |
| Effort | Lowest: days, not weeks | Low to moderate: NN/g notes a study can often run in about two days | Highest: two full rounds, separated by weeks |
| Skip it when | You already have recent findings for the same flows | Competitors serve a different audience or task | Traffic or budget cannot support 20+ participants twice; rely on Learn instead |
Worth noting
Five-user rounds find problems; they cannot prove a redesign is better. If someone needs a before-and-after number, plan the Prove column from the start. You cannot benchmark the old site once it has been replaced. For how much research to do before any build starts, see how much UX design to do before development.
A Pre-Redesign Test Script
This script works for Learn and Prove rounds. The key rule is that every task must make sense, word for word, on both the old site and the new one, so write tasks around user goals rather than current page names or menu labels.
- Introduction and consent (2 minutes): purpose, recording consent, “we're testing the site, not you”, and that there are no wrong answers.
- Context questions (3 minutes): how they normally do this task, what they use today, how often they visit the site.
- 4 to 6 top tasks: chosen from the tasks that matter most to users and the business, such as finding a service, comparing options, getting a quote or completing a purchase. Give each task a written success criterion before the first session.
- After each task: the Single Ease Question (“Overall, how difficult or easy was this task?” on a 1 to 7 scale), then one open question: “What, if anything, got in your way?”
- The keep question: “If this site changed tomorrow, what would you miss?” This protects the parts of the current design that already work.
- Competitor tasks (Compare rounds only): the same tasks on one or two competitor sites, with the order of sites rotated between participants.
- System Usability Scale (Prove rounds only): the 10-item questionnaire at the end, unchanged in wording.
- Wrap-up: anything they expected to find but didn't, then thanks and the incentive.
The Freeze Sheet: Keeping the Re-Test Fair
A before-and-after comparison only means something if the only thing that changed is the design. Nielsen Norman Group's benchmarking process (Alita Kendrick, 2020) is to choose what to measure, decide how, collect a baseline, redesign, then collect again under the same methodology before interpreting the results. A freeze sheet is a one-page record of that methodology, filled in at the baseline and copied unchanged for the re-test.
| Item to freeze | What to record at the baseline |
|---|---|
| Task wording | The exact text of each task, including any scenario details such as budget or location |
| Success criteria | What counts as success, partial success and failure for each task, written before testing |
| Metrics | The 2 to 4 metrics you will compare, and how each is calculated (for example, time on task for successful attempts only) |
| Questionnaires | SEQ and SUS wording and scale, unchanged |
| Participants | Screener criteria, user groups, number per group, and that re-test participants have not seen either version in a test |
| Devices and setting | Device types, browsers, screen sizes, remote or in person, moderated or unmoderated |
| Tool and setup | Testing platform, starting URL or start screen, account state (logged in or out), test data |
| Task order | Fixed or rotated, and how |
| Timing | Baseline date, launch date and planned re-test date |
Pro tip
If something has to change between rounds, such as a task that no longer exists in the new design, record it on the freeze sheet and exclude that task from the comparison rather than quietly rewording it.
When to Re-Test and How to Read the Result
Do not re-test on launch day. Kendrick notes that users often dislike change at first, so early scores from returning users can understate a good redesign. Her guidance is about 2 to 3 weeks after launch for products people use daily, and 4 to 5 weeks for products used once or twice a week. For a marketing website mostly used by first-time visitors, recruit participants who have not used either version and the wait matters less.
Keep the comparison to the 2 to 4 metrics you chose, and report them with confidence intervals rather than single numbers. For SUS, MeasuringU's Jeff Sauro reports an average score of 68 across around 500 studies, so a score above 68 is above average. That gives a rough reference point, but the comparison that matters is your old site against your new one under the same conditions. If the intervals overlap heavily, the honest conclusion is that the test did not show a difference, not that the redesign failed.
Usability metrics show whether people can complete tasks more easily. They do not show whether the redesign paid for itself. Enquiries, sales and retention are affected by traffic, season and pricing as well as design, so treat business metrics as a separate analysis.
Hypothetical example: a training provider plans a new website. A five-person Learn round shows that people cannot find course dates and that the existing course comparison table is well liked. The redesign brief keeps the table and fixes the dates. The team also runs a 24-person unmoderated baseline on four frozen tasks with SEQ and SUS, then repeats it five weeks after launch with new participants. No results are implied here; the example shows the sequence.
This level of rigour is not always worth it. For a small site rebuilt mainly for technical reasons, a Learn round and a launch-day check of key tasks is usually enough. Whether to redesign, rebuild or improve in stages is a separate decision, covered in website redesign vs rebuild.
Testing on Mobile
Test mobile experiences on phones, not desktop emulators. For remote tests, use tools that record the phone screen; in person, use a device camera or screen mirroring. Include tasks that involve typing, forms and switching apps, where mobile problems concentrate. See mobile app testing and mobile ecommerce UX.
Usability Testing vs A/B Testing
Usability tests explain why users struggle, using small samples. A/B tests measure which version performs better at scale, but not why. They complement each other: test to find and fix problems, then A/B test to measure impact where traffic allows. See A/B testing.
Usability Testing vs Heuristic Evaluation
A heuristic evaluation uses experts to predict problems quickly against usability principles. Usability testing confirms which problems real users actually hit. Running a heuristic review first clears obvious issues so test sessions uncover deeper ones.
Common Mistakes
- Leading tasks that tell users where to click
- Recruiting people who don't match your users
- Skipping the pilot session
- Helping participants when they struggle
- Testing only at the end, when changes are expensive
- Treating one participant's opinion as a finding
- Reporting percentages from five participants
- Reports that sit unread instead of changing the design
Want usability testing built into your process?
Talk to ZSpace Labs about UI/UX design with regular testing, not one-off studies.
Sources
Alita Kendrick, Nielsen Norman Group: 7 Steps to Benchmark Your Product's UX (13 September 2020).
Hoa Loranger, Nielsen Norman Group: competitive usability testing before a redesign (7 December 2014).
Jakob Nielsen, Nielsen Norman Group: Why You Only Need to Test with 5 Users.
Jakob Nielsen, Nielsen Norman Group: How Many Test Users in a Usability Study? (3 June 2012).
Jeff Sauro, MeasuringU: Measuring Usability with the System Usability Scale (SUS) (3 February 2011).
Accessed 14 October 2026.
Conclusion
Usability testing is the fastest way to see a design through users' eyes. Plan clearly, recruit the right people, write realistic tasks, pilot your script, observe without helping, analyse by severity and fix what you find, then test again. It's one of the core methods in user research.
If a redesign is coming, the next step is small: pick your top four to six tasks, write them so they will still make sense on the new design, and run a Learn round on the current site. Decide at the same time whether you will need a Prove benchmark, because the baseline has to be collected before anything changes.
Common questions.
Observing representative users as they attempt realistic tasks with a website, app or prototype, to find where the design causes confusion, errors or delay.
For qualitative testing, Nielsen Norman Group recommends around five users per round for a given user group, and running several small rounds rather than one large one. For quantitative benchmarks, NN/g suggests around 20 users or more.
In moderated tests a facilitator guides the session live and can ask follow-up questions. Unmoderated tests are completed by participants alone, usually through a testing platform, and are faster to scale.
For most websites and apps, remote testing works well and lets you reach participants in their own environment. In-person testing is better for physical products, specialist equipment, complex setups or when you need to see body language and device handling closely.
Yes. Testing prototypes before development is one of the most valuable uses of usability testing.
Asking participants to say what they're thinking as they work through tasks, which reveals their expectations and confusion.
A realistic scenario with a goal, such as "Find a waterproof jacket in your size under a set budget and add it to your cart", without telling users where to click.
Task success, time on task, errors, and post-task and post-test ratings such as the Single Ease Question and System Usability Scale, alongside qualitative observations. With small samples, focus on the observations.
A written guide for the session: introduction and consent, warm-up questions, task scenarios in order, follow-up questions and a wrap-up, so every participant gets the same experience.
Usability testing explains why users struggle with small samples. A/B testing measures which version performs better with large samples, but not why.
It's one user research method, focused on evaluating a design rather than discovering needs.
Yes, if the redesign is a real investment. Testing the current site first shows what is broken and what already works, so the new design keeps the good parts. If you also want to show the new version is better, record a baseline with fixed tasks now, because you cannot measure the old site once it has been replaced.
Not on launch day. Regular users often dislike change at first, which depresses early scores. Nielsen Norman Group suggests waiting about 2 to 3 weeks for products used daily and 4 to 5 weeks for products used once or twice a week. New-visitor websites can be re-tested sooner, with participants who have not seen either version.