Skip to content
CRO11 min read

How to Prioritise CRO Experiments When You Have More Ideas Than Time

How to prioritise CRO and ecommerce experiments: a traffic check before scoring, a copyable scoring worksheet, PIE, ICE and PXL compared, and lead-gen sites.

01

How should you decide which experiment to run next?

First remove every idea that cannot finish with your traffic, and send obvious problems straight to a fix list. Then score what is left on evidence, reach, noticeability and effort, using the same worksheet every time, and run the highest-scoring test on each template. Reprioritise monthly as research and results arrive. The order matters because each test holds a page's traffic for weeks: a weak test now means a stronger one waits.

This guide is for ecommerce and marketing teams with a long list of ideas from colleagues, agencies and leadership, and no agreed way to rank them. It includes a worksheet you can copy into a spreadsheet and a section for lead-generation sites, where the rules change.

02

Key takeaways

  • Check testability before scoring, not after.
  • Fix clear problems without testing them; research ideas with no evidence.
  • Score evidence with fixed criteria rather than a 1 to 10 confidence guess.
  • Measure reach by template and by revenue or lead value, not just sessions.
  • On low-traffic lead-generation sites, most ideas become fixes or research.
  • Record why ideas were ranked low so they are not reargued without new evidence.
03

Why Order Matters

Every test uses a page's traffic for weeks. A store that can run a couple of tests at a time on its product pages might complete a limited number there in a year. Choosing a low-value test means a higher-value one waits. Prioritisation is about making those slots count.

It's also about learning. Tests grounded in evidence produce clearer results, whether they win or lose. For the full experimentation process, see ecommerce experimentation; for the per-test standard, see A/B testing framework.

04

Step 1: can it finish with your traffic?

Run this check on every idea before anyone scores it. An idea that cannot reach a result is not a test, however good it sounds.

You need four numbers: the weekly sessions that reach the page or template, the current conversion rate for the metric the change affects, the smallest improvement worth detecting, and the longest you are willing to run a test (many teams use four to six weeks, to limit seasonal and promotional noise; that is a working rule, not a standard). Put the first three into a sample size calculator such as Evan Miller's, multiply the answer by the number of variants, and divide by weekly sessions.

Hypothetical example: a collection template gets 8,000 sessions a week and 3% of those sessions add to cart. Detecting a 10% relative lift (3.0% to 3.3%) at the usual 95% significance and 80% power needs roughly 53,000 sessions per variant, so about 106,000 for an A/B test. That is around 13 weeks. Under a six-week limit, the idea fails the check as written.

  • Passes: score it in the worksheet
  • Fails, but the change could be bolder: redesign the test to target a larger effect
  • Fails, but a busier template has the same problem: move it there
  • Fails, but a metric closer to the change would do (for example clicks on the size guide rather than orders): change the metric, accepting it says less about revenue
  • Fails and none of the above apply: treat it as a fix or a research item

Worth noting

If most of your ideas fail this check, testing is not your main tool yet. Our guide to CRO on low-traffic websites covers what to do instead.

05

Step 2: sort into fix, test or research

Not every idea should become a test. Fix: clear problems such as bugs, broken links, errors, slow pages and accessibility failures; testing them wastes traffic. Test: changes with uncertain outcomes that passed the traffic check. Research: ideas with weak evidence, or big questions, that need more understanding before a test is designed. See conversion research.

06

Step 3: score with the worksheet

Copy these columns into a spreadsheet. Evidence is one point for each source that independently shows the problem: analytics, user testing, surveys or support tickets, and session recordings. Reach is 1 (low), 2 (medium) or 3 (high) for the share of sessions, revenue or qualified leads passing through the template. Noticeability is 0 (below the fold or subtle), 1 (visible) or 2 (obvious within seconds). Effort is 1 (small), 2 (medium) or 3 (large), estimated by the people who will build it.

Score = (evidence + noticeability) × reach ÷ effort. Only ideas in the test stream get a score. The weights are a starting point; adjust them, then keep them fixed so scores stay comparable.

Idea (hypothetical)TemplateEvidence (A / UT / S / R)ReachNoticeEffortTestable?StreamScoreDecision and reason
Show delivery estimate beside add to cartProduct page1 / 1 / 1 / 0 = 3321Yes, about 5 weeksTest15Run first: strong evidence, high reach, small build
Filter chips above the grid on mobileCollection1 / 0 / 0 / 1 = 2312Yes, about 6 weeksTest4.5Next on this template
Payment icons broken on one mobile browserCheckout1 / 0 / 0 / 1 = 22n/a1Not neededFixn/aFix this week; no test
Cut quote form from 9 fields to 5 (lead-gen)Quote page1 / 0 / 1 / 1 = 3211No, about 600 sessions a weekFixn/aShip it and watch qualified leads for 8 weeks
New homepage headline (lead-gen)Homepage0 / 0 / 0 / 0 = 0222NoResearchn/aRun customer interviews and a five-second test first

Backlog ordered by opinion?

ZSpace Labs builds evidence-led testing backlogs with scoring your team can apply consistently.

Start a Project
07

The Core Factors

Whatever worksheet you use, it should capture these factors.

FactorQuestionHow to estimate
ReachHow many visitors and how much revenue pass through?Analytics by template, device, market
EvidenceWhat supports this idea?Analytics, research, support, past tests
ImpactHow much could it change the metric?Size of the problem, boldness of change
EffortWhat does it cost to build and run?Design, development, QA time
RiskWhat could go wrong?Revenue exposure, brand, legal, accessibility
TestabilityCan it reach a result with our traffic?Sample size vs traffic (Step 1)
08

PIE, ICE, PXL and RICE compared

Several published frameworks turn these factors into a score. All are simplifications; their value is consistency and discussion, not precision. Two attribution points are often garbled. PIE was described by Chris Goward of WiderFunnel in 2013 as a way to rank the pages or areas of a site to test, not individual ideas, and his description gives no fixed scale. ICE is commonly attributed to Sean Ellis, though we could not find an original source to cite.

FrameworkOriginCriteriaBest forWeakness
PIEChris Goward, WiderFunnel (2013)Potential (how much the page can improve), Importance (value and volume of its traffic), Ease (technical and organisational)Choosing which pages or areas to work onScores are subjective; no fixed scale
ICECommonly attributed to Sean EllisImpact, confidence, easeFast ranking of growth ideasSubjective and easy to inflate
PXLCXLMostly yes/no questions: above the fold, noticeable quickly, adds or removes an element, backed by user testing, analytics or other research, plus ease; some criteria count for more than othersReducing opinion in idea-level scoringTakes setup to adapt; designed to be customised
RICEProduct management practiceReach, impact, confidence, effortExplicit reachImpact still estimated
Custom weightedYour teamYour factors, your weights (like the worksheet above)Fits your contextNeeds calibration
09

Reach by Template

Ecommerce reach is usually about templates, not individual pages. A change to the product page template affects every product page view; a change to one landing page affects only its traffic. Estimate reach from sessions and revenue by template, device and market. Changes to cart and checkout reach fewer sessions but sessions with high purchase intent, so revenue share matters as well as session share.

TemplateReach profileTypical test value
Product pageHigh sessions, high intentHigh
Collection / listingHigh sessions, browsingHigh
CartFewer sessions, very high intentHigh per session
CheckoutFewest sessions, highest intentHigh but platform-constrained
HomepageVaries; often lower intentMedium
Single landing pageCampaign-dependentLow to medium
10

If you run a lead-generation site

The same process works, with three changes. First, measure reach by lead value, not sessions: a page that produces few enquiries but most of your closed deals has more reach than a busy blog post. Second, the outcome is qualified leads, not form submissions. A change that adds form fills but brings in poorer enquiries is a loss; see how to improve website lead quality for defining a qualified lead with your sales team.

Third, expect most ideas to fail the traffic check. Lead-generation sites usually have far fewer conversions than stores, and a qualified lead takes weeks to confirm. That means more of the backlog goes to the fix stream (ship clear improvements and watch qualified leads over a set period) and the research stream (interviews, usability tests, reviewing call notes). Testing is kept for the few pages with enough volume. The low-traffic CRO guide sets out what to work on first.

11

Balancing the Portfolio

A backlog of only small, safe tests produces small learnings. A backlog of only big bets risks long periods without results. Balance the two: most tests are well-evidenced, moderate changes on high-reach templates; a few are bolder changes that could shift understanding; some capacity goes to fixes and research. Review the balance quarterly.

12

Estimating Impact Without Guessing

Impact is the hardest factor to estimate. Anchor it in data rather than intuition. The size of the problem sets a ceiling: if only a small share of product page visitors open the size guide, a size guide change can only affect that share. Past tests on similar changes give realistic effect ranges. The boldness of the change matters: small tweaks produce small effects. Use bands (small, medium, large) rather than precise percentages, since precision here is false.

InputHow it informs impact
Share of sessions affected by the problemUpper bound on impact
Drop-off at the related stepSize of the opportunity
Past tests of similar changesRealistic effect range
Boldness of the changeSmall tweak vs substantial change
Research strengthConfidence the problem is real
13

Cost of Delay and Dependencies

Some ideas lose value if delayed: seasonal tests, changes linked to a product launch, or fixes to problems that grow with traffic. Others depend on prior work, such as tracking that must exist first or a design system component. Factor these into scheduling after scoring: a slightly lower-scoring seasonal test may need to run now, and a high-scoring idea may need to wait for its dependency.

14

Running the Prioritisation Meeting

Prioritise as a group with data visible. Each idea should arrive with its hypothesis and evidence already written. Score quickly using the agreed worksheet, discuss disagreements about evidence rather than opinions, and agree the next few tests for each template. Scoring invites bias: authors overrate their own ideas, senior stakeholders' ideas get generous scores, and vivid problems feel bigger than they are. See experimentation mistakes.

  • Every idea has a written hypothesis and evidence
  • Traffic check done before the meeting
  • Reach numbers pulled from analytics, effort estimated by the people who will build it
  • The author doesn't score their own idea alone
  • Scores compared with past test outcomes each quarter
  • Decisions and reasons recorded next to scores
15

Common Mistakes

  • Opinion-based scores that anyone can inflate
  • Testing obvious fixes
  • Scoring ideas before checking whether they can finish
  • Counting form fills instead of qualified leads
  • Prioritising pages instead of templates
  • Only small safe tests, or only big redesigns
  • Never reprioritising after results arrive
16

Limits of any scoring system

A score is a conversation aid, not a forecast. The worksheet ranks ideas by how well supported and how visible they are; it does not predict effect sizes, and two ideas a point apart are effectively tied. It also cannot rescue a backlog full of weak ideas: if nothing scores well, the next step is research, not more scoring. If you need help building the evidence, our CRO audit service page describes how we approach it.

17

Sources

Chris Goward, Practical Ecommerce: Use the PIE Method to Prioritize Ecommerce Tests (12 December 2013); CXL: the PXL prioritisation framework (the original article; check it for the current criteria and weights, which we have not reproduced); Evan Miller: sample size calculator.

Accessed 15 October 2026. The sample size in the hypothetical example uses the standard two-proportion calculation at 95% significance and 80% power.

18

Conclusion

Check whether each idea can finish with your traffic, sort the rest into fix, test and research, and score the tests with one worksheet your team keeps using. On lead-generation sites, weigh lead value and qualified leads, and expect more fixes and research than tests. Start by running your current backlog through Step 1; the list that survives is usually much shorter. Related: CRO testing roadmap and ecommerce CRO audit.

Want your backlog scored against your own data?

ZSpace Labs is a remote-first technology studio that works with clients globally on CRO audits and testing backlogs. If useful, we can run your current backlog through the traffic check and worksheet with your analytics and talk through what to test, fix or research first.

Start a Project
FAQ

Common questions.

Traffic and team time are limited. Each test occupies a page for weeks, so the order in which you test determines how much you learn and gain over a year.

Reach (how many visitors the change affects), strength of evidence, expected impact, effort and cost, risk and whether the test can reach a result with available traffic.

Scoring frameworks. PIE (Potential, Importance, Ease) was described by Chris Goward of WiderFunnel to rank which pages or areas of a site to test first. ICE (Impact, Confidence, Ease) is commonly attributed to Sean Ellis and is used to rank individual growth ideas. PXL, from CXL, uses mostly yes/no questions about visibility and evidence to reduce subjective scoring, and is meant to be customised.

The one your team applies consistently. Frameworks with objective criteria (like evidence sources and page traffic) tend to reduce bias compared with pure 1 to 10 opinion scores.

Use analytics: the share of sessions or revenue passing through the page or template, on the devices and markets in scope. On a lead-generation site, use the share of qualified leads or lead value, not just sessions.

Clear fixes (broken elements, errors, accessibility failures) should usually be fixed without a test. Test changes where the outcome is uncertain.

Tests on low-traffic pages may never reach a result. Check this before scoring: if a test cannot finish within your time limit, make it bolder, move it to a busier template, or treat the idea as a fix or a research question.

Regularly, such as monthly, and whenever new research or test results arrive.

They carry more risk and effort and are harder to learn from. Break them into testable parts where possible, or test the full redesign against control as a split test.

Require evidence for every idea, use scoring criteria that reward evidence, and review scores as a group with the data visible.

Not in Goward's original description, which gives the three criteria but no fixed scale or formula. Scoring each from 1 to 10 is a common team practice. Whatever scale you choose, write it down and keep it.

Get in touch

Have a project in mind?

Whether you're building a new digital product, improving an existing website, or looking to automate part of your business — let's talk.