How many testers do you need for a product test?

A practical guide to sizing a consumer test. We run consumer testing, so read the recommendations with that in mind: every rule of thumb below is standard practice you can check against any research methods text.

Almost every brand asks this question backwards. They pick a number that sounds respectable, run the test, and then try to work out what the result means. The order that works is the reverse: decide what you need to be able to say, and the sample size follows from it, often landing somewhere you did not expect.

Start from the decision, not the number

Three questions determine everything downstream.

Are you reading one product, or comparing two? Reading one means estimating a single quantity: what share of testers would buy this again. Comparing two means detecting a difference between two quantities, which is a much harder statistical job and needs a much bigger panel.

How big a difference would change your mind? If you would switch formulas over a 20-point gap in repurchase intent, you need far fewer people than if a 5-point gap matters. Small effects are expensive to detect. This is the single biggest driver of cost and the one most often left undecided.

How wrong can you afford to be? A directional read that guides an internal choice tolerates more uncertainty than a number you intend to print on a box.

Reading one product

When you are estimating a single proportion, the useful output is not the headline percentage. It is the confidence interval around it: the range the true value plausibly sits in. Sample size controls the width of that range, and it does so with sharply diminishing returns.

The width of a 95% interval shrinks with the square root of the sample size. That is the whole economics of this decision in one sentence. Quadrupling your panel only halves your uncertainty. Going from 25 to 100 testers is a real improvement; going from 100 to 400 to halve it again is rarely worth what it costs.

Work out the interval for your own numbers with our free sample size calculator before you commit to a panel.

Comparing two products costs much more

This is where budgets die. Detecting a difference requires enough data to be confident about two quantities at once, and the smaller the true difference, the more people it takes.

The relationship is brutal: halving the difference you want to detect roughly quadruples the sample you need. A test powered to catch a large, obvious gap between two formulas may need under a hundred people per arm. A test powered to catch a subtle preference can need many hundreds per arm, which is usually the moment a brand discovers the comparison it wanted was never affordable.

If you are running a genuine A/B comparison, our A/B significance calculator will tell you whether a gap you already have is distinguishable from noise.

Recruit for completions, not sign-ups

Sample size means completed responses. Everything above assumes you have that many usable debriefs at the end, not that many people who agreed to take part at the start.

Attrition in a two-week at-home test is real: people forget, move house, or lose interest. Recruit above your target with that in mind, and treat any panel where nobody dropped out as a reason to ask how the data was collected.

Composition matters as much as count. Thirty testers who all match your target skin profile tell you more than a hundred drawn at random, because a skincare result that is not broken down by skin type is an average of people the product was never for. That is the whole argument for matched in-home testing over volume.

How to report it honestly

Three habits separate a report that survives scrutiny from one that does not.

• Put the sample size next to the number, every time. "78% would repurchase" and "78% would repurchase (n=27)" are different claims, and only one of them is honest.
• Show the interval, not just the point estimate. A range communicates the actual state of your knowledge. A single number implies a precision you do not have.
• State the composition, including the unflattering parts. If the panel skewed young, or was partly recruited through your own channels, say so in the report rather than letting someone find out later.

The same discipline applies when a result is used as a marketing claim, where the sample and the method have to be available on request. We cover that in claims substantiation.

Common questions

How many testers do I need for a product test?

For a directional read on one product, 25 to 30 completed testers is usually enough to make an internal decision. For a number you intend to publish, 75 to 100 is a more defensible floor. For a head-to-head comparison between two products, expect to need several times more, because detecting a difference is much harder than estimating one value.

Is 30 testers enough?

For one product and an internal go or no-go, generally yes, provided the panel matches your target profile and you report the number alongside the result. For comparing two products or reading subgroups separately, 30 is too thin to support a conclusion.

Why does comparing two products need so many more people?

Estimating one quantity is easier than detecting a difference between two, and the smaller the true difference, the larger the sample needed. Roughly, halving the difference you want to detect quadruples the sample size required.

Does a bigger panel always give a better answer?

No. Uncertainty falls with the square root of sample size, so returns diminish quickly. Beyond a point, spending on better matching, a longer use period or a better-designed debrief buys more than adding people.

What if some testers drop out?

Plan for it. Sample size targets refer to completed responses, so recruit above your target. A two-week at-home test always loses some participants, and a panel reporting zero attrition is worth asking questions about.