customer-development, lean-startupWhen deciding how many customer interviews it will take to validate/disprove a single assumption within a value proposition hypothesis, how do you know when you’ve done enough?
Seems like there might be a number of factors such as:
You’re right — the answer is “it depends”.
The rule of thumb that has worked well for people in my experience is “stop doing interviewing when you stop learning”.
As soon as you have two people you’ll likely start seeing some common themes emerging. Every person after that will add more. At some point you hit a point of diminishing returns for your current learning context.
I rarely find that I stop learning useful new things before I’ve talked to 5 or 6 people. I rarely find that I’m still learning useful new things after about 15 interviews (even if the thing learned is “we don’t have a decent market segment identified since everybody is giving us different answers” ;-)
So if I was forced to pick a number I would say 10-12 people stands a fair chance of being useful.
I’m trying to encourage the folk I work with to move away from thinking of interviewing (and other custdev-ish things) as being one-time tasks. Instead approach them as continual task that you repeat to gather new insights, validate old ones, etc. Interviewing smaller groups more frequently, rather than a one-off batch at the start. With budget/time defined by the hypothesis we need de-risking — rather than an up-front guess on the number that will be “right”.
Note: The articles I link to below are AlertBox columns written by Jakob Nielson and relate to usability studies rather than marketing.
They’re relevant imho because, besides being interesting in their own right, they lay out the number of users you need to interview in order to test an assumption, an idea, a hypothesis, a model, etc., and why you need to test with that number of users.
This number of tests needed to get some qualitative results is not materially different from a field to the next. In particular, it applies to market studies.
Anyway, depending on the amount of insights you wish to gain, you’ll want to interview:
5 prospects or clients to validate your idea in broad strokes. (“We’re looking into providing [vague sketch out here], and we’d like your feedback on whether this might be something that you’d be interested in seeing us solve.”)
15 prospects or clients to collect deeper and more precise insights as the idea itself becomes more specific. (Diminishing returns then starts to seriously dent the usefulness of meeting more of them.)
20 prospects or clients to collect rudimentary statistics with a margin of error that isn’t completely off the charts (i.e. 20%, which is better than what you’d get with a sample of 5 or 15).
The above figures assume, of course, that your sample is reasonably sane – i.e. you’re asking random users in your target audience rather than cherry-picking if you’ve no idea what your audience is like, or indeed sampling your audience based on those stats if you do.
All content is licensed under CC BY-SA 3.0.