Startups Stack Exchange Archive

When should a startup do A/B testing?

• posted by: Esqarrouth on 2015-02-23
• tagged: product-market-fit, ab-testing
• score: 2

Getting product-market-fit is more about searching the “thing” that works and A/B testing is more about optimizing the current working product.

Ideally you should start A/B testing after you have something of a value. But the line is unclear. At what stage should a startup start doing A/B testing?

Answer 3502

• posted by: adrianh on 2015-02-24
• score: 3

It’s not really a “stage” thing. It’s when A/B testing is both possible and useful.

I’ve seen it used well at both late stages (testing to help optimise checkout processes) and early ones (testing alternate copy on a landing page MVP).

You start doing A/B testing when:

1. you have enough traffic for A/B testing to be statistically possible
2. when the thing you want to test is susceptible to A/B testing

Answer 3501

• posted by: Denis de Bernardy on 2015-02-24
• score: 2

It’s not about stage, it’s about quantity of data. You need a certain threshold of data before you can quickly obtain statistically significant results. Tools and more reading.

---

All content is licensed under CC BY-SA 3.0.