You have opened your sample-size calculator, entered your baseline conversion rate, kept the defaults for significance and power, and now the calculator is asking for one more number: the minimum detectable effect. It is the input that decides everything downstream — how many visitors you need, how many weeks the test runs, and whether the experiment is even worth starting. Yet it is the one field with no obvious "correct" value, so it tends to get a number typed into it that is really a wish rather than a decision.
This guide is about choosing that number deliberately. Not "what is MDE" in the abstract, but "what should I actually put in the box," why the answer is a business judgement as much as a statistical one, and how to avoid the two mistakes that quietly ruin experiment programs: an MDE so small the test never finishes, and one so large the test is underpowered to see the wins you care about.
What a minimum detectable effect actually is
The minimum detectable effect is the smallest true improvement you want your experiment to be able to reliably catch. Optimizely defines it as "the smallest improvement you are willing to detect" and notes that it "determines how 'sensitive' an experiment is." If you set a 5% MDE, you are telling the calculator: size this test so that a genuine 5% lift would show up as statistically significant. A real effect exactly at your MDE has a good chance of being detected; a real effect smaller than your MDE probably will not, because you did not buy enough traffic to see it.
The critical thing to internalize is that MDE is a planning input, not a result. It is not a prediction of how much your variation will lift conversions, and it is not the effect you will report at the end. It is a design lever you set beforehand to determine sample size. The test itself might measure a 3% lift or a 9% lift or nothing at all — the MDE only governed how sensitive the instrument was.
Three inputs work together to fix your sample size: the baseline conversion rate, the statistical significance and power you require, and the MDE. Hold the first two constant and MDE becomes the dial that trades detectable precision against how much traffic — and therefore time — the test consumes.
Relative vs. absolute MDE
Almost every calculator, Optimizely's included, expresses MDE as a relative lift, and this trips people up constantly. Optimizely states that MDE "represents the relative minimum improvement over the baseline." A 10% MDE does not mean "10 percentage points." It means a 10% change relative to your baseline.
Work through Optimizely's own example. If your baseline conversion rate is 20% and you set a 10% MDE, the test is sized to detect a move outside the absolute band of 18% to 22% — because 10% of 20% is 2 percentage points. So a "10% relative MDE" is a "2 percentage-point absolute MDE" at that baseline.
This distinction matters because absolute and relative framings diverge wildly at different baselines. A 10% relative MDE on a 2% baseline checkout rate is a 0.2 percentage-point move — invisible without enormous traffic. The same 10% relative MDE on a 40% baseline is a 4 percentage-point move, far easier to detect. Always confirm which convention your calculator uses before you type a number, and think in the units your business actually cares about. If stakeholders reason in percentage points ("we need to add two points of conversion"), convert to relative before entering it.
Why sample size scales with the inverse square of MDE
Here is the single most important fact for setting a realistic MDE: required sample size grows with roughly one over the square of the MDE. Optimizely puts the direction plainly — "the smaller your MDE is, the larger the sample size required to reach statistical significance" — and the standard fixed-horizon formula makes the rate precise.
Sample size per variation is approximately proportional to 1/MDE². The consequences are steep and non-linear:
- Halving your MDE (say 10% down to 5%) roughly quadruples the sample size, because (1/0.5)² = 4.
- Cutting MDE to a third (10% to 3.3%) needs roughly nine times the traffic.
- Going the other way, doubling the sample size only shrinks your detectable effect by about 29%, not half, because 1 − 1/√2 ≈ 0.293.
That last point is the one that surprises people. Traffic buys detectability at a punishing exchange rate. You cannot casually "just run it a bit longer" to chase a smaller effect — the marginal week of traffic detects a smaller and smaller increment of lift. This asymmetry is exactly why MDE has to be chosen against what your traffic can afford, not against what you would like to prove.
The 1/MDE^2 penalty (relative MDE, baseline and settings held constant)
Anchor point (from Optimizely's own worked example):
baseline = 15%, MDE = 10% relative -> ~8,000 visitors PER VARIATION
Apply the sample-size ∝ 1 / MDE^2 scaling law to that anchor:
MDE = 10% -> N ~ 8,000 / variation (baseline)
MDE = 5% -> N x (10/5)^2 = 4N ~ 32,000 / variation
MDE = 2.5% -> N x (10/2.5)^2 = 16N ~ 128,000 / variation
Reading it the other way:
To HALVE the effect you can detect, you need ~4x the visitors.
Doubling visitors only shrinks detectable MDE by ~29% (1 - 1/sqrt(2)).
Note: 8,000 is Optimizely's figure; the 32,000 and 128,000 are that
figure scaled by 1/MDE^2 — illustrative, not calculator-exact. Real
Stats Engine (sequential) durations differ, but the scaling holds.
At two visitors per test, the difference between a 10% and a 2.5% MDE is the difference between a test that concludes in a couple of weeks and one that needs several months of the same traffic. Same hypothesis, same page — a 4x to 16x difference in cost, driven entirely by the number you chose for one field.
How to choose an MDE that isn't wishful thinking
A defensible MDE sits at the intersection of two independent constraints. Set it too optimistically on either and the experiment fails you.
Constraint one — the smallest lift worth shipping. Ask what improvement would actually change a decision. If shipping the variation carries engineering cost, maintenance, or risk, a 0.5% lift might not be worth it even if it is real. The floor of your MDE should be the smallest effect that would still make you press "launch." Optimizely frames this as using "potential business impact to decide on the sensitivity of your experiment" — and notes that when conversions tie directly to revenue, a lower MDE can be justified precisely because each fraction of a percent is worth real money. A checkout step feeding millions in revenue earns a smaller MDE than a blog newsletter signup.
Constraint two — what your traffic can power in a reasonable window. Take your weekly traffic to the surface under test, decide the longest you are willing to run (commonly two to four weeks so you capture full business cycles without letting the test rot), and work out the largest sample you can realistically collect. Then find the smallest MDE that sample can power. Optimizely's prioritization guidance is explicit: divide total required sample by the traffic you can allocate to see how long a test will take, and judge the tradeoff — "a 4% lift in 5 weeks may be a reasonable tradeoff of impact for effort, but a 2% lift measured over several months may not be."
Your MDE is the larger of these two floors. If the smallest lift worth shipping is 3% but your traffic can only power an 8% MDE in a month, you have a mismatch to resolve — you cannot honestly detect the 3% you care about. Do not pretend otherwise by typing 3% into a test that cannot see it.
As rough orientation — not a rule — high-traffic pages often support a 2–5% relative MDE, while lower-traffic flows frequently bottom out around 10–20%. Treat these as sanity checks on your own calculation, not targets. And follow Optimizely's advice to work in "limits and ranges rather than looking for exact numbers": run the calculator at a few candidate MDEs and pick the point where detectable precision and runtime both look acceptable. The sample size calculator and velocity calculator turn each candidate MDE into a concrete sample size and a testing-cadence estimate, which is the fastest way to see the tradeoff in weeks rather than abstractions.
A decision framework for picking your MDE
flowchart TD
A[Start: which MDE do I enter?] --> B[What is the smallest lift<br/>worth shipping? = business floor]
A --> C[How much traffic can I collect<br/>in an acceptable window?]
C --> D[Smallest MDE that traffic<br/>can power = traffic floor]
B --> E{Is business floor<br/>>= traffic floor?}
D --> E
E -->|Yes| F[Set MDE = business floor.<br/>Test is worth running and powerable]
E -->|No: want to detect smaller<br/>than traffic allows| G[Mismatch — pick one]
G --> H[Reduce variance:<br/>CUPED, better metric,<br/>higher-traffic surface]
G --> I[Extend runtime or<br/>concentrate traffic allocation]
G --> J[De-prioritize: effect you care<br/>about is too small to power]
H --> E
I --> E
F --> K[Enter MDE in calculator,<br/>commit sample size, launch]
When you hit the mismatch branch, you have real options before you give up. Variance reduction is the highest-leverage one: techniques like CUPED lower the noise in your metric, which effectively lets you detect a smaller effect from the same traffic — the equivalent of shrinking your MDE for free. You can also choose a higher-traffic surface, pick a metric closer to the change, or concentrate allocation. Only when none of those close the gap should you de-prioritize, which is itself a legitimate and common outcome — Optimizely explicitly advises that "if the traffic cost is too high, consider de-prioritizing the hypothesis… or seek to measure lift with less granularity."
To see how much a given pre-period correlation shrinks the sample you need, try the CUPED savings estimator.
Common mistakes: too low and too high
Two symmetric errors account for most botched MDE decisions.
MDE set too low (the test that never ends). Chasing a tiny effect because "even 1% matters" without checking whether your traffic can power it. The 1/MDE² penalty means a 1% MDE can demand a hundred times the sample of a 10% MDE. The test drags on for months, business context shifts underneath it, and pressure builds to peek and stop early — which inflates false positives if you are not on a method built for it. If you genuinely need to detect small effects, plan for it with variance reduction or a sequential-testing approach, don't just lower the number and hope.
MDE set too high (the underpowered test). Entering a large MDE to make the sample size look affordable, then treating a non-significant result as proof the variation "didn't work." It proves no such thing. A test sized for a 15% MDE is simply blind to a real 6% lift — a genuinely valuable win — because it was never powered to see it. You will ship losers and kill winners on the strength of a test that was underpowered by construction. High MDEs are only honest when the smallest lift worth shipping really is that large.
Peeking as an MDE problem in disguise. When a test sized for a modest MDE runs longer than hoped, the temptation is to check daily and stop the moment significance flickers. On a fixed-horizon test this quietly wrecks your error rate. The disciplined fixes are to size honestly up front and, if you need the flexibility to stop early, adopt a method designed for it — see sequential testing and the peeking problem.
MDE is a planning input, not a result
The recurring theme is worth stating once more, cleanly: the MDE you enter shapes the experiment; it is not something the experiment gives back. Optimizely's own guidance is to "use MDE as a guide rather than an exact prediction." It sets the sensitivity of your instrument before you collect a single visitor. Confuse it with the expected or measured lift and you will either over-promise ("we'll get a 10% lift" — no, you sized to detect one) or misread a null result.
So treat the MDE field as a design decision with two owners: the business, who names the smallest lift worth shipping, and the traffic, which caps how small an effect you can afford to detect in a sensible window. Set your MDE at the larger of those two floors, verify the resulting sample size and runtime in a calculator, and only then launch. Done that way, the number in the box stops being a wish and becomes the honest specification of what your experiment can — and cannot — tell you.
Once your MDE is set, size the test with the sample size calculator, pace your roadmap with the velocity calculator, and if the effect you care about is too small to power, reach for CUPED variance reduction or sequential testing before you compromise on the MDE itself.
