Incrementality Calculator
Most marketers over-measure clicks and under-measure causality. Enter your test and control (holdout) results to get incremental lift, incremental CPA, incremental ROAS — and an actual statistical significance test, not just a percentage that might be noise.
Test vs. control results
Test group (exposed to ads)
Control group (holdout)
Test CVR
2.40%
Control CVR
1.90%
Incremental lift
+26.3%
Incremental conversions
250
Incremental CPA
$60
Incremental ROAS
1.33x
Statistically significant
z = 5.45 · p = 0.000 · ~100.0% confidenceYour test group converted at 2.40% versus 1.90% in the control group. At this sample size, that gap is unlikely to be random noise (p < 0.05) — there's roughly a 100% chance this lift is real rather than a coincidence of who happened to land in each group. Treat the 250 incremental conversions and $60 incremental CPA as decision-grade numbers.
Why platform-reported ROAS overstates reality
Every ad platform is graded on the conversions it can claim, so every ad platform's attribution model is structurally biased toward claiming more of them. Someone who searched your brand name anyway, who was already a repeat customer, or who would have converted from organic traffic still gets counted as an “ad-driven” conversion if they fell inside the attribution window. None of that is fraud — it's just what attribution windows do. Incrementality testing is the correction: hold out a comparable group from the ad, and the gap between the two groups is your actual causal effect.
A lift number without a significance test is a guess
A test group converting at 2.4% versus a control group at 1.9% looks like a 26% lift. At a sample size of 40,000 per group that's probably real. At a sample size of 400 per group, it's statistically indistinguishable from a coin flip. This calculator runs a two-proportion z-test so the lift number comes with a p-value attached — the same test used in clinical trials and A/B testing platforms, applied to holdout marketing tests.
Frequently asked questions
What is incrementality in marketing?
Incrementality is the share of conversions that happened *because* of your ad, not the share that merely occurred after someone saw it. Platform-reported conversions (Meta, Google) count anyone who converted within an attribution window, including people who would have converted anyway. A test vs. control holdout is the most reliable way to isolate the causal effect.
How do you set up a test and control group?
Split a comparable audience in two: the test group sees the ad, the control (holdout) group doesn't. Geo-based holdouts (some regions get ads, others don't) and platform-native conversion lift studies (Meta, Google) are the two most common methods. The key requirement is that the groups are otherwise similar — random assignment or matched geos, not 'people who clicked' vs. 'people who didn't.'
What does 'statistically significant' actually mean here?
This tool runs a two-proportion z-test comparing your test and control conversion rates. A p-value below 0.05 means there's less than a 5% chance the observed gap would occur if there were truly no difference between groups — the conventional (if somewhat arbitrary) bar for treating a result as real rather than noise.
My result isn't significant — what should I do?
Three options: run the test longer to accumulate more conversions, increase the audience size, or accept that the true effect may be smaller than your point estimate suggests. Don't cherry-pick a shorter window that happens to look significant — that's how false positives get made.
How is incremental CPA different from my normal CPA?
Normal (blended) CPA = spend ÷ total conversions, including ones that would have happened anyway. Incremental CPA = spend ÷ incremental conversions only. Incremental CPA is always higher than blended CPA, sometimes dramatically — a channel with a great blended CPA can have a terrible incremental CPA if most of its 'conversions' were already going to happen.
Related calculators
ROAS Calculator
Break-even and target ROAS
CAC Calculator
Blended CAC and payback period
LTV:CAC Ratio Calculator
Is your acquisition spend sustainable?
New to the terms? See incrementality, statistical significance, and holdout tests in the marketing glossary.