Incrementality Testing: The Gold Standard for Measuring Marketing Impact
Most marketing measurement answers the question "what happened?" Attribution models tell you which channels touched a customer before they converted. Dashboards show you spend, clicks, and revenue. But none of these tools reliably answer the harder question: "would this customer have converted anyway, without seeing our ad?"
That is the question incrementality testing is designed to answer. By comparing outcomes between groups that were exposed to your marketing and groups that were not, incrementality testing isolates the true causal impact of your advertising. It separates the conversions your ads created from the conversions that would have happened regardless.
According to a 2025 EMARKETER and TransUnion survey, incrementality testing has moved from niche practice to mainstream adoption, driven by privacy-related tracking limitations and growing pressure to prove that ad budgets generate real business impact. This article explains the core methods, when to use each, and how to get started.
Why Attribution Models Are Not Enough
Attribution models allocate credit to marketing touchpoints based on rules or statistical weighting. Last-click attribution gives 100% credit to the final touchpoint. Multi-touch models distribute credit across the journey. Data-driven models use machine learning to weight touchpoints based on their observed correlation with conversion.
The problem is that correlation is not causation. Just because a user clicked a branded search ad before converting does not mean the ad caused the conversion. That user may have been searching for your brand already, fully intending to buy. The ad captured existing demand rather than creating new demand. Attribution gives the ad full credit. Incrementality testing would reveal that the true lift was close to zero.
This distinction matters enormously for budget allocation. If branded search appears to drive a 10:1 return on ad spend in your attribution model, you might increase the budget. But if incrementality testing shows that 80% of those conversions would have happened without the ad, the true return is closer to 2:1. That changes the allocation decision entirely.
Attribution tells you where conversions occurred. Incrementality testing tells you which conversions your marketing actually caused.
How Incrementality Testing Works
The core principle is borrowed from randomised controlled trials in medical research. You create two groups: a treatment group that sees your marketing and a control group that does not. After a defined period, you compare outcomes between the groups. The difference is your incremental lift.
The formula is straightforward:
Incremental lift = (Treatment group conversions - Control group conversions) / Control group conversions x 100
If the treatment group converted at 5% and the control group at 4%, your incremental lift is 25%. Your marketing caused one additional conversion for every four that would have happened anyway.
Values above 20% generally indicate meaningful incrementality, though the threshold depends on your margins and the cost of the campaign being tested. The key is statistical significance: you need enough data to be confident the difference is real and not just random variation.
Three Core Methods for Incrementality Testing
1. Holdout Group Testing (User-Level)
The most precise method. You randomly split your target audience into two groups: one that sees your ads and one that does not. Because the split is random, any difference in outcomes between the groups can be attributed to the advertising.
How it works: Most major ad platforms support holdout testing natively. Meta offers Conversion Lift studies, Google Ads provides brand lift and conversion lift experiments, and programmatic platforms support audience-level holdouts. Typically, 5 to 10% of your audience is held out from ad exposure.
Strengths: The gold standard for precision. Random assignment eliminates selection bias. Results are directly attributable to the specific campaign or channel being tested.
Limitations: Requires platform support for audience splitting. Becoming harder to implement as user-level tracking erodes due to privacy restrictions. Not feasible for channels that cannot control who sees the ad (like television or outdoor).
Best for: Testing the incrementality of specific digital campaigns on platforms that support audience holdouts. Particularly valuable for always-on campaigns (retargeting, branded search) where the true incremental value is most questioned.
2. Geo-Lift Experiments (Market-Level)
When user-level randomisation is not possible, geo-lift testing offers a practical alternative. Instead of splitting users, you split geographic markets. Some regions receive your advertising; others do not. You compare business outcomes across regions.
How it works: Select a set of test markets (where advertising runs) and control markets (where it does not). The markets should be as similar as possible in demographics, purchasing behaviour, and seasonality. Run the test for a defined period, then compare outcomes. Open-source tools like Meta's GeoLift package and Google's CausalImpact help with market selection and statistical analysis.
Strengths: Works for any channel, including offline media. Does not require user-level tracking or platform integration. Privacy-friendly because it operates on aggregate regional data, not individual user data.
Limitations: Requires enough geographic markets to achieve statistical power. Results take longer to materialise because market-level data is noisier than user-level data. Confounding factors (a competitor launching in one region, for example) can skew results. Not practical for businesses that operate in only one or two markets.
Best for: Testing the incrementality of channels that cannot do user-level holdouts: TV, radio, out-of-home, or broad-reach digital campaigns. Also valuable for measuring the combined effect of multiple channels in a region.
3. Ghost Ads (Intent-to-Treat)
Ghost ads (also called predicted ad exposure or intent-to-treat analysis) are a hybrid method. Instead of physically withholding ads from a control group, the system identifies users in the control group who would have been eligible to see the ad based on targeting criteria, and tracks their behaviour as if they had been exposed.
How it works: The ad platform runs the targeting algorithm for both the treatment and control groups. The treatment group sees the actual ad. The control group sees either a public service announcement (PSA) or no ad, but the system records that they would have been eligible. Conversions are then compared between the two groups.
Strengths: Solves the selection bias problem inherent in simple exposed-vs-unexposed comparisons. More accurate than basic holdout tests because it accounts for the fact that people who are targeted by ads may already be more likely to convert.
Limitations: Requires sophisticated platform support. Not widely available outside of large programmatic platforms and walled garden environments. The "ghost" prediction is a model, and models can be wrong.
Best for: Display and programmatic advertising where selection bias is a known issue. Particularly useful for testing upper-funnel campaigns where the correlation between ad exposure and conversion is weakest.
Choosing the Right Method
The choice between methods depends on your channels, scale, and what you are trying to learn:
- Testing a specific digital campaign? Use holdout group testing if the platform supports it. This gives you the cleanest read on that campaign's incremental value.
- Testing a channel that cannot do user-level holdouts? Use geo-lift experiments. This includes TV, radio, out-of-home, and broad-reach campaigns.
- Running programmatic or display at scale? Ghost ads offer the most accurate incrementality read by controlling for targeting selection bias.
- Testing overall marketing effectiveness across channels? Geo-lift experiments work best because they measure the combined impact of all marketing activity in a region, not just one channel.
Many sophisticated marketing teams use all three methods at different times, choosing the right tool for the specific question they need to answer.
How to Run Your First Incrementality Test
You do not need a data science team to get started. Here is a practical sequence for running your first test:
- Pick a high-spend, high-question channel: Start with a channel where you spend a lot but are uncertain about the true incremental value. Branded search and retargeting are common first choices because they often show inflated returns in attribution models.
- Define the success metric: Choose a single, clear outcome metric: conversions, revenue, or new customer acquisitions. Avoid measuring too many things at once.
- Choose the method: If the platform supports audience holdouts, use that. If not, identify control markets for a geo-lift test. Ensure you have enough volume for statistical significance.
- Set the test duration: Two to four weeks is typical for digital campaigns. Geo-lift tests often need four to eight weeks. The test must run long enough to accumulate sufficient data, but not so long that external factors contaminate the results.
- Analyse and act: Calculate the incremental lift. If the channel shows strong incrementality, you have confidence to maintain or increase investment. If incrementality is low, consider reallocating budget to channels with higher proven lift.
Incrementality in the Broader Measurement Framework
Incrementality testing does not replace attribution or marketing mix modelling. It complements them. Attribution tells you how conversions are distributed across touchpoints. Marketing mix modelling tells you how budget changes might affect total outcomes. Incrementality testing tells you whether a specific activity is actually causing the outcomes attributed to it.
The most effective measurement frameworks use all three:
- Attribution for day-to-day campaign management and tactical optimisation.
- Marketing mix modelling for strategic budget allocation across channels.
- Incrementality testing to validate the assumptions underlying both. When incrementality results contradict attribution data, it is a signal that your attribution model needs recalibration.
As privacy restrictions continue to limit user-level tracking, incrementality testing and marketing mix modelling will become more important relative to attribution. Businesses that invest in these capabilities now will be better positioned to make evidence-based budget decisions as the measurement landscape evolves.
Start Testing What Matters
Every marketing team has at least one channel where the true incremental value is uncertain. Branded search, retargeting, and always-on display campaigns are common candidates. Running even a single incrementality test on one of these channels can reveal whether you are investing in demand creation or just paying to capture demand that already existed.
The insight is often uncomfortable, but it is always valuable. Knowing where your marketing genuinely drives outcomes means you can allocate budget with confidence rather than hope.
If you want to design and run incrementality tests for your marketing channels, our analytics team can help you set up the right testing framework. For businesses looking to connect incrementality insights with their paid search strategy, we can integrate test results directly into your campaign optimisation process.