Matched-Market Testing: The Gold Standard for Retail Media Proof

There's one way to know for certain whether a retail media campaign drove incremental sales. Not pretty certain. Not directionally correct. Certain enough to show a CFO and have them nod instead of ask follow-up questions.

It's matched-market testing. And almost nobody in brand marketing is doing it consistently, even though the methodology is well-established and the major networks now support it.

Here's how it works, why it's the gold standard, and what it takes to run one.

‍

"Matched-market testing is the methodology that changes the retail media conversation with finance permanently." — Five Eighty

‍

The core concept: two groups, one difference

A matched-market test divides your target geography (or your target shopper panel) into two groups that are as similar as possible on every variable that matters: category purchase rate, demographics, store format, price sensitivity, and baseline velocity for your product.

One group receives the campaign — the test markets. The other group doesn't — the control markets. Everything else stays the same: pricing, promotions, distribution, creative across other channels. The only difference is the retail media campaign.

At the end of the campaign period, you compare sales velocity between test and control markets. The lift attributable to the campaign — net of any natural market fluctuation — is your incremental sales result. That number is as close to a causal answer as marketing measurement gets outside of a randomized controlled trial.

Why it's hard to run — and why that's no longer a good excuse

The practical barriers to matched-market testing have historically been real. You need markets that are similar enough on baseline variables that the comparison is clean. You need a long enough test period for statistical significance — typically six to twelve weeks depending on purchase frequency. You need the network to hold back its inventory from the control markets, which requires a network partner willing to do that.

None of these barriers are theoretical anymore. Walmart Connect, Kroger Precision Marketing, and Albertsons Media Collective all support matched-market test design when asked. The methodology for constructing matched market groups is well-documented. The statistical significance calculations are standard.

The real barrier is organizational: someone has to plan the test before the campaign launches, not after. Someone has to resist the pressure to optimize in-flight and let the test run. Someone has to be willing to present a result that might be lower than the attributed ROAS. That's a culture question, not a methodology question.

‍

→ Incrementality vs. Attribution  —  agencyfiveeighty.com/retail-media-incrementality-vs-attribution

→ Data & Analytics  —  agencyfiveeighty.com/data-analytics

‍

How to construct a matched-market test

1. Define the measurement objective before you define the campaign: what incremental lift threshold would justify the spend? If you need 10% incremental lift to break even on the media cost, design the test to detect differences as small as 8% with 80% statistical power.
2. Select test and control markets based on baseline similarity: use 12-week pre-period sales velocity, category purchase rates, store count and format mix, and demographic composition. The goal is markets that behave identically before the test starts.
3. Agree on holdout methodology with the network: specify that control market stores will receive no impressions from the campaign. Get this in writing before the campaign launches.
4. Run the campaign for a minimum of six weeks, ideally eight to twelve: shorter tests in high-frequency categories can work, but longer tests produce cleaner signal and higher statistical confidence.
5. Analyze the result against your pre-defined threshold: did the test markets outperform control markets at the lift level you need? That answer — yes or no — is the only thing that matters.

What a positive result actually gets you

A clean matched-market test with a positive result gives you something almost no brand has: a defensible, network-independent proof that retail media drove incremental sales. Not a ROAS number that the network calculated using their own methodology. A causal claim supported by a clean experimental design.

That changes the budget conversation permanently. Instead of defending retail media spend against skeptics who suspect the ROAS is inflated, you're presenting evidence that holds up to scrutiny. Finance can't argue with a controlled experiment the way they can argue with last-click attribution.

Five Eighty designs measurement architecture before campaigns launch — because the proof of performance should be planned, not reverse-engineered from whatever the dashboard happened to show.

‍