Did the Ad Actually Sell Anything?

Yourretail media campaign ran. Sales went up. Your ROAS looks great. There's justone problem: you have no idea if the ad did anything.

Thefundamental question in retail media measurement isn't "did we see salesduring the campaign?" It's "would those sales have happened withoutit?" That gap — between sales your ad caused and sales your ad justhappened to be nearby for — is what separates real ROI from a very expensivecoincidence.

Most brands have no way to answer that question right now. That's the measurementcrisis hiding inside retail media's growth story.

"86%  of commerce-media decision-makers say stronger measurement is a high or  critical priority." — Forrester, State of Retail Media‍

The problem with last-click

Last-clickattribution is the default reporting method across most retail media networks.It works like this: a shopper sees your ad, then buys your product. The ad getscredit for the sale.

Here'sthe flaw. A shopper who was going to buy your cereal regardless — because theybuy it every two weeks, because it's on their list, because it's a habit —still gets counted as an ad conversion if they happened to see your bannerfirst. Your ROAS looks fantastic. Your actual return on the spend is a fractionof that number.

Thisisn't a rounding error. In high-frequency CPG categories with baseline purchaserates above 30%, last-click attribution can overstate performance by 3x ormore. You're not measuring the ad's impact. You're measuring your category'snatural momentum and giving the ad the trophy.

Incrementality is the right question

Incrementalitymeasurement asks something simpler and more honest: compared to people whodidn't see the ad, did the people who saw it buy more? The difference betweenthose two groups — net of the natural purchase rate — is what the ad actuallydid. That's the number worth optimizing.

Gettingto that number requires test-and-control methodology: one group exposed to thecampaign, a matched group held back as a control, and a clean measurement ofthe gap between them. It requires planning ahead. It requires a network willingto hold back inventory. And it requires a sample large enough to bestatistically meaningful.

Mostbrands never ask for it. The ones that do change the way they talk to their CMOpermanently — because suddenly they can show what retail media is actuallyworth, not what it happened to be nearby when people bought things.

→ Data& Analytics  —  agencyfiveeighty.com/data-analytics

→ 3Retail Media Fails & Fixes  —  agencyfiveeighty.com/insights---3-retail-media-fails-and-fixes

What "closed-loop" actually means vs. what networks claim

Everyretail media network says they offer closed-loop measurement. Almost none ofthem mean the same thing by it.

Areal closed loop: exposure data connected to purchase data at the individualshopper level, measured against a clean baseline, with a defensible controlgroup. That's the standard.

Whatmost networks actually deliver: a report showing impressions served to shopperswho later made a purchase. That is not a closed loop. That's a coincidencecounter dressed up in measurement language. The fact that someone saw your adand bought your product tells you nothing about whether the ad caused thepurchase.

Theway to pressure-test any network's claims is to ask five specific questionsbefore you sign anything.

Five questions to ask before you spend

• What is your incrementallymethodology — holdout cells, matched markets, or matched panels?
• How do you construct the controlgroup, and what's the minimum sample size for statistical significance?
• What's the baselinepurchase rate in this category, and how is it used to calculate lift?
• What's the lookback window,and why?
• Can I receive raw exposureand conversion data for independent verification?

Networksthat can answer these clearly and confidently deserve your money. Networks thatpivot, deflect, or send you a glossy one-pager instead of a methodologydocument deserve scrutiny.

FiveEighty works network-agnostically on purpose. We don't have spend commitmentsto fill. Our job is to figure out where the money actually works — and prove itwith numbers that hold up when someone in finance asks a hard question.

The brands that build incrementally-first measurement programs now will have acompounding advantage over the next three years. Because once you know what'sreal, every future dollar gets smarter.