A causal analysis of a real promotion run by a U.S. grocery retailer, using propensity matching and difference in differences on 1.47 million real transactions.
If this analysis were run the way most retail dashboards run it, comparing the spending of people who redeemed a coupon against everyone who didn't, the headline would write itself: a 78% increase in spend. That figure would make it into a slide deck, a board update, perhaps a case study. It would also be almost entirely wrong.
The problem surfaces the moment you look at these two groups before the campaign ever launched.
They are not an average customer converted by a coupon. They are a pre-existing heavy shopper who also happened to clip one.
People who went on to redeem this coupon were already spending 54% more than everyone else, weeks before the promotion existed. Comparing their post-campaign spend to the rest of the base doesn't measure what the campaign did. It measures who these people already were.
To isolate the campaign's true effect, each redeemer was matched to a non-redeemer with a near-identical pre-campaign spending history, a technique called propensity score matching. With comparable households on both sides, the change in spending from before the campaign to during it was measured for each group. The difference between those two changes is the effect the campaign actually caused, a method known as difference in differences.
The corrected effect is $139.99 in incremental spend per household, statistically robust (p is less than 0.0001, Cohen's d equals 0.58). Set against a coupon cost of just $2,509 across all 214 redeemers, that is an estimated 1,094% return. The campaign worked. It simply worked at roughly 44 cents on every naive dollar claimed.
Breaking the corrected effect down by income tells a story that is both a business insight and, unusually, a socially welcome one.
Households earning under $15,000 a year showed $244 in incremental spend, among the strongest response of any segment, and households with three or more children showed an equally strong $244 response. The highest income households showed no meaningful lift at all. Larger, lower income households are the most price sensitive shoppers in the base, and a well targeted coupon moves them furthest. The most profitable way to target this campaign and the most equitable way to target it are the same strategy.
The real test of a promotion built to earn loyalty isn't what happens while the discount is live. It's what happens the week after, once it's gone.
Only 16% of the lift survives once the promotion ends. This is consistent with pantry loading rather than habit formation. A single large campaign is unlikely to build lasting engagement on its own. A steadier cadence of smaller, rotating promotions would likely produce more durable results per dollar spent.
A common risk with coupon promotions is that shoppers buy only the promoted product and nothing else. The basket data says otherwise here.
Redeemers shopped 9.3 distinct departments on average during the campaign, against 8.4 for non-redeemers. The coupon produced a genuine cross sell halo: it drew people toward a wider basket, not merely toward the one item on sale.
Five findings are useful. A working tool is better. This is an uplift model trained on this campaign's real household data, using a two model (T-learner) approach: one model learns how redeemers' spending changed, a second learns how non-redeemers' spending changed under identical conditions, and the gap between their predictions for any given household profile is that household's personalized expected uplift from being targeted.