A/B Testing That Sticks

Incrementality testing: the measurement that tells you if a program actually works

Last-click attribution makes lifecycle look bigger than it is. Incrementality testing strips out users who would have converted anyway and surfaces the real number. This is how to design a test that produces a figure you can defend in front of a CFO.

9 min read

Holdout group design: the incrementality tool most lifecycle programs skip

Without a holdout, lifecycle ROI is attribution-model guesswork with a spreadsheet. With one, you get a defensible number you can actually put in front of finance. Here's how to size, run, and read a holdout — and the three mistakes that quietly invalidate the result.

9 min read

Attribution models for lifecycle: which one to defend in which room

Attribution debates are half epistemology, half politics. Last-touch is wrong but defensible. Multi-touch is more accurate but less defensible. Incrementality is the only one that answers the causal question — and it's the slowest. Here's which model to use for which question, and why.

10 min read

Price-testing through email: what's testable, what isn't

Email is the fastest place to try a new price, and the easiest place to learn the wrong lesson. What you can test cleanly, what you can't, and the measurement traps that quietly turn price tests into expensive false positives.

9 min read

Churn cohort analysis: the one chart that tells you if retention is actually improving

A cohort retention curve is the single most useful analytical artifact in lifecycle marketing. It isolates real program impact from the compounding noise that every other metric hides, and it's the one view that survives every limitation of the simpler numbers. Here's how to build one and how to read it without kidding yourself.

9 min read

Inbox placement testing: seed lists, their limits, and what to do instead

Seed-list tools report '82% inbox, 18% spam' and programs treat that as gospel. It's not. The number is directionally useful and literally wrong, and most programs read it more literally than they should. Here's what it can and can't tell you.

8 min read