8.1Real email campaign data from an open randomised experiment
The method gets the known answer
estimated effect, against a true 8.2; comparing who got the email with who did not said 11.8
- What was observed
- A public experiment emailed customers at random, so the true effect is known. We made it look like a targeted campaign, emailing the most promising customers first: for them, the email added 8.2 website visits per 100 customers.
- What a generic approach says
- Compare customers who got the email with those who did not: 11.8, overstating the effect by 45%.
- What the engine read
- A method that corrects for who was chosen for an action estimated 8.1, within 0.5% of the truth averaged over 10 targeted samples; single samples ranged from 6.9 to 9.3, and the 90% interval contained the truth in all 10. Run where there was no effect, it found none.
- What it meant for the decision
- The effect of a contact can be measured on records where contacts were targeted, which describes every collections book. The outcome here is a website visit, not a payment: no public dataset records collections contacts, and your own data closes that gap.