How to use this kit

Start with a question you’d act on. Something like, “Does this channel create sales, or just take credit?”

  1. Run a power analysis on your own geo-level history and log the test in the Geo Test Ledger with its Statistical power %. For an early, directional read, try the Geo Test Signal Planner (https://brinkeranalytics.com/geo-test-calculator). Its Synthetic signal score uses made-up sales data and is not statistical power. If your statistical power is low, run the test longer first. If that is not enough, add spend in the treatment cell (prorated to its share). Then grow the cell. For how to think about power, see brinkeranalytics.com/insights/geo-test-power-check.
  2. In the DMA Cell Planner, add each market you’re putting in the test, mark it Treatment or Control, and link it to the test in the Ledger. It starts empty, so you only list the markets you use.
  3. When results come in, add them to the Ledger and update the Channel Measurement Scorecard.
  4. Where your methods disagree, plan the next test.

Definitions: Treatment cell = the markets where media stays on. Control cell = the markets where the channel is paused or held out. Statistical power % = chance the test detects a lift of the planned size if it is real; it needs a power analysis on your own geo-level history. Planner signal score = directional only, from synthetic data, never a substitute for power. MDE = the smallest lift the test can reliably detect. Eric’s lever order when power is low: (1) extend the test, (2) add spend in the cell prorated to its share, (3) grow the cell. Rows marked “EXAMPLE DATA, not real results” are samples; filter on the Example data checkbox and delete them.

Geo Test Ledger = one row per test. DMA Cell Planner = one row per market you assign. Channel Measurement Scorecard = one row per channel, comparing lift test, MMM and platform ROAS (use the same revenue definition for all).

<aside> 💡

No model owns the truth. A test, an MMM, and platform reporting each see part of the picture. A gap between methods is a reason to dig in, not proof that one method is wrong.

</aside>

Channel Measurement Scorecard

Geo Test Ledger

DMA Cell Planner