Six marketing automation measurement mistakes, from hidden denominators to imported benchmarks

Marketing automation measurement mistakes often begin before analysis: the decision is vague, the population shifts or the evidence label overreaches. Fixing the chart afterwards cannot recover a missing counterfactual or an unknown denominator.

What to take away

How the list was chosen

Research ended 5 September 2026. A mistake was included when a current UK regulator, government evaluation or official-statistics record supplied a direct quality boundary for organisations operating in England. The list excludes product rankings, invented benchmarks, supplier performance and errors that only change presentation. It is non-ranked, with no affiliate, sponsor or supplier conflicts.

1. Choosing events before the decision

Collecting every available click produces activity, not a decision model. The GOV.UK Service Manual advises teams to define what they need to know and then select data. Write the action, mechanism and primary outcome first.

For an illustration, counting every page view might show 10,000 views, but it cannot answer whether a renewal reminder reduced cancellations. If the decision is whether to send that reminder, the primary outcome is the cancellation rate within 30 days, measured against accounts due to renew in that window.

2. Hiding the denominator

A rate can rise while its eligible base shrinks. Publish numerator, denominator and unresolved records beside the percentage. Government guidance on measuring service success also stresses matching methods to the question; a raw count and a rate are not interchangeable answers.

For an illustration, a 5 per cent rate from 1,000 eligible records is 50/1,000. If the eligible base later falls to 400 while the numerator stays at 50, the rate is 12.5 per cent even though the underlying count has not improved.

3. Calling platform credit an incremental effect

A rule that assigns an order to a touchpoint describes tracked credit. The Magenta Book distinguishes observing outcomes from determining whether an intervention caused them. Use experimental or justified causal methods before making an incremental claim.

For an illustration, a last-click rule might credit a closing email with £100 of revenue from a sale that would have happened after a paid search click. A holdout test or a justified causal design is needed before calling that £100 incremental.

4. Treating missing tracking as no behaviour

People can be absent from an event table because collection was unavailable, declined or blocked. ICO guidance covers storage and access technologies including cookies, pixels, tags and scripts. A qualified reviewer must assess the actual technology. Report unknown observation separately from observed non-response.

For an illustration, if 200 of 1,000 checkout events lack a consent record, reporting zero conversions for those 200 treats a tracking gap as no purchase. Report the 200 as unknown observation, not as observed non-response.

5. Importing an incompatible benchmark

A number from another geography, channel, period or denominator is not a like-for-like target. The ONS business-register methodology shows how enterprise, local-unit, register-coverage and geography definitions affect interpretation. Appraise the benchmark's population and collection method before comparing it.

For an illustration, a 7 per cent conversion rate from a national retail campaign may not apply to a local B2B software trial. That trial might have 300 invited accounts and a different definition of a conversion. Check whether the benchmark counts the same unit, period and population as your own metric.

6. Rewriting a metric after seeing the result

Changing the window, exclusions or identity rule can turn one question into another. The Code of Practice for Statistics centres trustworthiness, quality and value. Private reporting is not official statistics, but preserving definitions, changes and limitations follows the same useful discipline.

For an illustration, moving a reporting window from 30 to 90 days after seeing a conversion dip can raise the rate from 2 per cent to 4 per cent. No customer behaviour has changed. Keep the original definition and publish the break if you must change it.

Repair card for the affected metric

Record the population, numerator, denominator, unit, reporting period, timezone, inclusions, exclusions, source tables and query version. Add known missingness, model uncertainty, accountable owner and an explicit update trigger. For a count with no denominator, write "not applicable" rather than leaving the field empty.

For example:

Reproduce the old and corrected calculation on the same frozen extract. If they answer different questions, do not present the change as a routine correction. Give each definition its own version and mark the break on every chart that spans it. Check downstream exports, board packs and automated alerts, not only the source dashboard.

Stop publication when the eligible base cannot be reconstructed, a legal collection question remains open or causal wording exceeds the design. The correction note should preserve the old result, explain the defect and show when dependent decisions will be reconsidered.

Before you act

Common questions

Why is hiding the denominator a problem in marketing automation measurement?

A rising percentage can simply mean the eligible base has shrunk, so publish the base alongside the rate.

What is the difference between platform credit and incremental effect?

Credit is an attribution rule; incremental effect is a causal claim that needs an experimental or justified design.

How should missing tracking data be treated in analysis?

Treat absence from an event table as unknown observation, not as evidence of no behaviour.