Measurement
Part of Marketing automation measurement: a reporting framework
Marketing automation benchmark research: how to appraise
Marketing automation benchmark research: how to appraise populations, denominators, commercial interests and expiry rules before you trust an external figure.
Marketing automation benchmark research begins by asking whether a published number describes the same population, event and period as the proposed decision. An attractive percentage with no denominator or collection method is not a defensible target.
What to take away
- Official business statistics describe populations but do not measure automation results or platform performance.
- Record publisher, sample, event definition and commercial relationship before accepting any external benchmark.
- Vendor and analyst benchmark reports are appraised with the same record as official statistics.
- A versioned internal baseline can be more relevant than a broad external average for your decision.
- Leave the target unset until a comparable source or approved internal baseline exists.
Separate context from performance
Official business statistics can describe the economic population but do not automatically measure automation results. The ONS UK business activity, size and location dataset includes regional tables; its March 2024 edition reported 2.8 million enterprises in the UK, of which about 2.4 million were in England. Its quality and methodology information explains register coverage, enterprises, local units and geography.
Those sources may help an analyst decide whether a business sample resembles organisations in England. They contain no general email conversion, journey completion or marketing-automation ROI benchmark. Do not use a business count as evidence for platform performance.
Build a benchmark evidence record
Capture the publisher, title, URL, publication and fieldwork dates, geography, sample frame, achieved sample, weighting, collection mode, product or customer eligibility, channel, event definition and commercial relationship. Note whether the underlying data and calculation can be inspected. Where a benchmark rests on contact data, record the lawful basis for the campaign under the UK GDPR and the Privacy and Electronic Communications Regulations, which the Information Commissioner's Office administers.
Benchmark evidence record fields
- Publisher, title and URL
- Publication and fieldwork dates
- Geography and sample frame
- Achieved sample and weighting
- Collection mode and channel
- Event definition and commercial relationship
The 2026 UK Business Data Survey is official UK research with stated sampling, fieldwork and limitations. Its digital-data and AI findings can provide context when the exact population matches. They should not be translated into an automation-adoption or performance rate that the survey did not publish.
Use the Code of Practice for Statistics themes of trustworthiness, quality and value as appraisal questions. A vendor study does not become an official statistic by passing the checklist, and commercial sponsorship should remain visible.
The benchmark material a UK marketing team is most often handed comes from platform suppliers and research firms. HubSpot, Mailchimp and Klaviyo publish their own benchmark figures, and analyst houses such as Gartner and Forrester publish evaluations that teams use to compare platforms. Both are appraised with the same record: who published the figure, which population and event it measures, and what the publisher sells. A commercial source is not disqualified by being commercial, but the commercial interest belongs in the record.
Appraising a platform's own benchmark report
Take the kind of report a platform publishes about its own users, such as Mailchimp's email marketing benchmarks by industry or Klaviyo's benchmarks for e-commerce senders. Run it through the record:
- Population:the platform's own customers who chose to send through it, not UK organisations in general, and often weighted towards that platform's typical sender, such as online retailers for Klaviyo.
- Event:usually opens, clicks and unsubscribes. Since Apple introduced Mail Privacy Protection in 2021, an open can be recorded by the mail app rather than the reader, so an open-rate benchmark measures something different from a booking or purchase.
- Denominator:check whether rates are calculated on emails sent or on emails delivered; the two differ by the bounce rate.
- Geography and period:a global average may not match a list in England, and a year-old period may not match current inbox behaviour.
- Commercial interest:the publisher sells the platform, and a favourable average helps sales; record that, then judge the method.
A click-rate benchmark from such a report can be usable as context for a click-based internal measure with the same denominator. An open-rate benchmark is usually "context only", and neither can set a target for a booking or revenue event the report does not measure. The same appraisal applies to an analyst evaluation such as a Gartner Magic Quadrant or Forrester Wave: it rates vendors on the analyst's own criteria, which tells you how platforms compare on those criteria, not what response your journey should achieve.
Define the comparable metric first
For an "eligible response rate" benchmark, record:
- Populationunique eligible recipients in England under the named audience and channel rules.
- Numeratorpeople completing the defined response event once.
- Denominatorall eligible recipients, including non-delivery unless the benchmark uses another base.
- Unit, period and timezonepercentage for a fixed follow-up window with Europe/London cut-offs.
- Inclusions and exclusionsproduction traffic, named journey versions, test records, duplicates and invalid responses under stated rules.
- Source and ownerfrozen audience, delivery and response tables plus versioned calculation, with an analytics owner and statistical review.
Reject an external comparison that cannot be mapped to this card. If the source excludes non-delivery while the internal denominator includes it, show two separately labelled calculations rather than forcing a single league table.
Worked example (illustrative internal figures). A frozen May 2026 campaign sent 10,000 emails to eligible recipients in England. The defined event is a completed booking within 7 days. The response table shows 620 bookings and 380 hard bounces.
With all eligible recipients as the denominator, the internal baseline is 620 / 10,000 = 6.2%. With non-delivery excluded, it is 620 / 9,620 = 6.4%.
The external candidate is the ONS March 2024 figure of about 2.4 million enterprises in England. That count describes a business population, not campaign response, so it is marked context only. The internal baseline to use is 6.2%, with 6.4% labelled delivered only.
Give every accepted benchmark an expiry rule. Recheck it when the publisher changes methodology, the source period ages beyond the decision window, or the internal channel and audience no longer match. Preserve the version originally used so reviewers can reconstruct an earlier decision.
Prefer an internal baseline when it answers the question
A versioned internal history can be more relevant than a broad external average. Freeze definitions, annotate releases and compare like periods while noting seasonality and operational breaks. It remains observational evidence.
For claims about incremental effect, the 2026 Magenta Book provides evaluation concepts including counterfactual and theory-based approaches. It does not supply a private-sector benchmark. Ask a statistician to determine whether the design supports comparison or causation.
Finish the research note with "usable", "context only" or "reject", followed by reasons. Record the next update trigger and owner.
Before you act
- Check the benchmark describes the same population, event and period.
- Capture the publisher, sample frame, weighting and collection mode.
- Define the comparable metric before comparing any external number.
- Label separate calculations when denominators differ between sources.
- Give every accepted benchmark an expiry rule and owner.
- Ask a statistician whether the design supports comparison or causation.
Common questions
Why can an official business dataset not serve as a marketing automation benchmark?
It counts enterprises and local units, not campaign events. The ONS March 2024 edition reported about 2.4 million enterprises in England, but that figure says nothing about email conversion or journey completion. It cannot support an automation target.
What should a benchmark evidence record contain?
Record the publisher, dates, geography and sample frame. Add weighting, collection mode and any commercial relationship. Note whether you can inspect the underlying data and calculation. Use the Code of Practice themes as appraisal questions.
When should an internal baseline be preferred over an external benchmark?
When its frozen definitions, channel and audience match the decision better than a broad average. A versioned internal history is observational evidence. For incremental effect, ask a statistician whether the design supports comparison or causation.
