Marketing automation supplier comparison: a ready-to-run operator test script

Scripted task: A repeatable slice of the buyer's case that every supplier performs.

Quoted environment: The configuration and scope included in the supplier's quotation.

Buyer-side operator: An ordinary user who repeats the build without relying on the supplier's presenter.

Supplier intervention: Help or action from the supplier during the test, recorded with its time and reason.

Evidence log: The records kept to show what happened at each test step.

Mandatory failure: A pre-agreed result that rules out a supplier regardless of its other scores.

Walk-away point: The boundary the buyer will not trade away to retain a preferred bidder.

This guide gives the script and the scoring log, not a league table. Results depend on the edition, contact volume and region in each supplier's quote, so a published ranking would not transfer to your case. Run the script on each shortlisted supplier, for example HubSpot, ActiveCampaign, Brevo, Dotdigital or Klaviyo, and fill in the log yourself.

The scripted task sets a common starting point

A useful comparison starts with one task that each candidate must perform. It should be a manageable slice of the buyer's case, not a broad product tour, and it should test what the proposed service can do rather than what a supplier says it could do.

Build the task from a one-page case. Include the two initial journeys, current systems, users, channels, environments, integration frequency and support window. Decide the test order in advance, then send the same script to every supplier on the same day so each candidate receives equal notice.

The task should expose work that a presentation might leave out, including migration, controls and departure. Keep the script short enough to repeat consistently, but specific enough to show how the offered configuration behaves.

The script: inputs, steps and expected outcomes

Use the same test data in every supplier's environment. Prepare a file of 20 test contacts with fields for email, first name, consent status, source and sign-up date, plus three seed addresses you control. Mark two contacts as suppressed and one as unsubscribed. Never use real customer data in a trial.

A pass on step 4 is not optional. The ICO's direct marketing guidance explains why an unsubscribed contact must not receive marketing, so treat any message sent to a suppressed test contact as a mandatory failure.

The buyer-side operator tests repeatability

The buyer-side operator reveals whether the work can be repeated by someone who is not the supplier's best presenter. Give the proposed delivery team the task first, then ask an ordinary operator from the buyer's side to repeat it in the quoted environment.

The repeat build is the central test. If the proposed team completes it but the buyer-side operator cannot follow the same process, the product or its documentation may be the obstacle. The score should make that difference visible rather than treating the supplier team's performance as proof of buyer-side usability.

The evidence log makes each score traceable

Score actions that were demonstrated, not features described in a call or promised for later. Keep records from the quoted environment and connect each result to its test step. If a supplier offers no demonstration of a feature, that feature earns no credit in this comparison. Mark each script step pass, fail or not shown, and add the time taken.

Record supplier interventions alongside the evidence. This shows whether the result depended on help that a buyer would not have during ordinary operation. Keep source dates, assumptions, unanswered questions and rejected options with the test record, so the eventual decision can be traced back to what each candidate actually did.

Supplier intervention reveals delivery ownership

Name the legal entity responsible for every work package. The platform publisher, implementation partner and buyer may have different roles, so a successful test should be attributed to the entity that performed the work, not automatically to the brand selling the service.

Ask for a responsibility matrix covering data preparation, architecture, consent and suppression migration, templates, integrations, quality assurance, administrator training and post-launch support. References and certifications can inform diligence, but they do not prove delivery for this case. The contract should provide the buyer with the same people and environment used in the test.

Assurance checks stay with their specialists

Keep law, privacy, security and accessibility as distinct assessment areas. The NCSC cloud security principles can structure questions about identity, administration, data protection and audit, including audit information and alerting. WCAG 2.2 supplies technical accessibility criteria for representative authoring and output tests.

For personal information, the ICO guidance on contracts and liabilities describes processing terms, including subprocessors, security, audits and end-of-contract return or deletion. The page is under review following legislative change and does not cover the entire commercial bargain. Named UK legal, privacy, security, accessibility and procurement specialists should assess their own evidence areas.

Mandatory failures establish the walk-away point

Set mandatory failures before scoring suppliers; the quality checklist covers audience logic, permissions and recovery. Examples include an unpriced essential component, an unenforceable required publishing role, an unresolved restricted transfer, no acceptable export, or a test suppression that still receives a message. Do not waive a control simply to keep a favoured candidate in contention.

Place the quote expiry and a revalidation date beside the result, since packaging can change. Before approval, check the scope again, compare the price against the pricing models, and review the contract. The buying guide adds the wider evidence ladder, while the selection checklist sets out checks before contract.