Which Canvas checks a machine should own, and which it shouldn't

Picture the hour before a Canvas goes live. The build has been signed off, the copy is approved, someone has already told the wider team it launches this afternoon. Now a person opens a forty-item checklist and starts ticking. By item twelve they are pattern-matching rather than verifying. By item thirty they are ticking things they have not looked at, because everything so far was fine and the meeting is in ten minutes.

That is not carelessness. It is what a flat list does to a tired person — exactly the failure mode automation is good at removing. But automating the whole list is the wrong correction, because a meaningful fraction of the items are not verification at all. They are judgement wearing a checkbox.

The useful cut is decidability. If a check has one correct answer that can be established by reading something, hand it to a machine and stop spending human attention on it. If the check is really asking whether this is the correct call, no amount of tooling substitutes for someone who knows what the program is for. Sorting the list this way usually shrinks the human half to a size a person can actually hold at 4pm on a Thursday.

These have right answers. Every one of them compares what is configured against what was specified. A person reading them off a screen is strictly worse at it than a script.

Template binding, by content fingerprint. The single highest-value automated check, because it catches the mistake that costs the most. The Braze API's message-step title field is usually null, so you cannot verify a binding by name. You verify it by content: fetch the bound template, then fingerprint on subject line plus preheader plus one distinctive block that only exists in that variant — an upsell card, a tier-specific paragraph. That is how you catch the free-tier email bound to the paid branch, which looks completely normal in the flow diagram.

Liquid fallbacks on every variable. Every personalisation token needs a default. A missing fallback does not fail loudly at build time; it fails on the subset of users with a null value, which is always larger than anyone estimates. Scanning a body for unguarded variables is trivially mechanical and nobody does it reliably by eye.

Links, UTMs and the staging-domain check. Every CTA resolving, no test or staging hosts, tracking parameters consistent. Also worth automating: CTA parity, meaning two buttons with the same visible label cannot point at different destinations.

Compliance furniture. Unsubscribe link present, physical address present, sender and subscription group correct. These are pass or fail. They are legally load-bearing. They also vanish first when a template is rebuilt from a module that never had them.

Size after link-wrapping, and dark mode. The byte budget has to be measured after the ESP wraps every link for click tracking, because that is what actually gets delivered and it is meaningfully larger than what you built. Gmail clips past its limit. A clipped email loses its unsubscribe link — which turns a rendering nuisance into a compliance one. Dark-mode inversion is likewise a render fact, judged on a delivered seed send rather than a preview pane.

Conversion-event count and delay values, read back as configured.Braze caps conversion events at four per Canvas. Counting them is machine work. Reading each delay's actual duration back from the configuration — rather than from the build spec that says what it was supposed to be — is machine work too. That is where accidental zero-delay back-to-back sends get caught.

Now the half that does not automate, and will not, because the input a machine would need is intent — which lives in a person's head or a PRD, and if it lives in the PRD you are back to a human reading it.

The pattern across all six: the machine establishes what is true, the human decides whether it is right. Confusing those two is how a QA pass produces a confident green tick over a journey that is internally consistent and strategically wrong.

Most attempts at automating this break on a detail nobody writes down. Several of the machine-ownable checks cannot be done through the Braze API at all, because the API does not expose the configuration they inspect.

So "a machine can own this check" sometimes means a script hitting an endpoint, and sometimes means a model driving the Canvas Flow editor in a browser: opening each gate, clicking each step, reading the configuration off the screen. Both are automation. Only one of them works for delays and filters. Assuming the wrong one is how a QA report ends up asserting things nobody verified.

That gives a blunt rule for reading anyone's Canvas QA output, including your own tooling's: every quantitative claim — a segment size, a step count, a delay value — has to come from a live read performed during that pass. Not from memory, not from the build spec, not from an estimate that sounds plausible. A number in a QA report with no read behind it is the report's weakest point. It is usually the number someone acts on.

Whoever or whatever runs the pass, these hold. They are constraints on the QA process itself rather than checks on the Canvas.

Never launch during QA. Use plain Save. Do not click through toward a launch state to see what happens, because on a live Canvas what happens is that it launches. QA is not a licence to ship. A tool given QA scope should not be given launch scope in the same session.

Keep entry on a test audience for the whole pass. Restore it if you change it. The single most expensive mistake available here is leaving the real audience attached to a Canvas you are poking at. If you swap the audience to test, the pass is not finished until it is swapped back and confirmed.

Verify the settings that lock before anything else. Conversion events cannot be added or removed after launch. Re-entry settings are fixed once live. Those items belong at the top of the pass, not buried at item thirty-one, because a wrong conversion event discovered post-launch means rebuilding the Canvas to measure the thing you were always trying to measure.

Test sends deserve one specific trick: embed the step name or the user ID in the test body via Liquid. Six test emails in an inbox all look alike. Without a marker you cannot tell which step produced which, so the test send verified that mail arrives rather than that the right mail arrives from the right step.

Take your current Canvas checklist and mark each line M or H. Machine-decidable or human-judgement. Most teams discover the list is roughly two-thirds M, which means two-thirds of the attention currently spent on ticking boxes is being spent on the part a tool does better and never gets bored doing.

Then hand the M items over and protect the H items. Give them the front of the pass, while whoever is running it is still reading properly. The point of automating the mechanical two-thirds is not speed — it is that the conversion event, the group ranking and the catch-all routing finally get looked at by someone with attention left.

One structural note worth taking seriously: if the specification the Canvas is being checked against lives only in the head of the person doing the checking, this exercise has a ceiling regardless of tooling. A checklist compares configuration to intent, and intent needs a written home. That is the source-of-truth argument. It is upstream of everything here. If you are still deciding what access the QA tooling should have, start read-only — QA needs to see everything and change nothing.

The full severity-rated version, with the go / no-go call at the end, is in the Canvas QA skill.