Studio notes — agents built to be checked

Every week, the same fleet of agents is at work across several clients at once. This week, two separate builds share the same thread: at a courier company, the studio added a human review queue to an agent that triages and handles incoming email; at a training organization, it restructured a compliance audit into verifiable steps, with automatic reading of supporting documents. Two very different businesses, the same discipline underneath: an AI agent only earns trust if you can check, at any moment, what it did and why.

An email agent that proposes, but never decides alone

At the courier company, the agent reading the incoming mailbox no longer just sorts messages: it turns each email into a case, drafts a plan of action — reply, follow up, escalate to a person — and waits for sign-off. Every action sits in a queue until an operator accepts, rejects or edits it, and every AI decision is logged so it can be replayed or audited later. A landmark study by John D. Lee and Katrina A. See, Trust in Automation: Designing for Appropriate Reliance, published in 2004 in Human Factors, shows that trust in an automated system cannot simply be declared: it is built when the user can calibrate reliance on the machine — neither too much nor too little — which requires understanding what it actually did. Lesson for a business owner: an AI agent on a customer-facing channel (email, tickets, complaints) earns real adoption not by automating everything, but by keeping every action visible, reversible and logged before it executes.

A compliance audit broken into verifiable steps

At the training organization, the agent that prepares the compliance audit (Qualiopi-style) was restructured around roughly forty fixed steps: every document or link a user uploads is now read automatically — PDF, image, web page — and filed straight into the matching step, with an admin panel to correct, reassign or delete any piece of evidence if something goes wrong. A 2009 study published in the New England Journal of Medicine by Alex B. Haynes and colleagues, covering a surgical checklist deployed across eight countries, measured a drop in major post-operative complications from the simple act of breaking a complex procedure into short, individually checkable steps — a finding that holds just as well for an administrative audit: decompose, don’t just automate. Lesson for a CIO or quality manager: automating a compliance audit is only useful if every step stays individually verifiable and correctable by a human — an audit nobody can inspect isn’t a time saver, it’s one more risk at inspection time.

What connects both builds this week: an AI agent is only as good as what you can verify of its work — a review queue on one side, inspectable audit steps on the other. Before letting an agent act alone on a customer channel or a regulatory obligation, the studio always asks the same question: if this decision has to be checked tomorrow, do we actually have what it takes?