How we let our agents get smarter without letting them go rogue.

Everyone wants an AI that improves over time. Almost nobody wants one that quietly rewrites its own instructions overnight.

"Can it learn?" is one of the first questions we get asked. The honest answer is yes — but not in the way most people picture it, and the way most people picture it is the version you don't want.

AI AGENTSTHE APPROVAL GATEnightly transcriptsDE-IDENTIFIEDadd parking directionsclarify the free consultannounce it's an AIHUMANREVIEWapproved · live agentSHIPS TO PRODUCTIONdismissedNEVER SHIPSNothing ships unread.The first run proved why.

Nothing reaches a live agent unapproved.

— That's the whole guarantee.

How ours actually works

Every night, a process reads back through the day's call transcripts for each agent and files suggestions: questions the agent couldn't answer, places where callers got confused, gaps in the FAQ. De-identified. Those suggestions land in a pending queue on the client's admin page.

Then a person reads them. Approve, and it becomes part of the agent's instructions. Dismiss, and it disappears. Nothing touches a live agent automatically.

That last sentence is the entire guarantee, and the very first real run showed exactly why it's there.

It surfaced genuinely useful things. At one location, callers kept asking about parking and nearby landmarks — nobody had thought to include directions. At another, callers didn't understand what the free consultation actually covered, which is a booking problem disguised as an information problem. Both worth fixing, and neither was something we'd have guessed from the outside.

It also suggested that one agent should start proactively announcing it was an AI.

That's a change we had deliberately decided not to make for that client, for well-considered reasons. The suggestion wasn't stupid — you can argue for it. But it was a judgment call, and it belonged to a human.

An auto-applying system would have shipped it overnight. It would have quietly changed how a client's business introduces itself to its own customers, and nobody would have known until someone listened to a call.

Two things we learned running it

Treat approvals as edits, not as collecting. Every approved suggestion makes the instructions longer, and long instructions degrade how reliably any of them get followed. It's tempting to approve everything reasonable. Don't. Prune as you go.

Per-caller memory is a different question, and we've deliberately left it off. For agents that read from a CRM, a live lookup beats stored memory every time — memory is just a stale copy that will eventually contradict your system of record and confidently tell a customer something that isn't true anymore.

What this means for you

When a vendor tells you their AI "learns," ask exactly one question: what reaches the live agent without a human seeing it first?

If the answer is anything other than "nothing," you've handed your customer-facing voice to a process nobody is reading. That might be fine for a chatbot on a pricing page. It is not fine for the thing that answers your phone.

Want us to look at where a phone agent could take work off your plate? Grab a free workflow audit.

← Back to all posts Free Workflow Audit →