Our other articles are written for the owner deciding whether to use AI. This one is for the person who'll actually run it — the manager or designated "AI person" who has to make an agent behave on a Tuesday afternoon when a customer asks something nobody anticipated.
Every business AI agent that works in production — answering customers, taking bookings, triaging enquiries — is made of the same four layers. Vendor demos usually show you the first two. Whether the thing survives contact with real customers is decided by the other two.
Layer 1: Knowledge — what it's allowed to know
A business agent doesn't "know" your business the way a trained model knows language. It works by retrieval: when a question comes in, the system finds the most relevant passages from the documents you supplied and composes an answer constrained to them. No relevant passage, no answer — that's a feature.
Three things follow from this that practitioners get wrong less often than vendors imply:
One fact, one place. If your delivery fee appears in four documents, it will eventually appear in four versions, and retrieval will confidently serve the stale one. Structure your knowledge so every fact has a single home. This is unglamorous document work, and it determines more of the agent's quality than the model choice does.
Write for retrieval, not for reading. Short, self-contained sections with explicit headings beat long narrative pages. A passage that answers "do you open on public holidays?" should contain the words a customer would use, the answer, and nothing that contradicts another passage.
Someone owns freshness. Prices change, policies change. If nobody is named as the person who updates the knowledge base, the agent's accuracy has a half-life. Put a name and a review cadence on it from day one.
Layer 2: Rules — what it's allowed to say
The rules layer is the standing instruction the agent operates under: what it may discuss, what it must never compose, how it should sound, and what to do at the edges. A serviceable core looks like this:
"Answer only from the provided business information. Never state or invent prices, discounts, refunds, or commitments not written there verbatim. If the answer isn't in the material, or the customer is upset, or the request needs an exception — say so honestly and hand over to a person. Never guess."
Two practitioner notes. First, rules are strong defaults, not physics — a determined user can sometimes talk a model out of its instructions. That's precisely why layers three and four exist; rules alone are not a safety strategy. Second, keep the rules readable by humans. If your escalation policy lives only inside a prompt nobody on the team has seen, it isn't a policy — it's a secret.
Layer 3: The handoff — when it must stop talking
This is the layer that separates toys from production systems. Define, explicitly, the triggers that end the AI's turn: it isn't confident in the answer; the topic is sensitive (complaints about what actually happened, anything legal, anything safety-related); the request needs authority (refund, exception, discount); or the customer is escalating emotionally.
Then design what a good handoff actually delivers. The customer should never feel a dead end — "let me get a person" with a real route beats a loop of apologies. The human should receive context — who asked what, what the agent already said, what it couldn't resolve — not a bare notification. And handoffs should be fast to act on: an escalation nobody sees for six hours is a complaint in the making.
A useful design question to force clarity: what is the most expensive sentence this agent could say? Work backwards from that sentence until a rule, a knowledge constraint, or a handoff trigger makes it impossible.
Layer 4: Visibility — how you know it's still working
An agent nobody reviews degrades silently. Documents go stale, customers ask new things, edge cases accumulate. The visibility layer is the practitioner's actual job:
Transcripts you can read, and a habit of reading them — a weekly sample of ten conversations tells you more than any dashboard. Numbers that mean something: how many conversations were handled cleanly, how many were handed over, how many were abandoned mid-way — and which questions produced the handovers. And a loop: what you learn goes back into the knowledge base and the rules. The agent doesn't improve itself; you improve it. That's not a flaw — it's what "the business sets the rules" looks like in practice.
The checklist
Whether you're evaluating a vendor or building in-house, the four layers become eight questions: Where does the knowledge live, and who updates it? What happens when two documents disagree? Can I read the standing rules in plain language? What, verbatim, can it never say? What triggers a handoff, and what does the human receive? Where do transcripts go, and who reads them? Which numbers will tell us it's degrading? When we learn something, where does the fix go?
A vendor who has crisp answers to all eight is selling a system. A vendor who shows you a smooth demo and shrugs at question five is selling you a liability with good manners. (For transparency: this four-layer architecture is exactly how we build BizMate, the service agent we run ourselves — which is why we're opinionated about it.)
If you read our owner-level piece on stopping AI from making things up, you'll recognise the three boundaries — this is what they look like when someone has to operate them.
And if you'd rather someone ran the plumbing under all four layers for you — hosting, gateway, monitoring — that's what our Platform → is.