Gathering requirements for an AI agent: a guide for business analysts

You were asked to look into AI agents, and somewhere along the way you became the person who owns the requirements. The requirements for an AI agent are the set of decisions that determine what it says and does in conversation: the job it handles, the systems behind each answer, the data it touches, the line where a human takes over, and who signs off on all of it. A template cannot supply any of that. You get it by interviewing the people who own each decision, then writing their answers down in a form someone can veto.

What do you need to know before anyone builds?

Seven things. If you can answer all seven, an engineering team or vendor can build the agent without inventing behaviour on your behalf. If you cannot, they will invent it anyway, and you will discover their choices during UAT, when changing them is expensive.

1. The job-to-be-done

“Answer customer questions” is a wish. A job is specific and has an end state: taking a card-dispute report from first message to a filed case, for example, or moving a delivery date and confirming the new one. Pick one job for the first agent. Everything else in the requirements hangs off this choice, because a turn-by-turn conversation can only be written about a job with a beginning and an end.

2. The system behind each answer, and who owns it

Every useful thing an agent says is backed by a system: the order status lives in the order-management system, the balance in core banking, the delivery window with the carrier. For each answer the agent will give, you need the system it comes from, whether an API to it exists, and the name of the person who owns it. The owner matters more than the API. APIs get built; owners decide whether they will be, and they are the people missing from most kickoff meetings.

3. Data sensitivity

List what the agent reads and, separately, what it writes. Reading a balance and changing an address are different risk classes, and your security and legal colleagues will treat them differently. Flag anything personal, regulated, or destructive now. A data question that surfaces in month four can pause a project; the same question in week two is an agenda item.

4. The escalation line

Decide exactly where the agent stops and a human takes over: on explicit request, on detected anger, on legal keywords, after a set number of failed attempts. Then decide what the handover carries, because a customer who has to repeat their story to the human has just experienced the agent’s failure twice. The agent’s job ends at the handover. Do not script the human side.

5. What the agent must never do

Every stakeholder holds a short list of catastrophes: never quote a refund amount before the system confirms it, never speculate about a fee, never promise a callback time nobody can keep. These rules are cheap to collect in an interview and expensive to discover in production. Write each one as an explicit constraint with the reason attached, so the next person to read it understands why the rule exists.

6. Volume and channels

How many of these conversations happen in a week, in which channels, and what does the worst week look like? Channel shapes the design directly: an agent on SMS cannot show a form or a card, and a voice agent cannot show anything at all. Volume turns percentages into people. A failure path that catches two percent of conversations is forty customers a week at two thousand conversations, and that is a number a sponsor can react to.

7. The sign-off chain

Find out who can veto wording, who approves the data the agent touches, and who carries the exposure if a sentence turns out to be wrong. Get names, not role descriptions. The chain is only real if the last person on it will actually read what the agent says, which is one more argument for writing requirements as conversation.

What should you ask stakeholders?

Interview like a colleague. These questions get you most of the seven answers without asking anyone to “define requirements”.

• “Walk me through the last time a customer needed this. What actually happened, message by message?”
• “Where does that answer come from today? Who looks it up, and in which system?”
• “What happens when that system is slow, or down?”
• “What should this agent never say, even if a customer asks directly?”
• “When a customer gets angry today, who takes over, and what do they need to know at the moment they pick it up?”
• “If a sentence in this conversation turned out to be wrong, who gets the phone call?”
• “How many of these conversations happen in a normal week? What did the worst week look like?”
• “What would make you comfortable putting your name on this?”

Record the answers as conversation wherever you can. When someone describes the angry-customer case, do not write “agent responds empathetically”; write the sentence they would want the agent to say, and read it back to them. The read-back is the fastest validation loop available to you. People who nod at a bullet point will interrupt a sentence.

Where does requirements gathering go wrong?

Three traps do most of the damage.

Trap one: collecting intents instead of conversations

The classifier-era chatbot playbook says to gather intents: check_balance, dispute_charge, cancel_card, each with a stack of example utterances. An intent list describes what customers want; it says nothing about what the agent does next, asks for, or answers with. You cannot put an intent in front of a sponsor and ask for approval, and an engineer holding one still has to invent the entire conversation. Collect conversations instead: the actual sequence of turns, including the customer’s side.

Trap two: writing prose nobody can veto

“The agent should respond empathetically and escalate where appropriate.” Nobody in any review has ever objected to that sentence, and that is precisely its failure: a requirement that cannot be rejected cannot be agreed to either. Compare an example written as the actual turn: “When the second payment attempt fails, the agent says: ‘That card was declined again. I can hold your order for 48 hours while you sort it out. Want me to?’” A stakeholder can read that and say no. Early rejection is the product of good requirements, because every no you collect in a document is a no you did not collect in production.

Trap three: skipping the failure paths

The happy path drafts itself; the design lives in the branches. What does the agent say when the dispute system times out mid-filing? When the account is flagged? When the customer asks for a human in the second turn? If the requirements do not answer these, the engineer answers them at implementation time, alone and undocumented, and the steering committee finds out when a customer does. We wrote a separate piece on this: the happy path is not a design.

Why is a conversation design the requirements document?

Because it is the one format both audiences can consume. A conversation design is the agent’s behaviour written turn by turn: example turns for each situation, the tool call behind each answer with its owner and failure behaviour, the branches for the paths where things go wrong, and the explicit turn where a human takes over. Your sponsor can read it top to bottom like a transcript and veto individual sentences. Your engineers can build from it without a clarifying meeting, because the questions they would have asked are answered in the structure.

The seven categories map onto it directly. The job-to-be-done and the channels live in a project brief. Each system behind an answer becomes a specified tool call, carrying its owner, its data sensitivity, and what happens on timeout. The never-do list becomes behaviour guidelines linked to the turns that demonstrate them. The escalation line is a marked, terminal turn. And the sign-off chain becomes an actual approval: a named person reads the design and records their approval in its timeline before anyone builds. The step-by-step method is in how to design an AI agent conversation, and the approval mechanics are in getting sign-off on an agent design.

Klate keeps the turns, branches, tool calls, briefs, and approvals in one place. The Free plan is enough to draft the whole thing: write the interview answers as example turns and branch the failure paths, with the system behind each answer attached as a tool call. Putting the result in front of the person whose name goes on it takes a paid plan.