What should this agent do on the phone?
Plain words, the way you would tell a new hire. It guesses the whole flow first, then questions each step of it.
This tab is for whoever builds it. Everything above is the demo; this is what the demo is standing in for, and where the seams are.
Who owns what
Three things get confused for each other, so they are named separately here. The requirements agent is a gpt-oss-120b call — our own SGLang deployment — that runs at design time: once to draft the graph, then once per step as the operator answers that step’s questions. LangGraph holds that graph and validates it. LiveKit runs the call. Only the third one is on the phone with anybody.
What LangGraph is actually for
It is the compiler, not the runtime. That distinction is the whole answer, so it is worth being blunt about why: LangGraph assumes it owns the turn loop, and in a voice call LiveKit already owns it. Interruptions, barge-in, and a sub-second latency budget all live in LiveKit’s loop. Run LangGraph as the live executor and you have two loops arguing about whose turn it is.
So it holds structure and checks it. Build the StateGraph at deploy time, one node per step, and use it to answer the questions that are painful to answer by hand: is every node reachable, does any node bind a tool it was never granted, is there a path to take_payment that skips verification, does every terminal node actually terminate. Then emit the compiled node table and hand that to LiveKit.
What you get from the emitted table is small and boring, which is the point:
{
"verify": {
"instructions": "Confirm the account holder. Name and postcode.",
"tools": ["lookup_crm"],
"edges": [{"to": "state_invoice", "when": "account holder confirmed"},
{"to": "wrong_party", "when": "someone else answered"}]
}
}
What happens when the requirements agent names a tool
It does not name one. It picks one, from a closed list, and the difference is load-bearing. The response schema constrains tools to an enum of the ten ids in the catalogue, so an invented tool name is a schema violation rather than a runtime surprise. Four steps, in order:
- Selection. The model returns
tools: ["lookup_crm"]for a step. It never returns a signature, a description, or an implementation. Only an id that already exists. - Resolution. The compiler looks each id up in the registry. An id with no registered function is a build failure. This is the check that makes a hallucinated tool a deploy-time error instead of a dead call at 2am.
- Binding. The resolved functions are bound to that node only. This is where gating stops being a promise in a prompt and becomes a fact about the model’s context: on the greeting node,
take_paymentis not in the tool array, so there is no token sequence the model can emit that calls it. - Policy. Some rules are not the graph’s to enforce. DNC lists, calling windows, and spend caps sit in front of the tool, so a tool call passes through them whatever node it came from. A graph edit must not be able to switch them off.
How the system knows which step it is on
The edge labels on the canvas are not decoration; they are the transition conditions. At run time, each node’s instructions carry its own exit conditions and a move_to(step) tool whose enum is exactly that node’s outgoing edges. The model cannot jump to an unconnected step because the target is not in the enum.
Two things follow that are easy to miss. First, the swap does not restart the call: session.update_agent() replaces the agent object while the session, the audio path, VAD and STT all stay alive, and chat_ctx.copy(exclude_instructions=True) carries the dialogue across while dropping the previous system prompt. Second, when a step has exactly one exit, skip the model and advance on a deterministic check, which is cheaper and cannot drift.
The API note that will cost you a day if nobody tells you: Agent.update_instructions() does not exist. There is no supported way to mutate a live agent’s system prompt, so a node is a factory that builds a new Agent, not a mutation applied to an existing one. update_options() is models only. agent.update_tools() does exist but replaces all tools including decorator-registered ones, so dedupe by id. Verified against livekit-agents~=1.6.
async def enter_node(session, node, registry):
# a node transition constructs an Agent; it never mutates one
await session.update_agent(Agent(
instructions = node["instructions"],
tools = [registry[t] for t in node["tools"]],
chat_ctx = session.current_agent.chat_ctx.copy(
exclude_instructions=True),
))
What this demo fakes
| Here | In the real thing |
|---|---|
| gpt-oss-120b called from the browser on an open endpoint | Server-side, endpoint authed and rate-limited, never open |
| Ten tools that return nothing | A registry of real functions with schemas, timeouts, retries |
| Graph held in a JS object, lost on refresh | Versioned per tenant; a call pins the version it started on |
| Gating drawn as an orange wire | Gating enforced by which tools are bound to the node |
| The operator can edit any node freely | Open question. Editing a live graph mid-call is a footgun |
Where it will hurt
Three places, named now so nobody is surprised later. Swap latency is real time on the phone; if a swap costs 400ms the caller hears it, so the boring fix is fewer nodes and a deterministic advance where the branch is obvious. A wrong branch is unrecoverable unless you build the way back. The model deciding a caller confirmed something they did not is the failure mode that matters, and it argues for an explicit correction edge rather than trusting the classification. Ten tools is not the hard case; a tenant with sixty is, and at that size the requirements agent needs the catalogue filtered by what the tenant has actually connected before it ever sees the list.