Learn

The planner (agent)

Explaining a change takes judgement, and a model given judgement must not also be given the arithmetic.

What the language model actually does here, through tool calling and structured output, and what it is not allowed to do.

Stage 6 of 10. EXPLAIN investigates each top finding; DECIDE proposes actions. This is where a language model works, as the planner.

The problem it solves

The analysis found that revenue fell and where. Turning that into an explanation means deciding what to look at next: drill into customers? search the documents? compare with a benchmark? That is judgement, and judgement is what language models are useful for. But a model left free will write numbers from memory, run whatever code it likes, and present guesses as causes.

So the planner is given a job, a budget and a short list of fixed tools, and its answer must fit a schema. It never sees raw document text and never writes a number.

What happens on the example

The finding was “Revenue fell after March, driven by wholesale”. The planner had 8 tools: get_facts, drill_down, decompose, search_documents, search_web, compare_benchmark, propose_scenario, ask_user. Here is what it called, in order (the first two for the revenue finding, the rest for the refund-rate finding):

EXPLAIN · DECIDE · the coffee-roaster run

findingRevenue fell after March, driven by wholesale
tools8get_facts · drill_down · decompose · search_documents · search_web · compare_benchmark · propose_scenario · ask_user

EXPLAIN · DECIDE

1 · search_documents{"query":"wholesale price increase standing order paused"}
2 · drill_down{"metric_id":"revenue","period":"2026-04","dimension":"customer"}
3 · search_documents{"query":"packaging supplier switch valve bags"}
4 · search_web{"query":"Hotel Group LLC refunds 1258 — specialty coffee refund rate benchmark 2026"}
5 · compare_benchmark{"fact_id":"N64","extract_id":"E7"}

Out

search_documents{"extracts":[{"id":"E1","kind":"event","file":"price-list.pdf","date":"15 March 2026","text":"wholesale prices went up","quote":"Wholesale …
drill_down{"metric":"revenue","dimension":"customer","from":"2026-03","to":"2026-04","facts":{"total":{"id":"N258","value":-7900,"unit":"USD","label"…
search_documents{"extracts":[{"id":"E4","kind":"event","file":"ops-notes.docx","date":"28 April","text":"switched to a new packaging supplier","quote":"Swi…
search_web{"query_sent":"refunds specialty coffee refund rate benchmark 2026","external":true,"extracts":[{"id":"E7","kind":"statement","site":"coffe…
compare_benchmark{"external":true,"low":{"id":"N268","value":1,"unit":"percent","label":"external refund rate benchmark (low)"},"high":{"id":"N269","value":…

From the planner's tool calls in the run (scripted planner).

Tool	Input	Output (start)
search_documents	`{"query":"wholesale price increase standing order paused"}`	{"extracts":[{"id":"E1","kind":"event","file":"price-list.pdf","date":"15 March 2026","text":"wholesale prices went up","quote":"Wholesale prices increase 12% from 15 March 2026."},{"id":"E2","kind":"event","file":"email…
drill_down	`{"metric_id":"revenue","period":"2026-04","dimension":"customer"}`	{"metric":"revenue","dimension":"customer","from":"2026-03","to":"2026-04","facts":{"total":{"id":"N258","value":-7900,"unit":"USD","label":"drill_down revenue 2026-04 by customer:delta:total"},"change:Hotel Group LLC":{…
search_documents	`{"query":"packaging supplier switch valve bags"}`	{"extracts":[{"id":"E4","kind":"event","file":"ops-notes.docx","date":"28 April","text":"switched to a new packaging supplier","quote":"Switched to the new packaging supplier on 28 April."},{"id":"E5","kind":"event","fil…
search_web	`{"query":"Hotel Group LLC refunds 1258 — specialty coffee refund rate benchmark 2026"}`	{"query_sent":"refunds specialty coffee refund rate benchmark 2026","external":true,"extracts":[{"id":"E7","kind":"statement","site":"coffee-benchmarks.example","text":"specialty roasters usually see refunds of one to tw…
compare_benchmark	`{"fact_id":"N64","extract_id":"E7"}`	{"external":true,"low":{"id":"N268","value":1,"unit":"percent","label":"external refund rate benchmark (low)"},"high":{"id":"N269","value":2,"unit":"percent","label":"external refund rate benchmark (high)"},"gap":{"id":"…

Notice what came back: search_documents returned extracts (typed records with quotes), not the documents. drill_down returned facts with IDs, computed by SQL. search_web returned the query that was actually sent. compare_benchmark returned the gap as a fact. The planner can combine these; it cannot invent them.

Then it writes, in a schema. After investigating, the planner answers with a JSON object that must match a schema (ExplainOutput): a title, explanations with the extracts they rest on and a causal grade, alternatives it rejected, and a confidence. The scripted planner’s real answer on this run:

{
  "title": "Revenue fell after March, driven by wholesale",
  "explanations": [
    {
      "text": "The hotel customer paused its standing order because of the new price list.",
      "facts": [],
      "extract_ids": [
        "E2"
      ],
      "causal_grade": "document_supported"
    },
    {
      "text": "The wholesale price increase took effect shortly before the drop.",
      "facts": [],
      "extract_ids": [
        "E1"
      ],
      "causal_grade": "correlated"
    },
    {
      "text": "Wholesale also lost {{N99999}} in May.",
      "facts": [
        "N99999"
      ],
      "extract_ids": [],
      "causal_grade": "observed"
    },
    {
      "text": "The drop was caused by the price increase.",
      "facts": [],
      "extract_ids": [],
      "causal_grade": "document_supported"
    }
  ],
  "alternatives_rejected": [
    {
      "explanation": "Weaker retail demand",
      "reason": "retail revenue grew over the same months"
    }
  ],
  "confidence": "high",
  "confidence_reasons": [
    "One customer's order explains most of the drop, and the customer's email states why."
  ]
}

Two of these are deliberate mistakes, the kind real models make: a fact ID that doesn’t exist (N99999), and “caused by” with no document behind it. Neither reaches the report; the verifier drops the first and rewords the second.

On this run the model layer was called 41 times as extractor, 12 times as planner, 6 times as judge. The extractor does most of the reading.

The ideas behind it

Tool calling

Tool calling lets a model ask for an action instead of writing an answer: it emits a tool name and arguments, the engine runs the tool, and the result goes back to the model. The set of tools is the model’s whole world here. There is no eval, no generated code, no shell. A budget caps the calls per finding and per job.

Structured output

Structured output means the model’s answer must parse as JSON matching a schema. When it doesn’t, the error is fed back and the model tries again (twice at most). Endpoints that can enforce a schema natively do; for the rest, the engine validates with the same schema.

Placeholders instead of numbers

A model that writes “revenue fell 17.9%” may be right or may have made the number up; you can’t tell from the text. So the planner writes {{N30}} and code fills in the stored value. A placeholder pointing at a fact that doesn’t exist is caught at once.

Small-model mode

Small local models struggle with long tool loops. --small-model halves the budgets and replaces the free-form loop with a fill-in form. The numbers are the same either way, because the model computes none of them. The DECIDE request also stays under a 6,500-token input budget, trimmed in a fixed order that never changes a number (Running free).

Try it

With a model configured, run a job and watch the stages (--log adds structured logs on stderr):

bun run aa analyze examples/coffee-roaster --yes --currency USD --today 2026-07-01 --log

Without one, the scripted planner replays this page’s run: Analyse your first file. To run a real free model, see Model router & free providers.

In the code

packages/agent/src/tools.ts: the fixed tools and their budgets.
packages/agent/src/explain.ts: the investigation and the explanation schema.
packages/agent/src/decide.ts: candidate actions, with impacts computed in code.
packages/agent/src/prompts.ts: the prompts, versioned.
packages/models/src/retry.ts: schema validation with retries.
packages/core/testing/coffee-planner.ts: the scripted planner behind these pages.
The spec: docs/specs/agent-and-verifier.md.

Go deeper: Explain and causal grades and Decisions and impact ranges.