Learn

Confirm your metrics

Look at the drafted metric definitions, save them to a file, and review them one by one before a job measures anything.

Tutorial 2 of 5. About ten minutes, offline. Read Metric layer first if you want the why.

Every number in a report is computed from a metric definition. In the first tutorial --yes kept the drafts without asking, and the summary ended with “some metric definitions were assumed, not confirmed”. Here you check them.

1. See the drafts before running anything

aa metrics reads and profiles the files, drafts the definitions, and shows each one in words, with its SQL and its last six values. --save writes them to a YAML file:

Recorded output, offline run

$ bun run aa metrics examples/coffee-roaster --yes --currency USD --today 2026-07-01 --save agentic-analytics.metrics.yaml
Drafted 5 metric(s) (deterministic fallback). Note: the model was unavailable (…); used deterministic fallbacks.
  assumption: Amounts in revenue are taken to be in USD (no currency marked in the data).
  assumption: Amounts in retail_revenue are taken to be in USD (no currency marked in the data).
  assumption: Amounts in wholesale_revenue are taken to be in USD (no currency marked in the data).

Revenue = sum of amount where status is empty or status is not refunded, per month, in USD (assumed)
  sql: WITH m1 AS (SELECT strftime("date", '%Y-%m') AS period, sum("amount") AS value FROM "t_t1" WHERE NOT _is_total AND "date" IS NOT NULL AND (("status" IS NULL) OR ("status" NOT IN ($1))) AND strftime("date", '%Y-%m') BETWEEN $2 AND $3 GROUP BY ALL) SELECT period, value FROM m1 ORDER BY period
  params: ["refunded","2026-01","2026-06"]
  2026-01 41200 · 2026-02 42900 · 2026-03 44100 · 2026-04 36200 · 2026-05 35800 · 2026-06 37000
Orders = number of distinct order_id where status is empty or status is not refunded, per month (assumed)
  sql: WITH m1 AS (SELECT strftime("date", '%Y-%m') AS period, count(DISTINCT "order_id") AS value FROM "t_t1" WHERE NOT _is_total AND "date" IS NOT NULL AND (("status" IS NULL) OR ("status" NOT IN ($1))) AND strftime("date", '%Y-%m') BETWEEN $2 AND $3 GROUP BY ALL) SELECT period, value FROM m1 ORDER BY period
  params: ["refunded","2026-01","2026-06"]
  2026-01 980 · 2026-02 1010 · 2026-03 1050 · 2026-04 1020 · 2026-05 1005 · 2026-06 1040
Retail revenue = sum of amount where (status is empty or status is not refunded) and channel is retail, per month, in USD (assumed)
… (13 more lines)

Replaced in the recording because they change on every run: the reason no model answered (it depends on your setup).

No model was running here, so the definitions were drafted by deterministic rules, and the first line says so. With a model configured, it drafts them instead; either way they are validated by the same code. Read each sentence: is Revenue = sum of amount where status is not refunded what you mean by revenue? Are the values what you see in your own books?

2. Edit the file

The saved file is plain YAML, one entry per metric. Its first entry:

version: 1
grain: month
metrics:
  - id: revenue
    label: Revenue
    type: sum
    table: t_t1
    expression: amount
    filters:
      - status IS NULL OR status NOT IN ('refunded')
    time_column: date
    unit: currency
    currency: USD
    dimensions:
      - customer
      - channel
      - product
    status: assumed

Change what’s wrong: a filter, the column, the label. Then use your file instead of drafting (with --yes the statuses in the file are kept; without it you review each one):

bun run aa metrics examples/coffee-roaster --metrics agentic-analytics.metrics.yaml --yes --currency USD --today 2026-07-01

A definition that doesn’t fit the data (a column that doesn’t exist, a sum over a text column) is refused with the reason. The full format is in Metric YAML.

3. Review during a job

Without --yes, aa analyze stops after drafting and asks about each metric: accept, edit (type or paste new YAML, then an empty line), or drop.

AA_DEMO_MODEL=scripted bun run aa analyze examples/coffee-roaster --embedder hash --today 2026-07-01 --currency USD

In this recording we accepted the first four and dropped refund_rate:

Recorded output, offline run

$ AA_DEMO_MODEL=scripted bun run aa analyze examples/coffee-roaster --embedder hash --today 2026-07-01 --currency USD
… (8 lines before)
▸ AWAIT_CONFIRM

Review the drafted metric definitions. Every number in the report is computed from them.

Revenue = sum of amount where status is empty or status is not refunded, per month, in USD (assumed)
Metric revenue? [a]ccept / [e]dit / [d]rop: a

Orders = number of distinct order_id where status is empty or status is not refunded, per month (assumed)
Metric orders? [a]ccept / [e]dit / [d]rop: a

Retail revenue = sum of amount where (status is empty or status is not refunded) and channel is retail, per month, in USD (assumed)
Metric retail_revenue? [a]ccept / [e]dit / [d]rop: a

Wholesale revenue = sum of amount where (status is empty or status is not refunded) and channel is wholesale, per month, in USD (assumed)
Metric wholesale_revenue? [a]ccept / [e]dit / [d]rop: a

Refund rate = sum of amount where status is refunded ÷ Revenue, per month, as a percentage (assumed)
Metric refund_rate? [a]ccept / [e]dit / [d]rop: d
▸ AWAIT_CONFIRM
✓ AWAIT_CONFIRM (… ms)
▸ MEASURE
✓ MEASURE (… ms)
… (37 more lines)

Replaced in the recording because they change on every run: stage durations, the job id (a timestamp), the report folder (the current folder), your answers, written after each question (piped input isn't echoed).

and the job went on. The end of the same run:

Recorded output, offline run

$ AA_DEMO_MODEL=scripted bun run aa analyze examples/coffee-roaster --embedder hash --today 2026-07-01 --currency USD
… (53 lines before)
Findings:
• Revenue fell after March, driven by wholesale
  Revenue went from 44,100 in 2026-03 to 36,200 in 2026-04 (a change of −7,900, −17.9%). It stood at 37,000 in 2026-06.
• A change in revenue
  Revenue for “Green Leaf Bakery” went from 2,745.86 in 2026-05 to 1,966.87 in 2026-06 (a change of −778.99, −28.4%).
• Wholesale revenue fell sharply after March
  Wholesale revenue went from 15,900 in 2026-03 to 7,400 in 2026-04 (a change of −8,500, −53.5%). It stood at 7,300 in 2026-06.

Recommendations:
• Offer wholesale accounts a volume-tier discount to win back paused standing orders
  Expected impact: 3,995 to 7,191 next quarter

report: agentic-analytics/report.json
html:   agentic-analytics/report.html

Compare it with the first tutorial: the refund-rate finding and its recommendation are gone, because their metric was dropped, and the note about assumed metrics is gone, because every metric was confirmed. The report records each metric’s status, so a reader can always tell a confirmed definition from an assumed one.

You can also answer from a script (printf 'a\na\na\na\nd\n' | …); if the answers run out before the questions, the job stops and says so instead of guessing.

What you learned

aa metrics shows the definitions, their SQL and their values before any analysis.
Definitions live in a YAML file you can edit, keep and reuse with --metrics.
Without --yes, a job waits for you, and what you drop leaves the report.

Next: Add documents and emails as evidence.