Cost dashboards and budgets

Why this matters #

AI costs can change overnight. A prompt edit that adds a long example doubles input tokens on every request. A popular launch triples volume. A bug retries every failed call in a loop. A single customer’s integration starts sending enormous documents. The provider’s invoice will tell you about all of these, a month later, as one number.

A cost dashboard turns those surprises into decisions made the same day. It breaks spend down by feature, model, customer and outcome, sets budgets with owners, and alerts when something moves. This article covers what to track, how to build the view from data you already have, how to set budgets and alerts, and a monthly review that keeps spend tied to value.

What to track #

The provider’s console shows spend by API key and model. That’s necessary but not enough: you need to know why money was spent. Track cost broken down by:

The last two are the most often missing. Cost by outcome shows how much of your bill is waste you could recover by improving quality. Cost over time, compared with a baseline, catches problems while they’re small.

Building the dashboard #

You already have the data: every trace records the feature, prompt version, model, tokens and outcome, and your analytics events connect requests to customers and successes. The dashboard is an aggregation over those records:

This script does the aggregation and anomaly check over a day’s traces. In production you’d run the same logic as a query in your data warehouse or observability tool:

interface TraceRecord {
  day: string;
  feature: string;
  model: string;
  customer: string;
  outcome: 'success' | 'failure';
  costUsd: number;
}

/** Sums cost by any dimension of the trace records. */
function totalBy<K extends keyof TraceRecord>(records: TraceRecord[], key: K): Map<TraceRecord[K], number> {
  const totals = new Map<TraceRecord[K], number>();
  for (const r of records) totals.set(r[key], (totals.get(r[key]) ?? 0) + r.costUsd);
  return totals;
}

/** Flags a day whose spend is more than `threshold` above the average of the previous seven days. */
function isAnomaly(dailyTotals: number[], threshold = 0.5): { anomaly: boolean; today: number; baseline: number } {
  const today = dailyTotals.at(-1) ?? 0;
  const previous = dailyTotals.slice(-8, -1);
  const baseline = previous.reduce((a, b) => a + b, 0) / Math.max(1, previous.length);
  return { anomaly: baseline > 0 && today > baseline * (1 + threshold), today, baseline };
}

// A day's traces, already priced (tokens × prices from the model configuration).
const today: TraceRecord[] = [
  { day: '2026-10-17', feature: 'reply-drafts', model: 'opus', customer: 'acme', outcome: 'success', costUsd: 41.2 },
  { day: '2026-10-17', feature: 'reply-drafts', model: 'opus', customer: 'acme', outcome: 'failure', costUsd: 9.8 },
  { day: '2026-10-17', feature: 'reply-drafts', model: 'opus', customer: 'globex', outcome: 'success', costUsd: 12.5 },
  { day: '2026-10-17', feature: 'doc-analysis', model: 'opus', customer: 'acme', outcome: 'success', costUsd: 88.0 },
  { day: '2026-10-17', feature: 'ticket-routing', model: 'haiku', customer: 'globex', outcome: 'success', costUsd: 0.9 },
];

const usd = (n: number) => `$${n.toFixed(2)}`;
for (const key of ['feature', 'customer', 'outcome'] as const) {
  const lines = [...totalBy(today, key)].sort((a, b) => b[1] - a[1]).map(([name, cost]) => `${name} ${usd(cost)}`);
  console.log(`By ${key}: ${lines.join(', ')}`);
}

// Previous seven days of total spend, then today.
const daily = [64, 71, 68, 70, 66, 69, 72, today.reduce((sum, r) => sum + r.costUsd, 0)];
const check = isAnomaly(daily);
// Point the alert at the biggest contributors, so whoever receives it knows where to look first.
const top = (key: 'feature' | 'customer') => [...totalBy(today, key)].sort((a, b) => b[1] - a[1])[0]?.[0];
console.log(
  check.anomaly
    ? `ALERT: today ${usd(check.today)} vs 7-day average ${usd(check.baseline)}. Check ${top('feature')} and ${top('customer')} first.`
    : `Normal: today ${usd(check.today)} vs 7-day average ${usd(check.baseline)}.`,
);

The output points at where to look: a spike, with one feature and one customer accounting for most of the spend. That’s the value of breaking spend down before alerting. “Spend is up 80%” makes someone open the console; “doc-analysis spend for acme is up” makes someone open the right trace.

Price tokens with the same model configuration your code uses, so the dashboard stays correct when prices change. On this site, that’s the single model configuration every example reads from.

Budgets and alerts #

Budgets turn the dashboard into control:

• A provider-level spending limit on each API key, so no bug or attack can exceed a hard ceiling. Use separate keys for production, staging and CI, each with its own limit.
• A monthly budget per feature, owned by the feature’s team, with alerts at 50%, 80% and 100% of the budget’s pace.
• Anomaly alerts on daily or hourly spend compared with a trailing baseline, as in the script, broken down by feature so the alert says where to look.
• Per-customer thresholds for B2B products, so account managers know before a customer’s usage becomes a pricing conversation.

Every alert needs an owner and a first action: who gets it, and what they check first. An alert that goes to a shared channel with no owner is a notification, not a control.

A monthly cost review #

Once a month, for each AI feature, spend 30 minutes on these questions:

1. What did we spend, and what did we get? Total cost, successes, cost per success, and the trend.
2. Where is the waste? Spend on failures, retries and abandoned outputs. Which quality improvement would recover the most?
3. Is the configuration right? Could a lower effort, a smaller model for one step, or caching (opens in a new tab) cut cost without hurting quality? What do the evals say?
4. Who are the heaviest customers, and does their plan cover their usage? See pricing.
5. What changed? New prompts, models, prices or features since last month, and their effect.

Record decisions with owners and dates. Over a few months, the review builds the history that makes AI spend predictable, and it gives leadership a clear story: what AI costs, what it returns, and what you’re doing about the gap.

Common mistakes #

Try it #

Check your understanding #

Key takeaways #