2026-08-07 · engineering notes
Your LLM bill is a claim, not a fact
At the end of the month an LLM provider hands you one number. When that number doubles, the questions start: which model, which week, which feature? The usage export that could answer them is a CSV nobody opens. I spent two days building a spike called Explain My LLM Bill: a FastAPI service with a drag-and-drop dashboard that takes the export and gives back breakdowns, insights, and, most importantly, a recomputation of the bill from first principles. These are the notes, because the interesting problems were not the ones I expected.
Prices are a function of time
The first wrong assumption: a pricing table is a lookup from model to price. It is actually a lookup from model and date to price, because providers cut prices mid-year and your July usage was billed at July rates. The pricing file is date-versioned, and every lookup uses the price in force on the usage date, not today's price. A gpt-4o row from July 2024 prices at the old five dollars per million input tokens, which is the only way historical uploads reconcile against real invoices.
Money math never touches a float
Cost per row is tokens times price per million, and both stay Decimal end to end. No float sneaks in, and no rounding happens anywhere inside the analysis module; formatting rounds once, at the edge, for display. This sounds pedantic until you sum a few hundred thousand rows and your total disagrees with the provider's by an amount you cannot explain, which is precisely the product's job to avoid.
The discrepancy list is the point
Some exports include the provider's own cost column. The spike computes its own figure anyway and compares, flagging any row where the two disagree by more than a cent or half a percent. The bundled sample bill plants three such rows on purpose, plus five malformed ones, so a demo run exercises the discrepancy list and the skipped-row report in one shot. If a file parses but yields zero usable rows, the API returns a 422 with per-row reasons rather than an empty success. Recompute, compare, and show your work: a bill you cannot recompute is a claim, not a fact.
Insights only when the numbers clear a bar
The insights layer is four guarded heuristics, and each one stays silent unless its trigger holds. Concentration fires when one model carries more than half the spend. Substitution suggests the one obvious step down (gpt-4o to gpt-4o-mini, Opus to Sonnet) with wording that admits it is a cost estimate, not a quality promise. Anomaly flags days above the mean plus two standard deviations. Caching computes realized savings from cache-read pricing, and says nothing when the export has no cache columns at all. An empty insights list is a valid outcome, not a bug, and I think that restraint is what separates an analysis tool from a slide deck.
Limits, honestly
It parses OpenAI CSV exports today; the normalized schema already models Anthropic, Bedrock, and Gemini shapes, but those parsers are not written. The pricing table is maintained by hand and goes stale the day a provider changes rates. And heuristics are heuristics: the substitution insight cannot know whether the cheaper model is good enough for your workload. The same rule runs through everything I have shipped this month: generation is easy, verification is the product.
The spike runs locally with uv and FastAPI, dashboard included, sample bill bundled. If you want the deeper version of this argument applied to RAG, read A verifier that can't hallucinate.