LLM cost instrumentation as a first-class product feature
The default failure mode for AI products is not quality – it is invisible spend. The invoice arrives on the first of the month with a single line item; the discussion that follows is anecdotal because no one has per-product attribution. We have been through this loop enough times to make it the top house rule at Stratas: no untracked paid-API product ships.
What "instrumented" actually means
Every LLM call in every product we ship carries at least these tags:
- product slug – routes spend to the correct product row in the fleet dashboard
- user id (resolved via a shared helper) – enables per-user daily / monthly caps
- model – Opus / Sonnet / Haiku each have very different unit economics
- route or context – a
saga.drafts.batchcosts differently from asaga.chat.interactive; both live in the same Langfuse project but need to be reasoned about separately
The tagging is done in a shared resolver, not sprinkled at the call site. In saga, shared/api-key-resolver.ts handles it once for every Claude call in the product. In fabrication-quoting, recordLlmGeneration() wraps every extraction. If tagging drifts, one file changes; if a call slips through untagged, one function catches it.
Two-project Langfuse routing
Some products are billed to clients on cost-plus; some are internal ops. The agent-fleet-dashboard routes reads to two separate Langfuse projects – LANGFUSE_PUBLIC_KEY_CLIENT vs LANGFUSE_PUBLIC_KEY_STRATAS – based on each product's langfuse_project field. If a product isn't in the catalogue, its spend is silently nil. Silent-nil is deliberate: better to miss a new product than blend it into the wrong customer's invoice.
Cost gates before paid calls
Instrumentation tells you what you spent yesterday. Gates prevent you from spending recklessly today.
- preservation refuses to reach any paid endpoint unless
CLOUD_GATE_ACK=enabledis present. This has prevented at least two accidental full-scrape reruns that would have burned the month's enrichment budget. - agent-fleet-dashboard caps the Insights chat at $20/day per workspace. The rate cap is enforced via a Langfuse tag filter – the same telemetry that displays spend also enforces the ceiling.
- saga gates the expensive avatar-likeness eval on
AVATAR_LIKENESS_EVAL=1; the eval costs $4 and takes 20–30 minutes, so it never fires accidentally.
Cost as UX
The dashboard's Insights chat lets an operator ask "what did we spend on Saga last week?" and get an answer streamed from live Langfuse data. The chat itself is Claude-Sonnet-4.6 powered, rate-capped, and – crucially – traces its own cost back to the same Langfuse project. The AI feature has to be cheaper than the leak it is exposing, and the trace shows it.
Model stratification is a cost tool
Saga stratifies by use case: Opus 4.6 for user-facing chat (where quality is felt), Sonnet 4.6 for batch drafts (where throughput matters), Haiku 4.5 for cheap non-writing checks (scene inference, image verification, same-person confirmation). The rule is written into the source file that chooses the model, so the reasoning is visible when someone later thinks to "upgrade everything to Opus".
Why this matters commercially
For a two-founder consultancy shipping AI products for Nordic clients on cost-plus billing, per-product attribution is not a nice-to-have – it is the invoice. Cost instrumentation is also a signal of having shipped enough AI products to know what breaks first and cheapest. The first thing that breaks is not accuracy; it is the invoice.
The pattern is portable. The house rule is short: no product ships without a Langfuse tag on every call, a slug on the owning row, and a gate before any paid endpoint. Everything else follows.
Let’s have a coffee
Over fifteen years of consulting and growth, and hands-on AI delivery from discovery to daily operations. I can share my CV and talk through how I connect business outcomes with systems that run.
Available for a chat. In person in Stavanger, or remotely worldwide.