Guide
How to Track LLM Costs Per Customer
How to tag, meter, price and reconcile LLM requests so you can calculate AI cost by customer, and what a proxy does and does not cover.
The short answer: tag every LLM request with a customer_id set server-side, meter all routed calls at a single choke point (a proxy or a wrapped client) and know which paths bypass it, price each request from token counts at the moment it happens, and keep the result as an append-only ledger you can roll up by customer and reconcile against the provider invoice. Each of those clauses carries weight, and skipping any one of them is how teams end up with numbers finance won't sign.
Many teams aren't there yet. In CloudZero's State of AI Costs 2025 survey of 500 engineering professionals, 57% said they track AI costs manually in spreadsheets and 15% said they have no formal cost tracking. LLM cost tracking at the customer level is what turns an AI bill into AI unit economics: the ability to answer "which customers are profitable?"
Which tools can show cost per customer
Several tools can record what each customer costs. They differ on three things: how the customer gets attached to the cost, whether revenue sits next to it, and whether anything can stop a customer's calls. We make Spendline, so read this table with that in mind. Everything about another product comes from that vendor's own pages, checked on November 4, 2026. "Not described" means we didn't find it in the pages we read. That isn't proof the product can't do it.
| Tool | How the customer gets attached | Cost next to revenue, per customer | Can it stop a customer's calls? | Main limit | Published price |
|---|---|---|---|---|---|
| Spendline (ours) | Your server sends a customer ID on each routed call | Yes, where you enter or import one monthly revenue figure per customer | Yes. Strict budgets refuse before the call, from an estimate | Hosted only. Covers routed traffic on supported routes. No embeddings, images or audio. No tracing or evals. Young company | Guided 30-day free pilot. Paid terms agreed before you start |
| Langfuse | You set a user ID on traces. Set it to the paying account | Not described | No. It alerts on cost | Built for tracing and evals. Only sees what you instrument. | Free tier, Core $29/mo, self-hostable |
| LiteLLM | A customer ID in a header or the request body | Not described | Yes. Customer budgets, with pre-call reservations | You run it, with a database. Customer budgets apply across the whole deployment, not inside a team | Free and open source. Enterprise licence for some features |
| Portkey (now Prisma AIRS AI Gateway) | A _user metadata field on each request |
Not described | Yes, on the top plan. Per-user spend policies are listed as Enterprise Self Hosting only | Now part of Palo Alto Networks, so check the plan you'd be on | Free tier for testing, $49/mo |
| Cloudflare AI Gateway | Custom metadata on each request, up to five entries | Not described | Yes. One spend limit rule can split by customer ID | Spend limits are in beta. Cost is a best-effort estimate. Bursts of concurrent calls can briefly pass a limit | Core features free |
| Helicone | A user ID or custom property on each request | Not described | Partly. Cost-based rate limits per user or property, over a time window | Mintlify said in March 2026 it would run in maintenance mode. Helicone's site now says it's no longer part of Mintlify. Ask about support | Free tier, Pro $79/mo |
| OpenMeter by Kong | You send usage events per customer | Not described as cost against revenue | Partly. Quotas you enforce in your application | Built for metering what you bill customers. We didn't find provider cost pricing in the pages we read. OpenMeter Cloud is now Konnect Metering & Billing | Start for free, otherwise quoted |
| LangSmith | Metadata on a trace. Its docs give the user who started it as an example. Dashboards can group cost by metadata | Not described | Not per customer. Its gateway spend policies cover org, workspace, user and key, and the gateway is in beta | Built for tracing and evals | Developer $0, Plus $39 a seat a month, then pay as you go |
| CloudZero | Allocated from billing plus telemetry you send | Yes, it describes customer profitability | Not described. Works from billing and usage data | Built for company-wide cloud cost. Sees cost after the fact | Quoted |
| Vantage | Allocated. OpenAI costs arrive by project and API key, so a customer needs your own mapping | Yes, if you import revenue labelled by customer | No. Budgets send alerts | Provider data refreshes once a day | Free up to $2,500 tracked, Pro $30/mo, Business $200/mo |
| Finout | Allocated with virtual tags | Partly. Dashboards combine cost with revenue metrics | No. Plans track and alert | Built for company-wide cost allocation | Quoted |
| Your own logging | You log tokens and a customer ID, then price them in your warehouse | If you build it | If you build it | Only covers what you remember to instrument, and rates need upkeep | Your engineering time |
Short version: if you only need the number and you already trace calls, start with Langfuse or your gateway's logs. If you already run LiteLLM, its customer budgets may be all you need. If you want cost next to revenue with a check before each call, without running it yourself, that's the case for Spendline. If AI is a small part of a large cloud bill, look at CloudZero, Vantage or Finout first.
Why provider invoices can't show customer cost
Provider reports group usage by model, project or API key. Your application supplies the customer and workflow context needed to see cost per account. The provider view answers "what did we buy?" It cannot answer "who caused it?" And in a multi-tenant product, who caused it is the dimension that maps to revenue. Two customers on the same plan can cost very different amounts to serve, and a per-model total hides that. Per-customer cost allocation has to happen on your side of the API call, because only your systems know which customer a request belongs to.
The metadata contract
Customer-level cost attribution is a data contract, not a dashboard. Every request that reaches a model must carry, at minimum:
customer_id: the paying account (the org, not the end user). This is the join key to revenue.featureorworkflow: the product surface (chat,summarize,agent_research). Low-cardinality, safe to chart and alert on.agent_run_id: the user-initiated action that groups calls. One agent task can fan out into dozens or hundreds of model calls; without a run ID, a $4 agent run looks like 200 unrelated $0.02 requests.
Three rules make the contract hold:
- Set IDs server-side. Set customer IDs on your server from authenticated request context. A
customer_idaccepted from the client is a spoofable billing input, and no metering layer can tell a wrong ID from a right one. Timestamps are a separate question: the server's clock should decide which day and month a cost lands in, or your budget windows and month-end numbers drift. A server timestamp fixes the time, not the customer. - Meter all calls, not a sample. Sampling is fine for latency traces; it is wrong for money. If 5% of calls bypass the metering point, your per-customer numbers are estimates with an unknown error bar.
- Capture the token detail that pricing needs. Providers bill cached input at different rates than fresh input, and cache writes differently again. If you log only total input tokens, your computed cost will drift away from the invoice and you will discover it at month end.
What this looks like in code
With an in-path setup, the integration is a base URL change plus the metadata headers. Using the OpenAI SDK against Spendline's chat completions endpoint, with the stable Spendline key in the client configuration and the customer set per request from your server's own request context:
import OpenAI from "openai";
import { randomUUID } from "node:crypto";
const client = new OpenAI({
baseURL: "https://www.spendline.ai/v1",
apiKey: process.env.OPENAI_API_KEY,
defaultHeaders: {
"x-spendline-key": process.env.SPENDLINE_API_KEY,
},
});
// account comes from your authenticated session, never from the browser.
const response = await client.chat.completions.create(
{ model: "gpt-5.2", messages },
{
headers: {
"x-customer-id": account.id, // the paying account, per request
"Idempotency-Key": randomUUID(), // one value per logical request
"x-agent-id": "support-summarizer", // one stable value per agent or feature
"x-workflow-id": run.id, // groups this run's calls
"x-spendline-tags": JSON.stringify({ cost_center: "support" }),
},
}
);
Do not put x-customer-id in the client's default headers: every call would then be attributed to one customer. The same pattern applies with a self-hosted proxy or an SDK wrapper; what matters is that the identifiers ride with each request. Each call then lands in the ledger as one priced, attributed record:
{
"created_at": "2026-08-04T09:12:44Z",
"model": "claude-sonnet-4-5",
"customer_id": "acct_4821",
"agent_id": "support-summarizer",
"workflow_id": "run_9f3c",
"input_tokens": 2000,
"cached_input_tokens": 0,
"output_tokens": 600,
"cost_usd": 0.015
}
And the question "what does each customer cost us?" becomes a one-liner instead of a reconstruction project:
SELECT customer_id, SUM(cost_usd) AS ai_cost
FROM ai_calls
WHERE created_at >= '2026-08-01' AND created_at < '2026-09-01'
GROUP BY customer_id
ORDER BY ai_cost DESC;
The cost math, and why rates need versioning
The full per-request calculation covers every meter the provider bills:
cost = (uncached_input_tokens × input_rate)
+ (cache_read_tokens × cache_read_rate)
+ (cache_write_tokens × cache_write_rate)
+ (output_tokens × output_rate)
+ other provider-specific meters (batch, reasoning, regional)
A simplified worked example, using Claude Sonnet 4.5 list rates ($3 per million input tokens, $15 per million output, per Anthropic's published pricing) as a stable illustration rather than a current-flagship quote:
2,000 input tokens × $3 / 1M = $0.0060
600 output tokens × $15 / 1M = $0.0090
cost/request ≈ $0.015
At 50 requests per customer per day, that's roughly $22.50 per customer per month. That is the kind of number you want per customer, from records, not divided out of an invoice. Prices change with little notice, differ per model tier, and sometimes carry introductory or regional adjustments, so the rate table itself needs a source of truth with effective dates; hard-coding prices into application code is how computed cost quietly diverges from the bill. Current per-model rates and the multipliers stacked on top of them (cache tiers, batch, reasoning tokens) are broken down in our guide to Anthropic API pricing.
Cost per customer is then the sum over the period, and gross margin per customer follows once revenue is mapped in. Mapping that revenue side correctly is its own exercise, worked end to end in how to calculate AI gross margin by customer; the wider framework and benchmarks are in our unit economics guide.
Three ways to implement it
There are three honest routes, and the right one depends on what you need the numbers for.
| DIY tagging + warehouse | Observability SDK | In-path governance proxy | |
|---|---|---|---|
| How it works | Log token counts + IDs yourself, price in SQL/dbt | Wrap your client library; platform prices each trace | Point your base URL at a proxy; it meters everything routed through it |
| Coverage | Only what you remember to instrument | Only code using the wrapper | Calls routed through it, on the API shapes it supports; direct-to-provider paths are outside it |
| Engineering effort | Highest, ongoing | Low per service, per-codebase | Base URL change, plus metadata injection for full attribution |
| Can it enforce (block/cap)? | No | Generally no; it observes | Yes, for routed calls; budgets checked before the request is forwarded, from an estimate |
| Finance-ready (reconciliation, immutable ledger) | If you build it | Rarely the focus | Varies by product. Check for an append-only record, a closed period and an invoice comparison |
Tools like Langfuse and Braintrust are strong in the observability lane: they exist primarily to debug, trace, and evaluate LLM behavior, with cost as one attribute of a trace (the alternatives in that lane, and where they stop, are compared in Langfuse alternatives). LiteLLM is a popular open-source proxy with spend tracking if you want to run the in-path route yourself. A governance proxy like Spendline is a managed version of that third route, adding budget checks before the provider call, an append-only ledger, and customer revenue comparison where you share revenue data. (Where each layer stops is mapped in gateway vs. observability vs. governance.)
The practical test: if a call can reach the model without passing your metering point, that call is missing from your per-customer numbers. List those paths, and compare your ledger total with the provider invoice to see how large the gap is.
From tracking to control and close
Attribution is the foundation, not the finish line. Once every request is a priced, attributed record, two things become possible that aren't otherwise:
- Enforcement. Budgets per customer, team, or agent that are checked before the request is forwarded, not only alerts after the spend happened. A runaway agent loop is an in-flight problem; a weekly cost review cannot catch it in time, and the controls that stop a single run work on the run ID rather than the month. The scopes, the actions available when a cap is hit, and the concurrency trap are covered in LLM budget enforcement.
- The finance loop. Monthly reconciliation of your ledger total against the provider invoice, investigation of any gap, and a close that locks the period. A persistent reconciliation gap usually points to incomplete token-type capture, stale rate tables, side-door traffic, retries, or provider-specific pricing adjustments. The workflow is described in The AI month close, and the invoice comparison itself is worked line by line in how to reconcile OpenAI and Anthropic invoices.
Common failure modes
- Client-supplied attribution. IDs or timestamps trusted from the browser/SDK caller: spoofable. A wrong customer ID is recorded as faithfully as a right one.
- The side-door path. A batch job, cron script, new microservice, or a fallback that retries the provider directly when the proxy is unreachable never appears in your numbers. Metering at the network path narrows this, but only for traffic that actually takes that path.
- Missing labels. A routed call with no customer ID is still priced and recorded, but it lands in an unattributed bucket. Track the unattributed share, and give genuine internal jobs a stable synthetic ID such as
internal:nightly-reindex. - Cached tokens priced wrong. One common cause of invoice divergence is cache reads and writes being priced at the fresh-input rate, or not captured as separate fields at all.
- Agent fan-out without a run ID. Per-request numbers look healthy while a single user action quietly costs dollars.
- Mutable records. If cost rows can be edited in place, reconciliation can't prove anything; corrections should be new rows.
How Spendline does this
Spendline is a managed in-path route. Point a supported route's base URL at Spendline and send x-customer-id per request, plus x-agent-id and x-spendline-tags for agent and cost centre. Spendline records the supplied customer ID and server time with each routed call, prices it, and writes it to an append-only ledger. One ledger covers OpenAI, Anthropic and eight other providers on the supported request shapes: chat completions for OpenAI-compatible providers, Anthropic Messages, and non-streaming OpenAI Responses. Embeddings, images, audio and Google's native generateContent API are not supported, and calls sent directly to a provider are outside Spendline's records. Where you share revenue data, Spendline compares each customer's revenue with attributed AI cost over the same period. Other delivery costs are not included. An owner can close a month and compare a provider invoice total with it. That check compares totals, not line items, and records any difference as one adjustment.
The code sample earlier in this guide shows the request-level pattern. The integration guide lists the supported routes, their limits and how to check that the right customer receives the cost record.
FAQ
What tools can track LLM cost per customer? Observability tools such as Langfuse record it on traces. Gateways such as LiteLLM, Portkey and Cloudflare AI Gateway record it on each call and can limit it. FinOps platforms such as CloudZero, Vantage and Finout allocate it from billing and usage data. Spendline records it on each routed call, compares it with revenue where you share revenue data and checks budgets before the call. The table near the top of this guide gives the limits of each.
Can OpenAI or Anthropic show cost per customer? Not on their own. Provider reports group usage by model, project or API key. Your application supplies the customer and workflow context needed to see cost per account, via request tagging.
What metadata should every LLM request contain?
A customer_id, a feature or workflow name, and an agent run ID, all set server-side. That trio supports per-customer cost, per-feature economics, and per-run agent accounting.
Do you need an LLM proxy to track costs accurately? Not strictly. Wrappers and DIY logging work if every call path uses them. A proxy meters what is routed through it and can check budgets before a request reaches the provider, but calls sent directly to the provider, and API shapes it does not support, are outside it.
How should cached tokens be priced? At the provider's cache rates: on Anthropic models, cache reads run about a tenth of the base input price and cache writes carry a premium. Log cached and fresh counts separately.
How do you reconcile calculated LLM costs with provider invoices? Sum the ledger by provider for the billing period, compare to each invoice, investigate persistent gaps (token-type capture, stale rates, side-door traffic, retries, pricing adjustments), post corrections as new records, and lock the period.
Sources and method
Written by Aary Patnaik, Spendline co-founder, from building and testing Spendline's proxy, combined with public sources: adoption figures from CloudZero's State of AI Costs research; model pricing from Anthropic's published price list (illustrative rates; check current prices before relying on them); tool descriptions from each vendor's public documentation. The three-route comparison is our categorization judgment. Tool facts in the comparison table were checked on each vendor's own pages on November 4, 2026. Spendline's headers and supported routes were checked against the integration guide the same day.
See what this would look like in your product.
Start with a guided 30-day free pilot on live production traffic. The 30 days start when your traffic flows through Spendline, not at sign-up. We agree the scope, success criteria and paid terms before you start. There is no Spendline subscription fee during the pilot; your model-provider charges still apply.
Prefer email? fida@spendline.ai