Use case
How to cap tokens per API call
One runaway prompt or an unbounded max_tokens setting can turn a cheap workflow into an expensive one.
A policy token cap names a model and a token limit for it. A request naming that model, or served by it, that would exceed the limit is refused before being forwarded to the provider, unless the policy is set to warn instead of block, in which case the call goes through and the warning comes back on that response in an x-spendline-policy-warnings header.
How Spendline does this
The cap is evaluated as part of the same pre-call policy check used for model restrictions, matched against a model the same way a model-block policy is (a policy always names one model, matched exact, by prefix, or by substring), applied to the request's token parameters rather than to accumulated spend. It catches a failure mode a spending budget alone would not: one abnormally large individual call, rather than many calls adding up over time. It does not catch a large call on a model the policy does not name.
Why it matters
A monthly budget protects against sustained overspend; it does not stop a single call with an unbounded output size from consuming a large share of that budget in one request. A per-model token cap is the control for that specific shape of risk, for whichever model you point it at.
Frequently asked questions
Does this cap input tokens, output tokens, or both?
A token-cap policy is configured as either a max_input_tokens or max_output_tokens type, so you choose the dimension. Output is the more common choice, since that is what an unbounded generation setting would otherwise let run large.
Does this cap apply to every model, or just one?
Just the one the policy names. Like a model-block policy, a token cap always matches against a specific model (exact, by prefix, or by substring); a call for a model the policy does not name is not affected by it.
What happens to a call that would exceed the cap?
Refused before reaching the provider, the same pre-call pattern used for model-restriction policies and budget checks, unless the policy's enforcement is set to warn rather than block. A warn-mode policy lets the call through and returns the warning on that response, in an x-spendline-policy-warnings header. It is not stored anywhere afterwards, so treat warn mode as a signal to the calling code, not as a record you can review later.
See where this stands in your own setup
This page describes the mechanism. The 5 minute assessment scores your own attribution, enforcement, and reconciliation setup, so you know exactly which of these problems you actually have today.
Run the 5 minute assessment