Wire compatibility
The D8 conventions your bill is computed from.
This page is the published form of decision D8 — the conventions your bill is computed from. Changing anything here is treated internally as a billing-contract change and requires a recorded decision.
Two surfaces, one catalog
| Surface | Endpoint | Auth |
|---|---|---|
| OpenAI-compatible | POST https://api.routerplus.com/v1/chat/completions | Authorization: Bearer or x-api-key |
| Anthropic-compatible | POST https://api.routerplus.com/v1/messages | Authorization: Bearer or x-api-key |
| OpenAI Responses | POST https://api.routerplus.com/v1/responses | same |
| OpenAI Images | POST https://api.routerplus.com/v1/images/generations | same |
Every cataloged chat model is callable from both chat surfaces; image models answer on the Images surface only. The Responses surface serves OpenAI's models as a passthrough; Claude and the other chat models are a preview there, translated through the chat surface (see POST /v1/responses). GET https://api.routerplus.com/v1/models lists what your key can serve — OpenAI list shape by default, Anthropic shape when you send an anthropic-version header.
One distinction drives everything below:
- Passthrough route — your dialect matches the serving provider's (OpenAI format → OpenAI-dialect provider, Anthropic format → Anthropic-dialect provider). The parameters your dialect defines are forwarded as you sent them, and the gateway adds
stream_options.include_usageon streamed OpenAI-dialect dispatches. Whatever the provider accepts for those parameters, you get — including native features like Anthropic prompt caching (cache_control),thinking,top_k, and theanthropic-betavalues listed below. A top-level key outside your dialect's parameter set is dropped and recorded, never forwarded. Session ids and cache marks follow Sessions and prompt caching. - Translated route — dialects differ, and the gateway translates the request, the stream, and errors. Translation covers a pinned parameter set; everything else follows the rules below.
A "typed 400" in the translation tables is decided per deployment. When a translation refuses your request but another deployment of the model speaks your own dialect, the request goes there instead; you get the 400 only when no deployment can serve it. Content a model does not take follows the same rule: a deployment that does not take a part is skipped, and the 400 comes only when no deployment takes it (see Content is never silently dropped).
Parameter handling on passthrough routes
The forwarded set per dialect. Anything else at the top level is a recorded drop (see the note under the translation tables).
| Dialect | Forwarded as sent |
|---|---|
| OpenAI → OpenAI-dialect provider | model, messages, stream, stream_options, temperature, top_p, max_tokens, max_completion_tokens, stop, n, tools, tool_choice, parallel_tool_calls, response_format, reasoning_effort, seed, user, logprobs, top_logprobs, frequency_penalty, presence_penalty, logit_bias, metadata, store, prediction |
| Anthropic → Anthropic-dialect provider | model, messages, system, max_tokens, stream, temperature, top_p, top_k, stop_sequences, tools, tool_choice, metadata, thinking, output_config, cache_control, context_management (to Anthropic's own API; on Bedrock it is dropped and recorded) |
On the GPT-6 and GPT-5.6 models, the gateway calls OpenAI's Responses API for you and translates both ways (see GPT-6 and GPT-5.6 on the chat surface).
The reasoning object some providers take ({"enabled": false}, {"effort": "low"}) is not on that list: it is dropped and recorded. Send reasoning_effort instead. DeepInfra takes "none", "low", "medium" and "high", and "none" is its "enabled": false.
Some OpenAI-format keys go only to the providers that define them:
prompt_cache_retentionandsafety_identifiergo to OpenAI and Azure.- A
cache_controlmark on a content part passes unchanged to every OpenAI-dialect provider. OpenAI and Azure accept it and ignore it.
On Bedrock, the top-level cache_control becomes a mark on the last block that can carry one, because Bedrock's API does not take the top-level field. On every route, a top-level cache_control that Anthropic would refuse is dropped and recorded as cache_control: a field other than {"type": "ephemeral"} with an optional ttl of "5m" or "1h", a fifth mark, a 1-hour field after a 5-minute mark, or a field whose TTL differs from the mark on the last block.
Two Anthropic parameters are refused on every route, because stripping them would change who runs what: mcp_servers and container are a typed 400.
The anthropic-beta header is forwarded to Anthropic-dialect providers only for values that do not change how a token is billed: prompt-caching, token-efficient-tools, fine-grained-tool-streaming, interleaved-thinking, claude-code, oauth and computer-use prefixes. Anthropic's own API also gets the values that Claude Code needs on the Claude 5 models: the per-turn-control and mid-conversation-output-config prefixes (effort inside a message), context-management (context editing; compaction has its own beta, which is not forwarded), mid-conversation-tool-changes and inline-tools (tools added during a conversation), and thinking-display-updates (readable progress updates). Any other value is dropped and recorded as header:anthropic-beta:<value>. Two examples: advisor-tool runs a second model that bills at its own rates, and the auto-mode values (dangerous-tool-use, afk-mode, with the safeguards field) return a result field that the gateway does not pass back.
Claude Code's thinking: {"type":"adaptive","display":"updates"} is forwarded together with its thinking-display-updates-2026-08-18 beta on Anthropic's direct API. Its progress updates and thinking signatures survive streaming. Bedrock keeps its smaller beta set: thinking.display: "updates" is dropped along with that beta, while the thinking mode is preserved; both drops are reported, and require_parameters refuses the request.
Parameter handling on translated routes
OpenAI surface → Anthropic-dialect provider
| Parameter | Handling |
|---|---|
model, messages, stream, temperature, top_p | Translated / copied verbatim |
max_tokens, max_completion_tokens | Translated. On a request that carries neither, the gateway sets the model's own maximum output, fitted to its context window, before dispatch; see POST /v1/chat/completions for the bound (up to the model's declared maximum; 32,768 only when none is declared) |
stop (string or array) | → stop_sequences |
system / developer messages | → the Anthropic system string, joined in order with blank lines. If a part has a cache_control mark, system becomes text blocks with the same text, and each mark ends a block |
cache_control on a text part | Kept on the Anthropic block. A role:"tool" message's mark goes on its tool_result |
| Other keys of a text part | Not forwarded, and not recorded |
cache_control (top-level) | → the Anthropic top-level field (automatic caching). On Bedrock it becomes a mark on the last block |
tools, tool_choice | Translated: auto→{type:"auto"}, required→{type:"any"}, {function:{name}}→{type:"tool",name}, none→{type:"none"}. With none on a tool-free history, tools are omitted entirely so you don't pay for definitions you forbade using. A tool without parameters gets an empty object schema. The Claude 5 models refuse any and tool (a forced tool choice) with a 400 |
role:"tool" messages | → tool_result blocks; consecutive results merge into one Anthropic user message |
| Tool-call ids | Kept when Anthropic takes them. An id that Anthropic would refuse (a character other than a letter, a digit, _ or -, or more than 64 characters) is replaced by toolu_ and a hash of it, the same in the call and in its result and on every turn, so the pair still matches and a cached prompt start stays the same. Answers carry the provider's own ids |
image_url parts in user messages | → image blocks: a base64 data URL (JPEG, PNG, GIF or WebP; the image's own bytes decide its type) becomes a base64 source, an http(s) URL a URL source, which Anthropic fetches itself (some hosts refuse Anthropic's fetcher: that is a 400, with no failover). A cache_control mark is kept; a detail other than auto is dropped and recorded. Another image type is a typed 400 naming the part |
file parts in user messages | A PDF sent as a data URL → a document block, with the file's filename as its title. A file_id, or a file of another type, is a typed 400 naming the part |
An assistant message with content: null and no tool_calls | Dropped and recorded (OpenAI's own refusal shape); it has nothing Anthropic can carry |
stream_options | Usage is on by default; an explicit stream_options.include_usage: false is honored — you are still billed, but no usage chunk is emitted to you. The gateway acts on it itself: it is not forwarded, and not named in x-tm-dropped-params |
n | Rejected when n > 1: 400 invalid_request, on /v1/chat/completions and /v1/messages. The normalizer emits one choice; billing a garbled multi-choice response would be dishonest. The images route takes n up to 4 — see /docs/api-images |
temperature | Forwarded when ≤ 1. OpenAI's 0..2 range does not map to Anthropic's 0..1: temperature > 1 is a typed 400, never a silent clamp |
top_p | Forwarded only when temperature is absent (Anthropic documents them as mutually exclusive; temperature wins, the drop is recorded) |
parallel_tool_calls: false | → tool_choice.disable_parallel_tool_use: true |
tools[].function.strict | Mapped 1:1 |
response_format, reasoning_effort, prediction, audio, modalities | Typed 400 naming the param — the gateway does not map these across dialects yet and will not drop them silently: pinning JSON mode or a reasoning budget must not silently produce a different answer |
Trailing assistant message | Typed 400: Anthropic treats it as an assistant-prefill request, which current Claude models reject |
Everything else (logprobs, seed, penalties, …) | Dropped and recorded — never forwarded |
Anthropic surface → OpenAI-dialect provider
| Parameter | Handling |
|---|---|
model, stream, temperature, top_p | Copied verbatim |
max_tokens | → max_completion_tokens (the modern OpenAI field; legacy max_tokens is rejected by o-series/GPT-5 reasoning models). When absent, set by the gateway to the model's own maximum, fitted to its context window, as above |
system (string or text blocks) | → one leading system message |
stop_sequences | → stop |
messages | Translated: tool_use ↔ tool_calls, tool_result blocks → role:"tool" messages (one per result) |
image blocks in user messages | → image_url parts, from a base64 source or a URL source. An image in an assistant message or a tool result, or an image given by a file id, is a typed 400 naming the block |
document blocks in user messages | A base64 PDF → a file part. The block's title, context and citations are dropped and recorded. A document given by URL or as text is a typed 400 naming the block |
tools, tool_choice | Translated: auto→"auto", any→"required", none→"none", {type:"tool",name}→{function:{name}}. tool_choice.disable_parallel_tool_use: true → parallel_tool_calls: false; tools[].strict maps 1:1. Server tools (computer use, web search, bash, text editor) are a typed 400 — an OpenAI-dialect deployment cannot execute them |
thinking, output_config | Typed 400 naming the param (not mappable across dialects yet; native passthrough on Anthropic-dialect routes) |
top_k, metadata | Dropped and recorded (native passthrough on Anthropic-dialect routes). On a house route, metadata.user_id is used as the end-user id and is not recorded (see Sessions and prompt caching) |
cache_control (nested and top-level) | Dropped and recorded with the full path, e.g. messages[0].content[0].cache_control |
tool_result.is_error (nested) | Dropped and recorded with its full path |
"Dropped" means exactly that: the parameter is never forwarded — and every drop is recorded, per request. The full paths appear in the x-tm-dropped-params response header, in each attempt's dropped_params in GET /v1/generation, and in the (content-free) ledger. Send "provider": {"require_parameters": true} to turn any would-be drop into a typed 400 naming the first dropped parameter instead of a dispatch. Dropping only ever applies to parameters, never content.
GPT-6 and GPT-5.6 on the chat surface
The GPT-6 and GPT-5.6 models (GPT-6 Astra, GPT-6.1 Sol, GPT-6 Sol, GPT-6 Luna, GPT-5.6 Sol, GPT-5.6 Terra and GPT-5.6 Luna) take tools only on OpenAI's Responses API. The gateway calls them there and translates the request and the answer, so you keep calling them on /v1/chat/completions (or /v1/messages). This holds on the marketplace route and on your own OpenAI connection.
| Chat field | Handling |
|---|---|
messages | → Responses input items: text, image and file parts; an assistant's tool calls as function_call or custom_tool_call items; a tool message as the matching output item. Audio parts are a typed 400 |
max_tokens, max_completion_tokens | → max_output_tokens. A limit under 16 is sent as 16 (OpenAI's floor) and recorded as dropped |
tools with type: "function" | → function tools. A tool without strict is sent with strict: false, the Chat Completions default |
tools with type: "custom" | → custom (free-form) tools, with a format of text or a grammar in lark or regex syntax. Both the chat form ({type: "custom", custom: {name, description, format}}) and the flat form that Cursor sends are read. The model's call comes back as a tool_calls entry {id, type: "custom", custom: {name, input}}, in the body and in the stream, and you send its result back as a role:"tool" message. A call's input must be a string |
tool_choice | → the Responses form. It may name a custom tool |
| Tool-call ids | An id over 64 characters, which OpenAI refuses, is replaced by call_ and a hash of it, the same in the call and its result |
response_format, reasoning_effort | → their Responses forms |
store | Sent as false unless you set it |
n, stop, seed, logprobs, top_logprobs, logit_bias, prediction, the penalties | Dropped and recorded. A value equal to the default (n: 1, a penalty of 0) is dropped without a record |
temperature, top_p | Dropped and recorded, except temperature: 1 and top_p: 1: these models take only their default sampling |
The answer comes back as a chat completion, or as chat.completion.chunk records on a stream. finish_reason is tool_calls when the model called a tool and length when it reached max_output_tokens. When GPT-6 writes a short commentary before its answer, the two are joined with a blank line. The reasoning of one turn is not carried to the next: each turn reasons afresh.
Content is never silently dropped
Parameters are droppable because they don't get billed; content is not. A provider that drops a part still bills the request, so two rules apply:
- A part goes only to a model that takes it. Each listing declares what a message to the model may carry: text, plus any of image, file (a document such as a PDF), audio and video.
GET /v1/modelsshows what a model takes for your key inarchitecture.input_modalities. A model's page lists only the inputs that every listing of the model takes, so it can show less than the gateway accepts. The gateway sends a part only to a deployment whose listing takes its kind. A part that no deployment of the model takes is a typed 400 naming the exact field, on both surfaces, before anything is reserved or sent:messages[0].content[1]: model "gpt-4.1-mini" does not take audio input; it takes text, image and file. The parts checked areimage_url,file,input_audioandvideo_url(orinput_video) on the OpenAI surface, andimageanddocumentblocks on the Anthropic surface, also inside atool_result. A route on your own provider key (BYOK) declares nothing, so the gateway sends every part to it as you sent it. - A translation names what it cannot carry. Images (JPEG, PNG, GIF and WebP) and PDF files cross between the two formats in user messages, as the tables above show. On a translated route, content the translation cannot represent — audio parts, an image or a file given by file id, a file other than a PDF, an image in an assistant message or a tool result, unknown block types — is a typed 400 naming the exact field (
messages[2].content[0]and so on), never a silent omission.
One deliberate exception: assistant thinking / redacted_thinking blocks echoed back on the Anthropic surface are dropped rather than rejected. The gateway's own surface emits those blocks: an OpenAI-dialect model's reasoning reaches the Anthropic surface as thinking blocks on a stream, and as a leading thinking block on a reply that is not streamed. So echoing a transcript it produced must not 400 — but thinking content has no OpenAI-dialect equivalent and is never forwarded. A signed thinking block (Claude's own) and a redacted_thinking block are named in x-tm-dropped-params. An unsigned one is the gateway's own, so its drop is not recorded, and require_parameters does not refuse the next turn for it.
Sessions and prompt caching
A provider keeps a prompt's cache on one machine or host. The gateway passes on what each provider needs to send a conversation back there (decision D23).
Session id. The gateway takes the first valid value of these, in this order:
| Order | Input | Where |
|---|---|---|
| 1 | session_id | Body |
| 2 | x-session-id | Header |
| 3 | x-session-affinity | Header |
| 4 | x-claude-code-session-id | Header |
A valid value has 1 to 256 printable ASCII characters. The body field prompt_cache_key follows the same rule. An invalid value is ignored, never an error: the gateway uses the next input and records the ignored one as session_id (invalid), prompt_cache_key (invalid) or header:<name> (invalid). With provider.require_parameters: true, an ignored input is a 400, as for any drop. Valid session inputs are never recorded as dropped.
Each route gets its own field:
| Route | Field sent | Value |
|---|---|---|
| OpenAI (house or BYOK), Azure | prompt_cache_key | Your prompt_cache_key, else the session id. Nothing if you sent neither |
| Morph | Header x-session-id | The session id, else your prompt_cache_key. Nothing if you sent neither. Morph places the request's region and worker by it |
| Anthropic, Bedrock | Nothing | Anthropic has no session input: its cache matches the prompt prefix |
| DeepInfra | Nothing | Its cache matches the prompt prefix without a key |
Values go unchanged. A value longer than the provider accepts is replaced by tm- followed by 43 base64url characters: a hash of the value. prompt_cache_retention ("in_memory" or "24h") goes unchanged to OpenAI and Azure; other routes drop and record it.
End-user ids on house routes. A house route runs on the marketplace's own provider account, which every customer shares. On a house route, the gateway sends exactly one end-user id: a hash of your organization and your safety_identifier (else user, else, on /v1/messages, metadata.user_id), or of your organization alone.
| House route | Field sent | Not sent |
|---|---|---|
openai | safety_identifier | user |
anthropic | metadata.user_id | user, safety_identifier |
BYOK routes get your values unchanged: user and metadata as before, and safety_identifier on OpenAI and Azure from /v1/chat/completions. Anthropic and Bedrock BYOK routes, and any translation from /v1/messages, drop and record it.
Cache marks. The gateway never adds a cache_control mark to a block or a part. When a request carries no cache_control at all, the gateway may add the top-level field instead (Anthropic's automatic caching): see Automatic caching on Claude. The ledger prices every cache write at the 5-minute rate, so on a house route ttl: "1h" is removed from every mark (the cache then keeps 5 minutes) and recorded as <path>.cache_control.ttl, for example system[0].cache_control.ttl. ttl: "5m" passes. BYOK routes keep ttl: "1h": you pay that provider directly.
Staying on the first route. When the gateway's own quota pool for your first route is full, a follow-up request (one with an assistant or tool message after a user message) waits up to 5 seconds for it, instead of moving to a route with a cold cache. The response then carries x-tm-affinity-wait-ms with the milliseconds waited.
Automatic caching on Claude
Coding agents that call Claude through an OpenAI-compatible setting (Cline, Cursor, Kilo Code, OpenCode and others) send no cache marks, so each turn would pay the full input price for the same long prompt. The gateway turns on Anthropic's automatic caching for a request when all of these hold:
- 01Anthropic's own API serves it: the marketplace's
anthropicroute, or your own Anthropic connection. Not Bedrock, and not a dedicated endpoint. - 02The request carries no
cache_controlanywhere: not at the top level, and not on the system prompt, a tool, a message or a block inside a tool result. Your own marks always win. - 03It looks like a session that comes back soon: it has tools, or an assistant turn before its last message. A single question without tools is left alone, because for it the cache write would only cost more.
- 04The prompt (its system prompt, tools and messages, at four bytes a token) is at least the model's minimum cacheable length: 512 tokens on Claude Opus 5.5, Opus 5, Fable 5.1, Fable 5, Sonnet 5.5 and Haiku 5.5; 1,024 on Claude Sonnet 5 and Sonnet 4.5; 4,096 on Claude Haiku 4.5. Anthropic does not cache a shorter prompt.
- 05You did not turn it off with the request header
x-tm-auto-cache: off(any letter case).
The gateway then adds only the top-level field cache_control: {"type": "ephemeral"}, the standard form of automatic caching: Anthropic puts one cache point on the last block it can cache, and moves it forward as the conversation grows. The cache lasts 5 minutes. The gateway never adds marks to blocks and never a 1-hour cache, and caching does not change the answer. The first turn pays a cache write on its prompt (1.25 times the input price); later turns read it at the cache-read price, a tenth of the input price or less.
A response from a Claude route carries x-tm-auto-cache: applied, or skipped: followed by the reason (opted-out, client-markers, single-turn, short-prompt, bedrock, dedicated and a few more). A closed dedicated endpoint never sends it. Nothing is dropped, so x-tm-dropped-params does not change.
Marketplace requests reach Anthropic from one RouterPlus account, and Anthropic keeps its caches per account. A request reads a cache entry only when its prompt start is byte for byte the same as one sent before, but that earlier request can be another customer's. So someone who already knows your exact prompt start could see from their own cache reads that it was sent in the last few minutes. This was already so for requests with their own marks. If that matters for a prompt, send x-tm-auto-cache: off and no marks.
Usage accounting (how tokens are counted)
| Concept | OpenAI surface | Anthropic surface |
|---|---|---|
| Billable input | prompt_tokens — cache-inclusive: input + cache reads + cache writes | input_tokens — excludes cache reads and writes |
| Cache reads | prompt_tokens_details.cached_tokens | cache_read_input_tokens |
| Cache writes | prompt_tokens_details.cache_write_tokens | cache_creation_input_tokens |
| Output | completion_tokens | output_tokens |
| Reasoning | completion_tokens_details.reasoning_tokens (a subset of output) | not separately reported |
Cross-dialect translation converts exactly by this table; total_tokens = prompt_tokens + completion_tokens.
- Usage is always on. Streams and non-streams both report it. The gateway injects
stream_options.include_usage: trueinto every streamed OpenAI-dialect dispatch itself — without it those streams carry no usage at all. The onlystream_optionsvalue that changes what you receive is an explicitinclude_usage: false, which suppresses the usage chunk (you are billed the same). cache_write_tokensappears on both surfaces. Cache writes bill at their own rate, so the split is always visible: on billed OpenAI-surface responsesprompt_tokens_details.cache_write_tokensis always present (0 when the provider reports none), stream and non-stream alike.- Every billed response carries
usage.cost(USD) — in the final usage chunk on streams, in the body otherwise — computed with the exact integer micro-USD math the ledger settles with, and covering the full request debit, including attempts that failed over before your answer started. You can always recompute your bill from the wire. - Rounding favors you. Prices are micro-USD per million tokens; the pre-flight reservation rounds up, settlement rounds down.
- Web search has a fee per search. Anthropic's web search server tool, and OpenAI's hosted web search on
POST /v1/responses, are billed at the provider's price per search, on top of the tokens, and the fee is insideusage.cost— see Pricing & billing.
One documented limitation: an Anthropic-surface stream that dies mid-answer has no legal wire slot for usage (message_delta is the only one, and it never arrives). Recompute those from GET https://api.routerplus.com/v1/generation?id=REQUEST_ID, which returns every physical attempt with tokens and settled cost. The OpenAI surface has no such gap — known usage and cost are emitted before the terminal error event.
Streams
OpenAI-surface event order: role-priming delta (sent as soon as the upstream proves alive) → content deltas → finish chunk → usage chunk → data: [DONE]. [DONE] is never sent after an error, and nothing ever follows a terminal event.
- Keep-alives. After output has started, any silence of 15 seconds or more gets a keep-alive frame so proxies and clients don't kill an idle-but-healthy stream (slow reasoning models): an SSE comment (
: processing) on the OpenAI surface, a nativepingevent on the Anthropic surface. Both are spec-ignorable — SDKs skip them without code changes. - The commit boundary. Before any semantic output reaches you, provider failures are handled by invisible zero-backoff failover — you see one response, one monotonic event sequence, and
x-tm-attemptscounts what it took. After output has started, the gateway never switches providers: an upstream death becomes one terminal error event inside the 200 stream, and retry semantics stay yours. A refusal is never rerouted to another provider. - The request deadline. 450 seconds from dispatch, or a routing policy's
timeout_ms. A stream still open then gets the same terminal error event. - Mid-stream terminal shape: OpenAI surface — a full chunk envelope with
finish_reason: "error"and a top-levelerrorobject; Anthropic surface — anevent: errorframe.
finish_reason and native_finish_reason
Translated responses map finish reasons both ways:
Anthropic stop_reason | OpenAI finish_reason |
|---|---|
end_turn | stop |
stop_sequence | stop |
max_tokens | length |
tool_use | tool_calls |
refusal | content_filter |
The mapping is lossy (two stop reasons both become stop), so translated responses on the OpenAI surface carry the provider's raw value verbatim in choices[0].native_finish_reason — on the non-stream body and on the stream's finish chunk. When it differs from finish_reason, the native value is what the provider actually said — trust it for analytics. Passthrough responses are relayed verbatim, so their finish_reason is already native.
Errors
Every error body is generated by the gateway in the dialect you called with, and carries a stable error_type with the canonical class — error.metadata.error_type on the OpenAI surface, error.error_type on the Anthropic surface — plus the dialect's native fields so your SDK's built-in handling works untouched. On the Anthropic surface that means real native type strings (billing_error, rate_limit_error, authentication_error, not_found_error, api_error, invalid_request_error), so Anthropic SDK retry classification behaves. A provider's raw error body is never relayed, but the message carries the provider's own words: its error message, up to 1,000 characters, with keys, account ids and host names removed (never on a closed dedicated endpoint). The provider's HTTP status rides in x-tm-upstream-status. The body's metadata also names where the failure came from (origin, and the limit that refused it when one did) — see Errors.
A prompt too long for the model is a 400 in the standard shape of the surface, which coding agents look for before they compact and retry: code context_length_exceeded and OpenAI's wording on the OpenAI surface, prompt is too long: N tokens > M maximum on the Anthropic surface. The class stays context_overflow (see Context overflow).
Canonical classes and how routing treats them:
| Upstream signal | error_type | Failover |
|---|---|---|
| 429 (Retry-After honored, capped at 59 s; 10 s when the provider names none) | rate_limit | yes |
| 401/402/403 from a provider (our account problem, not yours) | auth | yes |
| Context/length errors | context_overflow | no — fix the request or pick a bigger window |
| Content policy / refusal | content_policy | never — a refusal is not rerouted |
| 404 / model not found upstream | model_unavailable | yes |
| 5xx / 529 / overloaded | upstream_error | yes |
| Other 4xx | upstream_error | no |
On POST /v1/decisions some of these signals read differently, by their status and the provider's own error code, never by the provider's name: Levanto's 402, Fastino's 402 or 403, a System One provider's 401, 402 or 403 on our account, and Cloudflare's 429 with code 3036 (our free daily allocation is used up) are a 503 model_unavailable with metadata.retryable: false. Cloudflare's 413 with code 5021 is a 413 context_overflow; Perplexity's 400 that names the model's maximum context length (on Perplexity Decider v1 27B) is a 400 context_overflow, by the context rule above; and a System One provider's 408 is a retryable 504 upstream_error. OpenAI's 404 model_not_found on GPT-6 Luna Decisions (our access to the model withdrawn) is a 404 model_unavailable by the 404 rule above, with no second provider to try. A decisions provider's 429 never counts against the deployment's health circuit. OpenAI can also decline one question on GPT-6 Luna Decisions and answer the rest. That is not a content_policy error: the response is a 200, that question's answer is {"type": "refusal"}, and the call is billed. See POST /v1/decisions.
Out-of-credits is deliberately a 429 insufficient_quota (not a 402): byte-compatible with what OpenAI's SDKs already recognize and back off on. Gateway-authored error bodies never echo flagged input content. The full per-status remediation table is on Errors.
Response headers
| Header | When | Meaning |
|---|---|---|
x-request-id | every call | the id GET /v1/generation?id= audits |
x-tm-provider | served requests | deployment that served the answer |
x-tm-attempts | served / failed dispatches | physical dispatches, failovers included |
x-tm-upstream-status | when an upstream responded | the provider's own HTTP status |
x-tm-error-code | failures | canonical class |
x-tm-error-origin | limit, provider and infrastructure failures | where it came from: gateway_admission, upstream_quota, upstream, gateway_infrastructure or authorization |
x-tm-limit-scope, x-tm-limit-kind, x-tm-limit-id | a refused limit | which limit refused the request — see Limits and capacity |
x-tm-cap-reset | spend-cap 429s | exact instant the monthly cap resets (first of next UTC month) |
x-should-retry | spend-cap 429s | false: do not retry before x-tm-cap-reset; OpenAI's and Anthropic's SDKs and Claude Code obey it |
x-tm-auto-cache | a Claude route served the request (never on a closed dedicated endpoint) | applied, or skipped: and the reason — see Automatic caching on Claude |
x-tm-dropped-params | when non-empty | comma-joined names of parameters the gateway stripped (D8 §2: drops are recorded, never silent) |
x-tm-upstream-model | when the route knows the model by another id | the id the gateway actually sent upstream |
x-tm-served-by | when a route reports one | the provider that ran the request |
x-tm-route-plan-id | billed requests | the id of the route plan that chose the deployments |
x-tm-admission-mode | billed requests | shared, or bounded_local while the shared limit store is unreachable |
x-tm-remaining-rpm, x-tm-remaining-tpm | admitted requests | the smallest remaining allowance across the limits the request claimed |
retry-after | 429/503 | whole seconds to wait, at most 59 (usage_limited sends 60 or 3,600); a provider's value is honored, capped at 59, and a provider's 429 without one gets 10 |
Browsers can read every header above except x-tm-error-code and x-tm-route-plan-id, which are not CORS-exposed; use the body's error_type in browser code.
Optional, content-free app attribution: send HTTP-Referer and X-Title to identify your app in per-app analytics.
Out of scope in v1
Honest errors or recorded drops — nothing silent:
- Audio and video input, and files other than PDFs, have no cross-dialect translation: typed 400 on translated routes. Images and PDFs cross (see Content is never silently dropped). On a passthrough route a part reaches the provider as sent when the model takes its kind, and is a typed 400 when no deployment of the model takes it;
GET /v1/modelslists the inputs a model takes. - Image generation is in scope on its own surface (D8 §7.6, D17); image edits, variations and streaming are not — see POST /v1/images/generations.
n > 1choices: rejected 400 on the chat surfaces (the images route takesnup to 4).- No model id variants — ids containing
:are not in the catalog and 404 like any unlisted id, and there is never a silent substitute. - On passthrough routes, the forwarded parameters reach the provider as you sent them, and the provider's behavior is the provider's. The guaranteed surface is exactly what this page documents.
See also: Migrate · Quickstart · Errors
Markdown source for agents: /docs/compat.md · index at /llms.txt