Dedicated endpoints

A dedicated endpoint is provider capacity reserved for your organization on one model. It has its own limits, prices and failover rules. We set one up with you under a contract; there is no self-serve setup. You call it with the model id we give you, in your organization's own namespace: <your-namespace>/<name>, for example acme/gemma-31b.

Calling an endpoint

Use the endpoint's model id in place of a catalog model id. Everything else stays the same: both surfaces (/v1/chat/completions and /v1/messages), streaming or not, and every request field the model accepts.

curl https://api.routerplus.com/v1/chat/completions \
  -H "Authorization: Bearer $TM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "acme/gemma-31b", "messages": [{"role": "user", "content": "Hello"}]}'

Only your organization's keys can call the endpoint. It can also be limited to named workspaces. For any other key the id does not exist: it gets the same 404 model_unavailable as an unknown model.

On an open endpoint, the response names the model that ran, for example "model": "google/gemma-4-31b-it", not the endpoint. A closed endpoint names the endpoint (see Closed endpoints). Two headers tell you how the request was served:

In /v1/generation, each attempt's route_context.dedicated records the endpoint id, the revision of its configuration in force, and the role.

GET /v1/models lists your organization's active endpoints, with owned_by routerplus, on the base URL the endpoint is served from. The console's Endpoints page lists them first, before the models you deployed from Optimize: for each endpoint its id, model, base URL, status, provider, price per million tokens (input, cached input and output) and limits.

What serves your request

1. 01
   Your dedicated capacity first.
2. 02
   Then the shared pool, for the same model only. The request moves on when the dedicated deployment fails before your answer starts, when its circuit is open, or when its capacity is full. It goes to public deployments of the same model that meet your endpoint's constraints. It never goes to a different model.
3. 03
   The commit boundary holds. Once the first token reaches you, the request stays with that provider. See Routing & failover.

The constraints are part of your endpoint's configuration and apply to every fallback:

A host that has not stated a value for a constrained property does not qualify. A fallback that reaches several hosts through one route sends the constraints with the request, so only hosts that meet them serve it. A region constraint excludes such a route, because it makes no promise about where a host runs.

An endpoint can also be set to primary_only. It then never falls back; when the dedicated deployment cannot serve, you get the error.

For one request you can narrow the routes but never widen them. "provider": {"allow_fallbacks": false} keeps the request on your dedicated capacity. On open endpoints, provider.only and provider.ignore filter by provider name, as on any request. Closed endpoints select their providers and hosts through the contract configuration: provider.only, provider.ignore, provider.order, provider.upstream and provider.require_parameters return 400 invalid_request, including empty selectors. Omit those fields on closed endpoints. allow_fallbacks remains supported.

Closed endpoints

An endpoint is open unless your contract says otherwise: like any request, it tells you which deployment and host of the shared pool served you, and your dedicated capacity shows as routerplus. A closed endpoint never names a deployment or a host. Every surface you can read names RouterPlus as the provider, whichever route served the request, and apart from the route role nothing you can read differs between the routes:

The records stay masked once an endpoint has been closed: what it served before it closed, and after it is opened again, still reads as above.

The fields of a closed chat completion:

Every other field is dropped, including any field a provider adds later, and a field without a value is left out (a message keeps content: null beside tool calls). A model's thinking arrives as reasoning_content on every route. A tool call's id is the gateway's own, call_ and 24 hex characters, the same on every chunk of the call and as the tool_use id on /v1/messages. In a stream, the first delta of a choice carries the role, the first delta of a tool call its id, type and name, and the finish and the usage each come in a chunk of their own.

Prices

A closed endpoint always bills your rate card on every route: a second price would tell you which route served. A fallback is tried only when its list price is known and at or below the endpoint's maximum price, if one is set, whichever price you then pay.

In /v1/generation, an attempt billed at your rate card has inference_price_source customer_rate and a price_snapshot_id of the form dedicated:<endpoint id>:r<revision>, so every charge traces back to the rate card that produced it. usage.cost on the response is the full debit across all attempts, as on every request.

Limits

For the endpoint's traffic, its own limits replace the default organization, workspace, principal and key limits:

A limit your organization set itself on /console/limits still applies on top. You can lower what one key or workspace may use; you cannot raise the endpoint's limits.

A refusal is the usual 429 rate_limit (see Rate limits & spend caps), with x-tm-limit-scope: endpoint:

GET /v1/limits?model=<your-namespace>/<name> returns your key's effective limits on the endpoint and a dedicated object: the endpoint, its revision, the model, the output default, the fallback caps and whether it spills over.

If the endpoint's outage mode is closed, requests are refused while the shared counter store is unreachable. In the default bounded_open mode they are admitted under the endpoint's own limits, divided among the gateways that serve it, for up to 10 minutes; see Limits and capacity.

The paced limit

A limit that counts each calendar second refuses requests in many seconds when traffic arrives at random, even at an average below the limit. A paced limit queues a short burst instead. It has three settings: the paced rate, a hard rate above it, and the longest wait.

• Requests leave admission no faster than the paced rate. A request that must wait waits in the gateway. x-tm-queue-ms on the response says how long, in milliseconds.
• A request whose wait would be longer than the bound gets 429, x-tm-limit-kind: rps_queue. Its retry-after says when its wait would fit.
• The hard rate refuses a request at once, x-tm-limit-kind: rps. It allows a burst as long as the bound (at least one second), so it never refuses a request the queue would hold: with a bound of a second or more, a full queue is what refuses.
• The wait comes before the request's own deadline starts, so it does not use the endpoint's total time.
• A waiting request holds no concurrency slot. Its place in the pace stays used even if you disconnect.

For example, with a paced rate of 6 requests a second, a hard rate of 6.5 and a 2 s bound:

A queue cannot hold a steady excess: over time the endpoint passes its paced rate.

Timeouts and defaults

A shorter header wait, or attempt limit for requests that are not streamed, makes failover faster when the dedicated deployment stalls. A stalled request still takes that limit plus the fallback's own time. The attempt limit applies only while another route remains: the last route may run for the rest of the total time.

Edges

• POST /v1/route does not explain dedicated endpoints yet.
• Health circuits are shared by every request to a deployment. Two failures in a row on the dedicated deployment send all its traffic to the fallbacks for 30 seconds. Per-endpoint circuit settings are not available yet.
• We change an endpoint's configuration on request. Each change is a new revision, and it reaches every gateway within seconds.