# Guardrails **Availability.** Guardrails are not available yet. This page describes how they work once RouterPlus turns them on, region by region, after the [management API](/docs/api-management). Until then no guardrail applies, the routes on this page answer `404 not_found` to an API key and `401` to a management key, and the console has no Governance pages. A guardrail is a set of rules for requests. You assign it to the organization, a workspace, a member or a key. Every guardrail that covers a request applies. When two guardrails disagree, the stricter rule wins. Guardrails apply to `POST /v1/chat/completions`, `/v1/messages`, `/v1/messages/count_tokens`, `/v1/responses`, `/v1/images/generations`, `/v1/videos`, `/v1/decisions`, `POST /v1/route` and new Parallel runs (`/v1/parallel/...`). They also apply to dedicated endpoints and to console playground requests. You manage them with the [management API](/docs/api-management) or in the console under Governance. RouterPlus serves a composite model (`tm/...`) and a server tool with model calls of its own. Each of those calls meets the same guardrails and budgets as your request. In the console playground, that includes the guardrails and budgets of the signed-in member. An organization that never had a guardrail or a grant, with HIPAA mode off, pays nothing for this: its requests read no rules. ## Create and assign a guardrail 1. Create the guardrail with a management key that has the `admin` scope: ```bash curl https://api.routerplus.com/v1/guardrails \ -H "Authorization: Bearer $TM_MANAGEMENT_KEY" \ -H "Content-Type: application/json" \ -d '{"name": "support-bot", "allowed_models": ["claude-*"], "zdr": true, "budgets": [{"id": "monthly", "limit_usd": 200, "period": "month"}]}' ``` The response is `201`. `data.id` is the guardrail's id. 2. Assign the guardrail. It has no effect until you do. `target_kind` is `org`, `workspace`, `member` or `key`: ```bash curl https://api.routerplus.com/v1/guardrails/$GUARDRAIL_ID/assignments \ -H "Authorization: Bearer $TM_MANAGEMENT_KEY" \ -H "Content-Type: application/json" \ -d '{"target_kind": "org", "target_id": "'$ORG_ID'"}' ``` A change applies to the next request on every gateway. With the guardrail above: - A call to a model that is not `claude-*` returns `403` with `error.code` `guardrail_blocked`. - A call goes only to routes that keep no prompts or completions. When no route of the model does, the call returns `403` `guardrail_blocked`. - Once the organization has spent $200 this month, a call returns `429` `insufficient_quota`. Each target has at most one guardrail. Assigning another guardrail to the same target replaces the first one there. ## Rules Unknown fields are refused. | Field | Type | Effect | |---|---|---| | `allowed_models` | string[] | Only these models. Empty means every model. See [Model patterns](#model-patterns) for the limits. | | `blocked_models` | string[] | Never these models. | | `allowed_providers` | string[] | Only these providers. Empty means every provider. | | `blocked_providers` | string[] | Never these providers. | | `zdr` (or `enforce_zdr`) | boolean | Only routes that keep no prompts or completions. | | `data_collection` | `"deny"` | Only routes that do not train on or keep request data. | | `allowed_regions` (or `allowed_data_regions`) | string[] | Only routes that process requests in these regions (`us`, `eu`, `in`, `apac` or a 2-letter code). `global` allows all. | | `blocked_infrastructure` | string[] | Never routes that run on these clouds (`aws`, `gcp`, `azure`, `oci`, `coreweave`, ...). | | `budgets` | object[] | Spending limits, at most 4. See [Budgets](#budgets). | | `limit_usd`, `reset_interval` | | One budget in a short form: a USD limit and its period. Its id is `limit`. | | `rpm`, `tpm` | integer | Requests and tokens per minute for each key the guardrail covers. See [Rate limits](#rate-limits). | | `max_output_tokens` | integer | The most output a request may ask for, from 16 (the smallest limit every provider takes). A larger limit is lowered to this value. A request with no limit gets this one. | | `server_tools` | `"allow"` or `"block"` | `block` refuses requests that ask for server tools. See [Server tools](#server-tools). | | `overridable` | `false` | Makes the guardrail a hard limit. See [Grants](#grants). | ### Model patterns - `*` matches any run of characters, `/` included. `?` matches one character. - An entry is at most 200 characters, with at most 8 `*`. `?` may stand before the first `*` and after the last one, not between two `*`. - One guardrail holds at most 200 entries in `allowed_models` and `blocked_models` together. - At most 16 of a guardrail's entries may have a `*` or `?`, the model filters of its budgets included. Exact model ids are not counted. - RouterPlus checks both the id you send and the catalog id it resolves to. A suffix such as `:free` or `:online` never hides a model. - A model id longer than 256 characters matches no pattern. ### Dedicated endpoints A rule sees a dedicated endpoint as you see it. - An open endpoint: `allowed_models` and `blocked_models` match its name (`/`) and its model. - A closed endpoint: they match its name only. A rule that names the model behind it does nothing. - In `allowed_providers` and `blocked_providers`, dedicated capacity and every route of a closed endpoint are the provider `routerplus`. A public fallback of an open endpoint keeps its own provider name. - Data rules use the facts of the route that serves the endpoint. - A refusal names the endpoint as you called it. - Budget filters: see [Budgets and dedicated endpoints](#budgets-and-dedicated-endpoints). ### How guardrails combine - Allow lists: every guardrail's non-empty list must allow the model or provider. - Block lists and blocked infrastructure: they add up. - `zdr`, `data_collection` and `server_tools: block`: one guardrail turns them on. - Regions: only the regions every guardrail allows. When none are left, no route qualifies. - `rpm`, `tpm` and `max_output_tokens`: the lowest wins. - Budgets: each one holds on its own. ### Provider facts A data rule (`zdr`, `data_collection`, `allowed_regions`, `blocked_infrastructure`) needs a fact about each route. RouterPlus leaves a route out when it does not know the fact. - A house route keeps no data, and does not collect it, when its provider's listing says it logs no prompts. - RouterPlus records a provider's no-training terms, regions and clouds only from the provider's own published terms, with the page and the date it read them. - A provider that states no region is left out by a region rule. A provider that names no cloud is left out by an infrastructure rule. - A route on your own provider key (BYOK) is judged by the facts you declared on the connection. These rules apply to a dedicated endpoint's own deployment and to its fallbacks alike. They also apply to what RouterPlus sends outside the model's route: - A `tako:` or `parallel:` server tool is judged by the facts of its provider (`tako`, `parallel`). `blocked_providers` and `allowed_providers` apply to it by that name. - A Parallel run (a task, a FindAll run, a monitor, and work you add to one) is judged by the facts of `parallel`. A tool or run that breaks a rule is refused with `403` `guardrail_blocked`, and the message names the rule. A provider's hosted tool, such as its web search, runs at the route's own provider. The request's own `provider.zdr`, `provider.data_collection` and `provider.regions` are different. They keep only routes on your own keys with matching facts, as before guardrails existed. ### Server tools `server_tools: "block"` refuses a request that asks someone other than you to run a tool: - the gateway's own tools (`x-tm-tools` and `tako:` or `parallel:` tools entries), - a provider's hosted tools: web search, web fetch, code execution, file search, image generation and remote MCP servers, - `web_search_options`, `plugins` and a `:online` model. Tools you run yourself are not server tools: functions and custom tools, Anthropic's `bash`, `text_editor`, `computer` and `memory` tools, and OpenAI's computer use, `local_shell`, `shell` and `apply_patch` tools. Every other typed tool counts as a server tool. ## Budgets ```json { "budgets": [ { "id": "monthly", "limit_usd": 500, "period": "month" }, { "id": "opus", "limit_usd": 50, "period": "week", "model": "claude-opus-*" }, { "id": "anthropic", "limit_usd": 100, "period": "month", "provider": "anthropic" }, { "id": "team-pool", "limit_usd": 2000, "period": "month", "shared": true } ] } ``` - `period` is `day`, `week` (from Monday), `month` or `lifetime`. Periods start at 00:00 UTC. A budget with no period is `lifetime`. - `model` counts only the spend on models that match the pattern. - `provider` counts only the spend on that provider's routes. - `shared: true` makes one budget for everything the guardrail is assigned to. Without it, each assignment has its own budget. A guardrail on a workspace counts that workspace's spend. - A budget counts what RouterPlus bills. Spend on your own provider keys (BYOK) is not counted, and `include_byok` is refused. - A guardrail on a member counts the spend of that member's keys, and the member's requests in the console playground from the time the budget applies to them. - A key also has its own limit: `limit` with `limit_reset` on the key. A budget holds like the monthly spend cap. Before RouterPlus sends a request, it reserves the request's worst-case cost. When settled spend plus every reservation in flight would pass the budget, the request is refused. So many requests at once, or long streams, cannot run past the limit. The refusal is the cap's `429` `insufficient_quota`. `x-tm-cap-reset` gives the time the period ends (none for a `lifetime` budget), and `x-should-retry` is `false`. See [Rate limits and spend caps](/docs/limits). A provider budget is reserved only on that provider's routes. When it is used up, RouterPlus leaves that provider's routes out and uses the model's other providers. When it has some room left but not enough for the request, that route is skipped and the next route serves. When no other provider serves the model, the request returns `429` `insufficient_quota`. A route on your own provider key (BYOK) stays: its spend is not in the budget. A budget for the provider `parallel` or `tako` also holds the fees of that provider's server tools, and Parallel runs. When it is used up, a request that asks for such a tool, and a new Parallel run, return `429` `insufficient_quota`. A `model` filter on such a budget matches the name the work is billed as, such as `parallel:search` or `parallel:task:base`, not the model of the request. `GET /v1/guardrails//status` shows each budget's limit, settled spend and reset time. In the console, open Governance > Budgets. ### When a budget can be set A budget must count all the spend of its period. RouterPlus keeps two records of spend: - **Monthly and lifetime totals** per organization and per key, complete from the first request. A `month` or `lifetime` budget with no `model` and no `provider` reads these. You can set one at any time, on any target. - **A daily record** per key, model and provider. A `day` or `week` budget, and any budget with `model` or `provider`, reads this. RouterPlus started it recently and is still building its earlier days. When the daily record does not yet hold a budget's whole period, RouterPlus refuses the guardrail with `409` `spend_history_building`. The message says from which day that kind of budget can be set. Nothing is stored, and nothing changes for the organization. A budget RouterPlus accepted stays ready. In the status, each budget has `ready`. `ready: false` means the budget's history is not whole, and RouterPlus refuses the requests it covers until it is. ### Budgets and dedicated endpoints A budget with no `model` and no `provider` counts everything, dedicated endpoints too. RouterPlus cannot yet count a dedicated endpoint by the names you see. So some filters are not available to an organization that has one: - With a closed endpoint, now or in the past: no budget can have a `model` or `provider` filter. - With any dedicated endpoint: no budget can have a `provider` filter. Dedicated capacity is the provider `routerplus`, and RouterPlus cannot count it under that name yet. - A `model` filter cannot name an endpoint (`/`). For an open endpoint, filter by the endpoint's model. A guardrail with such a budget is refused with `409` `budget_filter_unavailable`. Nothing is stored. Remove the filter, or put a budget with no filter on the keys that use the endpoint. A budget that was stored before the endpoint existed is suspended. A suspended budget holds nothing, counts nothing and refuses nothing. Its status has `suspended` with the reason, and the Budgets page says so. Remove the filter to make the budget count again. ## Rate limits - `rpm` counts requests per key, in a rolling minute. - `tpm` counts each request's input tokens plus the output limit the request set itself, at most 32,768. A default limit that RouterPlus adds is not counted. - One request never weighs more than the `tpm` limit, so a single large request can run when the minute is empty. - `POST /v1/messages/count_tokens`, `POST /v1/route`, new Parallel runs and the turns of a server-tool request are not counted. - `max_output_tokens` also lowers the size a request counts for in your key's, your organization's and the provider's token limits. A full limit returns `429` `rate_limit` with `x-tm-limit-scope: guardrail` and `x-tm-limit-kind` `rpm` or `tpm`. `retry-after` is 5 seconds for `rpm` and 10 for `tpm`. Retry after that time. ### Refused requests Requests that a guardrail refuses with `403` are counted too. On each gateway process: - a key gets at most 60 such refusals a minute, - an organization gets at most 600. Past that, a request that a guardrail refuses returns `429` `rate_limit` with `x-tm-limit-kind: guardrail_refusals` in place of the `403`, until the minute ends. `retry-after` gives the seconds left. Fix the requests before you retry. This limit never refuses a request that the guardrails allow. Such a request is served as always, for a key past its limit too. In the console playground, each member has a count of their own. ## HIPAA mode HIPAA mode is a setting of the organization (`PATCH /v1/org` with `"hipaa_mode": true`, or Data settings on the console's Guardrails page). While it is on: - Every request needs zero data retention and no data collection, as if a guardrail with `zdr` and `data_collection: "deny"` covered the organization. - Server tools are refused. - Video jobs and Parallel runs are refused: their content stays at the provider for later reads. Work added to an existing run or monitor (`extend`, `enrich`, `update`, `trigger`) is refused as well. - The organization's Parallel monitors are cancelled within minutes, for good. They show `service_disabled`. Create them again after HIPAA mode is off. When the organization has monitors, turning HIPAA mode on needs `"confirm": "cancel parallel monitors"` (the console shows a checkbox); without it the answer is `409` `confirmation_required` with their number. - A new model search (Optimize) is refused: a search keeps the eval's cases and each model's answers. - The request log keeps nothing. Turning HIPAA mode on also opts the organization out of the request log (`request_log: "off"`), and the opt-out stays when HIPAA mode goes off. Turning HIPAA mode off needs `"confirm": "turn off hipaa mode"`. See [Data policy](/docs/data-policy). ## Errors and headers | Status | `error.code` | When | |---|---|---| | 403 | `guardrail_blocked` | A model rule, server tool rule or HIPAA mode refused the request, or no route of the model meets the guardrails' data rules. The message names the guardrail. | | 429 | `insufficient_quota` | A budget is used up. `x-tm-cap-reset` gives the reset time. | | 429 | `rate_limit` | A guardrail's `rpm` or `tpm` is reached, or the key had too many [refused requests](#refused-requests). Retryable. | | 503 | `guardrails_unavailable` | RouterPlus could not read the organization's guardrails, and has no copy from the last 5 minutes. Retry after `retry-after` (2 seconds). | The `403` and the budget `429` carry `x-should-retry: false`. Do not retry them. Response headers: - `x-tm-guardrails`: the ids of the guardrails that applied. `;stale` at the end means RouterPlus could not read your guardrails just now and used the last copy it read. - `x-tm-grants`: the ids of the active grants that changed the rules. When RouterPlus cannot read your guardrails, it keeps using the last copy it read for up to 5 minutes, so a privacy rule does not lapse during a short outage. After 5 minutes it refuses the organization's requests with `503` `guardrails_unavailable` until it can read the guardrails again. It never serves a request on rules older than that. An organization with nothing to enforce is never refused this way: no guardrail is assigned, no key-limit grant is approved, and HIPAA mode is off. RouterPlus reads no guardrails for it. ## Grants A grant is a time-boxed exception to one rule. An approved grant applies from `starts_at` until `expires_at`. After `expires_at`, the base rule applies again. | Grant type | What it changes | |---|---| | `budget` | Raises one budget of a guardrail to a higher `limit_usd`. | | `key_limit` | Raises a key's own limit to a higher `limit_usd`. | | `guardrail` | Lets one model (`allow_model`, an id or pattern) pass that guardrail's `allowed_models` and `blocked_models`. Every other guardrail still applies. | - A grant lasts at most 90 days. - A budget or key limit grant must raise the limit. When two grants cover the same limit, the higher one applies. - A guardrail with `overridable: false` is a hard limit. It takes no grant. When it is assigned to the organization, its budgets also cap every grant for the same period. - An org owner or admin, or a management key with `admin`, approves or rejects a grant. Nobody approves a grant they asked for. A management key counts as the member who made it, so it cannot approve that member's request. A management key that files a request for a member (`requested_by`) cannot approve that request either: use another key, or the console. The person who asked may cancel it. - One requester may file 10 requests an hour. An organization may have 50 pending. - An API key requests and reads its own grants with `/v1/key/grants`. One API key may have 5 pending requests, and all API keys of the organization 40, so a member's request still goes through. An API key may make 20 requests and cancels a minute, and all API keys of the organization together may file 20 requests a minute and 60 an hour. - At most 8 approved `guardrail` grants whose `allow_model` has a `*` or `?` apply to one guardrail at a time. Approving one more returns `409` `too_many_pattern_grants`. Grants for exact model ids are not limited. - A rotated key keeps its grants, the requests it filed and its guardrail. In the console, open Governance > Grants. For the endpoints, see [Grants](/docs/api-management#grants) in the management API. ## Guardrail events `GET /v1/guardrails/events` lists the requests a guardrail refused, newest first. Each event holds the reason, the guardrails, the key, the model and a request id. It never holds request text. In the console, open Governance > Guardrail events. - An event can stand for many requests. Refusals with the same key, reason, model and guardrails within about a minute are one event. `detail.count` is how many, and the request id is the first one's. - When one organization has more than 8 kinds of refusal in a minute, the rest are one event per action with `reason: "other"`, no key and no model. `detail.reasons` counts each reason. - An event appears up to a minute after its first request. - RouterPlus keeps an organization's newest 20,000 events. When RouterPlus runs the retention job, it also deletes events older than 400 days, or older than the organization's `retention_days` when that is shorter.