Optimize

Optimize finds a cheaper model or mix that scores as well as yours on your own Braintrust evals, and deploys it as an endpoint: one model id you call like any other.

Before you start

1. 01
   Connect Braintrust on Integrations. Paste a Braintrust API key. The key is checked with Braintrust and stored encrypted, and the page shows only its last four characters. Owners and admins can connect, replace or disconnect it; viewers can only look. (Langfuse is on the same page as Request access: it is set up with the team, not self-serve.)
2. 02
   In Braintrust, have one of these to search on:

The baseline must be a catalog model or one of your own deployed endpoints (tm/...). A search takes up to 2,000 cases.

For an experiment or a dataset, these parts of the rows matter:

Code scorers run in Braintrust. Prompt scorers (LLM graders) run through the gateway and are billed to your credits.

Start a search

Click New search at the bottom of the searches list. Pick the source, name the search, and set the search budget. The budget is the most the search spends on model calls, from $5 to $100. Model calls are billed to your credits as the search runs, under a managed key made for the search (turn on Show playground keys on /console/keys to see it, with a managed tag). Viewers cannot start or change a search.

The search runs in stages. The budget pays for them in this order:

1. 01
   Shortlist. Public benchmarks pick the open models to try, as the Pareto frontier of benchmark score against price. Each model is tried on up to three of its hosts: the cheapest, the one at the highest precision, and the model maker's own.
2. 02
   Pilot. Every candidate answers a sample of the conversations.
3. 03
   Dev. The leaders answer all the dev conversations.
4. 04
   Mixes. Cascades, routers and votes are built from the saved answers; a search agent proposes mixes to try. These cost nothing extra.
5. 05
   Held-out. The finalists, including a fusion of the best models, answer conversations the search never saw. The winner is picked on these.
6. 06
   Customers. The finalists talk to a simulated customer who has the goal of a held-out conversation.

When the search reaches its budget it stops and keeps its results. Add $25 continues it from where it stopped. Stop ends it and keeps its results.

If a Braintrust read fails while the search loads its cases, the worker tries again 3 more times — after 2 s, 10 s and 30 s — before it stops the search. A stopped search says why; fix the cause (reconnect the key, say) and add budget to continue.

Read the results

The chart plots each candidate's cost against its score, with the best-value frontier as a line. The table below it:

The ★ row is the winner: the cheapest finalist that costs less than the baseline, is not worse than the baseline on a paired test over the same held-out cases, and scores within 2 points of it. Ties on cost go to the lower median latency. Rows marked dev or pilot were scored on fewer conversations and cannot win. If no row qualifies, there is no star.

Open a row to see the cases where it did worse than the baseline, side by side, and its score per tag.

Deploy the result

Deploy makes a result callable as tm/<name>-v1. The name is lowercase letters, digits and dashes, 40 at most:

curl https://api.routerplus.com/v1/chat/completions \
  -H "Authorization: Bearer $TM_KEY" \
  -H "content-type: application/json" \
  -d '{"model": "tm/billing-agent-v1", "messages": [{"role": "user", "content": "Hi"}]}'

• Only keys of your organization can call it.
• Each model call it makes is billed like a direct call to that model. The response's usage.cost is their sum.
• Send it on POST /v1/chat/completions. With "stream": true it returns the finished answer as one chunk, then usage.
• Deploying again gives a new version (-v2). Earlier versions keep working.

Endpoints lists everything your organization deployed, with the search it came from. Try it sends one held-out case of that search to the endpoint and bills your credits like any call.

What is stored

A search stores its cases and the answers candidates gave, for your organization only, until you delete the search. Deleting a search deletes them. A deployed model id keeps working after its search is deleted.