# Ori Eval: Find the Best Model for What You're Building

By Steven Van · 2026-08-03

Choosing the model inside your product is often decided without a systematic method.

OpenRouter has launched Ori Eval, a way to test which of the 500+ models on the platform actually performs best inside a specific app, rather than relying on a benchmark leaderboard or someone else's recommendation. Told to run a single command in a coding agent, [Ori Eval](<https://openrouter.ai/blog/announcements/ori-eval/>) scans a codebase for every place a model is called, asks what matters most (cost, latency, accuracy, tool call accuracy), and shortlists the five latest models that fit.

It then writes a review.eval.ts test file, runs the app's own agent against each candidate model in parallel, and returns a table scoring bug catch rate, p50 latency, and cost per run, along with a recommendation and the reasoning behind it. Evals check which tools an agent called or avoided, and use an LLM judge to grade open-ended answers against criteria the developer sets.

- No eval-writing experience is needed: Ori Eval interviews the developer and writes the eval file itself.

- Each run pins the harness, model, and effort level, so a changed score can only be attributed to a changed model.

- A described bug, such as an agent skipping an order lookup before issuing a refund, becomes an assertion that fails until the bug is fixed and then stays in the test suite.

- Eval files are plain code, so they can run in GitHub Actions to block regressions, or be scheduled to re-run automatically when new models ship.

Getting started means running a skill installer command in a coding agent (or the */spawn-ori-eval* slash command via the [OpenRouter](<https://creatorstoolbox.com/tools/openrouter>) MCP server), which installs Ori, asks for sign-in, and runs the interview. Running evals also requires Bun.

[![](<https://openrouter.ai/blog/images/ori-eval.png>)](<https://openrouter.ai/blog/announcements/ori-eval/>)

![OpenRouter](<https://openrouter.ai/favicon/glyph.png>)

OpenRouter

One API for 500+ AI models across 80+ providers — pay with credits that work anywhere, with automatic fallback when a provider goes down.

[View OpenRouter →](<https://creatorstoolbox.com/tools/openrouter>)

Sources

[Original source](<https://openrouter.ai/blog/announcements/ori-eval/>)

[Read Ori Eval: Find the Best Model for What You're Building on Creators Toolbox](<https://creatorstoolbox.com/blog/openrouter-ori-eval-find-the-best-model-for-what-you-re-building>)

---
Canonical source: https://creatorstoolbox.com/blog/openrouter-ori-eval-find-the-best-model-for-what-you-re-building