LOCAL FIRST · FROZEN EVALS · YOUR KEYS

Set the bar.
We find the system.

Give RunEvals a task, budget, and precision and recall target. It compares models, training, tools, and temporary agents under one frozen test, then returns the smallest system that passes.

Inspect the mechanism ↓

Current proof: local, curated evaluation demo. No production writes.

SYSTEM SEARCH
CODE-RISK-01HOLDOUT PASS
FROZEN CONTRACT

Detect unsafe code changes without blocking harmless patches.

MAX BUDGET$1,000
PRECISION≥ 92%
RECALL≥ 85%
01
CONTROLVanilla model

100% precision · 67% recall

RECALL MISS
02
LEARNEDLocal specialist

50% precision · 100% recall

PRECISION MISS
03
COMPOSEDSpecialist + inspectors

100% precision · 100% recall

RELEASE
UNTOUCHED HOLDOUT · 12 CASES · CURATED DEMORELEASE RECEIPT SEALED ✓
THE RELEASE PROCESS

The contract decides what ships.

  1. 01 / FREEZE

    Write the test first

    Define the task, budget, evaluation set, thresholds, and permissions before comparing systems.

  2. 02 / SEARCH

    Compare complete systems

    Test vanilla and post-trained models, tools, routes, and temporary agents on the same work at matched cost.

  3. 03 / RELEASE

    Ship evidence, not a score

    Return the winning graph with traces, costs, hashes, interventions, and a known rollback target.

FOR ENTERPRISE TEAMS

Buy an outcome, not a model.

Model selection is only one decision inside a production system. RunEvals keeps the business requirement fixed while the implementation changes underneath it.

01

Know when a cheaper system is actually safe

Replace expensive calls only when the candidate clears the same untouched holdout and permission gate.

02

Audit every improvement

Keep the evaluation version, request hashes, model routes, spend, and evidence needed to reproduce a promotion decision.

03

Change providers without changing the contract

Compare local models and external APIs against the same measurable release bar instead of rebuilding the workflow around one vendor.

WORKING BOUNDARY

The current demo runs locally and fails closed.

It compiles a task contract, executes a frozen evaluation, compares candidate systems, and emits an accepted or rejected release receipt. The displayed 12-case result is curated demonstration data, not a production benchmark.

Inspect the code ↗