Occam
A router with a scoreboard. It decides which agent should do each piece of work under a hard rupee budget, then proves the decision was right with numbers.
- Role
- Design and implementation
- When
- 2026
- Stack
- Python
- FastAPI
- Postgres
- Redis
- Langfuse
An agent that reaches for the largest model every time is trivial to build and expensive to run. Choosing a cheaper agent per subtask is easy to claim and hard to prove, because the two obvious things to measure turn out to answer different questions.
Occam picks the cheapest agent that can actually do each subtask, inside a budget it cannot exceed, and reports what that choice cost against a single-frontier-model baseline on the same tasks with the same graders.
A task is planned into a DAG of subtasks, each routed to an agent under a reserved budget, executed in parallel, checked by a critic that can send work back, and finally merged.
Plan
A planner breaks the task into a directed graph of subtasks, each with acceptance criteria a stranger could check. Many tasks need exactly one subtask, and the planner is expected to say so rather than decompose for its own sake.
Route
Four strategies sit behind one interface: a single-model baseline, an LLM router, an embedding router and a bandit. Every candidate is scored and stored, not just the winner, so a routing decision can be examined after the fact.
Over a stratified 24-task subset, the bandit reached 95.8% success at ₹0.4574 per task against the baseline's 100% at ₹0.9425. That is 96% of the baseline's success rate at 49% of its cost, with p95 latency of 16.2 seconds.
Execute
A DAG engine runs the graph in parallel where dependencies allow, with timeouts, transient retries and model-tier escalation.
Verify
A critic scores each output against that subtask's acceptance criteria before it reaches the merge, and never sees which agent produced the answer. A critic that knows the output came from a cheap model judges it differently.
One choke point for every model call
A single file talks to the model API. Planner, router, critic, agents and the benchmark judge all pass through it, which is what makes the cost column trustworthy: if a token was spent, it was spent there.
About 6,400 lines of Python and 2,800 of TypeScript, with 150 tests. One of them asserts that no run in the database has ever exceeded its budget, reading the real historical record, so it gets harder to pass over time rather than easier. Across 701 recorded runs the number that breached budget is zero.
Budget enforcement had to become reserve-then-settle rather than check-then-spend. With four nodes in flight, reading the total and then calling is a race: each node sees room for one more call and all four make it. A reservation is now taken under a lock and held for the duration of the call, and the reserved figure is deliberately the worst case, with output assumed to be the entire max_tokens ceiling. A bound the actual spend can exceed is not a guarantee.
That has a cost. The ceiling is charged in full before the call, so a generous max_tokens consumes budget even when the answer is three words long. At max_tokens 8192 a single cheap node reserves ₹3.6, and three parallel nodes cannot fit inside a ₹5 budget despite really costing about ₹0.01 each.
The critic had to be taught to reason before it rules. Structured output is generated in schema order, and with the boolean declared first the critic committed to false and then wrote its justification to match: it rejected a correct word count three times, explaining that the output of 9 was correct, and escalated it through two model tiers. Moving the reason field above the verdict cut that run's cost in half and its latency by 2.3 times. Two fields, reordered.
The two headline metrics measure different things, and only one of them is about money. Oracle agreement fell from 83% to 67% to 54% across the three routers while overspend stayed flat at 0 to 4%. The bandit disagrees with the answer key on nearly half its decisions and costs almost nothing, because most of those disagreements are between two equally cheap agents. Sending a word count to the wrong tier-1 agent is a wrong decision that is free.
Reported on oracle agreement alone, the bandit looks like the worst of the three routers. It is the cheapest one that matches the others' accuracy. Picking a single headline metric before understanding what it rewards would have buried the actual result.
Model ceilings should be sized to the work from the start. The reservation logic now halves them before a run gives up, which is a workaround for having set them generously in the first place.