CoeOS — the benchmark-composed virtual model & superagent

In one sentence: CoeOS routes every request to the model our benchmarks proved best at that specific skill — and, in its full form, orchestrates a pipeline of dedicated agents (plan, critique, execute) on top of that routing. Measured result: 97.2/100 on the TMB panel — ahead of Opus 4.8 (96.0) and Fable 5 (96.9), at a fraction of their cost.

CoeOS exists at three levels, all driven by the same benchmark data:

Level	What it is	Where
Virtual model `CoeOS`	One OpenAI-compatible model that classifies each request onto a skill axis and relays it to the axis champion.	Built into the OdyssAI-X engine.
CoeOS superagent	A pipeline of dedicated agents — triage, planner, grill, skeptic, executor — each role served by the model best on its competency pair.	`github.com/Odyssai-eu/coeos` (AGPL-3.0) + the assignment console (`:4800`).
CoeOS SE	The Simple Edition: the same skill routing as a standalone server over OpenRouter. `uvx coeos-se` and you have the endpoint.	`github.com/Odyssai-eu/coeos-SE` (MIT, on PyPI).

The routing core (virtual model)

CoeOS doesn’t run inference itself. It classifies, then relays:

Agent request  →  CoeOS
                    │  1. Which skill? (Python? GDPR? creative?…)
                    │     → explicit header, or the decider model classifies
                    │  2. Which model is best on that axis?
                    │     → the TMB Score Table says so (measured, not guessed)
                    │  3. Per-axis policy applied (thinking / reasoning effort)
                    ▼
              the axis champion (local or cloud)  →  response

Classification order:

Explicit header — an agent that knows its phase sends x-coeos-axis: plan_spec and skips classification entirely (faster, deterministic).
Decider — otherwise a small, fast model classifies the request into one axis. The decider is configurable (any local, Telemak-proxied, or cloud/OpenRouter model) and its calls are logged.
Default axis — if classification is ambiguous or the decider is unavailable, the configured default applies.

Then axis → bound model. If the champion can’t be served, CoeOS returns an explicit 503 — it never silently substitutes another model. (An optional cold-boot autoload can relax this for the default axis.)

The 18 skill axes

creative, redac_pro, legal_rgpd, legal_complex, reasoning, calc, python, code_general, debug, react, swift, refactoring, plan_decompo, plan_spec, plan_judgment, fast_tools, agent_exec, agent_safety.

The last two score agentic behaviour specifically: tooled execution discipline (agent_exec) and safety/scope discipline (agent_safety) — measured by a dedicated agent bench, not inferred from code scores.

Per-axis generation policy

Each axis can carry a thinking / reasoning-effort policy: a request routed to reasoning can automatically run with reasoning_effort: xhigh, while fast_tools stays lean. Priority order: explicit client setting → per-axis policy → server default. (Measured: reasoning_effort is the effective lever — ~1.6× more reasoning tokens between low and xhigh on GLM-class models.)

Observability

Every routed response carries x-coeos-axis, x-coeos-model and x-coeos-provider headers; compatible clients (Companion) display CoeOS · python — Kimi-K3. Routing decisions and decider calls are visible in the console’s logs tab.

The superagent (CoeOS full)

Routing alone picks the best model per request. The superagent goes further: it structures the work itself. A task submitted to coeos-run flows through dedicated agents, each served by the model best on its competency pair (pairs discriminate where single axes saturate):

Role	Function	Tools (restricted)	Competency pair
triage	judges complexity, prescribes the flow	—	format-reliable model
planner	closed, executable plan	read-only	plan_decompo × plan_spec × reasoning
grill	tightens the plan before execution	read-only	plan_judgment × reasoning
skeptic	attacks the result after execution	+ bash (verify)	debug × plan_judgment
executor	applies to the letter, fast	only role with write	agent_exec × code_general × speed

Two design rules, both proven the hard way:

Separation is structural, not prompt-based. Only the executor can write; the others physically lack the tools. A role cannot exceed its mandate.
Adaptivity from the model, rigor from the code. The triage decides which steps to engage (typed decision, pydantic-ai — no brittle JSON parsing), but deterministic floors override it: a task carrying a verifiable done-check always gets the skeptic, whatever the triage judged.

Hybrid by role: high-frequency execution runs on local models (zero marginal cost); low-frequency plan/critique goes to the best model for the job. That mix is where the cost advantage comes from.

The assignment console (`:4800`)

The control plane of the superagent. Import a TMB Score Table, and the console suggests a model per role from your engine’s live fleet (best competency-pair score, speed-weighted for the executor, ties broken by measured cost). Everything is overridable — any servable model is assignable, scored or not. Four tabs: assignment, criteria, decider (choose and push the decider model), and routing logs. Verbose models are flagged (token-burner: an outlier costs tokens and latency that quality scores don’t penalize — the flag makes it visible at assignment time).

The evidence — TMB benchmarks

The routing table is not a leaderboard opinion. It comes from the TMB panel: 32 tests across 5 suites (agentic, code, planning, playoffs, writing/legal/reasoning), 30+ models, aggregated into the 18 axes.

Judging protocol (hardened after proving a naive LLM judge unreliable): the judge — Opus 4.8, temperature 0 — scores each grading criterion in isolation (notes-only), and the harness computes every total mechanically. No LLM ever writes a verdict or adds numbers. When a test bundles several competencies, axes bind to the relevant criteria subset (e.g. creative = the five creative criteria of the fiction test, excluding length compliance).

Headline result (2026-07, equal-footing API runs):

#	Model	Global /100	Cost/test
1	CoeOS v1.3	97.2	$0.087
2	Fable 5	96.9	$0.426 — 4.9× more
3	Opus 4.8	96.0	$0.141 — 1.6× more

The residual biases run against CoeOS (the judge scores its own answers as a competitor), so the margin is a floor. Per axis, CoeOS’s curve tracks the envelope of the best specialists — no dips where a single model does better alone.

The TMB Score Table — data, not settings

The engine consumes raw data and ranks for itself (format tmb-score-table/1): per model — stable join keys (or_id, served_model, alias), per-axis scores with sample counts, measured cost-per-test, tok/s, verbosity and token-burner flags. The resolver picks champions per axis at import time against your registry; ties within a band go to the cheaper model.

{
  "format": "tmb-score-table/1",
  "models": {
    "Kimi-K3": {
      "or_id": "moonshotai/kimi-k3",
      "cost_per_test": 0.286, "tps_median": 31.2,
      "token_burner": true, "verbose_benches": ["C"],
      "axes": { "creative": {"score": 97.8, "n": 1, "verified": true},
                "code_general": {"score": 97.9, "n": 10, "verified": true} }
    }
  }
}

The table is regenerated from the bench harness when the panel moves (new model, new scores) and shipped to the console / SE. Pins always win: a model you pinned on an axis is never overridden by a suggestion.

Configuring

Engine (virtual model): import/push a score table via the console (:4800) or PUT /admin/coeos. Enable the virtual model; CoeOS appears in /v1/models. The decider and per-axis policies are set from the console’s decider tab.
Superagent: clone Odyssai-eu/coeos, ./install.sh, compose the role assignment in the console, run tasks with ./coeos-run "…". The repo ships an AGENTS.md playbook for AI-agent installs.
SE (no engine at all): uvx coeos-se → dashboard on :4600, paste an OpenRouter key. Settings ship from GitHub (one-click Update from GitHub — no manual imports).

⚠️ The decider must stay warm. Fine-grained routing depends on it: if it’s not servable, requests fall to the default axis. It’s small — reserve a slot.

Using the virtual model

export OPENAI_BASE_URL="http://<your-odyssai-x>:8000/v1"
export OPENAI_API_KEY="dummy"          # no key required on a LAN

curl "$OPENAI_BASE_URL/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{"model":"CoeOS","messages":[{"role":"user","content":"Write a Python script that…"}]}'

Force an axis when the agent already knows its phase:

  -H "x-coeos-axis: legal_rgpd"

Run the superagent pipeline on a task:

./coeos-run "fix the failing test in ./src and prove it passes"
./coeos-run --json "…"        # machine-readable report of every role's output

In short


What	Benchmark-composed routing (virtual model) + a dedicated-agent pipeline (superagent) + a standalone edition (SE).
Proof	TMB panel, Opus-judged, mechanical totals: CoeOS 97.2 vs Opus 96.0 / Fable 96.9 — at 1.6–4.9× lower cost.
How	Header or decider → axis → champion from the score table → per-axis policy → relay. Superagent: triage-prescribed flow, write restricted to the executor.
Config	Console `:4800` (score table, roles, decider, logs) · `PUT /admin/coeos` · SE dashboard `:4600`.
Usage	`model: "CoeOS"` on the engine · `./coeos-run` for the pipeline · `uvx coeos-se` standalone.
The edge	The envelope of the best open-weight specialists, grounded in measured evidence — not one frontier model’s average.