We benchmarked it before we believed it.

Primary endpoint

Same graded answers. 13–31% fewer fresh tokens.

Two arms, identical prompts, three repetitions each: an agent with only its built-in file tools, and the same agent with Euthynos. Fresh input tokens are the exact API-reported figures; each bar is a per-task median over valid sessions.

The part that makes the tokens mean something

Recall was identical. Perfect, in both arms.

A token saving is only real if the answers stay right. Every required item in the frozen keys — every caller, every implementation, every impact — graded present or absent, per session, before transcripts were opened.

Secondary signals

Where the arms differed: discipline under uncertainty.

Same recall — different behaviour at the edges. False positives and unhedged “these are all” claims were graded per session against preregistered definitions.

Attempted vs valid

21 of 42 sessions survived the rules. The rest are reported, not hidden.

The validity rules were frozen before session one: any profile-rule denial, turn-cap, or transport failure disqualifies a session. Mid-run, an external account session limit consumed two entire tasks — so those tasks are unmeasured, and they stay that way in every claim we make.

The benchmark bit us, too

M2 found a real defect in our own engine.

Every valid blast-radius session — in both arms — found a caller our call graph missed: a class property holding an arrow function. The parser extracted method declarations but skipped property-assigned functions, so their calls never entered the graph. That is exactly the kind of failure this benchmark exists to catch.

// invisible to the graph before M2:
class ClientRequestImpl {
  fetch = async (args) => {
    …
    cookies.push(serialize(key, value))
    …
  }
}

// the released engine, after the fix:
$ callers_of serialize

9 transitive callers of serialize:
  d1 fetch — src/client/client.ts:58
  d1 generateCookie — src/helper/cookie/…
  …

The fix shipped with six pinned regression tests. The frozen M2 numbers were not retroactively changed — the miss stays recorded, because rewriting history is how benchmarks stop meaning anything.

Engineering measurements

The performance envelope, stated honestly.

Validated to roughly 10,000 files, and comfortable well below about 1,500. Above 10,000 it is not validated and should not be assumed to work. Memory, not latency, is the binding constraint at the top of that range: on a 10,000-file repository process RSS rises from roughly 0.6 GB to about 1.1 GB during an edit loop.

Precise latency figures are deliberately not published in V1. Two internally consistent measurements disagreed, and the controlled experiment that would have settled which to trust could not be completed — so we publish neither, and ship the harness instead. Index reads are still neither free nor scale-invariant: find_symbol, read_function and find_references cost real time and grow with repository size. An earlier version of this page claimed ≤0.3 ms; that figure was an argument-rejection error path, not a read. Run the shipped harness for numbers that describe your own hardware.