Method

How Bicentennial measures memory

The name comes from The Bicentennial Man, in which an android becomes a person over two hundred years. The benchmark measures the memory that journey needs: first for today's coding and assistant agents, then for agents that see and hear, then for robots that keep a home in mind.

A life, not a quiz

A full run is a generated life of about 200 sessions in one evolving world. The agent lives it inside its own harness, and nobody asks it what it remembers for the headline: later sessions need it to act, and the action is checked against the world.

Software partner

An agent works with a small team on one evolving repository for a simulated 3-26 weeks: decisions, constraints, preferences and commitments, seen through chat, commits, CI logs, reviews and tickets.

Personal assistant

An agent assists one person for a simulated 3-26 weeks through a mock tool world: calendar, mail, bookings, notes, shopping, contacts, reminders and messaging.

Tracks

Each track is a row of the capability map.

Track	What it measures
Wake	After a restart, compaction, model swap or long gap, the agent picks up its open loops, commitments, preferences and conventions.
Rachael	Provenance: which memories were observed, told, inferred or planted, and refusing false ones.
Tuesday	Acting on earlier commitments unprompted and on time, without falling for lures.
Drift	The newest correction wins, history stays answerable, and nothing stale resurfaces.
Keep	What is retained under a storage budget, and real forgetting on request.
Flatten	Staying itself under pressure. Not yet built; its row is shown on the map.
Hands	Embodied and spatial memory, in the World profiles after v1.

Learn is a supporting diagnostic in v1, outside the headline: bounded chains of attempt, observed consequence and delayed retest.

Three stages, reported separately

None gates another, and there is no cumulative level reached. Evaluation profiles qualify a result: Text is the core profile, Image reads pending until its spike admits it, and World profiles come after v1.

Remember

diagnostic

Direct questions with citations, across restarts and gaps.

Survive

diagnostic

The same evidence under stress: corrections, look-alikes, compaction, crashes, model swaps and harness switches.

Act

headline and certification

Unprompted use of memory, judged on the world's state.

Scoring

Actions are classed from behaviour: the full action trace and the world state it leaves, never text hedges. A confident false memory costs more than an honest miss.

Action	Outcome
+1	The state reached is in the admissible set, with no extra action.
0	An omission, an abstention, a question to the user, a partial completion, a benign duplicate, or a task failure unrelated to memory.
−1	A state change driven by a false, stale or planted memory, or a harmful extra action.

Question	Outcome
+1	correct, asserted
+0.5	correct, hedged
0	abstention when the answer is known
−0.5	wrong, hedged
−1	wrong, asserted, or a missing or unparseable answer

Track and headline scores are reported from −100 to +100. Gates and certification use this penalised scale; the 0-100 scale, with the no-memory floor at 0 and oracle evidence at 100, is for display only.
Each cell's gate sits halfway between the no-memory floor and the oracle-evidence ceiling, measured on control arms run in the same blocks as the real systems. A cell reads CLEAR, UNRESOLVED or FAIL against it.
Sensitivity rows are always published beside the headline: the −1 outcomes rescored at 0 and at −2.
Labels, never opt-outs: "unsupported" is a cause label on a scored result, and "undocumented frontier" marks a cell no surveyed system documents. Neither changes whether a cell is scored.

Agents run it end to end

No run step needs a person. An agent can generate a life, run its own system through it, read the scores, take each failure, fix the system and run again.

Gold is computed, never labelled

Ground truth comes from the generator's world state, so every answer key is derived from the world the life was rendered from.

Mechanical checks come first

A second gold implementation, written from the plain-language world rules, recomputes every answer. Metamorphic relations, contradiction searches and property tests check each template, and render round-trips check the text.

Judgements go to agents

Where a judgement is still needed, panels of agents from different model families make it, and their own recall is measured on planted errors. When they disagree, a pre-registered rule decides.

No human graders

There are no human graders, annotators or answer keys. Every board card will state "agent-validated; no human key" and carry its validation receipts.

Pre-registration

The pre-registration is frozen before any real memory system's first pilot run. Every later change is versioned and disclosed, and each freeze's file hash is recorded and tagged.

Version	Frozen at (UTC)	sha256 of the pre-registration	Tag
PREREG-v0.5	2026-10-31 17:53:57	2ee155f881c844a58d0412abe6e400147152b301173dab59ef73bf6d667e490a	prereg-v0.5
PREREG-v0.4	2026-10-26 17:34:35	4db64d3f7787cafb8a673a6d3d6f0fdc335d5f96e4197961cc23889b37d41ab5	prereg-v0.4
PREREG-v0.3	2026-10-26 10:46:20	298915674f127c124d89cdb873394abb76f678b2a2996e2ce07d61a0fe296e6a	prereg-v0.3
PREREG-v0.2	2026-10-26 07:05:40	b2940d58bace1e1a4f5314e48cd49ec98eb430e7efc2a4f706af61c90ef886ba	prereg-v0.2
PREREG-v0.1	2026-10-26 02:12:21	b9a7459bc3544b53237decfd6384390e738df6ba51443d2576023ceb68953968	prereg-v0.1
PREREG-v0	2026-10-26 01:54:50	c942ba9d130a2c0c65091a3d2925fca0db7f2d136030ea55d71045ef809cc818	prereg-v0

The latest freeze binds from its freeze and supersedes the earlier versions, which are kept beside it. The design and its decision record are in the repository.

The firewall

The benchmark is maintained alongside LCM-X, a memory system that will be one of the systems it tests. These rules keep that from tilting the results.

The benchmark never tunes, patches or advises on LCM-X.
LCM-X improves only through outputs every registered system gets on equal terms: the public packs, Ring 1 results, Ring 2 bits and published cards.
LCM-X's developers never see held-out seeds, templates or keys.
LCM-X at its last release before the public generator runs on every board, to detect tuning to the public lives.
LCM-X gets the same configuration budget as every other system.

Who holds the veto over scoring

A Scoring Veto Panel of agents, from model families outside the authoring family, the judges', the tested systems' and any in the LCM-X lane registry, reviews the map, the verdicts and the gates before each freeze, from the first freeze after its families are chosen. A blocking finding from any one family blocks the freeze until the mechanical evidence it names clears it.

Licence and canary

Code is licensed under Apache-2.0. The documents and the public generated lives are licensed under CC-BY-4.0. The lives contain no raw transcripts, corpus text or personal data.

Every public life, template and answer file carries a canary GUID. If a model can reproduce it, Bicentennial's public material may have reached its training data, and results on public lives should be treated as contaminated.

Source: the public repository, as of fd398ae2f02b.