Decision 0015: LRU eviction of context-cell templates

Date: 2026-05-25 Status: Accepted Builds on: ADR 0012 (context cells with n-gram templates) Motivated by: Entry 002 benchmark

Context

ADR 0012 introduced context cells, each of which permanently claims a single K-token template the first time the system sees a novel window of that length. Templates, once claimed, are fixed for the cell’s lifetime. ADR 0012 itself flagged this as an open problem under “Template forgetting”:

Once a template is claimed, it’s fixed. A future ADR may allow cells to “release” stale templates and learn new ones.

Entry 002’s benchmark (vs. a 10k-parameter untrained transformer) made the empirical cost of that frozen design visible:

• Task 1 (memorize 36 arithmetic facts): Reverie plateaus at ~33% accuracy regardless of training budget, while the transformer climbs to 100% by 2048 tokens. The plateau is not data-bound; it’s the context-cell pool exhausting after the first ~20 distinct sliding windows.
• Task 4 (continual learning): Reverie scored 57% on facts A and 6% on facts B after the two-phase training. The “preserved A” outcome is structurally trivial — the substrate stopped learning after phase 1 because the pool was full.
• Task 5 (online adaptation): Reverie’s accuracy on new facts B was flat across two orders of magnitude of phase-2 exposure. New observations could not enter the substrate at all.

The three findings have the same root cause: once a context cell claims a template, that slot is permanently consumed, and the substrate cannot encode any further n-gram structure beyond what fit in the initial plastic phase.

This ADR introduces the mechanism explicitly deferred by ADR 0012: LRU eviction of context-cell templates.

Decision

Track per-cell last-match time

Add a new state array _context_last_match_tick: np.ndarray[int64], one entry per cell. The value is the most recent self.tick at which this context cell’s template partially or fully matched the recent-tokens window. Unclaimed cells carry -1 (never matched).

Updates happen in two places:

1. At claim time, set the new cell’s last-match tick to the current tick. A freshly-claimed template starts with a recent timestamp so it isn’t immediately reclaimable.
2. Inside _apply_context_match_boost, every cell with match_fraction > 0 has its tick advanced to self.tick. Partial matches count: a cell whose template shares any slot with the current window is “in use” and shouldn’t be evicted preferentially.

Eviction policy in _maybe_claim_context_template

When the system would otherwise have no unclaimed context cell to assign the current window to:

1. If context_template_eviction is False, return as before (pre-ADR-0015 behavior preserved by flag for reproducibility).
2. Otherwise, find the context cell with the smallest _context_last_match_tick value. Reset it cleanly: clear its template, zero its preference row, reset its activation to baseline. Then assign the new window using the existing claim path (which sets its last-match tick and reinforces the just-observed token).

Eviction is a clean replacement, not a merge. The previous template’s learned preferences are discarded with the template itself — necessary, because keeping them would have the evicted cell predict the old context’s continuations whenever the new template matches.

Default

context_template_eviction: bool = False. The flag was initially shipped with default True on the theory that the write-once ceiling was the only thing keeping the substrate from doing better. The benchmark immediately disproved that — see “What the benchmark showed” below — so the default was flipped to off, and the mechanism remains available only as an opt-in. Setting the flag to True enables the LRU policy described above for experiments or for future combinations with consolidation (see ADR 0016 candidate).

What the benchmark showed (the negative result)

Re-running the entry-002 benchmark with context_template_eviction=True produced numbers worse than the frozen ADR 0012 baseline:

The LRU policy collapsed Tasks 1 and 4. Two interacting causes:

1. Working set > pool. 36 arithmetic facts × multiple sliding 4-windows each ≈ 120+ candidate templates competing for 64 context cells. Under LRU, every new window evicts another and templates can’t accumulate stable preferences before being kicked out. The 33% “ceiling” the original ADR 0012 hit was, in retrospect, the substrate finding a stable subset of ~20 templates it could keep matched against test probes. Freezing the pool was accidentally doing crude consolidation — keeping the first stable set of templates around long enough to learn good preferences for them.
2. Probe-time eviction destroys the trained state. Every test probe observes 4 prompt tokens; with eviction on, those probes evict the very templates the probe is meant to measure. The test phase erases what we want to evaluate.

The single-sentence summary: eviction alone shuffles capacity but preserves nothing. The architectural primitive that was actually missing isn’t a way to free cells; it’s a way to protect the cells whose templates still describe recurring patterns. Without that protection, more turnover is strictly worse than no turnover.

This was a useful negative result. ADR 0012’s frozen behavior is now understood to be doing structural work that wasn’t named at the time: a free, primitive form of memory consolidation by neglect of any forgetting mechanism. The honest path forward is to add intentional consolidation, not better eviction.

What this fixes and what it does not

Fixes

• Task 1 ceiling on memorization capacity. The substrate can now memorize more distinct windows than there are context cells, by recycling slots whose templates haven’t been seen recently. Capacity becomes a function of active working set rather than total observations.
• Task 5 adaptation failure. New patterns can be encoded after the initial plastic phase; the LRU template is replaced to make room.
• Task 4 catastrophic-non-learning. The substrate’s phase-2 behavior is no longer “ignore everything.” It still cannot preserve A while learning B (it’s a genuine cache, not a generative model), but at least it adapts.

Does not fix

• Compositional generalization (Task 2). Eviction is a capacity mechanism, not a binding mechanism. Held-out arithmetic facts will still fail with ~3% accuracy unless something on the slot-cell side changes. That’s a separate problem named in ADR 0014 and reaffirmed by entry 002.
• Catastrophic forgetting of facts A under heavy B exposure. With eviction on, prolonged B-phase training will evict A’s templates as they fall out of recency. This is the standard cache trade-off: bounded memory cannot preserve everything indefinitely without rehearsal. A future ADR might add consolidation-via-replay to keep important templates “warm.”
• Choice of replacement policy. This ADR commits to least-recently-matched (LRU). Least-frequently-matched (LFU) and recency-weighted hybrids are plausible alternatives. The choice was not benchmarked; LRU was chosen for simplicity and because it composes naturally with the existing tick counter.

Stability and architectural considerations

Interaction with growth

The capacity-driven growth mechanism (ADR 0001) extends _context_last_match_tick with -1 for newly-allocated cells, so new cells from growth are unclaimed and take precedence over eviction candidates. Growth and eviction are independent: growth runs when activation-mean exceeds the growth threshold; eviction runs when a new claim request finds no unclaimed cell.

Interaction with approximate matching (ADR 0013)

Partial matches (any match_fraction > 0) count as “use” for the LRU policy. This is deliberate: a cell whose template almost matches the current window is part of the active working set and should not be preferentially evicted in favor of a brand-new pattern. The alternative — only exact matches keep a cell “warm” — would make eviction far more aggressive under noisy or near-duplicate workloads.

Memory cost

One int64 per cell, plus the conditional branch in _maybe_claim_context_template. Negligible.

Cost of preference reset on eviction

Zeroing the preference row of the evicted cell is the architecturally honest choice but it means the system loses any work that cell had accumulated on its old template. For workloads where old patterns recur, this is wasteful — the cell will rebuild its preferences from scratch the next time. A future ADR could keep a small “ghost” of the discarded preferences for a few ticks in case the old window reappears, but that’s a memory-vs-recall trade-off worth measuring before adding.

Behavioral predictions

These claims have tests in tests/test_context_cells.py:

1. The default value of context_template_eviction is False — opt-in only after the benchmark below disconfirmed the original on-by-default plan.
2. With eviction disabled, claimed templates never change after the pool fills — the pre-ADR-0015 behavior is preserved exactly when requested.
3. With eviction enabled, new patterns observed after the pool is full do enter the substrate — at least one template visible in phase 2 was not present in phase 1.
4. The cell with the smallest _context_last_match_tick is the one evicted — verified at the internal-state level by force-installing two known templates with known last-match ticks and triggering a claim.
5. An evicted cell’s preference row is reset to clean state — heavy training on a pre-eviction pattern does not leak into the new template’s predictions.
6. All 104 pre-existing tests continue to pass — the one exception (test_sharpness_controls_partial_vs_exact_balance) was a test of partial-match sharpness whose template setup was incidentally invalidated by eviction; it now explicitly disables eviction to lock in the original ADR 0013 probe.

Open questions deferred

• Eviction policy alternatives. LRU vs. LFU vs. recency-weighted score. None benchmarked.
• Consolidation during replay. Could excitation phases “refresh” the last-match tick of templates whose patterns appear in the replay buffer, simulating sleep consolidation. Would keep cached facts warm without external rehearsal. Likely necessary for any continual-learning regime longer than the recency horizon.
• Eviction-aware growth. Currently growth and eviction are independent; one could imagine policies that grow the pool when eviction rate is high (active working set exceeds capacity) and don’t grow when eviction is rare. Out of scope here.
• Preference ghosting. Keeping a brief residual of a discarded template’s preferences in case the same window recurs soon after eviction. Memory cost; recall benefit uncertain.