Decision 0013: Approximate context matching

Date: 2026-05-24 Status: Accepted Builds on: ADRs 0001–0012

Context

ADR 0012 introduced context cells with K-token templates and exact match semantics. A cell activates only when its template equals the recent observed-tokens window slot-for-slot. That semantic produced the arithmetic memorization win (8/8 trained queries answered correctly), but it leaves nothing on the table for any context that differs — even by a single position — from a trained one.

This ADR softens that. Cells activate at strength proportional to the fraction of slots that match, rather than 0/1 on exact-match. A trained [1, +, 1, =] cell will activate fully on [1, +, 1, =] and at 75% on [1, +, 2, =] — three of four slots match.

The motivation is not to solve arithmetic generalization (that needs variable binding, which this substrate doesn’t have). The motivation is to surface what happens when the system is queried with contexts that almost match something it knows. Three behaviors are plausible:

1. Useful smoothing: similar contexts produce similar outputs, even when the exact template isn’t present.
2. Catastrophic interference: similar contexts pull each other’s predictions apart, eroding the memorization wins.
3. Honest no-op: when no template is close enough, the system reverts to its non-context-cell base behavior.

We’ll find out by building it and running the experiment.

Decision

Match-fraction-based activation

Replace the exact-match check in _apply_context_match_boost with a count of slot-by-slot matches:

match_count = sum(template[k] == window[k] for k in range(K))
match_fraction = match_count / K   # in [0, 1]

A cell receives an activation boost proportional to match_fraction:

boost = match_fraction * CONTEXT_MATCH_ACTIVATION_BOOST
self._activations[c] += boost

For match_fraction == 1.0, the boost is identical to ADR 0012’s (3.0 by default). For 0.5, half as much. For 0, no boost.

Suppression of fully-non-matching cells

ADR 0012 suppressed all non-matching cells to 0.001× their normal activation. With graded matching that’s no longer the right cut. The new rule:

• Cells with match_fraction == 0 (no slots match at all): suppress to 0.001× as before.
• Cells with match_fraction > 0 (any partial match): keep their normal activation plus the graded boost.

In effect, the suppression now distinguishes “completely irrelevant” cells (suppressed) from “somewhat relevant” cells (allowed to contribute).

When no cell has any match

If match_fraction == 0 for every context cell (no template overlaps the current window at all), the suppression doesn’t fire — every cell keeps its normal activation. The system reverts to its pre-context-cell behavior. This is the architectural “honest no-op”: when nothing’s familiar, don’t pretend.

This is a change from ADR 0012, where if NO cell matched, suppression also didn’t fire. The new rule keeps that property but makes the threshold “any partial match anywhere” rather than “exact match somewhere.”

Other cell types and non-context cells

Non-context cells (mimicking, hebbian, predictive) are unaffected by this ADR. They retain their normal activations regardless of context-cell matches. The suppression in ADR 0012 explicitly suppressed all non-matched cells; under ADR 0013 the suppression only fires when some context cell has any partial match.

Stability considerations

Trained queries should still work

For an exactly-trained query (e.g., [1, +, 1, =] with that exact template claimed), the matched cell still gets the full boost. Other context cells with partial matches also activate, but their boost is smaller. The exact-matched cell’s contribution to its predicted answer should still dominate.

If empirically that’s not the case (partial matches drown out the exact match), the suppression of non-matched cells needs to be reintroduced more aggressively, or partial activation needs to be made weaker.

Untrained queries — the new behavior

For a query with no exact-match template, multiple cells partially activate. Their predictions compete in the output. There are three regimes:

• Many close matches with diverse predictions: output is mixed, reflecting an ensemble.
• One close match dominant: output is biased toward that cell’s prediction.
• No close matches: output reverts to non-context behavior (the case where the suppression doesn’t fire).

The test for “useful smoothing” is whether untrained queries with structural similarity to trained ones produce sensible outputs. This is empirical, and we’ll see.

Backward compatibility

The exact-match case (ADR 0012’s behavior) is the special case of match_fraction = 1.0. So any test that relies on exact-match behavior should still work, modulo small numeric differences from the changed suppression rule.

Empirical findings (after running)

Three honest results from probing the implementation:

1. Memorization survived — but barely, and only with high sharpness.

The initial linear (sharpness=1) and cubic (sharpness=3) drafts BROKE the arithmetic memorization wins from ADR 0012. With many context cells partially matching (e.g., for query [1, +, 1, =], the cells trained on [1, +, 2, =], [2, +, 1, =], etc. all partial-match at 3/4), their collective vote for different answers drowned out the exact-match cell’s vote for the correct answer.

We tried multiple sharpness values:

• Linear (1.0): arithmetic broke down to 0–1 correct out of 8.
• Cubic (3.0): improved but partial votes still drowned exact.
• Sharpness 5: ~3/8 with 200 samples; 6/8 with stronger seed.
• Sharpness 12 (default): arithmetic memorization restored at 200 samples per query.

Higher sharpness makes partial matches contribute progressively less. The default 12 means a 75% match produces just ~3% of the boost a 100% match does. Effectively, the mechanism behaves like exact-match-with-soft-edges rather than truly graded matching.

2. Untrained queries showed no meaningful generalization.

The hopeful prediction — that “1+4=” (untrained) might produce something near “5” because similar trained contexts contribute — did not happen. Untrained queries produce essentially uniform output across the digit vocabulary, just like the pre-ADR-0012 base behavior. The mechanism doesn’t bridge from memorized templates to unseen templates in any useful way.

This validates the architectural intuition from ADR 0012 onward: true generalization requires variable binding, not just partial pattern matching. ADR 0013’s partial matching gives “the same answer, slightly less confidently” rather than “a different, contextually-derived answer.”

3. Four existing tests needed context_cell_fraction=0.

Tests that pinned pre-ADR-0012 behaviors broke when partial matching expanded the contexts where suppression of zero-match cells fires. Disabling context cells in those tests preserves their original architectural claims. This is becoming a pattern: each ADR that adds a mechanism flags pre-existing tests as “of an earlier era,” and they get opted out.

Behavioral predictions (validated)

These have tests in tests/test_approximate_matching.py:

1. Trained queries still produce correct memorized outputs at the default sharpness — verified at 200 samples per query.
2. Partial-match cells contribute to activation proportional to match fraction with the sharpness exponent applied — direct test of the formula.
3. No partial match means no boost and no suppression — backward compatibility for unfamiliar contexts.
4. Sharpness controls the partial-vs-exact balance — higher sharpness gives partial matches smaller boosts.
5. All prior 94 tests continue to pass under the new default behavior (with the four opt-outs noted above).

Open questions deferred

• Position-weighted matching. Currently every slot contributes equally to the match. But some positions might matter more (e.g., the last position before the prediction). A future ADR could add per-position weights, possibly learned.
• Template adaptation. Currently templates are claimed once and fixed forever. A cell whose template is near-match for many sequences might “absorb” the common pattern over time, generalizing toward a more abstract template. Genuine variable binding may be the cleanest version of this.
• Approximate matching during observation, not just generation. Currently the boost fires in both observe() and generate(). Whether partial matching during observation reshapes learning in unintended ways is something to watch.