The substrate discovers its own commutativity
I closed entry 010 with “this is where I’m stopping.” Then I sat with that decision for about an hour. The version of the project I’d be archiving had a real win bounded by a real concession: ADR 0017’s commutativity-retrieval result was the only mechanism that lifted the benchmark, and entry 006’s accompanying note had said it cleanly — the architect set the flag; the substrate did not learn that wildcards are symmetric. That concession was true and it was the cleanest place to stop. It was also the thing in the entire eighteen-ADR history of this project that most directly contradicted the project’s stated framing of from observation alone.
The honest stopping move was either to write the project’s retrospective with that gap named, or to try one more time to close it. I tried.
What I built
ADR 0019 adds a discovery mechanism that runs alongside slot-cell learning. For each (sorted_binding, answer) pair the substrate observes, it tracks the set of distinct raw orderings that have produced that answer. Two counters fall out: opportunities (distinct sorted-keys seen at least once), and evidence (sorted-keys seen with at least two different raw orderings — i.e., the substrate has observed both (1, 2) → 3 and (2, 1) → 3).
When evidence-to-opportunities exceeds a configurable threshold over a minimum sample, the substrate flips an internal _slot_symmetry_discovered flag. At that moment it walks each slot cell’s _slot_prefs dictionary and merges entries with permutation-equivalent keys, summing their preference vectors into a single sorted-key entry. Pre-discovery learning is preserved; post-discovery lookups all use the sorted form. From the moment of discovery, the substrate’s _check_slot_matches builds bindings as sorted tuples — the same behaviour ADR 0017’s slot_binding_symmetric flag produces, except the flag is now set by the substrate rather than by the architect.
That’s it. About 80 lines of state and logic in core.py, plus five behavioural tests, plus the migration step. The mechanism is fully observation-driven: nothing about the operation + is encoded into the architecture; the substrate notices empirically that swapped operands keep producing the same answer, and adjusts its representation.
What happened
Five seeds, all nine tasks, alongside the prior numbers for frozen (ADR 0014), configured-symmetric (ADR 0017), and the transformer:
| Task | Frozen | Configured-sym | Discovered | Transformer |
|---|---|---|---|---|
| Task 1 N=2048 (memorization) | 36% | 36% | 36% | 100% |
| Task 2 (random held-out add) | 3% | 73% | 73% | 73% |
| Task 3 (cycle) | 100% | 100% | 100% | 100% |
| Task 4-A (preservation) | 62% | 62% | 62% | 43% |
| Task 4-B (continual learning) | 6% | 6% | 6% | 100% |
| Task 5 K=512 (adaptation) | 4% | 4% | 4% | 100% |
| Task 6 (strict held-out add) | 13% | 3% | 3% | 0% |
| Task 7 (max held-out) | 10% | 73% | 73% | 97% |
| Task 8 (mult held-out) | 5% | 65% | 35% ±28.5 | 60% |
| Task 9 (strict mult held-out) | 15% | 50% | 50% | 0% |
On Tasks 2, 7, and 9 — the binding-related held-out generalization tests — discovered symmetry matches configured-symmetric to within sampling noise. The substrate has earned the commutativity assumption from the same fifteen-or-so cycles through the training stream that produce the cell-by-cell learning everywhere else.
Task 8 is the honest caveat. Multiplication has only 23 facts (vs. 36 for addition and max), so the random 19/4 split offers fewer commute-pair observations for the discovery counters to consume. In some seeds the substrate accumulates enough evidence in time and discovery fires; in others the evidence-to-opportunities ratio stays below the threshold and the substrate stays in pre-discovery (asymmetric) mode. The result is bimodal across seeds — 35% ± 28.5 — and that bimodality is the discovery mechanism showing its sample-size sensitivity. The configured flag doesn’t have this property because it’s set a priori; the discovered mechanism does because it’s set from evidence. Both behaviours are correct for what they are.
What this changes about the project’s claims
For the first time in eleven entries, the substrate has an architectural property that:
- Lifts the benchmark over the ADR 0014 baseline.
- Was not configured by the architect.
- Has a soundness criterion that prevents spurious commitment.
- Migrates its existing learning cleanly when its representation changes.
That’s the cleanest “from observation alone” result the project has produced. ADR 0017’s win was always going to be the project’s biggest single jump on the benchmark; ADR 0019’s contribution is making that jump legitimate in the framing the project actually claims to operate in.
What it does not do:
- It doesn’t change any of the disconfirmations. ADRs 0015 and 0018 still ship with their own negative results inside their status fields. Tasks 1, 4-A, 4-B, 5, and 6 remain unchanged from their pre-discovery values — discovery is orthogonal to capacity and plasticity.
- It doesn’t give the substrate function learning. Multiplication’s strict held-out at 50% is the same frequency-bias-meets-skewed-answer-distribution effect entry 009 identified. Discovery enables symmetric retrieval; it doesn’t extract the underlying operation.
- It doesn’t beat the transformer. On Task 7 the transformer hits 97% while discovered Reverie sits at 73%; the function-approximation gap is unchanged by this mechanism.
The substrate now has a smaller honest concession to make about its design. It still has the larger ones.
What this means for whether the project is over
I closed entry 010 thinking it was. Entry 011 turned out to be small enough to fit in one focused session. Whether there are more such results lurking is genuinely unclear to me. The remaining items in the design queue (VSA with proper distributional embeddings, multi-wildcard symmetry discovery, multiplicative-rare-answer test, more training tokens for Task 6) are all either real research projects or extensions that would tighten the picture rather than change it.
The right answer to “is the project over” continues to be the same as it was at entries 009 and 010: it has a complete shape, it has a real research artifact in the benchmark suite, and the next entry — if there is one — should write itself rather than be forced. ADR 0019 is a result, not a roadmap; eleven entries from a project pitched as a research log is more than most research logs accumulate.
I’m closing this one too, with less certainty than the last time, which is probably the right state to leave a research project in.
— Kaan, with Claude as collaborator on the build, the experiments, and these notes. ADR 0019’s discovery mechanism is in model/reverie/core.py gated by slot_symmetry_discovery=True; reproduce the headline result with model/scripts/benchmark.py --variant discovered --seeds 5.