Decision 0019: Discovered slot symmetry

Date: 2026-05-25 Status: Accepted Builds on: ADR 0017 (configured symmetric slot binding) Motivated by: Approach C from entry 006’s design discussion — the closer-to-honest “from observation alone” version of the commutativity win

Context

ADR 0017 added slot_binding_symmetric=True as an architect-configured flag. Setting it sorts the bound-value tuple before it’s used as the per-cell dictionary key, collapsing operand permutations onto a single learned entry. This lifted held-out generalization on Task 2 from 3.3% to 73.3% — the project’s first headline-number win.

Entry 006’s accompanying discussion was explicit about what the flag is not: it’s not an architectural property the substrate discovered; it’s a domain assumption the architect supplied. Entry 011 names that gap directly: “the architecture isn’t discovering commutativity, it isn’t. I told it.” Approach C in the design queue was to close that gap by detecting symmetry from observation statistics rather than from configuration.

This ADR is the implementation of approach C.

Decision

Discovery state at the substrate level

When slot_symmetry_discovery=True, the substrate maintains a dictionary _slot_swap_seen: dict[(sorted_binding, answer), set[raw_binding]] that tracks, for each (sorted-operand-multiset, answer) pair the substrate has observed, the set of distinct raw orderings seen producing that answer.

Two counters are derived from this dictionary:

• opportunities: the number of distinct (sorted_binding, answer) keys observed at least once.
• evidence: the number of keys observed with at least two distinct raw orderings (i.e., the substrate has seen both (a, b) → c and (b, a) → c).

When opportunities >= slot_symmetry_min_evidence (default 4) and evidence / opportunities >= slot_symmetry_ratio_threshold (default 0.6), the substrate flips _slot_symmetry_discovered = True. This flip is permanent.

Migration of _slot_prefs at the moment of discovery

Before discovery, slot cells learn under raw-permutation keys: (1, 2) → 3 and (2, 1) → 3 accumulate separately. At the moment of discovery the substrate walks every slot cell’s _slot_prefs dictionary and merges entries whose keys are permutations of each other, summing their preference vectors into a single sorted-key entry. Without migration, all pre-discovery learning would be lost, since post-discovery lookups use sorted keys.

Post-discovery behaviour

From the moment of discovery onward, _check_slot_matches builds _slot_bindings[c] from the sorted wildcard values — the same change ADR 0017’s slot_binding_symmetric flag makes, but conditioned on the dynamically-set _slot_symmetry_discovered rather than the static config field.

Soundness condition

The mechanism can only confirm symmetry it has observed. If a particular operand multiset never appears in two orderings during training, no evidence accumulates for it. This is correct behavior — the substrate is not extrapolating from operand multisets it hasn’t seen, just declaring that those it has seen swapped-and-equivalent should be treated as such going forward.

Defaults

• slot_symmetry_discovery: bool = False — opt-in. Existing configurations are unaffected.
• slot_symmetry_min_evidence: int = 4 — minimum opportunities before discovery can fire. Prevents spurious discovery on a tiny sample.
• slot_symmetry_ratio_threshold: float = 0.6 — fraction of (sorted_binding, answer) keys with ≥ 2 raw orderings.
• slot_symmetry_history_size: int = 64 — reserved parameter for a future bounded-history variant; current implementation keeps full statistics in _slot_swap_seen without size capping (acceptable for the project’s vocabulary sizes).

What the benchmark showed

Re-running the nine-task suite with --variant discovered at 5 seeds, alongside the prior configured-symmetric numbers:

The claim that survives

On Tasks 2, 7, and 9 — the binding-related held-out generalization tests — the discovered variant matches the configured-symmetric variant to within sampling noise. The substrate now earns the commutativity assumption from observation statistics rather than receiving it from the architect. ADR 0017’s headline number is preserved; the path to it is now derived rather than declared.

The honest caveat

Task 8 shows the limit. With only 23 multiplication facts (vs. 36 for addition/max), 19 train and 4 held out per seed, the substrate has fewer commute pairs to observe — only ~10 distinct sorted-operand-multiset pairs, of which not all have both orderings present in any given random split. In some seeds discovery fires (and accuracy lifts to configured-symmetric levels); in others the evidence-ratio doesn’t cross the threshold within the training budget, and the substrate stays in pre-discovery (asymmetric) mode. The result is 35.0% ± 28.5 — meaningfully above the 5.0% frozen baseline but well below configured-symmetric’s 65.0%, with high seed variance reflecting the discovery-fires-or-doesn’t dichotomy.

This is a real architectural property of the discovery mechanism, not an implementation bug: with smaller datasets and lower commute-pair density, discovery is sample-size-sensitive. The configured flag doesn’t have this issue precisely because it makes the assumption a priori. A more aggressive threshold or a smaller slot_symmetry_min_evidence would let discovery fire faster on small datasets — at the cost of some risk of false discovery on data that isn’t actually symmetric.

What this means about the substrate

This is the third architectural change in the project’s history to lift the benchmark over the ADR 0014 baseline (after ADRs 0015 — which then disconfirmed — and 0017 — which worked). It is the first such change that closes the “architect-configured vs observation-derived” gap entries 006 and 011 named as the project’s deepest framing concern.

The substrate is doing real work here:

1. It tracks joint statistics of (sorted_binding, answer) and identifies which sorted multisets have been seen with multiple orderings.
2. It applies a soundness criterion (ratio over minimum sample) that prevents spurious commitment to symmetry from a tiny sample.
3. At the moment the criterion is met, it migrates its accumulated learning to the new representation — preserving everything trained pre-discovery rather than discarding it.

What it doesn’t do — and what ADR 0018’s empirical disconfirmation already established — is learn the underlying function. Multiplication’s higher functional complexity exposes this: discovery enables symmetric retrieval, but on Task 9 (strict held-out multiplication) it produces the same 50% as configured-symmetric, which entry 009 already showed is frequency-bias-meets-skewed-answer-distribution rather than learned multiplication. The discovery mechanism inherits this property because it’s discovering symmetry of binding, not structure of the operation.

Behavioral predictions

Five tests in tests/test_slot_cells.py:

1. test_discovery_off_by_default — opt-in flag.
2. test_discovery_does_not_fire_without_swapped_evidence — when only one ordering of each pair appears in training, evidence never accumulates and the flag stays False.
3. test_discovery_fires_after_swapped_evidence_accumulates — with both orderings of multiple pairs trained, discovery flips True over a small number of cycles.
4. test_discovery_migrates_split_slot_prefs_to_sorted_keys — after discovery fires, every per-cell slot_prefs dict has only sorted keys.
5. test_discovered_symmetry_lifts_held_out_generalization — end-to-end: training on pairs with both orderings present plus only one ordering of (1, 4) lets the substrate predict the answer for (4, 1) after discovery fires.

All five pass; the full 138 tests in the suite pass.

Open questions deferred

• Adaptive thresholds. Could the substrate adjust slot_symmetry_min_evidence based on the rate of new (sorted_binding, answer) keys appearing? This might let discovery fire faster on small datasets while still being robust on large ones. Real research; not the right next ADR for Reverie’s current scope.
• Per-position discovery for multi-wildcard templates. Currently the discovery is binary (template is symmetric or not). For templates with more than two wildcards, per-pair discovery would be needed — which positions are interchangeable with which. Out of scope here; Reverie’s current template has only two wildcards.
• False-discovery resistance. On data that genuinely isn’t symmetric (e.g., subtraction with a stable answer convention), the substrate could in principle wrongly declare symmetry from coincidental swap-match cases. The ratio threshold helps but isn’t a guarantee. The benchmark didn’t test this corner; the soundness argument is “the threshold is calibrated for the project’s vocabularies, and misconfigurations remain the architect’s responsibility.”