Decision 0005: Per-cell preference normalization (L1 budget)

Date: 2026-05-24 Status: Accepted Supersedes: none Builds on: ADRs 0001–0004

Context

While testing the v0.3 system interactively, Kaan ran a simple sequence:

observe("1") → excite 100 times (dream) → observe("2")

…and discovered that the system stayed locked on "1" regardless of how "2" was introduced afterward. The earlier ADR 0004 fix already addressed one piece of this (mimicking cells couldn’t even learn novel tokens), but a deeper property remained: dreaming reinforces what was already in the buffer, and what’s already in the buffer reinforces itself by activating cells that then learn more, in a positive-feedback loop. The system’s “voice” gets locked in by whatever it sees first.

The cause is structural. A cell currently has no constraint on its total preference — it can have pref[1] = 10, pref[2] = 10, pref[3] = 10 simultaneously. Nothing in the architecture forces tokens to compete within a cell. So:

• Reinforcement is purely additive: every observation grows preferences without trading anything off.
• The cell with the highest preference for the observed token activates most strongly, learns the most, and pulls further ahead — the rich-get-richer dynamic.
• Replay during dreaming amplifies this because cells with high preferences keep winning the lateral-inhibition field on every replay tick.
• A single new observation cannot displace a deeply-dreamed prior preference, because the budget is unbounded; the prior preference doesn’t have to give anything up.

This ADR introduces a per-cell preference budget — a fixed cap on each cell’s total positive preference — and a normalization step that enforces it. The effect: when a cell strengthens one token, others get implicitly weakened. The cell has finite “attention” to allocate across tokens.

Decision

Per-cell L1 budget — opt-in

Each cell has a preference_budget parameter. Default float("inf") — normalization disabled, v0.3 dynamics preserved. When set to a finite value, the budget enforces an L1 cap after every preference-learning step: cells whose positive preferences sum above the budget have their entire preference vector (positive and negative entries) scaled down by the ratio budget / total.

Mathematically, for each cell c:

total[c] = sum(max(0, pref[c, :]))
if total[c] > preference_budget:
    pref[c, :] *= preference_budget / total[c]

Note that negative preferences scale with the same factor, preserving the cell’s full preference structure (relative ordering of which tokens it likes and dislikes). Only the magnitudes shrink.

The budget is on positive preferences only. Negative preferences (from the mimicking penalty) represent the cell’s belief that a token is unlikely; they aren’t “uses of attention” in the same sense. They’re scaled along with the positives to keep the geometry consistent, but they don’t count against the budget.

What the budget means

With default activation_saturation = 3.0, the budget of 6.0 allows:

• One token at the saturation point (pref = 3) plus another at saturation. Two strong specialties.
• Or one token at pref = 6 (well above saturation; further preference adds no activation gain).
• Or three tokens at pref = 2 each. Balanced moderate specialization.
• Or any other distribution summing to ≤ 6.

The cell’s “voice” is shaped by how it allocates 6 units of preference across its observed vocabulary.

Why L1 (sum) rather than L2 (norm)

Several normalization schemes were considered:

• L2 / Oja-style norm. pref ← pref - α·y·(y·pref) where y is cell activation. This is the textbook choice for competitive learning. Rejected for v0.4 because the term y·(y·pref) is sensitive to activation magnitudes, which are already complex under the saturation + inhibition + propagation interactions. L1 is simpler to reason about and produces cleaner test behavior. We can revisit if dynamics demand it.

• Subtractive normalization (after each update, subtract a small constant from every preference). Rejected because it punishes well-learned tokens proportionally to nothing — a cell with one strong preference loses just as much as a cell with one weak preference. The constraint should depend on the cell’s state.

• Multiplicative decay (all preferences decay slowly toward zero on every tick). Rejected as the sole normalization mechanism because it doesn’t have a notion of capacity. A cell that has been observing for a long time will have larger preferences than one that just started, with no principled stop. (We may still add a small decay term later as a secondary mechanism — they’re not mutually exclusive.)

• Periodic renormalization (every N observations, rescale every cell). Rejected because behavior would have a discontinuity at the renorm tick, and would depend on N. The current per-step approach is smooth.

L1 budget is the simplest scheme that captures the architecturally meaningful constraint (“each cell has finite attention to allocate”) without introducing other dynamics.

Where the normalization applies

Normalization runs after every preference update, in both code paths:

1. After observe()’s learning step (full-rate observation).
2. After excite()’s replay learning step (replay-rate consolidation).

This means even dreaming respects the budget. A cell can’t accumulate unbounded preference for the dominant replayed token; it has to take from somewhere else in its own portfolio.

Why not normalize during the very first observation

When bets is None (first observation ever) or a token is novel, we do blanket reinforcement. Normalization still applies after this — but with only one token observed and small preferences, the budget won’t bite. The normalization is a no-op until preferences grow.

Why opt-in rather than on-by-default — an honest tension

The first implementation of this ADR defaulted to budget = 6.0, and three v0.3 tests broke. Two of them broke in the right direction (Hebbian cells started specializing under balanced input — which is what we wanted normalization to enable). One broke in a way that surfaced a deeper tension worth recording:

Lateral inhibition and connection formation, as currently implemented, rely on activation variance across cells. Normalization compresses that variance.

The variance came from tanh(pref / sat) having a finite linear-ish range. Under v0.3, cells with pref >> sat saturated their activation contribution at ~sat, while cells with low pref had activations near baseline. That spread of activations was what (a) lateral inhibition pruned to produce sparse responses, and (b) the centered-Hebbian connection update used to bind co-activating cells into assemblies.

Under L1 budget normalization, no cell can grow pref much above the budget. The runaway specialization that fed the variance is gone. Activations across cells become more similar, the centered activations become smaller, and connection formation slows — sometimes to near zero in heavy-repetition scenarios. Lateral inhibition still operates but silences fewer cells.

This is a real architectural tradeoff:

• Without normalization (v0.3): rich-get-richer drives specialization and assembly formation, but new tokens can’t easily compete with deeply-dreamed prior ones (Kaan’s finding).
• With normalization (v0.4 opt-in): preferences allocate fairly across tokens, but the connection-formation mechanism that depended on variance becomes weaker.

The opt-in default lets users choose which set of dynamics they want for a given experiment, and signals honestly that this ADR doesn’t yet fully solve the architectural problem it set out to address. A future ADR will need to make connection formation more robust to compressed variance (e.g., by using a different binding signal, or by combining normalization with a complementary forgetting mechanism such as decay).

Disabling normalization

The default — preference_budget = float("inf") — disables normalization in practice; the budget is never exceeded. To enable, pass a finite value (e.g., preference_budget=6.0).

Consequences

What we expect to change

1. Sustained dreaming on a single token no longer locks the system. With the budget capped, the dominant token saturates the budget but can’t grow past it. Subsequent observations of a new token claim a share of the budget proportional to how much they’re observed and replayed.

2. Hebbian cells stop all converging on the same winner. Under v0.3, all Hebbian cells learned identically and ended up sharing top preferences. Under L1 normalization, when a cell is forced to allocate its 6 units, small per-cell differences (from operational noise during early learning) get amplified into different allocation patterns. Different Hebbian cells should now end up specializing on different tokens — proper distributed assemblies.

3. Mimicking cells’ specialization sharpens further. With Rule B already producing per-cell divergence, normalization reinforces the effect: a mimicking cell that wins one token’s budget can’t simultaneously allocate to another.

4. Adaptation timescale is governed by the budget, not the unbounded growth rate. It now takes a defined number of observations of a new token to displace a dreamed preference — roughly budget · existing_pref_ratio / learning_rate observations to meaningfully shift the budget allocation. This is predictable in a way that the v0.3 dynamics were not.

5. Output distribution during dreaming is less skewed. With the budget allocated, less of it goes to whichever token was dreamed most; some always goes to other tokens. The system’s “voice” reflects its full observation history, not just the dominant fragment.

What broke when we tried on-by-default (worth recording)

When the first draft set preference_budget = 6.0 by default, three tests failed:

• test_cell_types_respond_differently_to_balanced_input — Hebbian cells now show concentration 0.107 instead of <0.1. This is correct architecturally: normalization makes Hebbian cells diversify their allocations. The v0.3 claim was wrong; the new behavior is what we wanted.
• test_lateral_inhibition_pushes_below_mean_cells_to_zero — 0 cells silenced (was: many). Inhibition still acts but doesn’t drive activations all the way to zero, because normalized preferences keep the activation field tighter.
• test_consistent_coactivators_form_positive_connection — only 41% positive (was: >70%). Connection formation depends on activation variance, which normalization compresses.

These outcomes prompted the opt-in design above. Future work will need to revisit lateral inhibition and connection formation to make them more robust to compressed variance.

Stability constraints

1. The budget interacts with activation_saturation. If preference_budget >> activation_saturation, the budget cap rarely bites (saturation already caps activation effect well below where total preferences hit the budget). If preference_budget << activation_saturation, cells can’t allocate enough to any one token to reach the activation saturation point — preferences stay below their useful range. Default budget = 2 · activation_saturation strikes a reasonable balance: cells can fully saturate two tokens or balance among more.

2. Mimicking penalty + normalization could shrink budget effectively. Negative preferences scale down with positive ones during normalization, so the geometry stays intact. But if a cell accumulates many negative preferences from wrong bets, its effective positive budget is the budget minus the dilution from carrying those negatives along the normalization. Tests should monitor whether this creates pathological cases.

3. First-observation blanket reinforcement plus normalization in the same tick. When the first observation ever arrives, every cell gets pref[idx] += learning_rate · activation (blanket). For 32 cells with activation ~1.0, that’s pref[idx] ≈ 0.1 per cell. Sum per cell = 0.1, well under budget. No-op normalization. As expected.

Behavioral predictions (validated by tests, with honest caveats)

These claims have passing tests in tests/test_normalization.py:

1. With preference_budget = inf (default), system behaves identically to v0.3. Backward compatibility, verified by all 39 prior tests passing unchanged and by direct preference-matrix equality.
2. With a finite budget, the per-cell positive preference sum stays at or below the budget after any sequence of observations and dreaming.
3. The normalization step preserves per-cell ordering. Scaling the preference vector cannot reorder its entries — the cell’s top-preferred token before and after one normalization step is identical.
4. Heavy reinforcement displaces existing preferences. Observing the same token many times under a finite budget pulls budget away from previously-strong tokens — preferences are now competitive, not purely additive.
5. Finite budget bounds preference magnitudes. Infinite-budget runs produce unbounded preferences; finite-budget runs cap them at the configured value.
6. Negatives scale proportionally with positives. The like/dislike geometry is preserved across the scaling operation.

Honest caveats — what this ADR does not deliver

• Kaan’s specific scenario (1 obs → dream → 1 obs) is not fixed. Preferences stay well below the budget at those input volumes, so normalization is a no-op. The lock-in there is a relative-magnitude problem (pref[1]/pref[2] ≈ 16), not an absolute-budget problem. Solving it requires the decay mechanism noted in “Open questions deferred” — a complementary, not competing, mechanism.

• “Cells diversify across tokens” was over-promised in an earlier draft. Normalization bounds magnitudes; it does not by itself break the rich-get-richer dynamic on which cells learn. Under balanced input + finite budget, Hebbian cells tend to spread their budget evenly across tokens, which makes them more uniform per cell, not less. The “all Hebbian cells collectively converge on one winner” v0.3 finding is therefore not directly fixed by this ADR. A future ADR addressing connection formation or per-cell biasing may revisit this.

• The architectural tension with lateral inhibition and connection formation is real and unresolved. Activation variance drives both of those mechanisms today; normalization compresses that variance. The opt-in default is an admission that we can’t make this ADR a strict improvement without further work on those subsystems.

Open questions deferred

• Whether budget should be adaptive — e.g., growing with vocabulary size, or per-token rather than per-cell. For v0.4, fixed budget is the simplest version; we can revisit.
• L2 / Oja-style normalization — may be revisited if L1 budget produces dynamics that feel too “hard-edged” in practice.
• Connection-matrix analog — connections already have a connection_max cap, but not a per-cell budget. The same logic could apply: a cell has finite connection capacity. Deferred until we see whether it matters.
• Decay term as a secondary mechanism — independent slow forgetting. Could be added alongside normalization in a future ADR if practice shows preferences become too rigid even with the budget.