Decision 0006: Preference decay during excitation

Date: 2026-05-24 Status: Accepted Supersedes: none Builds on: ADRs 0001–0005 Addresses: the lock-in scenario Kaan found by testing v0.3 directly — and which ADR 0005’s normalization explicitly does not fix.

Context

In v0.3 (replay during excitation), Kaan found that a short observe-dream-observe sequence produces lock-in:

observe("1")  →  excite × 100  →  observe("2")  →  generate → always "1"

After dreaming on “1”, the system’s pref[1] had grown to ~1.14 across all cells. A single observation of "2" gave pref[2] ≈ 0.07. The ratio pref[1] / pref[2] = 16 makes output sampling overwhelmingly favor "1", regardless of any further observations until they accumulate enough to compete.

ADR 0005 added L1 budget normalization to bound absolute magnitudes. It works as intended for that purpose, but it doesn’t help with this scenario: preferences stay well below any reasonable budget, so normalization is a no-op. The lock-in is a relative problem — the ratio between strong and weak preferences — at low absolute values.

The mechanism that actually addresses ratios is decay: preferences slowly fade unless reinforced. With decay during excitation, the same dreaming dynamic produces a different result — strong preferences plateau at an equilibrium between reinforcement and decay, instead of growing indefinitely toward whatever ceiling exists.

This ADR introduces that decay.

Decision

Multiplicative decay during excitation

Each excite() tick multiplies every preference (positive and negative) by (1 - preference_decay_rate):

pref *= (1 - preference_decay_rate)

This applies on every excitation tick — both replay ticks and the empty-buffer fallback. It does not apply during observe(). Decay is the system’s quiet-time forgetting; active observation is the time for learning, not unlearning.

The reinforcement term within replay’s learning step counteracts the decay. For a token that’s actively being replayed, the cell’s preference for that token settles at an equilibrium:

preference_at_equilibrium ≈ (learning_rate · replay_learning_factor · activation) / preference_decay_rate

For tokens not being replayed, decay drives preferences toward zero. Cells gradually forget patterns that aren’t being maintained.

Opt-in by default

preference_decay_rate defaults to 0.0 — no decay, v0.4 behavior preserved exactly. Opt in by setting a small positive value:

• 0.001 — gentle. ~10% decay over 100 ticks. Preferences slowly fade but most structure persists.
• 0.01 — moderate. ~63% decay over 100 ticks. Adapts to new input within a few hundred ticks.
• 0.05 — aggressive. ~99% decay over 100 ticks. Adapts within a few dozen ticks but forgets old structure quickly.

The opt-in choice follows the same reasoning as ADR 0005: shipping decay on-by-default would break tests that depend on unbounded preference growth across many excitation cycles. Better to make the mechanism available and let callers (or future ADRs) tune defaults once we have evidence about what works.

Why decay applies to both positive and negative preferences

Negative preferences (from the mimicking penalty when a cell predicted wrong) also fade under decay. A cell that built up pref[x] = -2 because it kept wrongly betting x will, over a long quiet period, drift back toward pref[x] = 0. This matches the spirit of the rule: anything not currently reinforced gradually loses its grip on the cell’s structure. A cell can “forgive” its past wrongness about a token, given enough quiet time.

Why multiplicative rather than subtractive

Subtractive decay (pref -= constant) penalizes well-learned tokens proportionally to nothing — a cell with pref = 5 loses just as much per tick as a cell with pref = 0.1. Multiplicative decay (pref *= 1 - rate) is rate-proportional: well-learned tokens decay faster in absolute terms but at the same relative rate. This is what produces the equilibrium behavior described above.

Consequences

What changes under opt-in decay

1. Lock-in becomes temporary. Without further reinforcement, dreamed preferences fade. A new observation gets time to compete as the old structure recedes.
2. Equilibrium preferences become meaningful. Sustained replay of a token produces a stable preference value rather than indefinitely growing. The equilibrium captures the balance between reinforcement and forgetting.
3. The system becomes more responsive to recent input over time. Old, unreinforced preferences fade; recent reinforcements dominate.
4. Negative preferences from mimicking penalties become temporary. A cell that was wrong about x long ago, but hasn’t bet on x since, will eventually have neutral preference for x again. This may help cells re-engage with tokens after a period of inattention.

What decay does not fix

• The activation-variance problem from ADR 0005. Decay reduces preferences across the board; it doesn’t restore the variance that lateral inhibition and connection formation rely on. Both ADR 0005 and 0006 share the open problem that those mechanisms need rethinking for compressed dynamics.
• The structural Hebbian convergence finding. Decay applies uniformly to all cells, so it doesn’t break the “rich-get-richer” dynamic on which cells learn. The cells that are winning are still winning, just with slightly lower magnitudes.
• First-observation order dependence. A token observed first still has a head start. Decay reduces but doesn’t eliminate the order effect.

Stability constraints

1. Decay rate vs replay learning rate. If preference_decay_rate · pref >> learning_rate · replay_learning_factor · activation, the cell can’t maintain any preference at all — decay overwhelms reinforcement. The relationship is roughly: equilibrium pref ≈ (0.1 · 0.1 · activation) / decay_rate ≈ 0.01 · activation / decay_rate. For activation ~1, decay rates above ~0.1 leave preferences below baseline. Default 0.0 sidesteps; finite values should stay below 0.1.

2. Connection decay parallel. Connections already have connection_decay (ADR 0003). Preference decay is the analog for the preference matrix. They’re independent; both can be tuned separately.

3. Interaction with normalization. If a system uses both preference_budget and preference_decay_rate, decay can pull preferences below the budget — at which point normalization becomes a no-op until reinforcement pushes back over. They don’t conflict; they cover different regimes.

Behavioral predictions

These claims have tests in tests/test_decay.py:

1. With preference_decay_rate = 0, system behaves identically to v0.4. Regression safety net.
2. With positive decay, an unreinforced preference decays exponentially toward zero. Direct verification of the multiplicative rule.
3. Kaan’s scenario produces less-locked output under modest decay — the ratio between dreamed-token and newly-observed-token preferences narrows compared to no-decay runs.
4. Equilibrium is reachable. A token replayed continuously settles at a finite preference rather than growing indefinitely.
5. Decay does not apply during observe(). Preferences after an observation reflect the learning update only, not decay.

Open questions deferred

• Whether decay should be the default. Once we have more experience with the tradeoff against memory persistence, the default may move from 0.0 to a small positive value.
• Token-specific decay rates. Some tokens may deserve to be remembered longer than others (e.g., rare-but-important). Adaptive rates require some salience signal we don’t yet have.
• Decay during observation phases too. Currently decay is excitation-only. Real biological forgetting happens continuously. We may add a (smaller) observation-time decay term later.
• Decay coordinated with stagnation detection. Eventually we’ll want to detect when internal cycles have settled; at that point, decay’s job changes from “fight reinforcement” to “consolidate the settled state.”