Decision 0003: Connections, lateral inhibition, and the asking protocol

Date: 2026-05-23 Status: Accepted Supersedes: none Builds on: ADRs 0001, 0002 Modifies: ADR 0002 (re-articulates the cell-type distinction; see “Consequences” below)

Context

After ADR 0002, the system has two cell populations with distinct learning rules but no way for cells to talk to each other. Every cell processes input in isolation. There are no assemblies — no distributed representations spanning multiple cells. Hebbian cells in particular are inert in any sense beyond “Hebbian-rule reinforcement,” because Hebbian learning’s interesting properties (assembly formation, pattern completion) all require connections.

This ADR introduces the substrate for that. It implements the “asking” protocol described in the original architecture sketch — cells ask for strengthening on inputs they want, broadcast inhibition to peers asking for the same input, and form connections with cells that ask for the same things.

The protocol bundles four mechanisms that don’t make sense in isolation but together produce the architecture’s first genuinely interesting dynamics:

1. Preference-driven activation — cells respond to inputs based on how strongly they prefer them, not uniformly.
2. Lateral inhibition — strongly-activating cells suppress peers.
3. Connection formation — cells co-activating on the same input form bidirectional connections.
4. Connection-mediated propagation — during excitation, activation spreads through connections.

Decision

Mechanism 1: Preference-driven activation

When an observation arrives, each cell’s activation depends on its current preference for the observed token, passed through a soft saturation:

raw_activation[c] = baseline + activation_saturation · tanh(preference[c, t] / activation_saturation) + noise[c]

Where baseline = 1.0, activation_saturation = 3.0 by default, and noise is per-cell operational noise at scale noise_scale.

The “ask strength” of a cell for a given input is its raw activation. Cells with strong preference ask loudly; cells with no preference ask quietly (baseline + noise).

On a brand-new token, all preferences are 0, so tanh(0) = 0 and activations reduce to baseline + noise (uniform). This preserves v0/v0.1 behavior for novel observations.

On familiar tokens, cells that previously learned this token activate more strongly than cells that didn’t. This is the substrate for everything else in this ADR.

Why the tanh saturation — a discovered constraint

A previous draft of this mechanism used the obvious linear form raw_activation = baseline + preference + noise. It produces exponential preference growth and overflows numerics within roughly 100 observations. The math:

pref[c, t] ← pref[c, t] + learning_rate · activation[c]
            = pref[c, t] + learning_rate · (baseline + pref[c, t] + ...)
            ≈ pref[c, t] · (1 + learning_rate) + small

With learning_rate = 0.1, that’s a 10% per-observation compounding rate. After 100 observations of a single token, pref ≈ pref_0 · 1.1^100 ≈ pref_0 · 13780. After 400 observations, the value space is far past anything numerically tractable, and downstream np.exp calls in the bet snapshot blow up.

The saturation keeps activation bounded as pref → ∞:

lim(pref→∞)  activation_saturation · tanh(pref / activation_saturation) = activation_saturation

So raw_activation is bounded above by baseline + activation_saturation = 4.0. Learning per tick is bounded by learning_rate · activation_max ≈ 0.4, so preferences grow at most linearly with experience. Strong cells still dominate the field (the response curve is near-linear for |pref| < activation_saturation), but they can’t run away.

This was discovered empirically — the test suite hung because of the runaway. The fix is recorded here so the constraint doesn’t have to be re-discovered.

Mechanism 2: Lateral inhibition

Cells with high raw activation suppress peers. We use a simple subtractive form:

activation[c] = max(0, raw_activation[c] - inhibition_scale · mean(raw_activation))

Below-mean cells get pushed below zero and clipped to zero. Above-mean cells survive but reduced. The strongest cell relative to its peers stays loudest. With inhibition_scale = 0, the rule reduces to no inhibition; with larger values, the effect approaches winner-take-all.

Why subtract-the-mean rather than divisive normalization or strict winner-take-all?

• Divisive normalization (activation /= sum) preserves relative ordering but flattens differences when most cells are excited. Subtractive inhibition is sharper.
• Strict winner-take-all eliminates the distributed-representation property — only one cell ever wins. We want multiple cells to remain active for any given input (the “assembly” idea), just with the field being narrower than without inhibition.

Mechanism 3: Connection formation

Connections are stored in a symmetric (n_cells, n_cells) matrix. The update rule on each observation is centered Hebbian binding:

centered[c] = activation[c] - mean(activation)
ΔC[i, j] = connection_learning_rate · centered[i] · centered[j]   (for i ≠ j)

Two cells that both activate above the mean on the same observation gain a positive connection between them. A cell that activates above mean and another that activates below mean acquire a negative connection (anti-correlated cells push apart). Cells both below mean strengthen too — they’re “co-quiet” — which is a quirk of pure outer-product Hebbian that we live with for v0.2.

To keep the matrix bounded, every observation also applies a tiny multiplicative decay:

C *= (1 - connection_decay)
C += ΔC
C = clip(C, -connection_max, +connection_max)

Self-connections (the diagonal) are forced to zero — a cell doesn’t excite itself.

Mechanism 4: Connection-mediated propagation (during excitation)

The excitation tick gains a term: activation spreads through connections to connected cells.

propagated[c] = Σ_j C[c, j] · activation[j]
activation[c] = activation_decay · activation[c] + propagation_scale · propagated[c] + noise[c]
activation[c] = max(0, activation[c])

This is the simplest possible propagation: linear, single-step, with a global scale parameter and a floor at zero. Cells that are connected to currently-active cells inherit some of their activation; isolated cells just decay. The propagation_scale is small (default 0.1) to keep dynamics stable.

This is not yet replay. There’s no replay of observed tokens; the system isn’t yet re-broadcasting past inputs to itself. What we have is connection-mediated spreading activation. Real replay (re-injecting past observations into the input stream during internal cycles) arrives in a later ADR.

Why these four together, not split across ADRs

Each mechanism is degenerate without the others:

• Preference-driven activation alone would just amplify existing biases; the activation profile would harden but cells wouldn’t communicate.
• Lateral inhibition alone would suppress activations uniformly, since with uniform activation there’s nothing to inhibit toward.
• Connections alone would form between random pairs, since with uniform activation co-activation is uniform.
• Propagation alone needs connections to propagate through.

Bundling them is the smallest coherent step that produces new dynamics. Splitting is possible but each split-out ADR would land something that doesn’t yet earn its keep.

Stability constraints worth recording

Lateral inhibition can starve the population

If inhibition_scale is too high (close to or above 1.0), almost every cell ends up below the mean and gets clipped to zero. The system goes silent. Default 0.3 keeps the field active while still producing meaningful suppression.

Connection growth must be slower than preference learning

Connections accumulate over many observations; preferences accumulate per observation. If connection_learning_rate ≈ learning_rate, connections grow as fast as preferences and the system becomes dominated by a positive-feedback loop where co-activated cells get connected, propagate activation to each other, get more co-activated, etc.

Default connection_learning_rate = 0.01 is ten times smaller than learning_rate = 0.1. This keeps connections as a slow-moving substrate that captures stable co-activation patterns rather than per-observation noise.

Connection decay prevents drift

Without decay, even small spurious co-activations would accumulate over millions of observations. Default connection_decay = 0.001 per observation gives a ~1000-tick half-life — connections persist over thousands of observations but eventually fade if not reinforced.

Propagation must be sub-unity to avoid runaway

If propagation_scale · max(|C|) ≥ 1, propagation amplifies rather than dissipates. Activations can blow up over multiple excitation ticks. Default propagation_scale = 0.1 with connection_max = 1.0 gives a per-tick amplification ceiling of 0.1 — safely sub-unity.

Consequences for the cell-type distinction

ADR 0002 claimed Hebbian cells “remain generalist” while mimicking cells specialize. That claim relied on activations being uniform. Under preference-driven activation, Hebbian cells do specialize:

• A Hebbian cell with pref[T] = 0.5 activates more strongly on T than a Hebbian cell with pref[T] = 0.
• Strong activation means more reinforcement on the next T observation (since the v0 Hebbian rule multiplies by activation).
• This is a positive-feedback loop that produces specialization through pure accumulation.

So both cell types now specialize. The distinction has to be re-articulated:

• Mimicking cells specialize via predictive selection: each cell makes a bet, and only successful predictions get reinforced. Specialization happens through error correction.
• Hebbian cells specialize via competitive amplification: cells with stronger initial preferences activate more, learn more, and outcompete peers in the lateral-inhibition field. Specialization happens through positive feedback.

Both produce specialized populations but the kind of specialization differs:

• Mimicking specialization is prediction-locked: a cell that bets X consistently becomes an X-specialist regardless of frequency, as long as it’s right more often than penalty/(1+penalty).
• Hebbian specialization is frequency-locked: cells specialize on the most-frequent tokens (since those drive activation most often), and the specialization is amplified by inhibition + connection formation.

This is a richer architecture than v0.1’s. It also means several tests from test_cell_types.py need updating — the “Hebbian cells remain generalist” prediction was wrong, and the test was passing partly because there was no mechanism for Hebbian specialization in v0.1.

Behavioral predictions (validated)

These claims have passing tests in tests/test_connections.py and tests/test_cell_types.py:

1. Cells with high preference for the observed token co-activate, and their pairwise connections grow positive. This is the substrate of assembly formation.
2. Activations on familiar tokens differentiate across the population. Standard deviation rises far above what noise alone would produce.
3. Lateral inhibition silences below-mean cells. With inhibition_scale > 0, the bottom of the activation distribution clips to zero.
4. With inhibition disabled, no cell is silenced by competition. Verifies the inhibition mechanism is the cause of silencing, not some side effect.
5. During excitation, stimulating one cell activates its connected peers. Connection-mediated propagation works.
6. Connection matrix stays symmetric, bounded, and zero-diagonal. Across observations, growth events, and decay.
7. With all ADR 0003 features disabled (inhibition_scale=0, connection_learning_rate=0, propagation_scale=0), the system behaves like v0/v0.1. Backward compatibility is preserved as a configurable mode.
8. Mimicking cells specialize under balanced input; Hebbian cells do not — the asymmetry between cell types is now sharper than under v0.1. Hebbian cells require frequency structure to specialize (test_hebbian_specialization_tracks_input_frequency validates that they do specialize under imbalanced input).

Consequences for the v0 long-excitation limitation

ADR 0001’s test_long_excitation_decays_toward_uniform_v0_limitation documented that prolonged excitation eroded frequency information from observed tokens. That limitation is removed by ADR 0003: connections preserve activation patterns across excitation cycles, and the frequency skew is in fact amplified beyond the underlying observation frequency (winner-take-all dynamics + connection reinforcement compound the dominant assembly). The test has been updated to assert preservation rather than decay.

Open questions deferred

• Replay during excitation — injecting past observations into the input stream during internal cycles. This is what makes the “stagnation dynamics” interesting; currently excitation is just slow decay + propagation.
• Stagnation detection — when does the system decide it has stopped processing and is ready for input?
• Asymmetric connections (C[i,j] ≠ C[j,i]) for directional binding (STDP-like, “A predicts B”).
• Type-restricted connectivity (Hebbian-only or mimicking-only assemblies).
• Connections between cells of different types — should mimicking connect to Hebbian, and what does it mean?
• Adaptive penalty / adaptive inhibition based on vocabulary size or population activity.