Decision 0002: Cell types — mimicking and Hebbian as distinct populations

Date: 2026-05-23 Status: Accepted Supersedes: none Builds on: ADR 0001

Context

ADR 0001 established a v0 kernel with a single, uniform cell type — every cell learning the same way. That was correct for proving the substrate works, but the architecture calls for two distinct populations:

• Mimicking cells — each has a local goal: produce outputs that match observed inputs. They are the system’s predictive / generative substrate.
• Hebbian cells — no goal beyond association. They discover what co-occurs and (in later ADRs) form connections between co-active cells, building distributed representations.

This ADR introduces that distinction. It does not yet introduce inter-cell connections — those are deferred to ADR 0003, where they’re load-bearing for Hebbian cells to do anything different from the v0 rule.

Decision

Two cell types as separate populations

Each cell is exclusively one type: either MIMICKING or HEBBIAN, never both. This is the more literal reading of the architecture (“there will be mimicking cells and there will be Hebbian cells” — two populations, not two rules per cell).

Alternatives considered:

• Mixed cells with both rules. Each cell runs both learning rules in parallel. Rejected because it muddies the question of what a cell is. If a cell is doing both, we can’t observe one rule’s effect cleanly. Separation gives us cleaner experimental signal.
• A continuum (cells more or less mimicking-ish). Each cell has a “mimicking weight” between 0 and 1. Rejected as premature parameterization — we haven’t yet shown the two pure rules behave usefully on their own.

Initial ratio: 50/50

The cell pool is split half-half between the two types at allocation time. The choice is provisional — we have no principled basis yet for any other ratio. The ratio is observable and adjustable; a future ADR can change it once we have data on how the populations contribute.

Mimicking learning rule: predict-then-correct (strict)

Each mimicking cell maintains an implicit bet — a token sampled from a softmax of its current preference vector. Sampled rather than argmax’d because deterministic bets across identical cells produce identical learning and prevent divergence; stochastic bets let cells with similar preferences bet differently and therefore diverge.

When an observation arrives, the cell compares its bet to the observed token:

• If the bet matched the observation: reinforce preference for that token (proportional to activation).
• If the bet did not match: apply a negative-reinforcement penalty to the wrongly-bet token, scaled by mimicking_penalty.
• The cell does not strengthen its preference for the observed token when its bet was wrong. Only correct predictions are reinforced.

Each mimicking cell is, in effect, a one-token “expert” — only learning about tokens it actively predicts. Different cells diverge because they happen to bet for different tokens early and ride that early reinforcement into specialization.

A path that didn’t work: the softer rule

Before landing on the strict rule above, we considered a softer variant: always reinforce the observed token, plus penalize wrongly-bet tokens. We rejected it for a concrete reason discovered during implementation.

Under that softer rule, with i.i.d. inputs, mimicking cells cannot stably specialize on any single token. The “always-reinforce-observed” component dominates and drags every cell’s preferences toward the input distribution regardless of what they bet. The penalty just adds noise. Hebbian cells (which only accumulate upward) end up more concentrated than mimicking cells under the softer rule — exactly opposite to the architectural intent.

The math (for a cell betting token b with probability q under uniform 2-token input):

d_pref[a]/dt = 0.05 - 0.025·penalty·q
d_pref[b]/dt = 0.05 - 0.025·penalty·(1-q)

For specialization on a to be stable when q=1, we’d need d_pref[a] > d_pref[b], which simplifies to 0 > 0 — never true. No penalty value rescues the softer rule.

The strict rule above doesn’t have this problem because it removes the “always reinforce observed” component; each cell only updates the dimension it predicted.

Stability constraint of the strict rule

The strict rule introduces its own constraint, worth recording. For a mimicking cell stably specializing on a single token under uniformly-distributed k-token input:

penalty < 2 / (k - 1)

Specifically: with k tokens uniformly observed and a cell always betting its preferred token, expected reinforcement per tick is (1/k)·rate and expected penalty is ((k-1)/k)·rate·penalty. Specialization is stable iff reinforcement exceeds penalty, i.e., penalty < 2/(k-1).

Default mimicking_penalty=0.1 covers vocabularies up to ~21 tokens. Larger vocabularies will need a smaller penalty (or a different rule entirely; a future ADR may make the penalty adaptive to vocabulary size).

Hebbian learning rule: unchanged from v0

Hebbian cells continue to use the v0 rule: strengthen preference for the observed token in proportion to activation. No bets, no comparison, no goal, no negative reinforcement.

In v0.1 (after this ADR, before ADR 0003), Hebbian cells are behaviorally close to v0 cells. Their distinct character will emerge in ADR 0003 with the introduction of inter-cell connections — that’s when “association” becomes a real mechanism rather than just a label.

Hebbian learning rule: unchanged from v0

Hebbian cells continue to use the v0 rule: strengthen preference for the observed token in proportion to the cell’s activation. No negative feedback, no comparison, no goal.

In v0.1 (after ADR 0002 but before ADR 0003), this means Hebbian cells are behaviorally identical to v0 cells. Their distinct character will emerge in ADR 0003 with the introduction of inter-cell connections — that’s when “association” becomes a real mechanism rather than just a label.

Output generation: unchanged from v0

All cells (both types) contribute to the activation-weighted vote across the vocabulary. Stochastic sampling selects the output token. The cell-type distinction affects what each cell prefers, not how preferences aggregate.

Why this ordering — types before connections

It would be tempting to introduce types and connections together. Two reasons we didn’t:

1. One architectural change per ADR is easier to reason about. If we ship both and behavior shifts unexpectedly, we don’t know which change caused it. Types are cheap to add; connections are the harder change. Land each separately.
2. Mimicking cells need no connections. Their entire learning rule is local to the cell + the current input. They can be developed and tested in isolation. Hebbian cells will be inert until 0003 — that’s fine; they’re labeled, just not yet doing their job.

Behavioral predictions (validated)

These are claims the implementation satisfies — each has a corresponding test in tests/test_cell_types.py:

1. Mimicking cells specialize under varied input within the stability range. Each mimicking cell ends up with a small number of strongly-preferred tokens — significantly more concentrated than Hebbian cells under the same input.
2. Hebbian cells remain generalist. Their preferences accumulate roughly uniformly across observed tokens, with low coefficient of variation.
3. The mimicking population diverges across the vocabulary. Different mimicking cells specialize on different tokens — not all collectively on one. This is the seed of distributed representation.
4. Mimicking cells can develop negative preferences; Hebbian cells cannot. A structural difference between the populations that holds across all input regimes.
5. All v0 acceptance tests still pass. The fundamental promises (observe → output, vocabulary tracking, growth) are unaffected by cell typing.

Open questions deferred

• Inter-cell connections and the “asking” competition protocol — ADR 0003.
• Replay during internal excitation — ADR 0003 or 0004.
• Whether the 50/50 ratio is right, and whether it should be observation-dependent — empirical, to be revisited.
• Predictive cells and abstraction cells as additional types — Kaan to specify.