Decision 0010: Multi-tick STDP — connections at multiple temporal lags

Date: 2026-05-24 Status: Accepted Builds on: ADRs 0001–0009

Context

ADR 0008 introduced asymmetric STDP — connections that strengthen when one cell fires just before another. The “just before” was strictly one tick. That’s enough to encode pairwise transitions (“A → B”), but it can’t capture longer-range patterns like “A then B then C → answer.”

Real neural systems integrate over longer temporal windows. ADR 0010 adds that to Reverie: instead of one connection matrix capturing “what fires at t-1 → what fires at t,” the system maintains K separate connection tensors, one for each lag from 1 to K. Each captures “what fired at t-k → what fires at t.”

A natural question is whether this delivers arithmetic. The honest answer is no — multi-tick STDP encodes marginal associations at each lag, not conjunctive ones. The system learns “1 was at lag-2 of an answer” and “1 was at lag-4 of an answer” as separate facts. It cannot directly learn “1 at lag-2 AND 1 at lag-4 → 2 specifically.” That conjunctive binding requires either higher-order tensor connections (very expensive) or variable binding (which this architecture does not have). What multi-tick STDP delivers is the substrate for multi-lag temporal sensitivity — a real capability, but not the whole capability needed for symbolic reasoning.

Decision

K lag matrices

Replace the single _connections matrix with a tensor of K matrices, indexed by lag:

• _connections[k] for k = 0, 1, …, K-1, where _connections[k] encodes connections from cells active k+1 ticks ago to cells active now.

For backward compatibility with code that reads self.connections, the property returns _connections[0] (the lag-1 matrix). Multi-lag access is exposed through a new property self.lag_connections.

Activation history

The system maintains a bounded queue of the last K activation vectors:

• _activation_history: deque-like structure holding [A_{t-K}, ..., A_{t-1}] (length K).

On every tick, after updating activations:

• The current activation vector is recorded as the new entry.
• The oldest is dropped if the queue is full.

Update rule per lag

The STDP update applies the same anti-symmetric outer-product rule from ADR 0008, but separately at each lag:

for k in 0..K-1:
  if activation_history has at least k+1 entries:
    pre_k = activation_history[-(k+1)]  # k+1 ticks ago
    asym_update_k = stdp_rate_k · lr_scale · (outer(post, pre_k) - outer(pre_k, post))
    _connections[k] += asym_update_k

The symmetric Hebbian update from ADR 0003 still happens at lag-0 (the current _connections[0] matrix), unchanged.

stdp_rate_k defaults to a single shared stdp_learning_rate across all lags, but the architecture permits per-lag rates (open question deferred to future ADR).

Propagation across lags

During excite(), the propagation term sums contributions from all lag matrices applied to the appropriate historical activation:

propagated = sum_k=0..K-1 of (_connections[k] @ activation_history[-(k+1)])

For lag-0, the “history” is the current activation (self._activations); this preserves ADR 0008 behavior at K=1.

Default K=3

The default stdp_lags = 3 captures associations at 1, 2, and 3 ticks back. This is a reasonable compromise between expressiveness and cost. For arithmetic-style problems requiring 4-token context, set stdp_lags=4. For backward compatibility with ADR 0008, set stdp_lags=1.

Memory cost: K · n_cells² floats. For default K=3 and n_cells=32: 3072 floats. Negligible.

Compute cost: K STDP updates + K matrix-vector products per tick. With K=3 and modest n_cells, well under a millisecond per tick.

What this fixes and what it doesn’t

Fixes

• Multi-tick temporal sensitivity: the system can now learn “C tends to come 3 ticks after A” as a distinct fact from “B comes 1 tick after A.”
• Sequence anticipation extends to longer ranges.
• Dreaming becomes more sequence-aware: propagation pulls activation from multiple past steps, not just the immediately previous one.

Doesn’t fix

• Arithmetic. Multi-tick STDP encodes marginal associations per lag, not conjunctive ones. The system learns “1 was sometimes at lag-2 of an answer” — but cannot represent “1 at lag-2 AND 1 at lag-4 → 2 specifically” as a unified condition. Closing this gap requires either:
  • Per-cell context-conditional preferences (a higher-order generalization of predictive cells), or
  • Variable binding (which this substrate has no mechanism for).
• The variance-compression problem from ADRs 0005/0006. Unchanged.
• Sequence prediction beyond K ticks. The lookback is bounded by stdp_lags. Patterns longer than K cannot be learned via this mechanism alone.

Stability considerations

Memory and decay

Each lag matrix decays independently under connection_decay. The rate is the same; lag matrices don’t get special treatment. This is the simplest choice and may need revisiting (older-lag connections might benefit from faster decay if they’re noisier).

Saturation across lags

With K=3 and per-lag STDP rate equal to ADR 0008’s, total connection growth per tick is roughly K times faster. To compensate, the per-lag stdp_learning_rate defaults to the ADR 0008 value divided by K. Future tuning may revisit.

Propagation magnitude

Summing propagation across K lags raises the magnitude of propagated by ~K compared to ADR 0008. The activation cap from ADR 0003 (2 · (baseline + saturation)) still catches runaway, but the typical activation level rises. This is acceptable; the cap is a hard ceiling, not a soft preference.

Behavioral predictions

These claims have tests in tests/test_multi_tick_stdp.py:

1. With stdp_lags = 1, behavior matches ADR 0008 — single-lag regression safety net.
2. With stdp_lags = K > 1, the system maintains K connection matrices and the property lag_connections returns a tensor of the right shape.
3. Patterns at different lags are encoded in distinct lag matrices. After training on “A B C A B C …”, the lag-1 matrix has A → B direction; the lag-2 matrix has A → C direction.
4. All prior 73 tests pass under default stdp_lags = 3.

Open questions deferred

• Per-lag STDP rates. Currently shared. A future ADR may add a stdp_rates[K] vector for fine-grained tuning.
• Adaptive lag depth. The system might learn at runtime which lags are most useful and grow/shrink K. Probably overkill.
• Lag-dependent decay. Older-lag connections might benefit from faster decay.
• Conjunctive lag binding. The core remaining gap. Closing it likely needs a substantively different architectural primitive (variable binding, hyperdimensional computing, or context-conditional preferences in cells).