Decision 0007: Stagnation detection

Date: 2026-05-24 Status: Accepted Supersedes: none Builds on: ADRs 0001–0006

Context

The original architecture sketch had a specific claim about excitation dynamics: “it will keep doing that while there is no external excitations or observations until there a stagnation point comes. Toward the stagnation point, the state changes should get minimalized.”

Up to ADR 0006, the system has all the mechanism for this — replay drives activation, decay (when enabled) drives unreinforced preferences down, the buffer is bounded — but no notion of when “enough is enough.” A caller has to pick a number of excitation ticks (“dream 100 times”) with no signal back from the system about whether it’s actually done thinking or just been cut off.

This ADR makes stagnation observable. The system tracks how much its activation state has been changing on recent ticks; when changes drop below a threshold consistently, it reports is_stagnant = True. New observations reset the tracker (an observation is by definition not stagnation).

Decision

Track per-tick preference change

After every excite() tick, compute:

delta = mean(abs(preferences_after - preferences_before))

Append to a bounded history of recent deltas. Default history length: stagnation_window = 10 ticks.

Stagnation criterion

The system is stagnant when:

1. The history has at least stagnation_window entries, and
2. Every entry in the history is below stagnation_threshold (default 0.008, calibrated empirically to sit between active-growth deltas (~0.011 without decay) and decay-equilibrium deltas (~0.006 with decay rate 0.01–0.1)). The “preference change” measure is the max absolute element-wise change across the preference matrix, not the mean — per-tick updates touch only one column per cell type, so averaging over the whole matrix would dilute the actual learning signal.

That is: stagnation requires a sustained run of small changes, not just one tick of low movement. A single quiet tick can happen by chance; ten in a row means the dynamics have actually settled.

Observations reset stagnation

observe() clears the activation-change history. The next excitation tick starts from a fresh state, and is_stagnant returns False until the system has had at least stagnation_window quiet ticks again. This matches the original architecture intuition: an observation is a perturbation, and the system needs time to re-settle.

excite_until_stagnant(max_ticks) convenience

A method that runs excite() in a loop and stops as soon as is_stagnant becomes True (or max_ticks is reached, whichever comes first). Returns the number of ticks executed. This is what callers actually want when they say “let it dream until done.”

Default-on, low cost

Unlike normalization (ADR 0005) and decay (ADR 0006), stagnation tracking is on by default. The cost is a single mean(abs(...)) per tick — negligible. The history is a Python list bounded at the window size. There’s no behavioral change to existing code paths; only new state is added.

A caller that wants the old “just run N ticks” behavior continues to call excite() directly in a loop and ignore is_stagnant. The tracking happens regardless.

Why preference change rather than activation change

Two candidate signals exist for “the system has stopped doing anything interesting”:

• Preference change (mean(abs(prefs_after - prefs_before))). What’s actually being learned.
• Activation change (mean(abs(activations_after - activations_before))). What’s actually firing.

The first draft of this ADR used activations. Empirical testing rejected that choice for two concrete reasons:

1. Activations have a noise floor. Per-tick operational noise at noise_scale = 0.01 produces activation changes around 0.01 even at equilibrium. Any stagnation threshold below the noise floor is unreachable; above the floor it triggers spuriously. The architectural intent is “no real change happening” — but noise is real change at the activation level, just not meaningful change.

2. Activations oscillate by design. Each replay tick samples a different token from the buffer. Cells respond differently to different tokens. So activations move every tick even after the system has fully settled in any meaningful sense — they’re cycling through the buffer, not stabilizing.

Preferences don’t have either problem. Noise doesn’t move them directly, and at decay equilibrium they barely move at all — exactly the architectural property we want to detect. Preferences only change when learning happens; when learning has reached equilibrium (reinforcement = decay), per-tick preference change drops to near zero independent of which token is currently being replayed.

A future ADR may add an activation-stability criterion as a secondary gate (e.g., “stagnation requires both preference stability AND that the system hasn’t oscillated wildly recently”). For now, preferences alone are the signal.

Stagnation requires decay to be meaningful

Without preference_decay_rate > 0 (ADR 0006), replay reinforcement keeps growing preferences indefinitely, which feeds back into activation through tanh(pref/sat). Activations don’t naturally settle — they’re driven by ever-changing preferences. The system may never reach stagnation.

With decay enabled, preferences plateau at equilibrium, activations stabilize, and stagnation becomes reachable. Stagnation and decay are complementary mechanisms that the architecture needs together.

This isn’t a problem — callers who don’t want decay also don’t typically want stagnation. They want a fixed number of dream ticks. Callers who want the “dream until done” behavior enable both.

Behavioral predictions

These claims have tests in tests/test_stagnation.py:

1. is_stagnant is False before sufficient history accumulates — the first stagnation_window ticks after any observation are never stagnant.
2. With decay enabled, the system reaches stagnation within a bounded number of ticks.
3. Without decay, the system does not stagnate (preferences keep growing, activations keep shifting).
4. Observations reset stagnation — calling observe() after stagnation makes is_stagnant False again.
5. excite_until_stagnant returns when stagnant — the count returned is at most max_ticks.
6. The activation-change measure is monotone with the threshold — raising stagnation_threshold makes stagnation easier to reach; lowering it makes stagnation harder.

Open questions deferred

• Combined activation + preference criterion. If preference learning is still happening (e.g., wrong-bet penalties accumulating slowly), should we count that as not-stagnant even if activations have settled? Likely yes, but the rule is unclear yet.
• Adaptive thresholds. A small, busy system has smaller absolute activation changes than a large one. Future versions may scale the threshold by 1/n_cells or by recent typical change magnitudes.
• Stagnation as a signal for action. Eventually the system itself may use stagnation to decide to generate output, to consolidate, to ask for input. For now stagnation is just observable; what to do with it is the caller’s choice.
• Hysteresis. Currently any tick with above-threshold change “breaks” stagnation, even briefly. A small grace period might be useful.