Decision 0011: Spontaneous output during excitation — the system dreams in tokens

Date: 2026-05-24 Status: Accepted Builds on: ADRs 0001–0010

Context

Through ADR 0010, the architecture has every mechanism described in the original sketch except one: spontaneous output during internal cycles. Currently excite() updates internal state (activations propagate through connections, replay reinforces patterns, decay erodes stale preferences) but produces no observable output. The system’s outputs are entirely caller-driven via generate().

The original architectural vision was different. The system was supposed to “generate outputs from its current state” continuously, between external observations. The outputs were a property of the system’s internal life, not something extracted by external query.

This ADR closes that gap. When enabled, each excite() tick (a) samples a token from current state (the existing generate() mechanism) and (b) feeds that token back as a soft self-observation — a perturbation at reduced learning rate. The system talks to itself. What it says during a dream is shaped by what it has seen, and the act of saying changes what it then thinks.

This is the difference between a system that processes and a system that expresses. It’s also what makes the project’s name literal: Reverie is now a dream you can listen in on.

Decision

Spontaneous output behavior

When self_observation_rate > 0 (default 0.0, opt-in), each excite() tick performs an additional sequence:

1. Run the existing excite logic — activation decay, propagation, replay-driven learning, connection updates, stagnation tracking.
2. After the state update, sample a token from current state via the same logic as generate(). Call this the self-output.
3. Treat the self-output as a soft observation: compute activations for that token (with propagation enabled, since we’re inside excitation), apply learning at self_observation_rate × learning_rate, update connections at the same scaled rate.
4. Record the self-output in a separate self_output_history deque for inspection. Do NOT add it to the replay buffer (replay is for external observations).

The self-output is not a free byproduct — it perturbs the system. With a positive self_observation_rate, the act of speaking reinforces what was spoken. Over many ticks, the system either settles into a recurring vocabulary (attractor states) or wanders through its full vocabulary (noisy regime), depending on parameters.

Why self-output uses reduced learning

Two architectural reasons:

1. Self-output is imagined, not external. Treating it as a full observation would let the system run away with its own confabulations — it would say something, reinforce that thing, say it again, reinforce again, ad infinitum. The reduced rate keeps the feedback loop bounded.
2. It preserves the asymmetry between observation and dreaming. External observations get full-rate learning (you should remember what actually happened). Replays get reduced learning (rehearsal). Self-outputs get even more reduced learning (you should remember what you imagined, but only weakly).

Default self_observation_rate = 0.05 — half the replay rate. Set higher for more “vivid” dreams (and more risk of runaway), lower for quieter ones.

Why a separate history rather than the replay buffer

The replay buffer drives what the system rehearses during dreams. Adding self-outputs to it would create a strong feedback loop where the system rehearses what it just imagined, reinforces it, imagines it again. By keeping self_output_history separate:

• The replay buffer continues to reflect external experience.
• Self-outputs are visible to the caller (for inspection) but don’t compound through replay.

If a future ADR wants to allow self-outputs into replay (a “lucid dreaming” variant where imagination becomes memory), the separation makes that a clean opt-in.

Disabling

self_observation_rate = 0.0 (default) disables self-output entirely. excite() behaves identically to v0.9. Backward compatible.

Interaction with stagnation

Stagnation detection (ADR 0007) is unaffected by self-output mechanics — it tracks preference change, which now also receives contributions from self-output learning. With a balanced self_observation_rate, this typically means the system settles slightly slower (because every tick adds a small learning event), but reaches stagnation when the self-output stream stabilizes into a fixed-point or limit-cycle.

That’s actually meaningful: stagnation now means “the system has converged on what it’s saying,” not just “internal state has stopped changing.” A richer notion of “done thinking.”

Why this matters for the “artist” framing

The original architectural sketch described a system whose outputs are not in service of any external goal — “it will be like an artist, making decisions on its own way and it will create its own goals.” That can only mean something if the system actually does things between observations. Until ADR 0011, the system was a thoroughly receptive substrate: it accepted observations, internally reshaped itself, and produced output only when asked.

After ADR 0011, the system has a behavioral life between observations. It generates. It hears itself generate. Its preferences shift in response to its own outputs. The stream of self-outputs across a long dream is the closest thing the system has to intention — what it chooses to say when no one’s asking.

This is not the same as solving arithmetic. The arithmetic experiment will not change with this ADR. But the artist hypothesis — that meaningful, content-dependent emergent behavior is possible from observation alone — now has a substrate that can actually exhibit it.

Behavioral predictions

These claims have tests in tests/test_spontaneous_output.py:

1. With self_observation_rate = 0, excite() produces no self-output and the self_output_history stays empty — backward compatibility.
2. With positive self_observation_rate, excite() populates self_output_history with one token per tick.
3. Self-output sequence reflects training distribution. After observing imbalanced input (“frequent” 80%, “rare” 20%), the system’s self-outputs during dreaming should also be imbalanced.
4. Self-output learning is at reduced rate — the preference change from a self-output is measurably smaller than from a full observation.
5. Self-outputs do not extend the replay buffer. The observation_buffer remains unchanged across excite() ticks.
6. All prior 80 tests continue to pass under default self_observation_rate = 0.

Open questions deferred

• Lucid dreaming — letting self-outputs into the replay buffer. A future opt-in.
• Output coherence over time — at what self_observation_rate does the system find stable attractor patterns vs wander through random tokens? Empirical work, not architectural.
• Self-output diversity — should the system get a “creativity boost” via higher noise during self-output generation? Open.
• Multi-modal generation — if the system has multiple vocabularies (e.g., tokens + actions), how does self-output across modalities work?