Finite-State Models for Continual Learning

The previous post ended on a claim: the data wall is structural, not logistical. The information exists — it is generated every second by agents and robots acting in the world — but transformers cannot consume it, because a transformer only ever learns from a static corpus assembled before training. To learn from a stream, you need a different shape of model. This post is about that shape.

Two properties a stream-learner must have

Start from what a transformer is. Attention compares every token to every other token 1. This gives two properties that are virtues for one-shot generation and liabilities for continual learning:

1. The state grows without bound. “Memory” is the context window — a buffer of past tokens. Cost grows quadratically with its length, and once it fills, the oldest content is simply dropped. There is no mechanism to consolidate what scrolled off; it is gone.
2. The weights are frozen at deployment. All learning happens during pre-training. At inference the model adapts only within the context window, and that adaptation evaporates when the session ends. What the model does, the corrections it receives, the outcomes it sees — none of it updates the model.

A model that learns from an unbounded operational stream needs the opposite of both: a bounded internal state whose cost does not grow with the length of experience, and an online update rule that folds new experience into the model permanently. Call a model with these two properties a finite-sized state model. Three research threads, usually discussed separately, are each solving one piece of it.

Thread 1 — A constant-size state can be enough (S4, Mamba)

The first objection is that you cannot compress an arbitrarily long history into a fixed-size state without crippling the model. Structured state-space models are the counter-evidence. S4 reframed sequence modeling as a linear state-space recurrence with a cleverly parameterized transition, and matched or beat attention on long-range benchmarks while scaling linearly in sequence length 2. Mamba added input-dependent (selective) state transitions, letting the model choose what to keep and what to forget, and reached transformer-quality language modeling with a recurrent, constant-memory inference path 3.

This settles the first property. A constant-size state, done right, is not a bottleneck — it is competitive. But an off-the-shelf Mamba is still trained once and frozen. Its recurrent state is a fast memory of the current sequence, not a place where lasting learning accumulates. That is the second thread.

Thread 2 — The state can be a learner (test-time learning)

The recent and more radical idea is to make the model’s internal state update itself during inference, by running a learning step on the data it is currently seeing. In Test-Time Training layers, the hidden state is the weights of a small model, and the recurrence’s update rule is literally a gradient step — so the longer the model runs, the more it has learned from the stream, all at inference time 4. Titans builds a neural long-term memory module that learns what to memorize at test time and reads it back, scaling to very long contexts while keeping a bounded, updatable store 5.

This is the conceptual hinge of the whole program: the forward pass becomes a learning step. Experience is not buffered and discarded; it is written into the state as it arrives. This is what “continual” has to mean architecturally — not periodic re-training in a lab, but learning in the loop, online.

Thread 3 — The hard problem is stability, not capacity

If updating online were free, this would already be solved. It is not, and the obstacle has a name. Neural networks trained sequentially on new data overwrite old knowledge — catastrophic forgetting, identified in connectionist models decades ago 6 and still the central failure mode of continual learning. Elastic Weight Consolidation showed forgetting can be mitigated by slowing changes to weights important for previous tasks, anchoring them with a quadratic penalty 7 — a real advance, but a partial one.

Worse, the problem is not only forgetting what you knew; it is losing the ability to keep learning at all. Dohare et al. demonstrated in Nature that standard deep networks trained continually suffer a steady loss of plasticity — units saturate, effective rank collapses, and the network gradually stops being able to learn new things — and that targeted interventions (continually re-injecting diversity into the weights) are needed to preserve it 8. Any architecture that claims to learn forever has to answer this directly.

So the bet is precise. The first two threads — bounded state and test-time updates — are maturing fast. The unsolved core is stability under an endless update stream: retaining old skills, preserving plasticity for new ones, all within a fixed-size state. That is the problem a finite-state continual learner is built around, not a feature bolted on afterward.

What it unlocks: data that compounds

Why is this worth a company rather than a paper? Because solving it inverts the data economics of the previous post. World-model research already shows that an agent can learn a compact, predictive internal model of its environment and improve by acting within it 9, including across many distinct domains from a single configuration 10. Couple that with online, non-forgetting updates and the operational stream stops evaporating: every robot, every shift, every correction makes the system better, and the data compounds instead of resetting to zero. The moat is not a single trained model — it is a self-improving system fed by a data source that frozen architectures cannot reach.

A falsifiable first step

Conviction is cheap; the bet has to be testable. The near-term target we hold ourselves to: a finite-state model that learns continually on an embodied or operational task and beats two baselines at once — a long-context model given the same history in its window, and a model that is periodically fine-tuned offline — while holding compute and state size bounded. Concrete, adversarial, and fast to falsify. If a bounded-state online learner cannot beat “just make the context longer” and “just re-train monthly,” the thesis is wrong and we want to know early.

That is the whole program in one line: the architecture every robot and agent ships with once the transformer era ends. The data wall says the old source is exhausted. This is the model built for the source that isn’t.

---