A model that learns while it reads

一个边读边学的模型

Test-time training (TTT) lets part of a model, its fast weights W_t, keep changing during inference. Information can then live in the weights after the text has scrolled out of the attention window. That is attractive for long documents and for agents that run for hours.

测试时训练（TTT）让模型的一部分参数，也就是快权重 W_t，在推理时继续更新。这样，文本即使已经滑出注意力窗口，信息仍能留在权重里。对长文档和长时间运行的智能体来说，这很有吸引力。

Two words will carry the rest of the story.

接下来的故事里，有两个词会反复出现。

Writing is not the villain. On real books, it helps:

写入本身不是坏人。在真实的书上，它是有帮助的：

The model writes its own homework

模型给自己出作业

Now let the model supply its own training data. The current weights W_t write a 1024-token chunk x_t. The model learns from x_t and becomes W_t+1. Then W_t+1 writes the next chunk. We call this the Closed Loop.

现在让模型自己提供训练数据。当前权重 W_t 写出一段 1024 token 的文本 x_t，模型从 x_t 中学习，变成 W_t+1；然后由 W_t+1 写下一段。我们把它叫做闭环（Closed Loop）。

To measure what the loop costs, we need a fair control. Writes Off reads exactly the same kind of generated text, keeping it in attention, but throws each update away.

要衡量这个闭环的代价，需要一个公平的对照组。关闭写入（Writes Off）读的是同样的生成文本，也把它们留在注意力里，但每次更新都直接丢掉。

Every check uses independent human-written passages from later, disjoint parts of the same PG-19 book. After each check, weights and attention are restored, so checking never changes the stream.

每次检查都用独立的人类文本：来自同一本 PG-19 书中靠后、互不重叠的段落。每次检查完都会恢复权重和注意力状态，所以检查本身不会改变数据流。

Nothing happens. Then everything does.

起初风平浪静，然后突然崩坏

We ran both policies for 128K tokens: an 8K real-text prefix, then 105 generated chunks. In the first few chunks the two look almost the same. Inside the 8K+8K window used by earlier evaluations, you would see nothing wrong. Only after many more self-writes does the Closed Loop pull away.

我们让两种策略各跑 128K token：先是 8K 的真实文本前缀，然后是 105 段生成文本。最初几段里，两者几乎一样。在以往评测采用的 8K+8K 窗口内，你看不出任何问题。只有经过更多次自我写入之后，闭环才开始明显偏离。

Three suspects

三个嫌疑人

There are three plausible culprits. Maybe generated text is simply bad training data. Maybe degraded text hurts while it sits in attention. Maybe learning from it stores the harm in the weights. We question each one by cutting one link at a time.

有三个看起来都说得通的嫌疑人。也许生成文本本身就是糟糕的训练数据；也许退化的文本只要待在注意力里就会造成伤害；也许从中学习会把伤害存进权重。我们每次只切断一条链路，逐个审问。

Now scroll through the investigation. The sketch follows along.现在往下滚动，开始审问。草图会跟着变化。

One update, two faces

一次更新，两副面孔

The endpoints above mix thousands of updates. To see what a single update does, copy the model right before a generated chunk: same weights, same attention, same text. One copy keeps the update. The other throws it away. Any difference afterwards is caused by that one update.

上面的终点混合了成千上万次更新。要看清单次更新做了什么，就在生成一段文本之前把模型复制一份：同样的权重、同样的注意力、同样的文本。一份保留这次更新，另一份丢掉。之后的任何差异，都来自这一次更新。

The cost also grows as the loop runs. Early on, keeping one update costs .006 nats. Late in a Closed Loop it costs .113, while after Writes Off or Fixed Generation it stays near zero.

而且随着闭环运行，代价越来越大。早期保留一次更新代价 .006 nats；闭环后期变成 .113，而在关闭写入或固定生成的历史之后几乎为零。

The gradients disagree梯度在“打架”

Before keeping an update, compare the gradient from the generated chunk with the gradient from the next real passage. The more they point in opposite directions, the more the update hurts. Over 96 paired updates per setting, the rank correlations are strong and every interval excludes zero.

在保留一次更新之前，比较生成片段的梯度和下一段真实文本的梯度。方向越相反，这次更新伤害越大。每种设置 96 对更新，秩相关都很强，所有区间都不包含零。

After a Closed Loop history, generated and real gradients actively oppose each other (mean cos ≈ −.10); after Writes Off they are nearly orthogonal. This explains which updates hurt, but it is not a usable filter, since it peeks at the very passage on which harm is measured.

经历闭环历史后，生成文本与真实文本的梯度明显相反（平均 cos ≈ −.10）；关闭写入之后则几乎正交。这解释了哪些更新有害，但不能直接当过滤器用，因为它偷看了用来衡量伤害的那段文本。

Most late updates are fine. A few are terrible.

大多数后期更新没事，少数是灾难

If late updates get more expensive on average, are they all bad? We replayed eight-chunk passages into identical receivers, once read-only and once read-and-write, 72 pairs at each position.

如果后期更新平均越来越贵，是不是每一个都坏？我们把 8 段长的片段回放给完全相同的接收者，一次只读、一次读并写，每个位置 72 对。

At the last position the mean is .7512 but the median is .0064. All twelve disasters come from 4 of 24 source sequences, and once a pair crosses the line it stays above it. So "late in the stream" is a warning sign, not a rule for which update to drop.

在最后一个位置，均值是 .7512，中位数却只有 .0064。全部 12 次灾难来自 24 条源序列中的 4 条，而且一旦某对越过阈值，之后就一直在阈值之上。所以“处在数据流后期”只是警示信号，不能用来决定该丢哪次更新。

Half-fixes

治标不治本

Before building something new, we tried the obvious remedies. Each one helps. None of them can tell you whether a particular update is safe to keep.

在设计新方法之前，我们先试了显而易见的补救办法。每一个都有点用，但没有一个能告诉你某一次更新是否值得保留。

Remedy 1 · Interrupt the loop with real text补救 1 · 用真实文本打断闭环

Swap some of the 105 generated chunks for real ones, still keeping every update. Try the schedules below and watch the longest stretch the model spends writing to itself.

把 105 段生成文本中的一部分换成真实文本，同时仍保留每一次更新。试试下面几种安排，观察模型连续“自说自话”的最长一段。

Remedies 2–4 · Write less, decode differently, down-weight repeats补救 2–4 · 写得更少、换解码方式、给重复降权

Ask before you remember

先问一句，再记下来

Every half-fix shares one blind spot: none asks whether the update actually helps beyond its own chunk. So ask directly. Keep each update pending. When independent real text q arrives, compare the current state with the candidate state on it. Keep the update only if the candidate predicts q at least as well. We call this Settlement.

所有半吊子方案都有同一个盲点：没有一个去问，这次更新在它自己那段之外是否真的有帮助。那就直接问。让每次更新先挂起，等独立的真实文本 q 到来时，在 q 上比较当前状态和候选状态，只有候选状态预测 q 至少一样好，才保留这次更新。我们把它叫做 Settlement（先验证，再提交）。

Check the state you will actually keep检验你真正要保留的那个状态

One subtlety. Updates that each look safe alone can be harmful together. So every candidate is judged on top of the ones already accepted, never against the original weights.

还有一个细节：单独看都安全的更新，合在一起可能有害。所以每个候选都在已接受的更新之上评判，而不是对着原始权重。

What if you just labeled the text?如果只是给文本打上标签呢？

A simpler rule, Source Masking, keeps updates from text labeled "real" and drops those labeled "generated". With perfect labels it is simpler and slightly better. Now corrupt the labels and drag the slider.

一个更简单的规则叫来源屏蔽（Source Masking）：保留标为“真实”的文本带来的更新，丢掉标为“生成”的。标签完全正确时，它更简单，也略好一点。现在把一部分标签弄错，拖动滑块看看。

Out in the world: agents

走进真实世界：智能体

Agents that learn from their own actions face the same loop, with a twist: every update carries the same "generated" label. A label rule can only keep them all (Closed Loop) or drop them all (Writes Off). A task score can tell them apart.

从自己的行动中学习的智能体，面对的是同一个闭环，还多一个麻烦：每次更新都带着同一个“生成”标签。基于标签的规则只能全部保留（闭环），或全部丢弃（关闭写入）。任务得分则能把它们区分开。

What we learned

我们学到了什么

1. 1The loop is the problem, not the text. A learner that eats synthetic text from a frozen writer stays close to Writes Off. Damage grows when the learner also writes its own future lessons.问题出在闭环，而不是文本。学习者从冻结的写手那里吃合成文本，表现仍接近关闭写入。只有当学习者也在给自己写未来的课本时，损害才会增长。
2. 2Damage is concentrated. Four of 24 source sequences produce every disaster above 0.5 nats. The loop makes some trajectories persistently harmful, not every late update.损害高度集中。24 条源序列中的 4 条造成了所有超过 0.5 nats 的灾难。闭环让某些轨迹持续有害，而不是让每次后期更新都有害。
3. 3Fitting its own chunk proves nothing. Neither source fit nor repetition tells you whether an update helps on new text. Real text helps most when it interrupts often.拟合好自己那段，什么也证明不了。源文本拟合度和重复度都无法说明一次更新在新文本上是否有用。真实文本最有效的用法，是频繁地打断自我写入。
4. 4Check on independent evidence, then commit. Settlement nearly matches Writes Off on generated streams, keeps useful real-text learning, and survives broken labels.先用独立证据检验，再提交。Settlement 在生成文本流上几乎追平关闭写入，保留了有用的真实文本学习，并且在标签出错时依然稳健。

BibTeX

@article{luo2026selfgenerated,
  title   = {Self-Generated Feedback Destabilizes Test-Time Training:
             A Causal Decomposition of Long-Horizon Adaptation},
  author  = {Luo, Cheng and Li, Bing and Ghanem, Bernard},
  journal = {arXiv preprint arXiv:2610.05076},
  year    = {2026}
}