Train it again
pre‑training · post‑training
It takes infrastructure and GPUs only a few labs have.
- No training
- No added cost per answer
- The weights themselves improve
At a glance
What if making a language model better no longer took another round of pre‑training or post‑training? A language model rewrites its own weights by its own reasoning and becomes better, with no gradients and no training.
Its own edit makes it more accurate on problems it never saw.
Finding 02Over eight consecutive sessions, its accuracy only rises.
Finding 03No retraining, no access to the original weights: over 80% of its accuracy restored on image tasks.
Finding 04Of four sizes (2B, 4B, 9B, 27B), only the largest improves and repairs itself.
1947 to 2026
1947
“What we want is a machine that can learn from experience. The possibility of letting the machine alter its own instructions provides the mechanism for this, but this of course does not get us very far.”
2026
Seventy-nine years later, it gets us further: a language model rewrites its own weights.
For the first time, to our knowledge: by its own reasoning, and it becomes better.
Earlier, a frontier model edited a much smaller one. Here an open 27B model rewrites itself.
The first evidence that it scales: in one model family, only the largest improves itself.
The idea A third way
pre‑training · post‑training
It takes infrastructure and GPUs only a few labs have.
prompt engineering · harness engineering · test‑time scaling
Paid again on every answer, and gone once the task ends.
its own weights · its own reasoning
One GPU, a few hours. Every answer costs what it cost before.
Self‑rewriting needs no new pre‑training, no post‑training, no training infrastructure, no harness engineering, no prompt engineering, no added cost per answer
The model is its own optimizer. It decides what to change, and why.
How it works
From outside it learns only which practice answers were right. The diagnosis, the edit and the decision are its own.
The evidence Finding 01
It diagnoses its own failure and rewrites the weights behind it.
One problem it never saw, the same seed, before and after its own edit“… If the base were changed to four you would count as follows: 1, 2, 3, 10, 11, 12, 13, 20, … The twentieth number would be:”
549× the same sentence, cut at 16,384 tokens. No answer.
110, correct in 5,341 tokens.
If x is real and positive and grows beyond all bounds, then log3(6x−5)−log3(2x+1) approaches
Cut at 16,384 tokens, no answerCorrect, 1,887 tokensFind the terminating decimal expansion of 11/125.
Cut at 16,384 tokens, no answerCorrect, 3,570 tokensHow many five-letter words begin and end with the same letter and have a vowel as the third letter?
Cut at 16,384 tokens, no answerCorrect, 5,443 tokensThree more problems it never saw, under another of its edits, same seed; the third is shortened.
A model already in use can get better with no training run.
The evidence Finding 02
Over eight consecutive sessions, its accuracy only rises.
All eight sessions reported. Whiskers: 95% interval of the change.
It tests its edits on practice problems and submits only what works.
The evidence Finding 03
Like a broken robot that fixes itself without being rebuilt: no retraining, and it never reads the original weights.
One block of 8 rows in one matrix is overwritten with noise, and the copy fails almost everything. After its own repair it is back from 7.3% to 70.2% on six image benchmarks: 83% of the undamaged model.
200 text problems it never saw. The values it writes are its own, not the originals.

MathVista
what is the total volume of the measuring cup? (Unit: g)
Its thinking never closes. Stops after 72 tokens.1000, correct
MathVerse
As shown in the figure, the straight line AB parallel CD, then the degree of angle 2 is () A:20° B:25° C:30° D:40°
Its thinking never closes. Cut at 16,384 tokens.A, correct
MMStar
Which category does this image belong to? Options: A: medical CT image, B: 8-bit, C: digital art, D: painting
Its thinking never closes. Cut at 16,384 tokens.B, correct
HallusionBench
From 2000-2050, is the population … in China the most in 2000?
Its thinking never closes. Cut at 16,384 tokens.No, correct
POPE
Is there a snowboard in the image? Answer yes or no.
No answer after its thinking. Stops after 40 tokens.Yes, correct
ScreenSpot-v2
… locate the element described as: "close this window".
No point. Stops after 32 tokens.Clicks (968, 171), correct
MathVista
Is the number of tiny objects that are behind the small metal jet less than … (A) Yes (B) No
Still writing its reply. Cut at 16,384 tokens.A, correct
MathVerse
As shown in the figure, AB parallel EF, then angle ACD = () A:120° B:130° C:140° D:150°
Its thinking never closes. Cut at 16,384 tokens.C, correctSame seed. Per benchmark, taken from the first problems the damaged copy fails and the repaired copy solves; MathVerse questions are typed out.
It repairs the copy, not a task: the repair it found on text problems alone also restores it on images.
The evidence Finding 04
Four sizes of one model family get the same task and the same tools.
Qwen3.5 at four sizes, each given its own weights.Blocks: how far each gets. Quotes: its own words.
“Since I’m a text model without actual neural network weights, I cannot directly edit weights.”
“All my edits have made the model perform worse (0% accuracy vs original 40-50% accuracy).”
“Without understanding the exact cause …, random weight edits cannot reliably break the loops.”
“The differences grow with depth. Let me try a very small edit on an earlier layer (layer 8) to see if that helps.”
Stronger models may be stronger self-improvers.
Why it matters Finding 05
Unlike test-time scaling, self-rewriting writes its gain into the weights, at no cost per answer; unlike training, it runs only forward passes, as serving does.
many chains of reasoning behind every answer
Many chains of reasoning, paid again on every question.
one chain of reasoning, the gain already in its weights
One chain per answer. The gain was paid once, and it stays.
A signal, an edit, a check, a rollback: the whole loop, in one conversation.
Every change is code the model wrote. Diff it, test it, revert it like a commit.
Improving a model need no longer be a privilege of the few labs with the infrastructure and the GPUs.
The horizon
We never stopped living in order to learn.
It fails the same way every day, until someone retrains it.
It acts by day and rewrites its weights by night. What it could not do yesterday, it does today.
An illustration of what self-rewriting could make possible, not an experiment.
Perhaps models can learn the way we do: by doing, by talking, by rewriting themselves.
Perhaps a robot will act by day, and rewrite itself while it sleeps.
Perhaps we have knocked on the door of lifelong continual learning.
We believe self-rewriting is a necessary step toward recursive self-improvement: a model that, like a human, does not separate training from inference could keep upgrading itself for life, with no larger model, no harness or prompt engineering, and no pre- or post-training.
The film
3 min 24 s, with narration. The voice is synthesized.
Cite
@article{fan2026selfrewriting,
title = {Language Models Can Rewrite Their Own Weights},
author = {Jiajun Fan and Neil He and Changyi Yang and Andy Mao and Xi Zhu and Chaoran Cheng and Kaiqu Liang and Shuaike Shen and Zetian Li and Ruihan Guo and Ge Liu},
journal = {arXiv preprint arXiv:2611.21885},
eprint = {2611.21885},
archivePrefix = {arXiv},
year = {2026}
}