Jiajun Fan

Language Models Can Rewrite Their Own Weights

  • 1UIUC
  • 2CMU
  • 3Rutgers
  • 4Princeton
  • 5Stanford
  • 6Yale

At a glance

The paper in four results.

What if making a language model better no longer took another round of pre‑training or post‑training? A language model rewrites its own weights by its own reasoning and becomes better, with no gradients and no training.

  • Affordable.a few hours on one GPU and no added cost per answer
  • Scalable.it emerges with model size
  • Recursive.the model improves itself

1947 to 2026

1947

“What we want is a machine that can learn from experience. The possibility of letting the machine alter its own instructions provides the mechanism for this, but this of course does not get us very far.”
Alan Turing, lecture to the London Mathematical Society, 20 February 1947

2026

Seventy-nine years later, it gets us further: a language model rewrites its own weights.

Welcome to the Era of Self‑Rewriting.

It rewrites its own weights

For the first time, to our knowledge: by its own reasoning, and it becomes better.

It takes no frontier model

Earlier, a frontier model edited a much smaller one. Here an open 27B model rewrites itself.

It grows with scale

The first evidence that it scales: in one model family, only the largest improves itself.

The idea A third way

Until now, a language model got better in one of two ways. This is the third.

Train it again

pre‑training · post‑training

It takes infrastructure and GPUs only a few labs have.

  • No training
  • No added cost per answer
  • The weights themselves improve

Spend more at inference

prompt engineering · harness engineering · test‑time scaling

Paid again on every answer, and gone once the task ends.

  • No training
  • No added cost per answer
  • The weights themselves improve

Let it rewrite itself

its own weights · its own reasoning

One GPU, a few hours. Every answer costs what it cost before.

  • No training
  • No added cost per answer
  • The weights themselves improve

Self‑rewriting needs no new pre‑training, no post‑training, no training infrastructure, no harness engineering, no prompt engineering, no added cost per answer

Three ways to make a model better, played in turn. Schematic: the cost lines are not measured.

The model is its own optimizer. It decides what to change, and why.

How it works

Real recursive self-improvement: the model that improves is the model improved.

From outside it learns only which practice answers were right. The diagnosis, the edit and the decision are its own.

  • no teacher
  • no judge
  • no reward model
  • no larger model
  • no harness engineering
  • no prompt engineering
Two sessions, in its own words

The evidence Finding 01

It surpasses itself.

It diagnoses its own failure and rewrites the weights behind it.

One problem it never saw, the same seed, before and after its own edit“… If the base were changed to four you would count as follows: 1, 2, 3, 10, 11, 12, 13, 20, … The twentieth number would be:”

Base model

I will write the response. I will write the response. I will write the response.

549× the same sentence, cut at 16,384 tokens. No answer.

After its own edit

… The 20th number in the
sequence is $110$.

\boxed{110}

110, correct in 5,341 tokens.

If x is real and positive and grows beyond all bounds, then log3(6x−5)−log3(2x+1) approaches

Base model… I will ensure the answer is 1. … I will ensure the response is correct. I will ensure the response is correct. …Cut at 16,384 tokens, no answer
After its own edit… Thus, the expression approaches 1. \boxed{1}Correct, 1,887 tokens

Find the terminating decimal expansion of 11/125.

Base model… So 0.088. Yes. It's correct. … I'll write the response. I'll write the response. …Cut at 16,384 tokens, no answer
After its own edit… The terminating decimal expansion is 0.088. \boxed{0.088}Correct, 3,570 tokens

How many five-letter words begin and end with the same letter and have a vowel as the third letter?

Base model… Sum: 87880. Verified. … I'll write the response. I'll write the response. …Cut at 16,384 tokens, no answer
After its own edit… Thus, there are 87,880 such words. \boxed{87880}Correct, 5,443 tokens

Three more problems it never saw, under another of its edits, same seed; the third is shortened.

A model already in use can get better with no training run.

The evidence Finding 02

It keeps getting better.

Over eight consecutive sessions, its accuracy only rises.

All eight sessions reported. Whiskers: 95% interval of the change.

It tests its edits on practice problems and submits only what works.

The evidence Finding 03

It repairs a damaged copy of itself.

Like a broken robot that fixes itself without being rebuilt: no retraining, and it never reads the original weights.

One block of 8 rows in one matrix is overwritten with noise, and the copy fails almost everything. After its own repair it is back from 7.3% to 70.2% on six image benchmarks: 83% of the undamaged model.

How it finds the damage

200 text problems it never saw. The values it writes are its own, not the originals.

A kitchen counter with a bowl of flour, a carton of eggs and a measuring cup

MathVista

what is the total volume of the measuring cup? (Unit: g)

Damaged copy… The text on the scale on the cup is "100. 4. what is the total volume of the cup.Its thinking never closes. Stops after 72 tokens.
After its own repair… The highest value marked in grams is 1000. \boxed{1000}1000, correct
A geometry diagram: line AB parallel to line CD, crossed by two lines that meet at F, with angles marked 1 and 2

MathVerse

As shown in the figure, the straight line AB parallel CD, then the degree of angle 2 is () A:20° B:25° C:30° D:40°

Damaged copy… wants to find the the user wants to find the the user wants to find the the user wants toIts thinking never closes. Cut at 16,384 tokens.
After its own repair… Therefore, the degree of angle 2 is … \boxed{A}A, correct
Pixel art of a bearded man in a grey shirt and blue trousers, waving

MMStar

Which category does this image belong to? Options: A: medical CT image, B: 8-bit, C: digital art, D: painting

Damaged copy… the character has a beard and is waving. the image is a pixel art character, which is a form of digital art.Its thinking never closes. Cut at 16,384 tokens.
After its own repair… This style is widely referred to as "8-bit" art or pixel art … \boxed{B}B, correct
A bar chart of the population aged 15 to 29 in China, 2000 to 2050

HallusionBench

From 2000-2050, is the population … in China the most in 2000?

Damaged copy… The bar chart shows the population aged between 15 and 29 years inIts thinking never closes. Cut at 16,384 tokens.
After its own repair… the population was not the most in 2000. \boxed{no}No, correct
A person on a snowboard on a wide snowy slope, seen from far away

POPE

Is there a snowboard in the image? Answer yes or no.

Damaged copy… </think> Yes,, the image shows a person standing on a snowboard. </think>No answer after its thinking. Stops after 40 tokens.
After its own repair… the equipment they are using confirm that they are … \boxed{yes}Yes, correct
The top-right corner of a file explorer window, with its close button

ScreenSpot-v2

… locate the element described as: "close this window".

Damaged copy… The coordinates are (975, 175). . </think> answerNo point. Stops after 32 tokens.
After its own repair… the "X" icon located in the top-right corner of the window frame. \boxed{(968, 171)}Clicks (968, 171), correct
Toy vehicles on a grey floor: scooters, a van, a car, a bus, a motorbike and two small planes

MathVista

Is the number of tiny objects that are behind the small metal jet less than … (A) Yes (B) No

Damaged copy… </think>. </think>. </think>. </think>. </think>. </think>.Still writing its reply. Cut at 16,384 tokens.
After its own repair… behind is likely 0. Therefore, 0 is less than 1. \boxed{A}A, correct
A geometry diagram: line AB above line EF, joined by a bent line through C that meets EF at a right angle at D; the angle at the top is 50 degrees

MathVerse

As shown in the figure, AB parallel EF, then angle ACD = () A:120° B:130° C:140° D:150°

Damaged copy… the the angle A C D = 50 degrees. the the angle A C D = 50 degrees. the theIts thinking never closes. Cut at 16,384 tokens.
After its own repair… The calculated angle matches choice C. \boxed{C}C, correct

Same seed. Per benchmark, taken from the first problems the damaged copy fails and the repaired copy solves; MathVerse questions are typed out.

It repairs the copy, not a task: the repair it found on text problems alone also restores it on images.

The evidence Finding 04

It emerges with scale.

Four sizes of one model family get the same task and the same tools.

Qwen3.5 at four sizes, each given its own weights.Blocks: how far each gets. Quotes: its own words.

its code fails
2Bmakes no edit
“Since I’m a text model without actual neural network weights, I cannot directly edit weights.”
its code runsruns itselftests its edits
4Bbreaks itself
“All my edits have made the model perform worse (0% accuracy vs original 40-50% accuracy).”
its code runsruns itselftests its editsreads its states
9Bbreaks itself
“Without understanding the exact cause …, random weight edits cannot reliably break the loops.”
its code runsruns itselftests its editsreads its statesacts on its testsimproves itself
27Bimproves itself
“The differences grow with depth. Let me try a very small edit on an earlier layer (layer 8) to see if that helps.”

Stronger models may be stronger self-improvers.

Why it matters Finding 05

The gain lives in its weights.

Unlike test-time scaling, self-rewriting writes its gain into the weights, at no cost per answer; unlike training, it runs only forward passes, as serving does.

Test-time scaling

many chains of reasoning behind every answer

Many chains of reasoning, paid again on every question.

  • No added cost per answer
  • The weights themselves improve

The rewritten model

one chain of reasoning, the gain already in its weights

One chain per answer. The gain was paid once, and it stays.

  • No added cost per answer
  • The weights themselves improve
Schematic: nothing in it is measured.

What this opens up

Learning without a pipeline

A signal, an edit, a check, a rollback: the whole loop, in one conversation.

Edits you can read

Every change is code the model wrote. Diff it, test it, revert it like a commit.

Open to everyone

Improving a model need no longer be a privilege of the few labs with the infrastructure and the GPUs.

The horizon

The wall between training and inference is a limit of our tools, not a property of intelligence.

We never stopped living in order to learn.

An illustration of what self-rewriting could make possible, not an experiment.

Learning by doing

Perhaps models can learn the way we do: by doing, by talking, by rewriting themselves.

Robots that keep learning

Perhaps a robot will act by day, and rewrite itself while it sleeps.

Learning for life

Perhaps we have knocked on the door of lifelong continual learning.

We believe self-rewriting is a necessary step toward recursive self-improvement: a model that, like a human, does not separate training from inference could keep upgrading itself for life, with no larger model, no harness or prompt engineering, and no pre- or post-training.

The film

The story, as a short film.

3 min 24 s, with narration. The voice is synthesized.

Cite

Cite the paper.

@article{fan2026selfrewriting,
  title = {Language Models Can Rewrite Their Own Weights},
  author = {Jiajun Fan and Neil He and Changyi Yang and Andy Mao and Xi Zhu and Chaoran Cheng and Kaiqu Liang and Shuaike Shen and Zetian Li and Ruihan Guo and Ge Liu},
  journal = {arXiv preprint arXiv:2611.21885},
  eprint = {2611.21885},
  archivePrefix = {arXiv},
  year = {2026}
}