Does Scaling Reinforcement Learning Really Require More Training?
Published in arXiv preprint, 2026
Recommended citation: Bangji Yang, Jiajun Fan, Hongbo Ma, Ruihan Guo, Ge Liu. "Does Scaling Reinforcement Learning Really Require More Training?" arXiv preprint arXiv:2610.01133, 2026. https://arxiv.org/abs/2610.01133
A finished RL training history can yield policies stronger than any checkpoint its optimizer visited, with no more training and no more inference per query. SURGE fuses two checkpoints of the same run, an accurate anchor and a donor that answers more briefly, in the spectral space of their updates; it is evaluated on two 1.5B math-reasoning histories and a 7B coding history.