AudioRSI: Recursively Self-Improving Audio Reasoners

TL;DR — Reinforcement learning has made audio LLMs better reasoners, but the reward still comes from outside the model, and an external judge stays frozen while the student it trains keeps improving. In AudioRSI one model is both student and teacher: it writes reasoning paths, then, hearing the same recording, judges them pair by pair; every 200 steps the improved student becomes the teacher of its successor. To our knowledge, it is the first recursively self-improving audio LLM.

WATCH THE FILM

The problem: a teacher that never learns

In audio RL, the reward that judges reasoning is always supplied from outside the policy. A final-answer check gives a lucky guess and a sound derivation the same credit. A hand-crafted process score adds terms that must be tuned against each other. An external LLM judge gives richer feedback, but it is frozen: the standard it enforces does not move while the student keeps improving.

The key finding: the student surpasses its teacher

Train an audio LLM with a frozen external judge as its teacher. Here the teacher is Ke-Omni-R, an outcome-only RL model on the same backbone, which scores 74.6% on MMAU test-mini. Its student ends at 75.1%, above the teacher that trained it: a teacher does not have to be better than its student to help it.

So the student can teach itself, and keep doing so. Let the improved student become the teacher, again and again: that is AudioRSI, at 76.1%.

Method: one model, two roles

AudioRSI runs inside GRPO. For each audio question the model, as the student (actor), writes K = 8 reasoning paths. The same model, as the teacher (critic), hears the same recording and compares the paths pair by pair: win, tie or lose. A path’s self-critique reward is how often it beats the rest of its group, added to the usual reward for a correct answer. There is no hand-crafted process reward and no external judge.

The teacher is a copy of the student’s weights, held fixed between handovers so that the reward does not drift from one step to the next. Every T = 200 steps the current student replaces it, but only if its accuracy has not regressed, so the teacher’s role never passes to a weaker checkpoint. A 6,400-step run has 32 such sync points: the teacher learns too, so student and teacher co-evolve.

A teacher that listens

A text-only judge has to take a claim like “I hear a train” at face value. AudioRSI’s teacher hears the recording, so it can reject reasoning that cites a sound the audio does not contain, instead of rewarding reasoning that merely sounds plausible.

One of four double-blind audit cases in the paper; in every case the teacher’s choice matched all three annotators. Four cases illustrate the behavior; they are not an agreement study.

Why the recursion matters

Keep everything else fixed and change only whether the teacher is refreshed. With a frozen teacher, accuracy on MMAU test-mini, which is never used for training, climbs and then stalls. When the student keeps becoming the teacher, accuracy keeps rising, and AudioRSI stays ahead at every checkpoint in the second half of training.

How often to hand over matters too. Hand over too often (T = 50), close to a reward that shifts with every update, and the whole gain over outcome-only training is lost; too rarely (T = 1000, the closest setting to a frozen copy) and most of it is. Periods of 200 to 500 steps work best.

Results

All four benchmarks are out of distribution: none of them overlaps the AVQA training data. AudioRSI is a 7B open model with no external reward model.

Best among the systems in the paper’s main MMAU table. AudioRSI trails on Music, where Audio Flamingo 3 leads. A variant of Omni-R1 trained on different data (VGGSound-GPT instead of AVQA) reports 77.0%.

The clearest lead is on the hardest split of MMAU-Pro, where a question needs inference across several recordings at once. There AudioRSI is ahead of every system tested, and Gemini 2.5 Flash, the strongest system on MMAU-Pro overall, sits at the bottom of the column.

On MMSU, AudioRSI ranks first overall by a few tenths of a point, and its edge over outcome-only RL is on the Reasoning split (80.54% vs. 78.06%). On MMAR it leads the 7B open-source tier among methods without external reward models, with its clearest margin on Music.

Reasoning nobody asked for

No reward term asks for any connective, template or phrase. But the teacher prefers traces that are explicit and grounded in what was heard, and the student moves toward them. Across 1,000 MMAU test-mini reasoning paths per model, every tracked behavior is most frequent under AudioRSI:

An outside judge agrees. GPT-4o Audio, never used in training, prefers AudioRSI’s reasoning in 81.2% of 1,000 order-randomized comparisons against the base model and 74.7% against outcome-only RL. That judge scores below all three models on MMAU and may favor explicit wording, so we read it as corroboration, not proof.

What this does not show yet

Each configuration is trained once, with a single seed, on one backbone and one training corpus, so where AudioRSI leads by a fraction of a point, as on MMSU, it is comparable rather than decisively better. Testing the recursion across backbones and data is the natural next step. We expect it to carry over wherever the teacher can perceive what the reasoning is about, and to weaken where it must take the reasoning’s premises on trust.

AudioRSI follows CESAR, which showed that rewarding the reasoning process, not just the answer, makes audio reasoning scale. CESAR’s process rewards were written by hand; in AudioRSI the model supplies them itself.

BibTeX

@article{fan2026audiorsi,
  title   = {{AudioRSI}: Recursively Self-Improving Audio Reasoners},
  author  = {Jiajun Fan and Roger Ren and Jingyuan Li and Rahul Pandey and
             Prashanth Gurunath Shivakumar and Ankur Gandhe and Yile Gu and
             Ge Liu and Ivan Bulyko},
  journal = {arXiv preprint arXiv:2610.36578},
  year    = {2026}
}