Outcome-only RL
rewards the answer and its format
B is rewarded as much as A, so its broken reasoning is reinforced too.
- Checks the answer and its format
- Checks the reasoning
- Tells sound reasoning from a lucky guess
At a glance
Why do audio language models get worse the longer they think? Their training never rewards how they reason. Reward the reasoning itself, and thinking turns from a liability into a gain.
Where the base model loses accuracy by reasoning, CESAR gains it, up to a sweet spot.
Finding 02A 7B model ahead of Gemini 2.5 Pro and GPT‑4o Audio.
Finding 03A GPT‑4o Audio judge prefers its reasoning over both baselines.
Finding 04It beats its base model even without reasoning, and has the best perception average of any model on MMSU.
The problem
Chain-of-thought helps language models on text. In audio it backfires: the longer the base model reasons, the lower its accuracy, and a model trained with outcome-only RL gains nothing clear from reasoning.
The paper’s test-time scaling sweep: the limit on thinking length is raised step by step, and each point is placed at the average length the model actually used.
The culprit is not reasoning itself. It is training that never looks at the reasoning.
The idea From outcomes to process
Outcome-only RL cannot tell a sound chain of reasoning from a lucky one. CESAR scores how the model got to its answer, so sound reasoning is what gets reinforced.
rewards the answer and its format
B is rewarded as much as A, so its broken reasoning is reinforced too.
also rewards consistency, structure and calibrated length
Sound reasoning ranks first, and that is what GRPO reinforces.
Reasoning stops being a side effect and becomes a skill the model is trained for.
How it works
For each training question the model writes eight answers. Each gets one score, the weighted sum of five terms, and GRPO reinforces the answers that beat the group’s average.
MMAU Test-mini questions. The highlights show what each term looks at; they illustrate the rewards, they are not the computed scores.
MMAU Test-mini. The paper’s progressive ablation, read from the base model up; every model reasons, at its own sweet spot.
The three reasoning terms target the ways reasoning fails: drifting from the question, contradicting its own answer, wandering without structure, overthinking.
The evidence Finding 01
Turn reasoning on, and the base model loses accuracy while CESAR gains it. Give it more room to think, and its accuracy peaks at a sweet spot.
MMAU Test-mini, total accuracy. The sweep raises the thinking-length limit from 0 to 250 in steps of 25; the second chart shows each model without reasoning and with reasoning at its best length.
A model must first learn to think well before thinking more pays off.
The evidence Finding 02
With 7B parameters, CESAR is ahead of Gemini 2.5 Pro, Gemini 2.5 Flash and GPT‑4o Audio on MMAU Test-mini, a benchmark that shares nothing with its training data.
Qwen2.5-Omni-7B and Ke-Omni-R were run by the paper under the same protocol, each shown in its better mode; the other scores come from the MMAU paper. CESAR also has the best Sound and Music scores in the paper’s table.
The gain comes from scaling the reasoning, not the parameter count.
The evidence Finding 03
Asked which of two reasoning traces is more logical and more faithful to the audio, a GPT‑4o Audio judge picks CESAR’s most of the time.
MMAU Test-mini. The judge sees the audio, the question, the choices, the correct answer and both traces; ties are split evenly between the two.
Against the base model
Based on the given audio, identify the source of the speech.man · woman · child · robot
Answers robot: it invents a missing choiceAnswers man, correctAgainst outcome-only RL
What is the characteristic of the chord played from 0:02 to 0:03 in the audio?A:min/P5 · E:maj/P1 · D:maj/M3 · G:maj/P1
Answers E:maj/P1: it goes in circlesAnswers A:min/P5, correctAgainst CESAR without the overthinking penalty
What is the sound event taking place in the audio?a person crying · laughing · coughing · yawning
Answers crying: it thinks itself into a mistakeAnswers laughing, correctAgainst outcome-only RL
How many times does the telephone ring in the audio?2 · 4 · 5 · 3
Answers 2: its reasoning says threeAnswers 3, correctAgainst outcome-only RL
Given that an infant is crying, name a famous person known for their association with the animal heard in the background.George Orwell · Ernest Hemingway · Charles Darwin · Charles Dickens
Answers “Ernest Hemmanway”: a made-up factAnswers Ernest Hemingway, correctAgainst the base model
What kind of event could be taking place in the audio?a quiet library · a heated argument · a military parade · a gentle breeze
Answers a heated argument: a leap from “noise”Answers a military parade, correctReasoning traces from the paper’s qualitative comparison on MMAU Test-mini; … marks a cut. Choices are shortened where the full option is longer.
Its answers follow from its reasoning, and its reasoning follows the audio.
The evidence Finding 04
Trained to reason well, it also answers better than its base model when it does not reason at all. On MMSU it has the best perception and reasoning averages of any model, and passes humans on semantic reasoning.
MMSU separates perception tasks from reasoning tasks. Ke-Omni-R is the paper’s own run; the other MMSU scores come from the MMSU paper.
Better thinking, it turns out, leads to better hearing.
Why it matters Finding 05
One reward term changes how the model reasons. Without the overthinking penalty it becomes a depth specialist that does best with long chains on hard questions; with it, a calibrated generalist that peaks with short ones and gains more from reasoning.
MMAU Test-mini, from the same sweep as Finding 01. Dashed: each model without reasoning.
First teach the model to reason well. Then sweep its thinking length at test time to find the sweet spot, with no further training.
A 7B model ahead of proprietary systems, by scaling how it reasons rather than how big it is.
A judge that compares reasoning traces head to head shows what accuracy alone cannot.
Cite
International Conference on Learning Representations 2026 (ICLR 2026). Featured on Amazon Science.
@inproceedings{fan2026incentivizing,
title = {Incentivizing Consistent, Effective and Scalable Reasoning
Capability in Audio {LLM}s via Reasoning Process Rewards},
author = {Jiajun Fan and Roger Ren and Jingyuan Li and Rahul Pandey and
Prashanth Gurunath Shivakumar and Ivan Bulyko and Ankur Gandhe
and Ge Liu and Yile Gu},
booktitle = {The Fourteenth International Conference on Learning Representations},
year = {2026},
url = {https://openreview.net/forum?id=DUr48hxO2h}
}