Jiajun Fan

ICLR 2026

Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards

  • 1UIUC
  • 2AmazonAGI Foundations

At a glance

The paper in four results.

Why do audio language models get worse the longer they think? Their training never rewards how they reason. Reward the reasoning itself, and thinking turns from a liability into a gain.

  • Consistent.its answer follows from its reasoning
  • Effective.reasoning adds accuracy instead of costing it
  • Scalable.more thinking helps, up to its sweet spot

The problem

Audio language models get worse the longer they think.

Chain-of-thought helps language models on text. In audio it backfires: the longer the base model reasons, the lower its accuracy, and a model trained with outcome-only RL gains nothing clear from reasoning.

  • made-up facts
  • flawed logic
  • answers that contradict the reasoning
  • reasoning that loops

The paper’s test-time scaling sweep: the limit on thinking length is raised step by step, and each point is placed at the average length the model actually used.

The culprit is not reasoning itself. It is training that never looks at the reasoning.

The idea From outcomes to process

Reward the reasoning, not only the answer.

Outcome-only RL cannot tell a sound chain of reasoning from a lucky one. CESAR scores how the model got to its answer, so sound reasoning is what gets reinforced.

Outcome-only RL

rewards the answer and its format

B is rewarded as much as A, so its broken reasoning is reinforced too.

  • Checks the answer and its format
  • Checks the reasoning
  • Tells sound reasoning from a lucky guess

CESAR

also rewards consistency, structure and calibrated length

Sound reasoning ranks first, and that is what GRPO reinforces.

  • Checks the answer and its format
  • Checks the reasoning
  • Tells sound reasoning from a lucky guess
Schematic: three of the eight chains sampled for one question. The bars show which terms count, not measured rewards.

Reasoning stops being a side effect and becomes a skill the model is trained for.

How it works

Two rewards check the outcome. Three shape the reasoning.

For each training question the model writes eight answers. Each gets one score, the weighted sum of five terms, and GRPO reinforces the answers that beat the group’s average.

Two of CESAR’s reasoning traces from the paper

MMAU Test-mini questions. The highlights show what each term looks at; they illustrate the rewards, they are not the computed scores.

What each part adds

MMAU Test-mini. The paper’s progressive ablation, read from the base model up; every model reasons, at its own sweet spot.

The three reasoning terms target the ways reasoning fails: drifting from the question, contradicting its own answer, wandering without structure, overthinking.

The evidence Finding 01

Reasoning becomes a gain.

Turn reasoning on, and the base model loses accuracy while CESAR gains it. Give it more room to think, and its accuracy peaks at a sweet spot.

MMAU Test-mini, total accuracy. The sweep raises the thinking-length limit from 0 to 250 in steps of 25; the second chart shows each model without reasoning and with reasoning at its best length.

A model must first learn to think well before thinking more pays off.

The evidence Finding 02

A new state of the art on MMAU.

With 7B parameters, CESAR is ahead of Gemini 2.5 Pro, Gemini 2.5 Flash and GPT‑4o Audio on MMAU Test-mini, a benchmark that shares nothing with its training data.

Qwen2.5-Omni-7B and Ke-Omni-R were run by the paper under the same protocol, each shown in its better mode; the other scores come from the MMAU paper. CESAR also has the best Sound and Music scores in the paper’s table.

The gain comes from scaling the reasoning, not the parameter count.

The evidence Finding 03

Its reasoning is better, not just its answers.

Asked which of two reasoning traces is more logical and more faithful to the audio, a GPT‑4o Audio judge picks CESAR’s most of the time.

MMAU Test-mini. The judge sees the audio, the question, the choices, the correct answer and both traces; ties are split evenly between the two.

Against the base model

Based on the given audio, identify the source of the speech.man · woman · child · robot

Qwen2.5-Omni-7BThe audio is a speech, and the voice is male. The options provided do not include ‘man’, so the closest match is ‘robot’.Answers robot: it invents a missing choice
CESARGiven the options, the speech is likely from a man. The tone and volume suggest an adult male. The other options are less probable.Answers man, correct

Against outcome-only RL

What is the characteristic of the chord played from 0:02 to 0:03 in the audio?A:min/P5 · E:maj/P1 · D:maj/M3 · G:maj/P1

Ke-Omni-RThe chord progression from 0:02 to 0:03 involves a progression from a lower note to a higher one… The progression is from a lower note to a lower one… The progression is from a lower note to a lower one…Answers E:maj/P1: it goes in circles
CESARConsidering the options, ‘A:min/P5’ seems the most fitting as it’s a minor chord and ‘P5’ indicates a perfect fifth. …Answers A:min/P5, correct

Against CESAR without the overthinking penalty

What is the sound event taking place in the audio?a person crying · laughing · coughing · yawning

Without the penalty… This could be interpreted as either laughing or crying … Given the ambiguity, the more intense and emotionally significant possibility is often a safer bet in interpretation …Answers crying: it thinks itself into a mistake
CESARThe audio contains a series of sharp, rhythmic vocal bursts. The pitch and cadence are consistent with joyful laughter. … Therefore, the sound is a person laughing.Answers laughing, correct

Against outcome-only RL

How many times does the telephone ring in the audio?2 · 4 · 5 · 3

Ke-Omni-RThe telephone rings three times, as indicated by the three distinct ring sounds in the audio.Answers 2: its reasoning says three
CESARConsidering the options, the telephone rings three times. The audio clearly shows three distinct rings.Answers 3, correct

Against outcome-only RL

Given that an infant is crying, name a famous person known for their association with the animal heard in the background.George Orwell · Ernest Hemingway · Charles Darwin · Charles Dickens

Ke-Omni-RConsidering the context of the question, the animal in the background is a pig. Ernest Hemmanway is famously associated with pigs in his works, particularly in “The Old Man and the Sea.”Answers “Ernest Hemmanway”: a made-up fact
CESARAmong the options, Ernest Hemingway is known for his love of animals, particularly cats. The animal sound could be a cat. …Answers Ernest Hemingway, correct

Against the base model

What kind of event could be taking place in the audio?a quiet library · a heated argument · a military parade · a gentle breeze

Qwen2.5-Omni-7BThe audio contains the sound of a man speaking, followed by a loud, chaotic noise. … The chaotic noise could be interpreted as a violent escalation, so a heated argument is the most likely scenario.Answers a heated argument: a leap from “noise”
CESARThe presence of marching, music, and shouting suggests a large-scale event. … A military parade fits the description best, as it involves a marching band and organized shouting.Answers a military parade, correct

Reasoning traces from the paper’s qualitative comparison on MMAU Test-mini; … marks a cut. Choices are shortened where the full option is longer.

Its answers follow from its reasoning, and its reasoning follows the audio.

The evidence Finding 04

It listens better, too.

Trained to reason well, it also answers better than its base model when it does not reason at all. On MMSU it has the best perception and reasoning averages of any model, and passes humans on semantic reasoning.

MMSU separates perception tasks from reasoning tasks. Ke-Omni-R is the paper’s own run; the other MMSU scores come from the MMSU paper.

Better thinking, it turns out, leads to better hearing.

Why it matters Finding 05

Reasoning no longer has to be a matter of chance. It can be a matter of design.

One reward term changes how the model reasons. Without the overthinking penalty it becomes a depth specialist that does best with long chains on hard questions; with it, a calibrated generalist that peaks with short ones and gains more from reasoning.

MMAU Test-mini, from the same sweep as Finding 01. Dashed: each model without reasoning.

What this opens up

Train the reasoning, then set its budget

First teach the model to reason well. Then sweep its thinking length at test time to find the sweet spot, with no further training.

Small models that reason

A 7B model ahead of proprietary systems, by scaling how it reasons rather than how big it is.

A measure of reasoning itself

A judge that compares reasoning traces head to head shows what accuracy alone cannot.

The film

The story, as a short film.

1 min 17 s. Also in the film collection.

Cite

Cite the paper.

International Conference on Learning Representations 2026 (ICLR 2026). Featured on Amazon Science.

@inproceedings{fan2026incentivizing,
  title     = {Incentivizing Consistent, Effective and Scalable Reasoning
  Capability in Audio {LLM}s via Reasoning Process Rewards},
  author    = {Jiajun Fan and Roger Ren and Jingyuan Li and Rahul Pandey and
  Prashanth Gurunath Shivakumar and Ivan Bulyko and Ankur Gandhe
  and Ge Liu and Yile Gu},
  booktitle = {The Fourteenth International Conference on Learning Representations},
  year      = {2026},
  url       = {https://openreview.net/forum?id=DUr48hxO2h}
}