TL;DR — Audio LLMs trained only on outcome rewards produce hallucinatory, inconsistent, and unscalable reasoning chains. CESAR instead rewards the reasoning process itself via online RL (GRPO), resolving test-time inverse scaling and reaching SOTA on MMAU — outperforming Gemini 2.5 Pro and GPT-4o Audio.
The problem: test-time inverse scaling
Adding chain-of-thought reasoning to Audio LLMs often degrades performance — a phenomenon we term test-time inverse scaling. Longer reasoning chains yield progressively worse results, because outcome-only rewards leave the reasoning process completely unsupervised.
Without process rewards
The reasoning chain is invisible to the reward. Models produce hallucinatory, inconsistent reasoning that accumulates errors over longer chains — reasoning becomes a liability.
With CESAR
Process rewards score consistency, structured analytical patterns, causal reasoning, and calibrated depth — turning reasoning into a genuine, scalable capability.
Method
CESAR uses Group Relative Policy Optimization (GRPO) with a multi-faceted reward suite that goes beyond simple correctness — every component targets a specific failure mode of unsupervised reasoning.
1
Correctness
Standard outcome reward — is the final answer right?
2
Consistency
Reasoning ↔ question and reasoning ↔ answer alignment.
Overthinking penalty — calibrate reasoning length to the task.
Figure 1. Outcome-only RL (top) rewards only final-answer correctness and format. CESAR (bottom) adds process rewards — consistency, structure, and an overthinking penalty — on the reasoning chain itself.
Results
SOTA
MMAU Test-mini
> 2.5 Pro
beats Gemini 2.5 Pro
> GPT-4o
beats GPT-4o Audio
+0.038
positive scaling slope
Model
MMAU (total)
Test-time scaling
Qwen2.5-Omni-7B (base)
65.2%
Inverse (slope −0.51)
Ke-Omni-R (outcome-only RL)
74.6%
Flat (slope −0.007)
CESAR w/o overthinking penalty
76.5%
Positive
CESAR (Ours)
77.1% Best
Positive (slope +0.038)
Figure 2. Per-category MMAU accuracy. CESAR leads the base model (Qwen2.5-Omni-7B) and the outcome-only baseline (Ke-Omni-R) on Sound, Music, Speech and overall; its no-penalty variant does too, except on Music where it matches Ke-Omni-R — process rewards improve perception, not just reasoning.
Test-time scaling
With process rewards, more reasoning stops hurting and starts helping. The base model collapses as reasoning length grows; CESAR stays flat-high and even rises.
Figure 3. Accuracy vs. average reasoning tokens on MMAU. Qwen2.5-Omni-7B suffers inverse scaling — accuracy collapses — while CESAR, its no-penalty variant, and Ke-Omni-R stay flat and high.Figure 4. Fitted scaling slopes (Δaccuracy per reasoning token). Qwen2.5-Omni-7B is strongly negative (−0.51); Ke-Omni-R is flat (−0.007); only CESAR is clearly positive (+0.038) — reasoning genuinely helps.
Publication journey
Oct 2025
arXiv preprint released
arXiv:2510.20867.
Oct 2025
Submitted to ICLR 2026
The Fourteenth International Conference on Learning Representations (Submission #8335).
@inproceedings{fan2026incentivizing,
title = {Incentivizing Consistent, Effective and Scalable Reasoning
Capability in Audio {LLM}s via Reasoning Process Rewards},
author = {Jiajun Fan and Roger Ren and Jingyuan Li and Rahul Pandey and
Prashanth Gurunath Shivakumar and Ivan Bulyko and Ankur Gandhe
and Ge Liu and Yile Gu},
booktitle = {The Fourteenth International Conference on Learning Representations},
year = {2026},
url = {https://openreview.net/forum?id=DUr48hxO2h}
}