TL;DR — An audio LLM is a black box in a specific way: we see what it says, never what it worked out on the way there, and chain-of-thought monitoring only helps if the model writes its reasoning down. Reading a base Qwen3-Omni with a logit lens at the audio-token positions, the answer to a spoken question becomes legible — in words — in the middle layers, before a single token is emitted.
WATCH THE FILMHow to read it. Each column is a layer, each row a position in the input; the audio-token positions come before the question and options and cannot see them. Before anything is emitted, the logit lens takes the hidden states at the audio-token positions of a middle layer and passes them through the model’s own output head, so they read out as a word — here 美国总统 (“US president”), the role that reaches rank 1 at the audio positions of the Nixon clip below. The answer slot is still empty: nothing has been said yet. Layers and positions are drawn for illustration. Schematic.
A hidden multi-hop chain, with no chain-of-thought
On a clip whose verbatim transcription is empty garbling, the readout reconstructs Watergate and scandal; then the person and the role surface together: Nixon and president both appear across the transition into the middle layers, the role persisting there as Chinese 总统 — an event-then-actor chain, not a strict word-by-word order. Watergate and scandal appear in neither the question, the options, nor the model's own transcription; Nixon is a printed option, but it is decoded over the audio positions, not merely read off the choice. The reasoning happened; it simply was never written down.
How to read it. The question asks which president was forced to resign; the model's own transcription of the clip is empty garbling. Before it answers, a logit lens at the audio positions already reads the event (water + gate, scandal) in the early layers; Nixon and the role president both by about a third of the depth, the role persisting through the middle layers (event, then actor; not a strict word-by-word order). Schematic of the paper's flagship clip; layer positions are approximate.Figure 1. Where the task-relevant concept becomes readable, for seven curated cases. The strongest readout lands in the middle band of the network — and the word that surfaces is often not in the language of the question.
What the workspace looks like
How to read it. Reading an audio LLM with a logit lens at the audio positions, the middle-layer state is verbalizable, multilingual (38% of top-1 readouts are Chinese on English inputs), paralinguistic, and causal under activation patching. Schematic; words from the paper's Nixon case.
A waveform-swap control runs throughout: identical text, only the sound changed. That is what separates a genuinely audio-driven signal from a prior over the printed answer options.
How to read it. The two columns on the right are two possible explanations, not two outputs of the model. The control keeps the printed text identical and changes only the sound. A readout that only reflects a prior over the printed answer options cannot tell the two inputs apart and would come out the same (dashed); a genuinely audio-driven readout differs between the sounds. In the paper the readout follows the sound: it is best with the real clip, weaker with a mismatched clip, and at chance with silence. Option letters are placeholders. Schematic.
Deleting layers maps the pipeline
Reading is correlational; deleting is causal. Bypassing one layer at a time gives a clean functional map: listening is localized at the entry, delivery at the output, and retrieval is spread across the interior — no single interior layer is essential.
Figure 2. Accuracy lost when a single layer is deleted (identity skip, averaged over clips). Only the encoder-to-text entry layer is critical; every interior layer costs at most ten points.
The same tooling shows where hallucinations come from: in text mode the wrong concept usually takes over early, and is never revised afterwards.
Why it matters, and what it is not
If what a model is about to say is legible mid-stack, monitoring need not wait for it to speak, and need not trust it to narrate its own reasoning. But this is a deliberately qualitative account of one model family: a readable correlate is not a complete mechanism, and the paper is conservative about what the lens can and cannot establish.
BibTeX
@article{fan2026readmind,
title = {Can We Read the Mind of an Audio {LLM}? A Verbalizable,
Multilingual Middle-Layer Workspace},
author = {Jiajun Fan and Jingyuan Li and Prashanth Gurunath Shivakumar
and Qi Luo and Jia-Hong Huang and M. Maruf and Roger Ren
and Yile Gu and Rahul Pandey and Ge Liu and Ivan Bulyko},
journal = {arXiv preprint arXiv:2608.24958},
year = {2026}
}