Kyutai Introduces "Voice of Reason": A Speech-Native Model Tackling Spoken Mathematics Through Reinforcement Learning

Kyutai has released Voice of Reason, two open-weight speech-to-speech models that solve math problems entirely out loud—no transcription step, no separate text LLM in the loop. Built on GLM-4-Voice-9B and refined with supervised fine-tuning (SFT) followed by reinforcement learning (RL), the models push spoken GSM8K accuracy from the base model's 27.3% up to 77.1%. It is, according to the team, the first application of RL to math reasoning in speech-native models.

Why Speech Models Struggle with Math

Cascaded pipelines—speech-to-text, a text LLM, then text-to-speech—still lead on reasoning benchmarks, but every stage adds latency and strips out paralinguistic cues like tone. Speech-native models face a different constraint: they must emit audio at regular intervals to stay interactive, which limits how many hidden reasoning tokens they can afford.

The starting point was low. Base GLM-4-Voice scores 27.3% on GSM8K, a widely used grade-school math benchmark. An earlier method called STITCH (Chiang et al.) had raised that to 58.7% by inserting reasoning chunks.

How the Two-Stage Training Works

GLM-4-Voice interleaves its output: 13 text tokens followed by 26 audio tokens, repeating.

Stage 1 — SFT. Training used 150,616 problems from Orca-Math, each rewritten for speech by Qwen3-235B and voiced by Kyutai's DSM TTS in many voices. SFT alone lifted accuracy from 27.3% to 61.7%.

Stage 2 — RL. For each spoken question, the model sampled 4 replies at temperature 0.9. A judge model, Qwen3-235B-A22B-2507, scored the decoded text with a binary reward—without ever seeing the reference answer. On 100 hand-checked cases it agreed with humans 88% of the time. Rewards were centered within each group, forming a group-relative REINFORCE objective related to GRPO but without PPO clipping or KL regularization. Training ran on 16 H100 GPUs over 1,500 updates.

Two design choices proved decisive:

• Temperature correction: dividing logits by the sampling temperature before the loss's log-softmax. Without it, GSM8K accuracy collapsed from 65.5% to 12.3%.
• Audio-token merging: summing all audio-vocabulary probabilities into one abstract token at each audio position, so the loss only asks whether audio came next. The paper proves this estimator is unbiased and lower-variance under a value-invariance assumption.

The Two Checkpoints and Their Scores

The released glm-4-voice-of-reason-9b answers directly, with any step-by-step working spoken aloud. glm-4-voice-of-reason-stitch-9b inserts silent 100-token reasoning chunks between spoken blocks; later chunks are generated while earlier speech plays, so thinking adds no extra latency. In the STITCH-R layout used here, the first reasoning chunk precedes the first spoken block.

On spoken GSM8K (top-k 50 decoding, averaged over 3 seeds):

Removing top-k decoding yields the released checkpoints' 70.3% and 77.1%. Kyutai notes the omni and cascaded systems are larger top lines, not matched comparisons.

Other findings worth noting: transcribed with Qwen3-ASR-1.7B, the Stitch model still scores 72.0 ± 1.9%, showing the gains survive real speech. Naturalness held steady—UTMOSv2 moved only from 4.067 to 4.069 (direct) and 4.174 to 4.164 (Stitch). The improvements don't come from longer answers: RL actually cut the direct model's average reply from 41.9 to 36.4 seconds, and Stitch reasoning tokens rose just from 167 to 176. RL also helps most when data is scarce—with 10% of SFT data, a longer RL run reached 58.5% versus 43.9% for SFT alone.

There are trade-offs. Spoken TriviaQA dropped from 40.6% to 34.0% for the direct model, attributed mainly to full-data SFT rather than RL. A contamination check dropped AddSub, MultiArith, SingleEQ and SVAMP from evaluation after finding 54.0% of 678 checked questions overlapped with Orca-Math at the paraphrase level. Evaluation audio came from GPT-4o-mini-TTS—a different TTS than training—and GPT-4o served as evaluation judge.

What This Stage Means

Voice of Reason currently sits at the "open release, self-hosting only" stage: Kyutai ran both BF16 checkpoints on a single H100, but you also need the GLM-4-Voice repo for its speech tokenizer and decoder. Weights inherit the GLM-4-Voice license, and no Hugging Face inference provider hosts them yet. While text-output omni models and cascaded systems still lead on raw accuracy, the gap is narrowing fast—the question to watch is whether RL-based speech-native reasoning closes it further in future releases.