berdo26@interspeech_2026@ISCA

Total: 1

#1 Post-Training Speech Enhancement Language Models with Perceptual Rewards [PDF] [Copy] [Kimi] [REL]

Authors: Frédéric Berdoȥ, Luca A. Lanzendöerfer, Antonis Asonitis, Roger Wattenhofer

Speech enhancement language models achieve strong results when trained on discrete audio tokens, but their optimization relies on token-level cross-entropy rather than the perceptual metrics used for evaluation. We introduce a post-training stage for autoregressive speech enhancement language models using Group Sequence Policy Optimization (GSPO) with multi-metric perceptual rewards. Our method directly optimizes non-differentiable quality metrics (DNSMOS, WER, and UTMOS) as reward signals, without learned surrogates or offline preference pairs. Applied to two autoregressive base models, UniSE and GenSE, our approach achieves state-of-the-art results on the DNS2020 benchmark. A human evaluation ablation further shows that the composite multi-metric reward is preferred over any single-metric variant, confirming that multi-reward optimization avoids the reward hacking observed with single-metric training.

Subject: INTERSPEECH.2026 - Speech Processing