ren26d@interspeech_2026@ISCA

Total: 1

#1 AuDirector: A Self-Reflective Closed-Loop Framework for Immersive Audio Storytelling [PDF] [Copy] [Kimi] [REL]

Authors: Yiming Ren, Ziyang Zhang, Wen Wu, Baoxiang Li, Chao Zhang, Xuenan Xu

Despite progress in text and visual generation, coherent long-form audio storytelling remains challenging. Existing systems often suffer from mismatches between character settings and voice performance, weak self-correction, and limited user interaction. We propose AuDirector, a self-reflective closed-loop multi-agent framework for audio narrative generation. Its identity-aware pre-production mechanism converts narratives into character profiles and utterance-level emotion instructions, retrieves suitable voices, and guides expressive speech synthesis. A collaborative synthesis and correction module audits and regenerates defective audio via closed-loop self-correction. A human-guided interactive refinement module further interprets natural language feedback to revise scripts interactively. Experiments show that AuDirector outperforms state-of-the-art baselines in structural coherence, emotional expressiveness, and acoustic fidelity. Samples: https://github.com/Riddae/AuDirector.

Subject: INTERSPEECH.2026 - Speech Synthesis