2409.11003

Total: 1

#1 Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillation [PDF4] [Copy] [Kimi] [REL]

Authors: Gerard I. Gállego, Roy Fejgin, Chunghsin Yeh, Xiaoyu Liu, Gautam Bhattacharya

Audio token modeling has become a powerful framework for speech synthesis, with two-stage approaches employing semantic tokens remaining prevalent. In this paper, we aim to simplify this process by introducing a semantic knowledge distillation method that enables high-quality speech generation in a single stage. Our proposed model improves speech quality, intelligibility, and speaker similarity compared to a single-stage baseline. Although two-stage systems still lead in intelligibility, our model significantly narrows the gap while delivering comparable speech quality. These findings showcase the potential of single-stage models to achieve efficient, high-quality TTS with a more compact and streamlined architecture.

Subjects: Sound , Artificial Intelligence , Audio and Speech Processing , Signal Processing

Publish: 2024-09-17 09:08:43 UTC