fujita24b@interspeech_2024@ISCA

Total: 1

#1 Lightweight Zero-shot Text-to-Speech with Mixture of Adapters [PDF1] [Copy] [Kimi2] [REL]

Authors: Kenichi Fujita ; Takanori Ashihara ; Marc Delcroix ; Yusuke Ijima

The advancements in zero-shot text-to-speech (TTS) methods, based on large-scale models, have demonstrated high fidelity in reproducing speaker characteristics. However, these models are too large for practical daily use. We propose a lightweight zero-shot TTS method using a mixture of adapters (MoA). Our proposed method incorporates MoA modules into the decoder and the variance adapter of a non-autoregressive TTS model. These modules enhance the ability to adapt a wide variety of speakers in a zero-shot manner by selecting appropriate adapters associated with speaker characteristics on the basis of speaker embeddings. Our method achieves high-quality speech synthesis with minimal additional parameters. Through objective and subjective evaluations, we confirmed that our method achieves better performance than the baseline with less than 40% of parameters at 1.9 times faster inference speed.