2410.15573

Total: 1

#1 OpenMU: Your Swiss Army Knife for Music Understanding [PDF] [Copy] [Kimi] [REL]

Authors: Mengjie Zhao ; Zhi Zhong ; Zhuoyuan Mao ; Shiqi Yang ; Wei-Hsiang Liao ; Shusuke Takahashi ; Hiromi Wakaki ; Yuki Mitsufuji

We present OpenMU-Bench, a large-scale benchmark suite for addressing the data scarcity issue in training multimodal language models to understand music. To construct OpenMU-Bench, we leveraged existing datasets and bootstrapped new annotations. OpenMU-Bench also broadens the scope of music understanding by including lyrics understanding and music tool usage. Using OpenMU-Bench, we trained our music understanding model, OpenMU, with extensive ablations, demonstrating that OpenMU outperforms baseline models such as MU-Llama. Both OpenMU and OpenMU-Bench are open-sourced to facilitate future research in music understanding and to enhance creative music production efficiency.

Subjects: Sound ; Artificial Intelligence ; Computation and Language ; Multimedia ; Audio and Speech Processing

Publish: 2024-10-21 01:36:42 UTC