2512.02593

Total: 1

#1 Spoken Conversational Agents with Large Language Models [PDF4] [Copy] [Kimi6] [REL]

Authors: Chao-Han Huck Yang, Andreas Stolcke, Larry Heck

Spoken conversational agents are converging toward voice-native LLMs. This tutorial distills the path from cascaded ASR/NLU to end-to-end, retrieval-and vision-grounded systems. We frame adaptation of text LLMs to audio, cross-modal alignment, and joint speech-text training; review datasets, metrics, and robustness across accents and compare design choices (cascaded vs. E2E, post-ASR correction, streaming). We link industrial assistants to current open-domain and task-oriented agents, highlight reproducible baselines, and outline open problems in privacy, safety, and evaluation. Attendees leave with practical recipes and a clear systems-level roadmap.

Subjects: Computation and Language , Multiagent Systems , Neural and Evolutionary Computing , Sound , Audio and Speech Processing

Publish: 2025-12-02 10:02:10 UTC