N18-5018@ACL

Total: 1

#1 SMILEE: Symmetric Multi-modal Interactions with Language-gesture Enabled (AI) Embodiment [PDF] [Copy] [Kimi] [REL]

Authors: Sujeong Kim ; David Salter ; Luke DeLuccia ; Kilho Son ; Mohamed R. Amer ; Amir Tamrakar

We demonstrate an intelligent conversational agent system designed for advancing human-machine collaborative tasks. The agent is able to interpret a user’s communicative intent from both their verbal utterances and non-verbal behaviors, such as gestures. The agent is also itself able to communicate both with natural language and gestures, through its embodiment as an avatar thus facilitating natural symmetric multi-modal interactions. We demonstrate two intelligent agents with specialized skills in the Blocks World as use-cases of our system.