When should animated agents give additional instructions to users? - Monitoring user's understanding in multimodal dialogues -

Kazuyoshi Murata, Mika Enomoto, Yoshiko Arimoto, Yukiko Nakano

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

In multimodal communication, verbal and nonverbal behaviors such as gestures and manipulating objects in a workspace occur in parallel, and are coordinated in proper timing to each other. This paper focuses on the interaction between a beginner user using a video recorder application on PC and a multimodal animated help agent, and presents a probabilistic model of fine-grained timing dependencies among different behaviors of different modalities. First, we collect user-agent dialogues using a Wizard-of-Oz experimental setting, and then the collected verbal and nonverbal behavior data will be used to build a Bayesian network model, which can predict the likelihood of successful mouse clicks in near future, given evidence associated with the status of speech, agent's gestures and user's mouse actions. Finally, we attempt to determine proper timing when the agent should give additional instructions by estimating the likelihood of a mouse click occurrence.

Original languageEnglish
Title of host publicationICCAS 2007 - International Conference on Control, Automation and Systems
Pages733-736
Number of pages4
DOIs
Publication statusPublished - 2007 Dec 1
EventInternational Conference on Control, Automation and Systems, ICCAS 2007 - Seoul, Korea, Republic of
Duration: 2007 Oct 172007 Oct 20

Publication series

NameICCAS 2007 - International Conference on Control, Automation and Systems

Other

OtherInternational Conference on Control, Automation and Systems, ICCAS 2007
CountryKorea, Republic of
CitySeoul
Period07/10/1707/10/20

Keywords

  • Animated agent
  • Bayesian network
  • Instruction dialogues

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'When should animated agents give additional instructions to users? - Monitoring user's understanding in multimodal dialogues -'. Together they form a unique fingerprint.

  • Cite this

    Murata, K., Enomoto, M., Arimoto, Y., & Nakano, Y. (2007). When should animated agents give additional instructions to users? - Monitoring user's understanding in multimodal dialogues -. In ICCAS 2007 - International Conference on Control, Automation and Systems (pp. 733-736). [4406995] (ICCAS 2007 - International Conference on Control, Automation and Systems). https://doi.org/10.1109/ICCAS.2007.4406995