TY - GEN
T1 - When should animated agents give additional instructions to users? - Monitoring user's understanding in multimodal dialogues -
AU - Murata, Kazuyoshi
AU - Enomoto, Mika
AU - Arimoto, Yoshiko
AU - Nakano, Yukiko
PY - 2007/12/1
Y1 - 2007/12/1
N2 - In multimodal communication, verbal and nonverbal behaviors such as gestures and manipulating objects in a workspace occur in parallel, and are coordinated in proper timing to each other. This paper focuses on the interaction between a beginner user using a video recorder application on PC and a multimodal animated help agent, and presents a probabilistic model of fine-grained timing dependencies among different behaviors of different modalities. First, we collect user-agent dialogues using a Wizard-of-Oz experimental setting, and then the collected verbal and nonverbal behavior data will be used to build a Bayesian network model, which can predict the likelihood of successful mouse clicks in near future, given evidence associated with the status of speech, agent's gestures and user's mouse actions. Finally, we attempt to determine proper timing when the agent should give additional instructions by estimating the likelihood of a mouse click occurrence.
AB - In multimodal communication, verbal and nonverbal behaviors such as gestures and manipulating objects in a workspace occur in parallel, and are coordinated in proper timing to each other. This paper focuses on the interaction between a beginner user using a video recorder application on PC and a multimodal animated help agent, and presents a probabilistic model of fine-grained timing dependencies among different behaviors of different modalities. First, we collect user-agent dialogues using a Wizard-of-Oz experimental setting, and then the collected verbal and nonverbal behavior data will be used to build a Bayesian network model, which can predict the likelihood of successful mouse clicks in near future, given evidence associated with the status of speech, agent's gestures and user's mouse actions. Finally, we attempt to determine proper timing when the agent should give additional instructions by estimating the likelihood of a mouse click occurrence.
KW - Animated agent
KW - Bayesian network
KW - Instruction dialogues
UR - http://www.scopus.com/inward/record.url?scp=48349133973&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=48349133973&partnerID=8YFLogxK
U2 - 10.1109/ICCAS.2007.4406995
DO - 10.1109/ICCAS.2007.4406995
M3 - Conference contribution
AN - SCOPUS:48349133973
SN - 8995003871
SN - 9788995003879
T3 - ICCAS 2007 - International Conference on Control, Automation and Systems
SP - 733
EP - 736
BT - ICCAS 2007 - International Conference on Control, Automation and Systems
T2 - International Conference on Control, Automation and Systems, ICCAS 2007
Y2 - 17 October 2007 through 20 October 2007
ER -