Figure 1 summarizes the type of human-computer interaction system that we believe is necessary to support proactive computer interaction. (1) At the center is the user who wishes to carry out some task. (2) A set of sensors (video and audio, plus keyboard and mouse) senses the user's behavior (face expression, gestures, body motion, speech signal, eye position) and task actions. (3) This information is fused into state detectors, including emotional, motivational, cognitive and task states, as well as user recognition. This component may be either bypassed or accomplished by computer learning algorithms. (4) A current state table, plus state history, are maintained and updated, from which persistent patterns are identified indicating user personality characteristics (timid, impulsive, reflective, or forceful) and characteristic strategies. (5) A computer action decision module, including a dialogue generation system, makes decisions concerning the proactive actions for the computer to take, based on current state and past history, or directly from the state detectors themselves. (6) Computer actions are synthesized and presented to the user. (7) Computer learning occurs based on the user's responses to computer actions, thus molding the system to the user's personality and preferences.
Copyright 2001 Beckman Institute,
University of Illinois at Urbana-Champaign
Computer learning occurs based on the user's responses to computer actions, thus molding the system to the user's personality and preferences.
