Visual Focus Of Attention Recognition Research Articles

Student engagement study based on multi-cue detection and recognition in an intelligent learning environment

Student engagement has great impact on learning performance. It’s necessary to investigate student engagement objectively from learning behavior. In this paper, we propose a student engagement study approach in an intelligent learning environment, which automatically detects and analyses multiple learning behavioral cues based on five modules, i.e., attendance management, teacher-student (T&S) communication, visual focus of attention (VFOA) recognition, smile detection and engagement analysis. Attendance management matches the student’s identity and locates his/her profile using face recognition. T&S communication provides an additional channel of Question and Answer (Q&A) between a teacher and students for students’ behavioral engagement analysis via their cell phones. VFOA recognition is used to recognize students’ cognitive engagement through capturing students’ attention based on the estimated head poses, visual environment cues and prior states in class. Smile detection achieves students’ affective engagement through spontaneous smile expression classification. Finally, a tree-structural engagement model is proposed to decide student engagement based on multi-cues of one’s behavioral, cognitive and affective engagement. We thoroughly evaluated each module for engagement study on some public available datasets and practical video sequences in class applications. The experimental results suggest that the proposed approach can automatically detect and analyze student class engagement objectively and effectively.

Multiperson Visual Focus of Attention from Head Pose and Meeting Contextual Cues

This paper introduces a novel contextual model for the recognition of people's visual focus of attention (VFOA) in meetings from audio-visual perceptual cues. More specifically, instead of independently recognizing the VFOA of each meeting participant from his own head pose, we propose to jointly recognize the participants' visual attention in order to introduce context-dependent interaction models that relate to group activity and the social dynamics of communication. Meeting contextual information is represented by the location of people, conversational events identifying floor holding patterns, and a presentation activity variable. By modeling the interactions between the different contexts and their combined and sometimes contradictory impact on the gazing behavior, our model allows us to handle VFOA recognition in difficult task-based meetings involving artifacts, presentations, and moving people. We validated our model through rigorous evaluation on a publicly available and challenging data set of 12 real meetings (5 hours of data). The results demonstrated that the integration of the presentation and conversation dynamical context using our model can lead to significant performance improvements.

Recognizing Visual Focus of Attention From Head Pose in Natural Meetings

We address the problem of recognizing the visual focus of attention (VFOA) of meeting participants based on their head pose. To this end, the head pose observations are modeled using a Gaussian mixture model (GMM) or a hidden Markov model (HMM) whose hidden states correspond to the VFOA. The novelties of this paper are threefold. First, contrary to previous studies on the topic, in our setup, the potential VFOA of a person is not restricted to other participants only. It includes environmental targets as well (a table and a projection screen), which increases the complexity of the task, with more VFOA targets spread in the pan as well as tilt gaze space. Second, we propose a geometric model to set the GMM or HMM parameters by exploiting results from cognitive science on saccadic eye motion, which allows the prediction of the head pose given a gaze target. Third, an unsupervised parameter adaptation step not using any labeled data is proposed, which accounts for the specific gazing behavior of each participant. Using a publicly available corpus of eight meetings featuring four persons, we analyze the above methods by evaluating, through objective performance measures, the recognition of the VFOA from head pose information obtained either using a magnetic sensor device or a vision-based tracking system. The results clearly show that in such complex but realistic situations, the VFOA recognition performance is highly dependent on how well the visual targets are separated for a given meeting participant. In addition, the results show that the use of a geometric model with unsupervised adaptation achieves better results than the use of training data to set the HMM parameters.