Detecting &amp; interpreting self-manipulating hand movements for student’s affect prediction

Akhtar Hussain,Abdul Rehman Abbasi,Nitin Afzulpurkar

doi:10.1186/2192-1962-2-14

Detecting & interpreting self-manipulating hand movements for student’s affect prediction

Akhtar Hussain, Abdul Rehman Abbasi + Show 1 more

Open Access

https://doi.org/10.1186/2192-1962-2-14

Copy DOI

Abstract

Abstract Background In this paper, we report on development of a non-intrusive student mental state prediction system from his (her) unintentional hand-touch-head (face) movements. Methods Hand-touch-head (face) movement is a typical case of occlusion of otherwise easily detectable image features due to similar skin color and texture, however, in our proposed scheme, i.e., the Sobel-operated local binary pattern (SLBP) method using force field features. We code six different gestures of more than 100 human subjects, and use these codes as manual input to a three-layered Bayesian network (BN). The first layer holds mental state to gesture relationships obtained in an earlier study while the second layer embeds gesture and SLBP generated binary codes. Results We find it very successful in separating hand (s) from face region in varying illuminating conditions. The proposed scheme when evaluated on a novel data set is found promising resulting with an accuracy of about 85%. Conclusion The framework will be utilized for developing intelligent tutoring system.

Highlights

Introduction & related workCommunication is an important source of conversational interaction between the human beings
The research in psychology shows that the majority of communications among the human beings are being done through non-verbal communication [1,2]
It is important to find that Sobel-operated local binary pattern (SLBP) performs better when compared to the Local Binary Patterns (LBPs) alone

Summary

Introduction

Introduction & related workCommunication is an important source of conversational interaction between the human beings. Non-verbal communication play a vital role in our daily life because we convey a lot of non-verbal communicative signals through body language to whom we interact and these communicative signals carry valuable information about the intention of the person in that particular context. These communicative signals can be recognized by human beings through body language such as gestures and postures, body movements, facial expression and eye contact, etc. We report on development of a non-intrusive student mental state prediction system from his (her) unintentional hand-touch-head (face) movements

Methods

Results

Discussion

Conclusion