Gesture phase segmentation using support vector machines

Abstract

An interaction between humans or between a human and a machine will be more effective if it is supported by gestures. In different levels of complexity, the communication system used in human interaction includes the use of gesture. In natural conversation, for instance, speakers use gestures for both to enhance the expressiveness of their speech and to support their own linguistic reasoning. The audience absorbs the content being transmitted also based on the speakers’ gesticulation. Thus, an analysis of gestures should add value to the purpose of the interaction. One of the concerns in the analysis of gestures is the problem arising from the segmentation of phases of a gesture (rest position, preparation, stroke, hold and retraction), which, from the standpoint of Gesture Theory, may reveal information on prosody and semantics of what is being said in a discourse. Finding an automation solution to this problem involves enabling the development of theoretical and application areas that are based on the analysis of human behavior and on the interpretation and generation of natural language. In this study, the problem of gesture phase segmentation is modeled as a problem of classification, and then support vector machine is employed to design a model able to learn the patterns of gesture that are inherent to each phase. This work presents two main highlights. The first is to address the limitations of the segmentation approach through the study of its performance in different scenarios that represent the complexity of analyzing patterns of human behavior. In this study, we reached an F-score around 0.9 for rest position and around 0.8 for stroke and preparation as segmentation results in the best cases. Moreover, it was possible to investigate how classification models are influenced by human behavior. The second highlight refers to the conduction of an analysis by considering the standpoint of specialists concerned with gesture phase segmentation in the area of Linguistics and Psycholinguistics, through which we obtained impressive results. Thus, in regard to the suitability of our approach, it is a feasible means of supporting the development of the Gesture Theory as well as the Computational Linguistics and Human Machine Interaction fields.