Intelligent affect regression for bodily expressions using hybrid particle swarm optimization and adaptive ensembles

Abstract

This research focuses on continuous dimensional affect recognition from bodily expressions using feature optimization and adaptive regression. Both static posture and dynamic motion bodily features are extracted in this research. A hybrid particle swarm optimization (PSO) algorithm is proposed for feature selection, which overcomes premature convergence and local optimum trap encountered by conventional PSO. It integrates diverse jump-out mechanisms such as the genetic algorithm (GA) and mutation techniques of Gaussian, Cauchy and Levy distributions to balance well between convergence speed and swarm diversity, thus called GM-PSO. The proposed PSO variant employs the subswarm concept and a cooperative strategy to enable mutation mechanisms of each subswarm, i.e. the GA and the probability distributions, to work in a collaborative manner to enhance the exploration and exploitation capability of the swarm leader, sustain the population diversity and guide the search toward an ultimate global optimum. An adaptive ensemble regression model is subsequently proposed to robustly map subjects’ emotional states onto a continuous arousal–valence affective space using the identified optimized feature subsets. This regression model also shows great adaption to newly arrived bodily expression patterns to deal with data stream regression. Empirical findings indicate that the proposed hybrid PSO optimization algorithm outperforms other state-of-the-art PSO variants, conventional PSO and classic GA significantly in terms of catching global optimum and discriminative feature selection. The system achieves the best performance for the regression of arousal and valence when ensemble regression model is applied, in terms of both mean squared error (arousal: 0.054, valence: 0.08) and Pearson correlation coefficient (arousal: 0.97, valence: 0.91) and outperforms other state-of-the-art PSO-based optimization combined with ensemble regression and related bodily expression perception research by a significant margin.