Optimized hybrid deep learning pipelines for processing heterogeneous facial expression datasets

Abstract

Emotions, central to human experience, have been categorized through various theories. Biological theories, such as the James-Lange model, suggest that physiological reactions are the precursors to emotions. Cognitive theories, as highlighted by the Schachter-Singer model, argue that emotions emerge from a combination of physiological responses and cognitive appraisal. Evolutionary theories, rooted in Darwin's work, view emotions as adaptive mechanisms that have evolved to promote survival. In the realm of psychology, expressing emotions via facial expressions is termed face reading, a vital communication method. With the swift advancement of Deep Neural Networks (DNN) across various fields, they are increasingly used to derive exclusive features for automated Facial Expression Prediction and Recognition (FEPR). This research employs a Three-staged Deep Learning (TsDL) strategy for FEPR. Recognizing the pivotal role of datasets in determining system performance, we incorporate both laboratory-controlled and real-world data. Initially, the Generative Adversarial Networks (GAN) model, leveraging datasets like CK+ and Radboud Faces Database (RFD), is introduced. This model successfully identifies expressions across seven categories with an accuracy of 92.15 %. Subsequently, another model based on the Attention-Long Short Term Memory (A-LSTM) is trained using the Real-world Affective Face Database, given its capability to manage spatio-temporal features. The TsDL approach predicts and classifies six primary emotions, achieving an 89.22 % accuracy rate. In the culmination of our work, we harness Ensemble Learning Strategies to craft a refined hybrid model, ensuring systematic performance and optimal accuracy. The diverse theoretical perspectives on emotions greatly enrich our comprehension, especially when developing our FEPR model.