Emotion Recognition Experiments Research Articles

Multi-Representation Joint Dynamic Domain Adaptation Network for Cross-Database Facial Expression Recognition

In order to obtain more fine-grained information from multiple sub-feature spaces for domain adaptation, this paper proposes a novel multi-representation joint dynamic domain adaptation network (MJDDAN) and applies it to achieve cross-database facial expression recognition. The MJDDAN uses a hybrid structure to extract multi-representation features and maps the original facial expression features into multiple sub-feature spaces, aligning the expression features of the source domain and target domain in multiple sub-feature spaces from different angles to extract features more comprehensively. Moreover, the MJDDAN proposes the Joint Dynamic Maximum Mean Difference (JD-MMD) model to reduce the difference in feature distribution between different subdomains by simultaneously minimizing the maximum mean difference and local maximum mean difference in each substructure. Three databases, including eNTERFACE, FABO, and RAVDESS, are used to design a large number of cross-database transfer learning facial expression recognition experiments. The accuracy of emotion recognition experiments with eNTERFACE, FABO, and RAVDESS as target domains reach 53.64%, 43.66%, and 35.87%, respectively. Compared to the best comparison method chosen in this article, the accuracy rates were improved by 1.79%, 0.85%, and 1.02%, respectively.

Multi-Region and Multi-Band Electroencephalogram Emotion Recognition Based on Self-Attention and Capsule Network

Research on emotion recognition based on electroencephalogram (EEG) signals is important for human emotion detection and improvements in mental health. However, the importance of EEG signals from different brain regions and frequency bands for emotion recognition is different. For this problem, this paper proposes the Capsule–Transformer method for multi-region and multi-band EEG emotion recognition. First, the EEG features are extracted from different brain regions and frequency bands and combined into feature vectors which are input into the fully connected network for feature dimension alignment. Then, the feature vectors are inputted into the Transformer for calculating the self-attention of EEG features among different brain regions and frequency bands to obtain contextual information. Finally, utilizing capsule networks captures the intrinsic relationship between local and global features. It merges features from different brain regions and frequency bands, adaptively computing weights for each brain region and frequency band. Based on the DEAP dataset, experiments show that the Capsule–Transformer method achieves average classification accuracies of 96.75%, 96.88%, and 96.25% on the valence, arousal, and dominance dimensions, respectively. Furthermore, in emotion recognition experiments conducted on individual brain regions or frequency bands, it was observed that the frontal lobe exhibits the highest average classification accuracy, followed by the parietal, temporal, and occipital lobes. Additionally, emotion recognition performance is superior for high-frequency band EEG signals compared to low-frequency band signals.

EEG based emotion recognition by hierarchical bayesian spectral regression framework

Spectral regression (SR), a graph-based learning regression model, can be used to extract features from graphs to realize efficient dimensionality reduction. However, due to the SR method remains a regularized least squares problem and being defined in L2-norm space, the effect of artifacts in EEG signals cannot be efficiently resisted. In this work, to further improve the robustness of the graph-based regression models, we propose to utilize the prior distribution estimation in the Bayesian framework and develop a robust hierarchical Bayesian spectral regression framework (named HB-SR), which is designed with the hierarchical Bayesian ensemble strategies. In the proposed HB-SR, the impact of noises can be effectively reduced by the adaptive adjustment approach in model parameters with the data-driven manner. Specifically, in the current work, three different distributions have been elaborately designed to enhance the universality of the proposed HB-SR, i.e., Gaussian distribution, Laplace distribution, and Student-t distribution. To objectively evaluate the performance of the HB-SR framework, we conducted both simulation studies and emotion recognition experiments based on emotional EEG signals. Experimental results have consistently indicated that compared with other existing spectral regression methods, the proposed HB-SR can effectively suppress the influence of noises and achieve robust EEG emotion recognition.

Audio-Visual Emotion Recognition With Preference Learning Based on Intended and Multi-Modal Perceived Labels

This paper introduces a novel preference learning framework that simultaneously considers both the intended and the perceived labels while addressing the mismatches between them. Based on analyzing the discrepancies and agreements between the intended and the perceived labels in different modalities of audio-only, visual-only, and audio-visual, as well as the consistency among the perceptual ratings of all raters, we propose three sets of pair-wise ranking rules to generate multi-scale relevant scores for preference learning, scaling from sketchy manner to detailed manner. Three ranking models with support vector machine (SVM), deep neural networks (DNN), and gradient boosting decision trees (GBDT) are developed. Our results demonstrate that all three preference learning models significantly outperform the conventional classifiers baselines, and the LambdaMART model with gradient boosting decision trees achieves the best performance. The improvement from the preference learning models confirm the benefits of complementary information provided by different types of labels. We also observe additional improvement from the detailed ‘complex ranking rules’, particular with the best LambdaMART model, which suggests that we should treat intended and perceived labels in single-model & multi-modal differently. We further discuss the complementary of different ranking models, and obtain the best overall accuracy of 85.06% on CREMA-D dataset when combining the two best ranking models–LambdaMART and RankNet–together, which is significantly better than the 76.19% accuracy attained by the baseline models. Finally, we perform the cross-corpus emotion recognition experiments by training emotion rankers on CREMA-D dataset and tested the ranking-based emotion classifier on the SAVEE dataset that do not have perceived labels annotated.

Atypical scanning strategies of emotional faces for individuals with high autistic traits.

Autism has become one of the primary diseases causing disability in children, and the incidence has risen rapidly in recent years. The preclinical study on individuals with high autistic traits is extremely important to reduce genetic risks of autism because high autistic traits is the susceptibility marker of autism. However, few studies explored the face scanning pattern of people with high autistic traits in typical developing populations. In this study, we designed a facial emotion recognition experiment including four emotions (happy, neutral, sad, angry) and three angles (0°, 45°, 90°) , and informed the participants to identify the facial emotion. Forty-two college students with typical development were recruited and divided into high autistic traits (HAT) group and low autistic traits (LAT) group by the Autism-Spectrum Quotient, and we collected the eye movement data using eye-tracking technology when they performed the task. The response time, recognition accuracy, AOI based proportional fixation time and pupil diameter were computed and analyzed for both groups. HATs showed significantly lower recognition accuracy and lower pupil diameter than LATs when recognizing negative emotions (P<0.05) , indicating HATs kept poor autonomic nervous arousal. What ' s more, the proportional fixation time of HATs were significantly more in mouth area but less in eye area than that of LAT group (P<0.05) , revealed HATs had an atypical emotional faces scanning strategies that paid less attention to eyes and more attention to mouth. Our research provides a feasible objective biomarker for screening high autistic traits population.

MetaFERA: a meta-framework for creating emotion recognition frameworks for physiological signals

Recognizing emotions from physiological signals has proven to be important in various scenarios. To assist in developing emotion recognizers, software frameworks and toolboxes have emerged, offering ready-to-use components. However,these have limitations regarding the type of physiological signals supported, the recognition steps covered, or the acquisition of multiple physiological signals. This paper presents metaFERA, an architectural meta-framework for creating software frameworks for end-to-end emotion recognition from physiological signals. The modularity and flexibility of the meta-framework and the resulting frameworks allow the fast prototyping of emotion recognition systems and experiments to test and validate new algorithms. To that end, metaFERA offers: (i) a set of pre-configured blocks to which we can add behavior to create framework components; (ii) an easy way to add behavior to the pre-configured blocks; (iii) a channel-based communication mechanism that transparently and efficiently supports the exchange of information between components; (iv) a simple and easy way to use and link components from a resulting framework to create applications. Additionally, we provide a set of Web services, already configured, to make the resulting recognition systems available as a service. To validate metaFERA, we created a framework for Electrodermal Activity, an emotion recognizer to identify high/low arousal using the aforementioned framework, and a layer to offer the recognizer as a service.

Convolutional Neural Network-Based Automated System for Dog Tracking and Emotion Recognition in Video Surveillance

This paper proposes a multi–convolutional neural network (CNN)-based system for the detection, tracking, and recognition of the emotions of dogs in surveillance videos. This system detects dogs in each frame of a video, tracks the dogs in the video, and recognizes the dogs’ emotions. The system uses a YOLOv3 model for dog detection. The dogs are tracked in real time with a deep association metric model (DeepDogTrack), which uses a Kalman filter combined with a CNN for processing. Thereafter, the dogs’ emotional behaviors are categorized into three types—angry (or aggressive), happy (or excited), and neutral (or general) behaviors—on the basis of manual judgments made by veterinary experts and custom dog breeders. The system extracts sub-images from videos of dogs, determines whether the images are sufficient to recognize the dogs’ emotions, and uses the long short-term deep features of dog memory networks model (LDFDMN) to identify the dog’s emotions. The dog detection experiments were conducted using two image datasets to verify the model’s effectiveness, and the detection accuracy rates were 97.59% and 94.62%, respectively. Detection errors occurred when the dog’s facial features were obscured, when the dog was of a special breed, when the dog’s body was covered, or when the dog region was incomplete. The dog-tracking experiments were conducted using three video datasets, each containing one or more dogs. The highest tracking accuracy rate (93.02%) was achieved when only one dog was in the video, and the highest tracking rate achieved for a video containing multiple dogs was 86.45%. Tracking errors occurred when the region covered by a dog’s body increased as the dog entered or left the screen, resulting in tracking loss. The dog emotion recognition experiments were conducted using two video datasets. The emotion recognition accuracy rates were 81.73% and 76.02%, respectively. Recognition errors occurred when the background of the image was removed, resulting in the dog region being unclear and the incorrect emotion being recognized. Of the three emotions, anger was the most prominently represented; therefore, the recognition rates for angry emotions were higher than those for happy or neutral emotions. Emotion recognition errors occurred when the dog’s movements were too subtle or too fast, the image was blurred, the shooting angle was suboptimal, or the video resolution was too low. Nevertheless, the current experiments revealed that the proposed system can correctly recognize the emotions of dogs in videos. The accuracy of the proposed system can be dramatically increased by using more images and videos for training the detection, tracking, and emotional recognition models. The system can then be applied in real-world situations to assist in the early identification of dogs that may exhibit aggressive behavior.

Emotion recognition based on group phase locking value using convolutional neural network

Electroencephalography (EEG)-based emotion recognition is an important technology for human–computer interactions. In the field of neuromarketing, emotion recognition based on group EEG can be used to analyze the emotional states of multiple users. Previous emotion recognition experiments have been based on individual EEGs; therefore, it is difficult to use them for estimating the emotional states of multiple users. The purpose of this study is to find a data processing method that can improve the efficiency of emotion recognition. In this study, the DEAP dataset was used, which comprises EEG signals of 32 participants that were recorded as they watched 40 videos with different emotional themes. This study compared emotion recognition accuracy based on individual and group EEGs using the proposed convolutional neural network model. Based on this study, we can see that the differences of phase locking value (PLV) exist in different EEG frequency bands when subjects are in different emotional states. The results showed that an emotion recognition accuracy of up to 85% can be obtained for group EEG data by using the proposed model. It means that using group EEG data can effectively improve the efficiency of emotion recognition. Moreover, the significant emotion recognition accuracy for multiple users achieved in this study can contribute to research on handling group human emotional states.

DESCU: Dyadic emotional speech corpus and recognition system for Urdu language

Speech signal contains the emotional state of a speaker along with the message. The recognition of the emotional state of a speaker helps in determining the true meaning of a message and allows for more natural communication between humans and machines. This paper presents the design and development of a dyadic emotional speech corpus for the Urdu language. The corpus is developed by recording dialog scenarios for anger, happy, neutral, and sad emotions. The performance of frame-level features, utterance-level features, and spectrograms have been evaluated in this work. Emotion recognition experiments have been conducted using classifiers including Support Vector Machine, Hidden Markov Models and Convolutional Neural Networks. Experimental results show that the utterance-level features outperform the frame-level features and spectrograms. The combined feature set of cepstral, spectral, prosodic, and voice quality features performs better than the individual feature sets. The unweighted average recalls of 84.1%, 80.2%, 84.7% have been achieved for speaker-dependent and speaker-independent and text-independent emotion recognition, respectively.

Novel techniques for improving NNetEn entropy calculation for short and noisy time series

Entropy is a fundamental concept in the field of information theory. During measurement, conventional entropy measures are susceptible to length and amplitude changes in time series. A new entropy metric, neural network entropy (NNetEn), has been developed to overcome these limitations. NNetEn entropy is computed using a modified LogNNet neural network classification model. The algorithm contains a reservoir matrix of N=19625 elements that must be filled with the given data. The contribution of this paper is threefold. Firstly, this work investigates different methods of filling the reservoir with time series (signal) elements. The reservoir filling method determines the accuracy of the entropy estimation by convolution of the study time series and LogNNet test data. The present study proposes 6 methods for filling the reservoir for time series. Two of them (Method 3 and Method 6) employ the novel approach of stretching the time series to create intermediate elements that complement it, but do not change its dynamics. The most reliable methods for short time series are Method 3 and Method 5. The second part of the study examines the influence of noise and constant bias on entropy values. Our study examines three different time series data types (chaotic, periodic, and binary) with different dynamic properties, Signal to Noise Ratio (SNR), and offsets. The NNetEn entropy calculation errors are less than 10% when SNR is greater than 30 dB, and entropy decreases with an increase in the bias component. The third part of the article analyzes real-time biosignal EEG data collected from emotion recognition experiments. The NNetEn measures show robustness under low-amplitude noise using various filters. Thus, NNetEn measures entropy effectively when applied to real-world environments with ambient noise, white noise, and 1/f noise.

Cross-Corpus Speech Emotion Recognition Based on Multi-Task Learning and Subdomain Adaptation.

To solve the problem of feature distribution discrepancy in cross-corpus speech emotion recognition tasks, this paper proposed an emotion recognition model based on multi-task learning and subdomain adaptation, which alleviates the impact on emotion recognition. Existing methods have shortcomings in speech feature representation and cross-corpus feature distribution alignment. The proposed model uses a deep denoising auto-encoder as a shared feature extraction network for multi-task learning, and the fully connected layer and softmax layer are added before each recognition task as task-specific layers. Subsequently, the subdomain adaptation algorithm of emotion and gender features is added to the shared network to obtain the shared emotion features and gender features of the source domain and target domain, respectively. Multi-task learning effectively enhances the representation ability of features, a subdomain adaptive algorithm promotes the migrating ability of features and effectively alleviates the impact of feature distribution differences in emotional features. The average results of six cross-corpus speech emotion recognition experiments show that, compared with other models, the weighted average recall rate is increased by 1.89~10.07%, the experimental results verify the validity of the proposed model.

Multisource Associate Domain Adaptation for Cross-Subject and Cross-Session EEG Emotion Recognition

Emotion recognition is important in the application of brain-computer interface (BCI). Building a robust emotion recognition model across subjects and sessions is critical in emotion based BCI systems. Electroencephalogram (EEG) is a widely used tool to recognize different emotion states. However, EEG has disadvantages such as small amplitude, low signal-to-noise ratio, and non-stationary properties, resulting in large differences across subjects. To solve these problems, this paper proposes a new emotion recognition method based on a multi-source associate domain adaptation network, considering both domain invariant and domain-specific features. First, separate branches were constructed for multiple source domains, assuming that different EEG data shared the same low-level features. Secondly, the domain specific features were extracted by using the one-to-one associate domain adaptation. Then, the weighted scores of specific sources were obtained according to the distribution distance, and multiple source classifiers were deduced with the corresponding weighted scores. Finally, EEG emotion recognition experiments were conducted on different datasets, including SEED, DEAP, and SEED-IV dataset. Results indicated that, in the cross-subject experiment, the average accuracy in SEED dataset was 86.16%, DEAP dataset was 65.59%, and SEED-IV was 59.29%. In the cross-session experiment, the accuracies of SEED and SEED-IV datasets were 91.10% and 66.68%, respectively. Our proposed method has achieved better classification results compared to state-of-the-art domain adaptation methods.

TMLP+SRDANN: A domain adaptation method for EEG-based emotion recognition

Electroencephalogram (EEG) contains emotion information, but usually undergoes severe signal variations. In the literature, for the issue of EEG-based emotion recognition, not only have diverse features been designed to capture the representative characteristics of EEG, but also various domain adaptation techniques have been devised to reduce the distributional discrepancy between the source and target domains. However, existing domain adaptation techniques for this issue treat all the source samples equally, but ignore their transferability difference, which more or less limits the performance of learned domain-invariant and class-discriminative EEG features for emotion recognition. To cope with this problem, we propose Transposition Multi-Layer Perceptron (TMLP) and Sample-Reweighted Domain Adaptation Neural Network (SRDANN) in one whole learning framework. TMLP extracts the robust multi-channel EEG features related to emotions; SRDANN transfers the discriminative knowledge from the source domain to the target domain for classifying emotions. By concentrating on the source samples with stronger transferability in domain adaptation, TMLP+SRDANN can learn the more domain-invariant and class-discriminative features from EEG, thus providing a more effective solution to emotion recognition in the challenging cross-subject scenario. The subject-independent emotion recognition experiments on two benchmark datasets have demonstrated the effectiveness of our method: an accuracy of 81.04% for emotion classification on SEED, 61.88% for valence classification and 57.70% for arousal classification on DEAP.

Face memory and facial expression recognition are both affected by wearing disposable surgical face masks

Faces are fundamental stimuli for social interactions since they provide significant information about people’s identity and emotional states. With the outburst of the COVID-19 pandemic, global use of preventive measures, such as disposable surgical face masks (DSFMs), has been imposed. The massive use of DSFMs covering a large part of the face could interfere with identity and emotion recognition. Thus, the main aim of the current study was (i) to assess how DSFMs affect identity recognition (Experiment 1), (ii) how DSFMs affect emotion recognition (Experiment 2), and (iii) whether individual empathy levels correlate with emotion recognition with DSFMs. The potential relation between identity and emotion recognition with and without DSFMs was also investigated. Two tasks were administered to 101 healthy participants: (i) the Old-new face memory task aimed to assess whether the learning context (i.e., DSFMs on/off) affects recognition performance, whereas (ii) the Facial affect task explored DSFMs’ effect on emotion recognition. Results from the former showed that the stimuli’s features in the learning stage affect recognition performances; that is, faces wearing DSFMs were better recognized if wearing DSFMs at first exposure and vice versa. Results from the Facial affect task showed that DSFMs lead to reduced disgust, happiness, and sadness recognition. No significant correlation emerged between identity and emotion recognition. The Interpersonal Reactivity Index (IRI) was administered to assess affective and cognitive empathy; however, IRI scores did not correlate with either face memory recognition or facial affect recognition. Overall, our results demonstrate (a) a “context effect” for face memory with and without DSFMs; (b) a disruptive effect of DSFMs depending on the expressed emotion; and (c) no correlation between empathy and emotion recognition with DSFMs.

An Approach to Emotion Recognition Using Brain Rhythm Sequencing and Asymmetric Features.

Emotion can be influenced during self-isolation, and to avoid severe mood swings, emotional regulation is meaningful. To achieve this, efficiently recognizing emotion is a vital step, which can be realized by electroencephalography signals. Previously, inspired by the knowledge of sequencing in bioinformatics, a method termed brain rhythm sequencing that analyzes electroencephalography as the sequence consisting of the dominant rhythm has been proposed for seizure detection. In this work, with the help of similarity measure methods, the asymmetric features are extracted from the sequences generated by different channel data. After evaluating all asymmetric features for emotion recognition, the optimal feature that yields remarkable accuracy is identified. Therefore, the classification task can be accomplished through a small amount of channel data. From a music emotion recognition experiment and a public DEAP dataset, the classification accuracies of various test sets are approximately 80–85% when employing an optimal feature extracted from one pair of symmetrical channels. Such performances are impressive when using fewer resources is a concern. Further investigation revealed that emotion recognition shows strongly individual characteristics, so an appropriate solution is to include the subject-dependent properties. Compared to the existing works, this method benefits from the design of a portable emotion-aware device used during self-isolation, as fewer scalp sensors are needed. Hence, it would provide a novel way to realize emotional applications in the future.

Transfer EEG Emotion Recognition by Combining Semi-Supervised Regression with Bipartite Graph Label Propagation

Individual differences often appear in electroencephalography (EEG) data collected from different subjects due to its weak, nonstationary and low signal-to-noise ratio properties. This causes many machine learning methods to have poor generalization performance because the independent identically distributed assumption is no longer valid in cross-subject EEG data. To this end, transfer learning has been introduced to alleviate the data distribution difference between subjects. However, most of the existing methods have focused only on domain adaptation and failed to achieve effective collaboration with label estimation. In this paper, an EEG feature transfer method combined with semi-supervised regression and bipartite graph label propagation (TSRBG) is proposed to realize the unified joint optimization of EEG feature distribution alignment and semi-supervised joint label estimation. Through the cross-subject emotion recognition experiments on the SEED-IV data set, the results show that (1) TSRBG has significantly better recognition performance in comparison with the state-of-the-art models; (2) the EEG feature distribution differences between subjects are significantly minimized in the learned shared subspace, indicating the effectiveness of domain adaptation; (3) the key EEG frequency bands and channels for cross-subject EEG emotion recognition are achieved by investigating the learned subspace, which provides more insights into the study of EEG emotion activation patterns.

Design of Interactive Vocal Guidance and Artistic Psychological Intervention System Based on Emotion Recognition.

The research on artistic psychological intervention to judge emotional fluctuations by extracting emotional features from interactive vocal signals has become a research topic with great potential for development. Based on the interactive vocal music instruction theory of emotion recognition, this paper studies the design of artistic psychological intervention system. This paper uses the vocal music emotion recognition algorithm to first train the interactive recognition network, in which the input is a row vector composed of different vocal music characteristics, and finally recognizes the vocal music of different emotional categories, which solves the problem of low data coupling in the artistic psychological intervention system. Among them, the vocal music emotion recognition experiment based on the interactive recognition network is mainly carried out from six aspects: the number of iterative training, the vocal music instruction rate, the number of emotion recognition signal nodes in the artistic psychological intervention layer, the number of sample sets, different feature combinations, and the number of emotion types. The input data of the system is a training class learning video, and actions and expressions need to be recognized before scoring. In the simulation process, before the completion of the sample indicators is unbalanced, the R language statistical analysis tool is used to balance the existing unbalanced data based on the artificial data synthesis method, and 279 uniformly classified samples are obtained. The 279∗7 dataset was used for statistical identification of the participants. The experimental results show that under the guidance of four different interactive vocal music, the vocal emotion recognition rate is between 65.85%-91.00%, which promotes the intervention of music therapy on artistic psychological intervention.

A deep subdomain associate adaptation network for cross-session and cross-subject EEG emotion recognition

Developing robust cross-subject or cross-session EEG-based affective models is a key issue in affective brain-computer interfaces, which often suffer from the individual differences and non-stationarity of EEG. Aiming at generalizing the affective model across subjects and sessions, this paper proposes a novel transfer learning strategy with Deep Subdomain Associate Adaptation Network (DSAAN) for EEG emotion recognition. Domain was divided into subdomains according to the sample labels, and the source domain use the true sample labels while the target domain use the predicted pseudo-labels. DSAAN was established as a transfer network by aligning the relevant subdomain distributions based on Subdomain Associate Loop (SAL). The adaptation of networks was achieved by minimizing the summation of source domain classification loss and SAL loss. For the purpose of verifying the generalization of DSAAN, we carried out the cross-session and cross-subject EEG emotion recognition experiments on benchmark SEED and DEAP. Compared with existing domain adaptation methods, the DSAAN achieved outstanding classification results.

Single-Channel Selection for EEG-Based Emotion Recognition Using Brain Rhythm Sequencing.

Recently, electroencephalography (EEG) signals have shown great potential for emotion recognition. Nevertheless, multichannel EEG recordings lead to redundant data, computational burden, and hardware complexity. Hence, efficient channel selection, especially single-channel selection, is vital. For this purpose, a technique termed brain rhythm sequencing (BRS) that interprets EEG based on a dominant brain rhythm having the maximum instantaneous power at each 0.2 s timestamp has been proposed. Then, dynamic time warping (DTW) is used for rhythm sequence classification through the similarity measure. After evaluating the rhythm sequences for the emotion recognition task, the representative channel that produces impressive accuracy can be found, which realizes single-channel selection accordingly. In addition, the appropriate time segment for emotion recognition is estimated during the assessments. The results from the music emotion recognition (MER) experiment and three emotional datasets (SEED, DEAP, and MAHNOB) indicate that the classification accuracies achieve 70-82% by single-channel data with a 10 s time length. Such performances are remarkable when considering minimum data sources as the primary concerns. Furthermore, the individual characteristics in emotion recognition are investigated based on the channels and times found. Therefore, this study provides a novel method to solve single-channel selection for emotion recognition.

Featural vs. Holistic processing and visual sampling in the influence of social category cues on emotion recognition

ABSTRACT Past research demonstrates that emotion recognition is influenced by social category cues present on faces. However, little research has investigated whether holistic processing is required to observe these influences of social category information on emotion perception, and no studies have investigated whether different visual sampling strategies (i.e. differences in the allocation of attention to different regions of the face) contribute to the interaction between social cues and emotional expressions. The current study aimed to address this. Participants categorised happy and angry expressions on own- and other-race faces, and male and female faces. In Experiments 1 and 2, holistic processing was disrupted by presenting inverted faces (Experiment 1) or part faces (Experiment 2). In Experiments 3 and 4 participants’ eye-gaze to eye and mouth regions was also tracked. Disrupting holistic processing did not alter the moderating influence of sex and race cues on emotion recognition (Experiments 1, 2, 4). Gaze patterns differed as a function of emotional expression, and social category cues, however, eye-gaze patterns did not reflect response time patterns (Experiments 3 and 4). Results indicate that the interaction between social category cues and emotion does not require holistic processing and is not driven by differences in visual sampling.