Human Facial Expression Recognition Research Articles

Real-time Facial Expression Recognition to Enhance Emotional Intelligence in Autism

The ability to interpret emotions from facial expressions is essential in social interaction. However, these abilities are complex for individuals with autism spectrum disorder (ASD). This study aims to develop a real-time human facial expression recognition system on a mobile phone application to help individuals with ASD. The expression recognition model was developed using the FER-2013 dataset with the VGG-16 architecture. The experimental results show that the model obtained from the VGG-16 architecture can produce the best accuracy and average F1 scores with 0.91 and 0.91, respectively. We also experiment with three individuals with ASD, resulting in increasing interest in learning the expression.

Matters of development and experience: Evaluation of dog and human emotional expressions by children and adults.

Emotional facial expressions are an important part of across species social communication, yet the factors affecting human recognition of dog emotions have received limited attention. Here, we characterize the recognition and evaluation of dog and human emotional facial expressions by 4-and 6-year-old children and adult participants, as well as the effect of dog experience in emotion recognition. Participants rated the happiness, anger, valence, and arousal from happy, aggressive, and neutral facial images of dogs and humans. Both respondent age and experience influenced the dog emotion recognition and ratings. Aggressive dog faces were rated more often correctly by adults than 4-year-olds regardless of dog experience, whereas the 6-year-olds' and adults' performances did not differ. Happy human and dog expressions were recognized equally by all groups. Children rated aggressive dogs as more positive and lower in arousal than adults, and participants without dog experience rated aggressive dogs as more positive than those with dog experience. Children also rated aggressive dogs as more positive and lower in arousal than aggressive humans. The results confirm that recognition of dog emotions, especially aggression, increases with age, which can be related to general dog experience and brain structure maturation involved in facial emotion recognition.

MOOD PLAYER: EMOTION BASED MUSICRECOMMENDATION SYSTEM

Music can create, change and lighten one’s mood, and when technologies of artificial intelligence meet with music it can be a great combination. Selecting appropriate songs according to our emotion from such a wide range is a very tedious and timeconsuming task. It takes a lot of time and effort to manually group songs on a playlist and annotate them according to the user’s emotional condition at the moment. To automate this process, many algorithms have been proposed. However, the current algorithms are inefficient, use more hardware, which raises the system’s overall cost, and are less accurate. This paper aims at recognition of human facial expressions through a model based on 68 landmarks approach of the training dataset, further selecting the appropriate playlist for the user. The Emotion Based Music Recommendation System has the potential to enhance the music listening experience by providing users with music that not only matches their preferences but also their emotional state, thereby increasinguser engagement and satisfaction.

Human Face and Facial Expression Recognition Using Deep Learning and SNet Architecture Integrated with BottleNeck Attention Module

Human Face and Facial Expression Recognition Using Deep Learning and SNet Architecture Integrated with BottleNeck Attention Module

Facile and Controllable Ultrasonic Nebulization Method for Fabricating Ti3C2Tx‐Based Strain Sensor and Monitoring of Human Motion and Sound Wave

AbstractFlexible and wearable electronic devices hold great potential in electronic skins, health monitoring systems and soft robotics. Among them, flexible strain sensors with high performance are key components for wearable health monitoring devices. However, the facile and controllable preparation of highly sensitive sensors still faces significant challenges. By virtue of excellent conductivity of 2D transition metal carbids (MXenes), this work reports a facile and low‐cost fabrication strategy for large‐scale production of strain sensors. The sensitive layer is deposited on flexible interdigital electrodes by ultrasonic nebulization of Ti3C2Tx nanosheets. By controlling the nebulization time, different thicknesses of Ti3C2Tx films has a great influence on the performance of strain sensors. The Ti3C2Tx‐based strain sensor exhibits good sensing performances such as high GF (19.1) in the low strain range (≈0.25%–1.14%), short response time (0.7 s), and stable durability (over 1000 cycles). In practice, the potential applications of the strain sensor in sound frequency detection, human physiological signal monitoring and facial expression recognition are demonstrated. Finally, this work integrates the strain sensor with a miniaturized analyzer to assemble a wearable motion monitoring device for mobile healthcare. This study provides a facile strategy for fabricating flexible strain sensors in the field of wearable electronics.

Human Facial Expression Recognition based on Attention Mechanism

In the current study of facial expression recognition, CNNs have been extensively used and proven to be useful models. In this work, we analyze and improve VGG-16 network by embedding SENet between the convolutional layers and three fully connected layers of the original network to form the SE-VGG-16 network. The proposed network architecture is able to extract additional spatial information regarding interdependencies among feature channels, which is complementary to facial expression recognition. The experiment is performed on dataset FER2013 and improve the accuracy with top-1 error 33.5.

Combination of Adaptive Neuro Fuzzy Inference System and Machine Learning Algorithm for Recognition of Human Facial Expressions

Combination of Adaptive Neuro Fuzzy Inference System and Machine Learning Algorithm for Recognition of Human Facial Expressions

Deep Learning on Facial Expression Detection : Artificial Neural Network Model Implementation

The moods, emotions, and even medical issues of a person can frequently be seen directly reflected in their facial expressions. The fields of social science and human-computer interaction have recently begun to pay more attention to facial emotion detection as a result of this. The primary focus of this study is on the automatic recognition of human facial expressions using an artificial neural network (ANN) model and a technique based on straightforward convolution. The dataset utilized is a self-mined dataset that was obtained by utilizing the web scraping approach on Google Image with the help of the Selenium package for Python. A dataset containing six categories of fundamental human expressions that are likely to be met on a daily basis, namely anger, confusion, contempt, crying, sadness, disgust, and happiness, with a total of 6,016 photos being used. The goal of this research is to determine how accurate the model of artificial neural networks can be in predicting.

An Improved Face-Emotion Recognition to Automatically Generate Human Expression With Emoticons

Any human face image expression naturally identifies expressions of happy, sad etc.; sometimes human facial image expression recognition is complex, and it is a combination of two emotions. The existing literature provides face emotion classification and image recognition, and the study on deep learning using convolutional neural networks (CNN), provides face emotion recognition most useful for healthcare and with the most complex of the existing algorithms. This paper improves the human face emotion recognition and provides feelings of interest for others to generate emoticons on their smartphone. Face emotion recognition plays a major role by using convolutional neural networks in the area of deep learning and artificial intelligence for healthcare services. Automatic facial emotion recognition consists of two methods, such as face detection with Ada boost classifier algorithm and emotional classification, which consists of feature extraction by using deep learning methods such as CNN to identify the seven emotions to generate emoticons.

Early Education Application Software Based on Artificial Intelligence VR Technology.

In order to solve the problems of restricted classroom and lack of repeated training in good curriculum, the design and development of a school life adaptation curriculum based on VR is proposed. This study identified six life adaptation themes for design and development (recognizing facial expressions, crossing the road, how to get lost, shopping, taking public transport, and job interview) and set the objectives, design principles, specific content, teaching evaluation, and other aspects of virtual reality life adaptation course. Combined with the characteristics of virtual reality, the implementation suggestions and matters needing attention are put forward. The results showed that, in the recognition of human facial expressions, 51 experts thought it was appropriate, 3 experts thought it needed to be modified, and no expert thought it was inappropriate. On the topic of crossing the road, 46 experts thought it was appropriate, eight thought it needed to be modified, and no expert thought it was inappropriate. On the theme of taking public transportation, 48 experts thought it was appropriate, 6 experts thought it needed to be modified, and no expert thought it was inappropriate. For the three major topics of how to get lost, shopping, and job interview, 2-3 experts put forward different opinions and think that the topic is not appropriate mainly because the grade arrangement is not appropriate. Conclusion. The VR-based curriculum for school life adaptation has been reviewed by experts and revised by researchers. It is scientific, interesting, and operable, solves the teaching problems of life adaptation teachers, and provides a safe, reliable, and effective practice channel for students.

Image and Speech Recognition Technology in the Development of an Elderly Care Robot: Practical Issues Review and Improvement Strategies.

As the world's population is aging and there is a shortage of sufficient caring manpower, the development of intelligent care robots is a feasible solution. At present, plenty of care robots have been developed, but humanized care robots that can suitably respond to the individual behaviors of elderly people, such as pose, expression, gaze, and speech are generally lacking. To achieve the interaction, the main objectives of this study are: (1) conducting a literature review and analyzing the status quo on the following four core tasks of image and speech recognition technology: human pose recognition, human facial expression recognition, eye gazing recognition, and Chinese speech recognition; (2) proposing improvement strategies for these tasks based on the results of the literature review. The results of the study on these improvement strategies will provide the basis for using human facial expression robots in elderly care.

Multi-Feature Integrated Concurrent Neural Network for Human Facial Expression Recognition

<p>Facial expression helps to communicate between the people for conveying abundant information about human emotions. Facial expression classification is applied in various fields such as remote learning education, medical care, and smart traffic. However, due to the complexity and diversity of the facial emotions, the present facial expression recognition model causes a low recognition rate and it is hard to extract the precise features that are related to facial expression changes. In order to overcome this problem, we proposed Multi-feature Integrated Concurrent Neural Network (MICNN) which is significantly different from the single neural network architectures. It aggregates the prominent features of facial expressions by integrating the three kinds of networks such as Sequential Convolutional Neural Network (SCNN), Residual Dense Network (RDN), and Attention Residual Learning Network (ARLN) to enhance the accuracy rate of facial emotions detection system. Additionally, Local Binary Pattern (LBP) and Principal Component Analysis (PCA) are applied for representing the facial features and these features are combined with the texture features identified by the Gray-level Co-occurrence Matrix (GLCM). Finally, the integrated features are fed into softmax layer to classify the facial images. The experiments are carried out on benchmark datasets by applying k-fold cross-validation and the results demonstrate the superiority of the proposed model.</p> <p> </p>

A Bird\'s Eye View of Human Facial Expression Recognition Techniques

bstract: Human Face Expression Recognition is one of the most remarkable and promising errands in friendly interaction. For the most part, facial emotions are normal and straight forward means for human creatures to impart their expressions and goals. Facial emotions are the critical attributes of non-verbal interactions. In this paper depicts the review of Face Expression Recognition (FER) strategies which incorporate the three significant steps, for example, preprocessing, face detection and emotion classification. In this review work, clarifies the different kinds of FER procedures with its significant handout. Till now many works is done in Facial expression recognition. This paper will in general concentrate on the work done in recognition utilizing various methods. Facial recognition is significant viewpoint in making more human like PCs where human intelligent machine can be construct. Various methods are utilized till now on various viewpoints; in this paper we will be mainly focusing on feature based facial expression recognition. This paper additionally centers on classification strategies.

Facial Emotion Expressions in Human–Robot Interaction: A Survey

AbstractFacial expressions are an ideal means of communicating one’s emotions or intentions to others. This overview will focus on human facial expression recognition as well as robotic facial expression generation. In the case of human facial expression recognition, both facial expression recognition on predefined datasets as well as in real-time will be covered. For robotic facial expression generation, hand-coded and automated methods i.e., facial expressions of a robot are generated by moving the features (eyes, mouth) of the robot by hand-coding or automatically using machine learning techniques, will also be covered. There are already plenty of studies that achieve high accuracy for emotion expression recognition on predefined datasets, but the accuracy for facial expression recognition in real-time is comparatively lower. In the case of expression generation in robots, while most of the robots are capable of making basic facial expressions, there are not many studies that enable robots to do so automatically. In this overview, state-of-the-art research in facial emotion expressions during human–robot interaction has been discussed leading to several possible directions for future research.

Face detection and grimace scale prediction of white furred mice

Studying the facial expressions of humans has been one of the major applications of computer vision. An open question is whether common machine learning techniques can also be used to track behaviors of animals, which is a less explored research problem. Since animals are not capable of verbal communication, computer vision solutions can provide valuable information to track the animal’s state. We are particularly interested in pain neurobiology research, where rodent models are extensively used to investigate pain interventions. A grimace scale is used to understand the suffering of a mouse in the presence of interventions, which is inferred from various facial features such as the shape of the eyes and ears. In this work, we automate the prediction of the grimace scale on white furred mice using a machine learning approach, following the same principles used for human facial expression recognition: face detection, landmark region extraction, and expression recognition. We demonstrate the use of the you only look once (YOLO) framework for face detection of the mice with outstanding results. For eye region extraction and grimace pain prediction, we propose a novel structure based on a dilated convolutional network. The experimental results are promising, showing that it is possible to differentiate among the pain scale of the mice.

Real Time Facial Expression Recognition for Online Lecture

In order to better improve online teaching during the epidemic, teachers can adjust the teaching according to the students’ understanding. During the epidemic period, online teaching has become a basic way for teachers to teach. One big problem with online teaching is that the lecturers find it hard to see the students’ facial expressions. According to the study, the students’ facial expressions are essential for the instructor to understand the students’ understanding of the course material. There are some relevant studies on the recognition of human facial expressions. However, most of them did not consider some expressions such as “Enlightened,” “Confused,” or “Bored.” This study helps teachers to improve the quality of teaching by designing a program to provide them with students’ facial expressions. The study, implemented on students’ computers, can capture their faces from a camera, identify some common and useful facial expressions, and send them to lecturers regularly during online lectures. The main research methods used in this paper are as follows: (1) Use the face recognition algorithm provided by OpenCV to capture faces. (2) Train a convolutional neural network model to recognize the expression “Happy,” “Surprised,” “Neutral,” “Enlightened,” “Confusion,” and “Boredom.” (3) Use the message protocol EMQX to transmit the expression information. This study can successfully capture faces, and about 80% accuracy can identify expressions and successfully transmit expression information. This study contains some expressions rarely studied by others and is innovative in the field of facial expression recognition. Datasets, deep learning models, and results are available for other research teams.

Multi-Pose Facial Expression Recognition Using Hybrid Deep Learning Model with Improved Variant of Gravitational Search Algorithm

The recognition of human facial expressions with the variation of poses is one of the challenging tasks in real-time applications such as human physiological interaction detection, intention analysis, marketing interest evaluation, mental disease diagnosis, etc. This research work addresses the problem of expression recognition from different facial poses at the yaw angle. The major contribution of the paper is the proposal of an autonomous pose variant facial expression recognition framework using the amalgamation of a hybrid deep learning model with an improved quantum inspired gravitational search algorithm. The hybrid deep learning model is the integration of the convolutional neural network and recurrent neural network. The applicability of the hybrid deep learning model can be considered as significant if the feature set is efficiently optimized to have the discriminative features respective to each expression class. Here, the Improved Quantum Inspired Gravitational Search Algorithm (IQI-GSA) is utilized for the selection and optimization of features. The IQI-GSA method is significant for optimizing the features compared to quantum-behaved binary gravitation search algorithm for handing the local optima and stochastic characteristics. Comparing with state-of-art techniques, the proposed framework exhibits the outperformed recognition rate for experimentation on Karolinska Directed Emotional Faces (KDEF) and Japanese Female Facial Expression (JAFFE) datasets.

A Review on Human Facial Expression Recognition and the Governing Physics

A Review on Human Facial Expression Recognition and the Governing Physics

Recognition of Human Facial Expressions Using DCT-DWT and Artificial Neural Network

Facial expressions are a term that expresses a group of movements of the facial fore muscles that is related to one's own human emotions. Human–computer interaction (HCI) has been considered as one of the most attractive and fastest-growing fields. Adding emotional expression’s recognition to expect the users’ feelings and emotional state can drastically improves HCI. This paper aims to demonstrate the three most important facial expressions (happiness, sadness, and surprise). It contains three stages; first, the preprocessing stage was performed to enhance the facial images. Second, the feature extraction stage depended on Discrete Wavelet Transform (DWT) and Discrete Cosine Transform (DCT) methods. Third, the recognition stage was applied using an artificial neural network, known as Back Propagation Neural Network (BPNN), on database images from Cohen-Kanade. The method was shown to be very efficient, where the total rate of recognition of the three facial expressions was 92.9%.

An Effective Combination of Textures and Wavelet Features for Facial Expression Recognition

In order to explore the accompanying examination goals for facial expression recognition, a proper combination of classification and adequate feature extraction is necessary. If inadequate features are used, even the best classifier could fail to achieve accurate recognition. In this paper, a new fusion technique for human facial expression recognition is used to accurately recognize human facial expressions. A combination of Discrete Wavelet Features (DWT), Local Binary Pattern (LBP), and Histogram of Gradients (HoG) feature extraction techniques was used to investigate six human emotions. K-Nearest Neighbors (KNN), Decision Tree (DT), Multi-Layer Perceptron (MLP), and Random Forest (RF) were chosen for classification. These algorithms were implemented and tested on the Static Facial Expression in Wild (SWEW) dataset which consists of facial expressions of high accuracy. The proposed algorithm exhibited 87% accuracy which is higher than the accuracy of the individual algorithms.