Smile Detection Research Articles

In-orbit detection of the spectral smile for the Mars Mineral Spectrometer

As a payload of Tianwen-1 (TW-1), the Mars Mineral Spectrometer (MMS) is tasked with acquiring hyperspectral data of the Martian surface to detect material composition. Microdeformations in optical, mechanical, and thermal components result in the MMS experiencing spectral response distortion in orbit, leading to systematic changes in pixel central wavelengths and full width at half maximum (FWHM). Known as the spectral smile, this distortion compromises the accuracy of reflectance inversion and material composition detection. This study introduces a method for detecting the spectral smile through the Martian atmospheric absorption channel, capitalizing on the distinct characteristics of the atmospheric composition and absorption patterns of Mars. A suitable technical route for in-orbit spectral smile detection was established and tested using simulation experiments and MMS-acquired hyperspectral data. Results suggest that the proposed method can attain central wavelength shifts with a maximum error of 0.32 nm and FWHM variations with a maximum error of 1.95 nm. Employing in-orbit spectral smile detection markedly enhances the correction of Martian atmospheric absorption and provides technical support for Martian surface reflectance inversion. https://github.com/wubingnote/MMS-Spectral-Smile.

Personalized teaching of college psychology based on information fusion technology

Abstract This paper explores the cognitive attention mechanism in mental activities, analyzes the learning emotion contained therein as well as the information of thinking activity that carries out the whole cognitive process of mental learning, and obtains the information construction data of the three-dimensional interest model after the estimation of the learner’s head gesture and the recognition of dynamic expression. A conditional random forest model is built using multimodal information fusion technology, and a natural smile detection method is proposed based on it. After training, it realizes the information estimation of the learner’s head pose data, generates the smiley face classifier based on conditional random forest, and determines the teaching decision boundary using K-Means clustering. To analyze the psychological personalized teaching results of the model, an empirical study is conducted through a controlled experiment. The experimental results show that the learning efficiency of the model on the CelebA dataset and SMILEsmileD dataset is improved, the accuracy rate is stabilized at 95% after the number of iterations 10, and the model’s performance is superior. The majority of the students in the experiment have a mastery of psychological knowledge that is around 0.85, and there is a significant positive correlation, and personalized teaching has a more significant effect.

Biometric Authentication based on Liveness Detection Using Face Landmarks and Deep Learning Model

This paper describes the approach to active liveness detection of the face using facial features and movements. The project aims to create a better method for detecting liveness in real-time on an application programming interface (API) server. The project is built using Python programming with the computer vision libraries OpenCV, dlib and MediaPipe and the deep learning library Tensorflow. There are five modules in active liveness detection progress related to different parts or movements on the face: headshakes, nodding, eye blinks, smiles, and mouths. The functionality of modules runs through face landmarking through dlib and MediaPipe and detection of face features through Tensorflow Convolutional Neural Network (CNN) trained in two different approaches: smile detection and eye-blink detection. The result of implementing face landmarking shows an accurate result through the pre-trained model of MediaPipe and the pre-trained parameter of the dlib 68 landmarking model. And more than 90% classification model accuracy in precision, recall, and f1-score for both trained CNNs in detecting smiles and eyes blinking through the Scikit-Learn classification report. In addition, the prototype API is also implemented using the Python RESTful API library, FastAPI, to test the detection functionality in the prototype Android application. The prototype result is outstanding, as the model excellently requests and retrieves from the API server. The possible research path gives the success of real-time detection on API servers for easy implementation of liveness detection on low-spec client devices.

Deep learning framework for real-time face recognition using multimodal facial expression with feature optimization and hybrid classification

Smile detection plays key role in recognizing human facial expressions, especially in inter-personal relations. In modern biometrics and security applications, face recognition is widely used. In real-life applications, automated face recognition has always been challenging because people express themselves differently. Real-world scenarios present many challenges, such as low-quality images, temporal variations, and disguises that alter facial characteristics. Face recognition has been accomplished using deep learning frameworks, but there are still several challenges to overcome, such as expression variations and lack of training data. In these methods, margins are used to enforce intra-class compactness and inter-class discrepancy to prevent overfitting. Multimodal facial expression detection is used for real-time face recognition to solve the aforementioned problems. First, we utilizes the Prewitt edge detector for eyes, nose, mouth, and brow detection, and design a deep transfer learning model to extract deep features from face images. Next, feature optimization is done by harpy eagle search optimization (HESO) to selects optimal best among multiple features which reduce data dimensionality problem. Then, we employ a semi-region based convolutional neural network (SR-CNN) to analyzes the facial expressions from multimodalities such as eyes (blinking), nose (shapes), mouth (smiling), and brow (shapes). The analyzed facial expressions and corresponding input images are fed into the benchmark SVM classifier for face recognition. The facial expression based face recognition framework is most suited for real-time scenario like video surveillance. Finally, we validate the performance our framework using benchmark multi-modal Cohn-Kanade and real-time 5671 person images datasets. The simulation results show that the proposed framework can effectively suppress data dimensionality issues and achieve high recognition rate compare to state-of-art frameworks.

Hippocampal volume mediates the relationship of parental rejection in childhood with social cognition in healthy adults

Childhood abuse reduces hippocampal and amygdala volumes and impairs social cognition, including the ability to recognize facial expressions. However, these associations have been studied primarily in individuals with a history of severe abuse and psychiatric symptoms; researchers have not determined whether these associations can also be observed in healthy adults. In the present study, we analyzed data from 400 healthy adults (208 men and 192 women) at Tamagawa University. Parental rejection reflecting childhood abuse was assessed using the short form of Egna Minnen Beträffande Uppfostran, while social cognition was assessed using the “Fake Smile Detection Task.” Hippocampal and amygdala volumes were extracted from T1-weighted magnetic resonance imaging data using FreeSurfer. We found that greater parental rejection resulted in smaller hippocampal and amygdala volumes and poorer performance in the Fake Smile Detection Task. Structural equation modeling analysis supported the model that hippocampal volume mediates maternal rejection effect on performance on the Fake Smile Detection Task, with involvement of the amygdala. These findings are in line with the structural and functional connectivity found between the hippocampus and amygdala and their joint involvement in social cognition. Therefore, parental rejection may affect hippocampal and amygdala volumes and social cognitive function even in symptom-free adults.

Deeply Smile Detection Based on Discriminative Features with Modified LeNet-5 Network

Deeply Smile Detection Based on Discriminative Features with Modified LeNet-5 Network

Automated detection of smiles as discrete episodes.

Patients seeking restorative and orthodontic treatment expect an improvement in their smiles and oral health-related quality of life. Nonetheless, the qualitative and quantitative characteristics of dynamic smiles are yet to be understood. To develop, validate, and introduce open-access software for automated analysis of smiles in terms of their frequency, genuineness, duration, and intensity. A software script was developed using the Facial Action Coding System (FACS) and artificial intelligence to assess activations of (1) cheek raiser, a marker of smile genuineness; (2) lip corner puller, a marker of smile intensity; and (3) perioral lip muscles, a marker of lips apart. Thirty study participants were asked to view a series of amusing videos. A full-face video was recorded using a webcam. The onset and cessation of smile episodes were identified by two examiners trained with FACS coding. A Receiver Operating Characteristic (ROC) curve was then used to assess detection accuracy and optimise thresholding. The videos of participants were then analysed off-line to automatedly assess the features of smiles. The area under the ROC curve for smile detection was 0.94, with a sensitivity of 82.9% and a specificity of 89.7%. The software correctly identified 90.0% of smile episodes. While watching the amusing videos, study participants smiled 1.6 (±0.8) times per minute. Features of smiles such as frequency, duration, genuineness, and intensity can be automatedly assessed with an acceptable level of accuracy. The software can be used to investigate the impact of oral conditions and their rehabilitation on smiles.

Discriminating Fake and Real Smiles Using Electroencephalogram Signals With Convolutional Neural Networks

Genuineness of smiles is of particular interest in the field of human emotions and social interactions. In this work, we develop an experimental protocol to elicit genuine and fake smile expressions on 28 healthy subjects. Then, we assess the type of smile expressions using electroencephalogram (EEG) signals with convolutional neural networks (CNNs). Five different architectures (CNN1, CNN2, CNN3, CNN4, and CNN5) were examined to differentiate between fake and real smiles. We transform the temporal EEG signals into normalized gray-scale images and perform three-way classification to classify fake smiles, genuine smiles, and neutral expressions in the form of subject-dependent classification. We achieved the highest classification accuracy of 90.4% using CNN1 for the full EEG spectrum. Likewise, we achieved classification accuracies of 87.4%, 88.3%, 89.7%, and 90.0% using Beta, Alpha, Theta, and Delta EEG bands respectively. This paper suggests that CNNs models, widely used in image classification problems, can provide an alternative approach for smile detection from physiological signals such as the EEG.

Facial Feature Extraction for Emotion Classification Using Fuzzy C-Mean Clustering

Background: Automatic human emotion recognition system is an active area of research due to its wide applications in the field of Human Computer Interaction(HCI) systems, driver fatigue monitoring systems, surveillance systems, human assistance systems, smile detectors etc. Objective: The study presents a fuzzy based approach to extract facial features from input image and builds different classification models to classify the image into two emotion classes i.e. happy and neutral. The system has potential implications in the field of smile detection systems, customer experience analysis and patient monitoring systems. Methods: The proposed system determines the dimensional attributes (l-attribute and w-attribute) of mouth region extracted from the facial image using viola-jones algortithm. The feature set is generated by using a total of 136 images from JAFFE, NimStim and MUG dataset. The differentiating power of the attribures is then evaluated using five different classification models. Results: The accuracy, precision and recall is determined for each classification model. The results show good accuracy of 70% for the grayscale JAFFE and NimStim databases and 95% for the coloured MUG database. Conclusion: The mouth features calculated in the study are based on the geometric coordinates which eliminates the possibility of false distance measurements due to presence of noise or shadows.

Review on Smile Detection

Facial expressions are a result of specific movement of face muscles, and these face expressions are considered as a visible sign of a person’s internal thought process, intensions, and internal emotional states. Smile is such a face expression which often indicates, satisfaction, agreement, happiness, etc. Though, a lot of studies have been done over detection of Facial Expression in last decade, smile detection had attracted researcher for more deeper studies. In this review paper, different type of available smile detection so far has been discussed such as Deep Convolutional Neural Network (CNN), Hidden Marcov Model(HMM), K-Nearest Neighbours(KNN), Self Similarity of Gradient(GSS), Histogram of Oriented Gradients (HOG), Gabor-Energy Filters and Local Binary Pattern(LBP) etc and classifier like HAAR Classifier, Hidden Markov Model(HMM), Adaboost Support Vector Machine (SVM),Softmax Classifier and Extreme Learning Machine(ELM).This review paper will prove beneficial for learning about smile detection and its application.

Multiple Smile Detection Using Histogram of Oriented Gradient and Support Vector Machine Methods

The face or the front of the head consists of the eyes, nose and mouth. Each face has its uniqueness from the many human faces. The face is used to show happy and sad expressions and feelings. Smiling also includes self-expression from others. The analysis of facial expressions plays a key role in analyzing human emotions and behavior. Smile detection is a specialized task in facial expression analysis with a variety of potential applications such as photo selection, user experience analysis, smile payments, and patient monitoring. Conventional approaches often extract low-level face descriptors and smile detection based on strong binary classifiers. In this paper, we propose an effective Histogram of Oriented method supporting vector engines for smile detection. The experimental results show that our proposed network outperforms the current state of the method. The test results using the Histogram of Oriented Gradient and Support Vector Machine method for smile detection were 87% for a precision value of 88% and a recall value of 83% and accuracy. In the future, we want to exploit some of the latest effective designs. we will try to update our mouth model so we can support a bigger head turn and face size scale.

Assessment of the Viability of a Biometric Characteristic in the Context of Biometric Authentication on Mobile Devices

The issue of safe utilization of mobile devices is becoming an increasingly important problem, among others due to the widespread use of such devices to access sensitive data (such as electronic documents or banking data). In our work we analyze the use of biometric techniques in order to secure a mobile device, with particular emphasis on the viability of selected biometric characteristics. For this purpose, we investigate the possibility of applying machine learning models to assess the authenticity of a biometric characteristic. Results of our tests have shown that the most effective method of assessing the viability of a biometric characteristic involves blink and smile detection.

Faded Smiles? A Largescale Observational Study of Smiling from Adolescence to Old Age

A relatively large body of work exists examining sex differences in expressiveness; however, there remains little research of differences in expressiveness associated with aging. Observational studies of facial expressivity across ages are limited in part due to the poor scalability of traditional research methods. We collected over 17,000 videos of natural facial behavior using the Internet and performed a large observational study of smiling responses of people ages 18 to 70 years. Using automated facial coding we quantified the presence of smiles as people watched a set of controlled mundane online content. The likelihood of smiles and the duration of smiles increased with age. We attribute this to greater expression of positive emotion in older people. Women smiled more than men over all and gender differences increased significantly with age. We question whether results may be influenced by the effect of age on the accuracy of the automated smile detection; however, validation on a large set of human coded videos shows that the observed effects were not due to smile detection performance.

WITHDRAWN: Face smile determination using face and smile detection for perceptual user interfaces (PUIs) for real-time interaction

Face recognition and biometric analytics is a novel domain of research and enormous algorithms going in this dimension. The identification of smile is a two-stage operation. You sense a face first and then wait for a smile and a motion detector breaks the shot in thousands of zones, Criteria for analysis such as white balance and facial level. If a human smile, by specifying various parameters, the camera detects a facial defect, including closing your eyes, clear teeth, bending your lips and raising your lips. In order to improve the sensitivity of the smile automatic function you should change the camera parameters. Authentic smile identification is more successful when participants, particularly their eyes, (with bangs, etc.) do not cover their faces. Hats, masks or sunglasses may also obstruct the work. You should have a big and open-mouthed grin on your subjects. When your teeth are clear, your camera can still detect smiling better. The presented work is focusing on the face smile analytics with the Perceptual User Interfaces for cumulative performance in the biometric analytics.

An Unconventional Framework for Smile Detection using Eye StatesThe Description of Wood’s Lamp and Dermoscopy on Patients suspected of Melasma in Adam Malik Hospital Medan

An Unconventional Framework for Smile Detection using Eye StatesThe Description of Wood’s Lamp and Dermoscopy on Patients suspected of Melasma in Adam Malik Hospital Medan

Smile Detection Using Data Amalgamation

A smile is a specific movement of face muscles to relay an optimistic feeling. A smile represents satisfaction and happiness. Many application created using smile detection technology, for example product rating, patient monitoring, image capturing, video conferencing and interactive systems. There are many smile detections techniques have been proposed for smile detection in the unconstrained scenarios. However, the dimensions of most notable feature descriptors are humongous, which is challenging in real-time applications. Besides, feature should be more powerful to identify between smiling and non-smiling face. The proposed method has two consecutive actions: 1) amalgamation of geometric feature extraction (GFE) and regional local binary pattern (LBP) features extraction using autoencoders; 2) Kohonen selforganizing map (KSOM) is adopted to classify smile based on these features. The proposed method is mathematics more dynamic and performance wise more precise. The performance of the propounded approach is proved on GENKI-4K database.

Interactive video-player to improve social smile in individuals with autism spectrum disorder

PurposeThe purpose of this paper is to propose and develop a live interaction-based video player system named LIV4Smile for the improvement of the social smile in individuals with autism spectrum disorder (ASD).Design/methodology/approachThe proposed LIV4Smile intervention was a video player that operated by detecting smile using a convolutional neural network (CNN)-based algorithm. To maintain a live interaction, a CNN-based smile detector was configured and used in this system. The statistical test was also conducted to validate the performance of the system.FindingsThe significant improvement was observed in smile responses of individuals with ASD with the utilization of the proposed LIV4Smile system in a real-time environment.Research limitations/implicationsA small sample size and clinical utilizing for validation and initial training of ASD individuals for LIV4Smile could be considered under implications.Originality/valueThe main aim of this study was to address the inclusive practices for children with autism. The proposed CNN algorithm-based LIV4Smile intervention resulted in high accuracy in facial smile detection.

Fast facial smile detection using convolutional neural network in an intelligent working environment

Infrared facial expression recognition is very important to understand the human emotion and behaviors. Smile detection plays a key role in facial expression recognition. In this paper, we have developed a fast facial smile detection algorithm with convolutional neural network and infrared imaging technology. Due to random noise and band overlap in the infrared imaging spectroscopy, we propose a maximum likelihood (ML) restoration algorithm with spectral-spatial L1-norm constraint for low-resolution infrared spectrum. In this model, we also employ the spectral-spatial L1 norm to contrain the smoothness of the infrared spectrum data. With the aid of spectral-spatial L1-norm minimization as compelling constraints, the proposed method can not only remove noise dramatically but also compute the instrument blurry function simultaneously. The results show that the developed method can recover a noise-free infrared spectrum, and the spectral structure is richer than the compared methods. The high-resolution infrared spectrum could be utilized for fast facial smile detection task in the intelligent learning environment.

Real-Time Smile Detection using Deep Learning

Real-time smile detection from facial images is useful in many real world applications such as automatic photo capturing in mobile phone cameras or interactive distance learning. In this paper, we study different architectures of object detection deep networks for solving real-time smile detection problem. We then propose a combination of a lightweight convolutional neural network architecture (BKNet) with an efficient object detection framework (RetinaNet). The evaluation on the two datasets (GENKI-4K, UCF Selfie) with a mid-range hardware device (GTX TITAN Black) show that our proposed method helps in improving both accuracy and inference time of the original RetinaNet to reach real-time performance. In comparison with the state-of-the-art object detection framework (YOLO), our method has higher inference time, but still reaches real-time performance and obtains higher accuracy of smile detection on both experimented datasets.

A hybrid framework for smile detection in class imbalance scenarios

In this study, we consider the problem of smile detection in both an imbalanced data scenario, in which the number of smile images is in the minority compared with the number of neutral images, and a balanced data scenario. We first propose a smile detection model using a convolutional neural network (SD-CNN) to improve the performance in the balanced data scenario, and then a hybrid deep learning framework (HF-SD) that uses a modification of the SD-CNN model to learn and then extracts the features from dataset. These extracted features are then used to train an extreme gradient boosting approach to handle the imbalanced problem. An experiment shows that the proposed model has impressive discriminative ability for smile detection, in both balanced and imbalanced data scenarios, compared with existing approaches. HF-SD yields an accuracy of 93.6% and outperforms the state-of-the-art approaches for the original GENK14K database in the balanced data scenario. The results of the second experiment show that HF-SD also achieves better AUCs (area under the receiver operating characteristic curve) compared with the state-of-the-art methods for smile detection in an imbalanced data scenario with different balancing ratios.