Facial Image Analysis Research Articles

Facial image analysis for automated suicide risk detection with deep neural networks

Accurately assessing suicide risk is a critical concern in mental health care. Traditional methods, which often rely on self-reporting and clinical interviews, are limited by their subjective nature and may overlook non-verbal cues. This study introduces an innovative approach to suicide risk assessment using facial image analysis. The Suicidal Visual Indicators Prediction (SVIP) Framework leverages EfficientNetb0 and ResNet architectures, enhanced through Bayesian optimization techniques, to detect nuanced facial expressions indicating mental state. The models’ interpretability is improved using GRADCAM, Occlusion Sensitivity, and LIME, which highlight significant facial regions for predictions. Using datasets DB1 and DB2, which consist of full and cropped facial images from social media profiles of individuals with known suicide outcomes, the method achieved 67.93% accuracy with EfficientNetb0 on DB1 and up to 76.6% accuracy with a Bayesian-optimized Support Vector Machine model using ResNet18 features on DB2. This approach provides a less intrusive, accessible alternative to video-based methods and demonstrates the substantial potential for early detection and intervention in mental health care.

Implementation of Convolutional Neural Networks (CNN) in An Emotion Detection System for Measuring Learning Concentration Levels

Technological advancements have had a significant impact on the education sector, including the application of Convolutional Neural Networks (CNN) for facial image analysis. This research aims to implement CNN to measure students' learning concentration levels. The FER2013 dataset, which includes seven emotion classifications and comprises 28,709 images for training data, is used as the database. The data is processed through rescaling and augmentation to prepare the CNN model. The model consists of several convolutional layers, pooling layers, and fully connected layers designed to extract crucial features from facial images. Evaluation results demonstrate a very high accuracy of 94.95% on training data, indicating that the model effectively recognizes complex patterns within the data. Although there is a higher loss value of 157% and a decreased accuracy of 62.75% on validation data, this suggests that the model possesses a strong foundational capability and can still be improved through further adjustments. With high accuracy in training and promising validation results, the model shows substantial potential for real-world application, where it can assist teachers in understanding students' emotional responses in real-time. The implementation of CNN aids educators in comprehending students' emotional responses and adapting their teaching methods more effectively, thereby creating a more conducive learning environment and enhancing students' academic and social development. These findings also open opportunities for further research to improve the performance and generalization of the model on unseen data, making this technology an increasingly reliable tool in education

Detection and Segmentation of Mouth Region in Stereo Stream Using YOLOv6 and DeepLab v3+ Models for Computer-Aided Speech Diagnosis in Children

This paper describes a multistage framework for face image analysis in computer-aided speech diagnosis and therapy. Multimodal data processing frameworks have become a significant factor in supporting speech disorders’ treatment. Synchronous and asynchronous remote speech therapy approaches can use audio and video analysis of articulation to deliver robust indicators of disordered speech. Accurate segmentation of articulators in video frames is a vital step in this agenda. We use a dedicated data acquisition system to capture the stereovision stream during speech therapy examination in children. Our goal is to detect and accurately segment four objects in the mouth area (lips, teeth, tongue, and whole mouth) during relaxed speech and speech therapy exercises. Our database contains 17,913 frames from 76 preschool children. We apply a sequence of procedures employing artificial intelligence. For detection, we train the YOLOv6 (you only look once) model to catch each of the three objects under consideration. Then, we prepare the DeepLab v3+ segmentation model in a semi-supervised training mode. As preparation of reliable expert annotations is exhausting in video labeling, we first train the network using weak labels produced by initial segmentation based on the distance-regularized level set evolution over fuzzified images. Next, we fine-tune the model using a portion of manual ground-truth delineations. Each stage is thoroughly assessed using the independent test subset. The lips are detected almost perfectly (average precision and F1 score of 0.999), whereas the segmentation Dice index exceeds 0.83 in each articulator, with a top result of 0.95 in the whole mouth.

Artificial Intelligence-Driven Facial Image Analysis for the Early Detection of Rare Diseases: Legal, Ethical, Forensic, and Cybersecurity Considerations

Artificial Intelligence-Driven Facial Image Analysis for the Early Detection of Rare Diseases: Legal, Ethical, Forensic, and Cybersecurity Considerations

ENHANCING DEEPFAKE DETECTION: LEVERAGING MTCNN AND INCEPTION RESNET

In response to the escalating threat posed by manipulated facial imagery in the digital age, our research project is dedicated to developing an advanced framework for the detection and classification of such content, augmented by a proactive user reporting mechanism. Leveraging state-of-the-art deep learning models like Multi-Task Cascaded Convolutional Networks (MTCNN) and InceptionResnetV1, our framework achieves an impressive accuracy rate of 92% in distinguishing between genuine and manipulated faces. The integration of explainability methods such as Grad-CAM enhances model interpretability, empowering users to understand model predictions. Additionally, our user-centric reporting interface enables active user participation in identifying and flagging potentially manipulated content, fostering transparency and accountability in digital media platforms. With the continued proliferation of deepfake technology, our research endeavors not only advance facial image analysis techniques but also uphold principles of trust and integrity in the digital realm, aiming to safeguard the credibility of information dissemination through vigilance, innovation, and collaborative action. Keywords: Deepfake detection, Facial image manipulation, Deep learning models, User reporting mechanism, Model interpretability, Transparency in digital media, Trust and integrity, Information dissemination, Vigilance, Collaborative action.

Facial expression recognition for stress detection: A Conv-XGBoost Algorithm approach

Facial Emotion Recognition (FER) is a powerful tool for gaining insights into human behaviour and well-being by precisely quantifying a wide range of emotions especially stress, through the analysis of facial images. Detecting stress using FER entails meticulously examining subtle facial cues, such as changes in eye movements, brow furrowing, lip tightening, and muscle contractions. To assure effectiveness and real-time processing, FER approaches based on deep learning and artificial intelligence (AI) techniques was created using edge modules. This research introduces a novel approach for identifying stress, leveraging the Conv-XGBoost Algorithm to analyse facial emotions. The proposed model sustain rigorous evaluation techniques, for employing key metrics examination such as the F1 score, validation accuracy, precision, and recall rate to assess its real-world reliability and robustness. This comprehensive analysis and validation proved the model’s practical utility in facial analysis. Integrating the Conv-XGBoost Algorithm with facial emotion analysis represents a promising and highly accurate solution for efficient stress detection. The method surpasses existing literature and demonstrate significant potential for practical applications based on well-validated data.

Xception transfer learning with early stopping for facial age estimation

The rapid development of deep learning attracts more attention to the analysis of person's face images. Deep learning methodsof facial age estimation are more effective compared to methods based on anthropometric models, models of active appearance, texture models, subspace of aging patterns. However, deep learning networks require more computing power to process images. Pre-trained models do not need a large training set and their training time is less. However, the parameters obtained as a result of transfer learning of the pre-training network significantly affect its efficiency. It is also necessary to take into account the properties of the processed images, in particular, the conditions under which they were obtained.Recently, the facial age estimation is implemented in applications in devices with limited resources of computing, for example, in smartphones. The memory size and power consumption of such applications are limited by the computing power of mobile devices. In addition, when photographing a person's face with a smartphone camera, it is very difficult to ensure the uniform lighting. The aim of the research is reducing the error of facial age estimation from uneven illuminated images by applying an early stopping of transfer learning of the Xception network. The proposed technique of transfer learning includes an early stopping of training, if the improvement of the results is not observed within a certain number of epochs. Then the network weights from the epoch with the lowest validation loss are saved. As a result of the proposed technique applying, the average absolute error of age estimation was about five years from unevenly illuminated test images. A number of parameters of the used in this case Xceptionnetwork is less than that of other deep learning neural networks which solved the age estimation problem. Then applying of the Xception network reduces the resource consumption of devices with limited computing power. Prospects for further research are reducing the unevenness of facial image lighting to decrease the error of age estimation. Also, to reduce the computing resources, it is promising to use fast transforms in the Xception convolutional layers.

Predicting human chronological age via AI analysis of dorsal hand versus facial images: A study in a cohort of Indian females.

Predicting a person's chronological age (CA) from visible skin features using artificial intelligence (AI) is now commonplace. Often, convolutional neural network (CNN) models are built using images of the face as biometric data. However, hands hold telltale signs of a person's age. To determine the utility of using only hand images in predicting CA, we developed two deep CNNs based on 1) dorsal hand images (H) and 2) frontal face images (F). Subjects (n = 1454) were Indian women, 20-80 years, across three geographic cohorts (Mumbai, New Delhi and Bangalore) and having a broad variation in skin tones. Images were randomised: 70% of F and 70% of H were used to train CNNs. The remaining 30% of F and H were retained for validation. CNN validation showed mean absolute error for predicting CA using F and H of 4.1 and 4.7 years, respectively. In both cases correlations of predicted and actual age were statistically significant (r(F) = 0.93, r(H) = 0.90). The CNNs for F and H were validated for dark and light skin tones. Finally, by blurring or accentuating visible features on specific regions of the hand and face, we identified those features that contributed to the CNN models. For the face, areas of the inner eye corner and around the mouth were most important for age prediction. For the hands, knuckle texture was a key driver for age prediction. Collectively, for AI estimates of CA, CNNs based solely on hand images are a viable alternative and comparable to CNNs based on facial images.

Utilization of Facial PPG Signals as a Novel Source for Blood Pressure Estimation

Hypertension, a prevalent global health concern, necessitates accurate and non-invasive blood pressure estimation techniques for effective monitoring and management. This paper proposes a novel machine learning approach utilizing Photo plethysmography (PPG) signals for precise blood pressure estimation. PPG signals, obtained conveniently through wearable devices, offer valuable physiological information related to cardiovascular activity. Leveraging advanced machine learning algorithms, including deep learning architectures and feature extraction methods, our proposed technique aims to establish a robust model for blood pressure estimation using facial image analysis. The methodology involves preprocessing PPG signals, extracting relevant features, and employing sophisticated machine learning models for regression analysis. The evaluation of this novel approach involves comprehensive experimentation with diverse datasets, ensuring its efficacy across various demographic groups and conditions. Results demonstrate promising accuracy and reliability in estimating blood pressure values, suggesting the potential for practical implementation in healthcare settings. The proposed technique showcases a promising avenue for non-invasive and accessible blood pressure monitoring, contributing significantly to personalized healthcare and continuous health monitoring systems.

Cold sensitivity classification using facial image based on convolutional neural network

Objectives: Facial diagnosis is an important part of clinical diagnosis in traditional East Asian Medicine. In this paper, we proposed a model to quantitatively classify cold sensitivity using a fully automated facial image analysis system.Methods: We investigated cold sensitivity in 452 subjects. Cold sensitivity was determined using a questionnaire and the Cold Pattern Score (CPS) was used for analysis. Subjects with a CPS score below the first quartile (low CPS group) belonged to the cold non-sensitivity group, and subjects with a CPS score above the third quartile (high CPS group) belonged to the cold sensitivity group. After splitting the facial images into train/validation/test sets, the train and validation set were input into a convolutional neural network to learn the model, and then the classification accuracy was calculated for the test set.Results: The classification accuracy of the low CPS group and high CPS group using facial images in all subjects was 76.17%. The classification accuracy by sex was 69.91% for female and 62.86% for male. It is presumed that the deep learning model used facial color or facial shape to classify the low CPS group and the high CPS group, but it is difficult to specifically determine which feature was more important.Conclusions: The experimental results of this study showed that the low CPS group and the high CPS group can be classified with a modest level of accuracy using only facial images. There was a need to develop more advanced models to increase classification accuracy.

Cultivating Insight: Detecting Autism Spectrum Disorder through Residual Attention Network in Facial Image Analysis

Revolutionizing Autism Spectrum Disorder Identification through Deep Learning: Unveiling Facial Activation Patterns. In this study, our primary objective is to harness the power of deep learning algorithms for the precise identification of individuals with autism spectrum disorder (ASD) solely from facial image datasets. Our investigation centers around the utilization of face activation patterns, aiming to uncover novel insights into the distinctive facial features of ASD patients. To accomplish this, we meticulously examined facial imaging data from a global and multidisciplinary repository known as the Autism Face Imaging Data Exchange. Autism spectrum disorder is characterized by inherent social deficits and manifests in a spectrum of diverse symptomatic scenarios. Recent data from the Centers for Disease Control (CDC) underscores the significance of this disorder, indicating that approximately 1 in 54 children are impacted by ASD, according to estimations from the CDC's Autism and Developmental Disabilities Monitoring Network (ADDM). Our research delved into the intricate functional connectivity patterns that objectively distinguish ASD participants, focusing on their facial imaging data. Through this investigation, we aimed to uncover the latent facial patterns that play a pivotal role in the classification of ASD cases. Our approach introduces a novel module that enhances the discriminative potential of standard convolutional neural networks (CNNs), such as ResNet-50, thus significantly advancing the state-of-the-art. Our model achieved an impressive accuracy rate of 99% in distinguishing between ASD patients and control subjects within the dataset. Our findings illuminate the specific facial expression domains that contribute most significantly to the differentiation of ASD cases from typically developing individuals, as inferred from our deep learning methodology. To validate our approach, we conducted real-time video testing on diverse children, achieving an outstanding accuracy score of 99.90% and an F1 score of 99.67%. Through this pioneering work, we not only offer a cutting-edge approach to ASD identification but also contribute to the understanding of the underlying facial activation patterns that hold potential for transforming the diagnostic landscape of autism spectrum disorder.

Computer aided diagnosis of neurodevelopmental disorders and genetic syndromes based on facial images – A systematic literature review

Neurodevelopment disorders can result in facial dysmorphisms. Therefore, the analysis of facial images using image processing and machine learning techniques can help construct systems for diagnosing genetic syndromes and neurodevelopmental disorders. The systems offer faster and cost-effective alternatives for genotyping tests, particularly when dealing with large-scale applications. However, there are still challenges to overcome to ensure the accuracy and reliability of computer-aided diagnosis systems. This article presents a systematic review of such initiatives, including 55 articles. The main aspects used to develop these diagnostic systems were discussed, namely datasets - availability, type of image, size, ethnicities and syndromes - types of facial features, techniques used for normalization, dimensionality reduction and classification, deep learning, as well as a discussion related to the main gaps, challenges and opportunities.

Hybrid Techniques of Facial Feature Image Analysis for Early Detection of Autism Spectrum Disorder Based on Combined CNN Features.

Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder characterized by difficulties in social communication and repetitive behaviors. The exact causes of ASD remain elusive and likely involve a combination of genetic, environmental, and neurobiological factors. Doctors often face challenges in accurately identifying ASD early due to its complex and diverse presentation. Early detection and intervention are crucial for improving outcomes for individuals with ASD. Early diagnosis allows for timely access to appropriate interventions, leading to better social and communication skills development. Artificial intelligence techniques, particularly facial feature extraction using machine learning algorithms, display promise in aiding the early detection of ASD. By analyzing facial expressions and subtle cues, AI models identify patterns associated with ASD features. This study developed various hybrid systems to diagnose facial feature images for an ASD dataset by combining convolutional neural network (CNN) features. The first approach utilized pre-trained VGG16, ResNet101, and MobileNet models. The second approach employed a hybrid technique that combined CNN models (VGG16, ResNet101, and MobileNet) with XGBoost and RF algorithms. The third strategy involved diagnosing ASD using XGBoost and an RF based on features of VGG-16-ResNet101, ResNet101-MobileNet, and VGG16-MobileNet models. Notably, the hybrid RF algorithm that utilized features from the VGG16-MobileNet models demonstrated superior performance, reached an AUC of 99.25%, an accuracy of 98.8%, a precision of 98.9%, a sensitivity of 99%, and a specificity of 99.1%.

New Trends in Emotion Recognition Using Image Analysis by Neural Networks, A Systematic Review.

Facial emotion recognition (FER) is a computer vision process aimed at detecting and classifying human emotional expressions. FER systems are currently used in a vast range of applications from areas such as education, healthcare, or public safety; therefore, detection and recognition accuracies are very important. Similar to any computer vision task based on image analyses, FER solutions are also suitable for integration with artificial intelligence solutions represented by different neural network varieties, especially deep neural networks that have shown great potential in the last years due to their feature extraction capabilities and computational efficiency over large datasets. In this context, this paper reviews the latest developments in the FER area, with a focus on recent neural network models that implement specific facial image analysis algorithms to detect and recognize facial emotions. This paper's scope is to present from historical and conceptual perspectives the evolution of the neural network architectures that proved significant results in the FER area. This paper endorses convolutional neural network (CNN)-based architectures against other neural network architectures, such as recurrent neural networks or generative adversarial networks, highlighting the key elements and performance of each architecture, and the advantages and limitations of the proposed models in the analyzed papers. Additionally, this paper presents the available datasets that are currently used for emotion recognition from facial expressions and micro-expressions. The usage of FER systems is also highlighted in various domains such as healthcare, education, security, or social IoT. Finally, open issues and future possible developments in the FER area are identified.

Combined genome-wide association study of 136 quantitative ear morphology traits in multiple populations reveal 8 novel loci.

Human ear morphology, a complex anatomical structure represented by a multidimensional set of correlated and heritable phenotypes, has a poorly understood genetic architecture. In this study, we quantitatively assessed 136 ear morphology traits using deep learning analysis of digital face images in 14,921 individuals from five different cohorts in Europe, Asia, and Latin America. Through GWAS meta-analysis and C-GWASs, a recently introduced method to effectively combine GWASs of many traits, we identified 16 genetic loci involved in various ear phenotypes, eight of which have not been previously associated with human ear features. Our findings suggest that ear morphology shares genetic determinants with other surface ectoderm-derived traits such as facial variation, mono eyebrow, and male pattern baldness. Our results enhance the genetic understanding of human ear morphology and shed light on the shared genetic contributors of different surface ectoderm-derived phenotypes. Additionally, gene editing experiments in mice have demonstrated that knocking out the newly ear-associated gene (Intu) and a previously ear-associated gene (Tbx15) causes deviating mouse ear morphology.

The future role of facial image analysis in ACMG classification guidelines.

The use of next-generation sequencing (NGS) has dramatically improved the diagnosis of rare diseases. However, the analysis of genomic data has become complex with the increasing detection of variants by exome and genome sequencing. The American College of Medical Genetics and Genomics (ACMG) and the Association for Molecular Pathology (AMP) developed a 5-tier classification scheme in 2015 for variant interpretation, that has since been widely adopted. Despite efforts to minimise discrepancies in the application of these criteria, inconsistencies still occur. Further specifications for individual genes were developed by Variant Curation Expert Panels (VCEPs) of the Clinical Genome Resource (ClinGen) consortium, that also take into consideration gene or disease specific features. For instance, in disorders with a highly characerstic facial gestalt a "phenotypic match" (PP4) has higher pathogenic evidence than e.g. in a non-syndromic form of intellectual disability. With computational approaches for quantifying the similarity of dysmorphic features results of such analysis can now be used in a refined Bayesian framework for the ACMG/AMP criteria.

Exploring the Potential of A-ResNet in Person-Independent Face Recognition and Classification

Abstract This study offers a novel face recognition and classification method based on classifiers that use statistical local features. The use of ResNet has generated growing interest in a variety of areas of image processing and computer vision in recent years and demonstrated its usefulness in several applications, especially for facial image analysis, which includes tasks as varied as face detection, face recognition, facial expression analysis, demographic classification, etc. This paper is divided into two steps i.e. face recognition and classification. The first step in face recognition is automatic data cleansing which is done with the help of Multi-Task Cascaded Convolutional Neural Networks (MTCNNs) and face.evoLVe, followed by parameter changes in MTCNN to prevent dirty data. The authors next trained two models: Inception-ResNetV1, which had pre-trained weights, and Altered-ResNet (A-ResNet), which used Conv2d layers in ResNet for feature extraction and pooling and softmax layers for classifications. The authors use the best optimizer after comparing a number of them during the training phase, along with various combinations of batch and epoch. A-ResNet, the top model overall, detects 86/104 Labelled Faces in the Wild (LFW) dataset images in 0.50 seconds. The proposed approach was evaluated and received an accuracy of 91.7%. Along with this, the system achieved a training accuracy of 98.53% and a testing accuracy of 99.15% for masked face recognition. The proposed method exhibits competitive outcomes when measured against other cutting-edge algorithms and models. Finally, when it comes to why the suggested model is superior to ResNet, it may be because the A-ResNet is simpler thus it can perform at its best with little data, whereas deeper networks require higher data size.

Comparative analysis of original face and skin-warped average face images for the scoring of skin attributes.

Representative of a panel, an average face image could be used to analyse/display skin changes while alleviating image rights constraints. Therefore, we used landmark-based deformation (warping) of individual skin images onto their panel's average face, evaluating this approach's relevance and possible limits. An average front face image was constructed from images of 71 Japanese women (50-60 years old). After warping individual skin images onto this average face, the resulting skin-warped average faces were presented to three experts who graded: forehead wrinkles, nasolabial fold, wrinkle of the corner of the lips, pore visibility and skin pigmentation homogeneity. Two experts estimated subjects' age. Results were compared to gradings performed on original images. Inter-expert grading shows excellent to good correlation whatever image type: from 0.918 (forehead wrinkles) to 0.693 (visibility of pores). Correlations between scoring of both image types are almost always higher than inter-expert correlations (maximum: 0.939 for forehead wrinkles-minimum: 0.677 for pore visibility). Frequencies of grades/ages are similar when scoring original and skin-warped average face images. Experts scores are similar in 90.6%-99.3% of the cases. Average deviations upon scoring both image types are smaller than average inter-expert deviations on original images. Scoring facial characteristics in original images and skin-warped average face images show an excellent agreement, even for perceived age, a complex feature. This opens the possibility of using this approach to grade facial skin features, monitor changes over time, and to valorise results on a face deprived of image rights.

Analysis of Genetic Face Images with Respect to Reflexology for Prediction of Diseases

Analysis of Genetic Face Images with Respect to Reflexology for Prediction of Diseases

Identification of Autism Spectrum Disorder using Residual Attention Network for Facial Image Analysis.

The goal of the present paper is to apply deep learning algorithms to identify autism spectrum disorder (ASD) patients from a face image dataset based solely on the patient's face activation patterns. We investigated ASD Patients' face imaging data from a worldwide multispecialty database known as Autism Face Imaging Data Exchange. ASD is a brain-based disorder characterized by social deficiency and the symptoms are different scenarios. According to recent Centers for Disease Control data, ASD affects About 1 in 54 children who have been identified with autism spectrum disorder (ASD) according to estimates from CDC’s Autism and Developmental Disabilities Monitoring (ADDM) Network. We investigated patterns of functional connectivity that objectively identify ASD participants from functional face imaging data and attempted to unveil the facial patterns that emerged from the classification. With the proposed module, standard CNNs are made, like ResNet-50 have more discriminative power for deep face recognition, and results improved the state-of-the-art by achieving 99% accuracy in the identification of ASD versus control patients in the dataset. We present the results and identify the areas of facial expressions that contributed most to differentiating ASD from typically developing controls as per our deep learning method. For verification purposes, the videos collected in real-time manually from different children we retested and an accuracy score of 99.90% and an F1 score of 99.67% were achieved.