Facial Feature Extraction Research Articles

Deep aligned feature extraction for collaborative-representation-based face classification with group dictionary selection

Face recognition plays an important role in many robotic and human–computer interaction systems. To this end, in recent years, sparse-representation-based classification and its variants have drawn extensive attention in compress sensing and pattern recognition. For image classification, one key to the success of a sparse-representation-based approach is to extract consistent image feature representations for the images of the same subject captured under a wide spectrum of appearance variations, for example, in pose, expression and illumination. These variations can be categorized into two main types: geometric and textural variations. To eliminate the difficulties posed by different appearance variations, the article presents a new collaborative-representation-based face classification approach using deep aligned neural network features. To be more specific, we first apply a facial landmark detection network to an input face image to obtain its fine-grained geometric information in the form of a set of 2D facial landmarks. These facial landmarks are then used to perform 2D geometric alignment across different face images. Second, we apply a deep neural network for facial image feature extraction due to the robustness of deep image features to a variety of appearance variations. We use the term deep aligned features for this two-step feature extraction approach. Last, a new collaborative-representation-based classification method is used to perform face classification. Specifically, we propose a group dictionary selection method for representation-based face classification to further boost the performance and reduce the uncertainty in decision-making. Experimental results obtained on several facial landmark detection and face classification data sets validate the effectiveness of the proposed method.

A Hybrid Method of Face Feature Extraction, Classification Based on MLBP and Layered-Recurrent Network

A Hybrid Method of Face Feature Extraction, Classification Based on MLBP and Layered-Recurrent Network

Smart face identification via improved LBP and HOG features

Smart face identification is widely used in smart city and smart healthcare. However, smart face identification technology is susceptible to envirnmental factor, such as illumination, mask, and expression. In order to fully extract facial feature information, we fuse an improved local binary pattern (LBP) and the histogram of oriented gradients (HOG) to extract the texture and detailed features on the face. The 2DPCA + PCA is used to reduce the dimensionality of the extracted features. The 2DPCA sloves the issue that the model is too complex when the feature dimension is very high. The feature reduction reduces the calculation scale and increases the calculation speed. Finally, experimental results on ORL and Yale face databases show that the feature extraction based on the fusion of improved LBP and HOG complement with each other. Compared with other recognition algorithms, the improved algorithm has higher recognition and identification rate.

Fusion Strategies for Multimodal Biometric System Using Face and Voice Cues

Deployment of biometric systems in the applications of real world includes the most of unimodal biometric systems. Unimodal biometric system based on the information collected from single source. Sometimes single source of information may not identify the individual correctly because of some limitations such as Non-universality, Noisy data, Intra-class variation, Spoof attacks and Intra-class similarities. Various limitations of unimodal biometric systems are overridden by the multimodal biometric systems which involves multiple sources of information. Multimodal systems can be constructed by fusing of information of multiple modalities. This fusion can take place at various steps of processing such as at image acquisition, extraction of features of the traits, matching of test vectors with trained vectors and during decision taking based on classification. This paper presents a system of multimodal biometrics using face and voice biometric traits by including four fusion methods. Fusion takes place at i) feature level using concatenation of face and voice features, ii) score level using method involving the maximum of mode of scores obtained from two matchers, iii) rank level using borda count & iv) decision level fusion using logical conjunction (AND). Fusing of Log Gabor & Local Binary Pattern (LBP) takes place at the facial feature extraction. The voice features are also fused using Mel Frequency Ceptral coefficients (MFCCs) and Linear Predictive Coefficient features (LPC). Computation of similarity between test feature vectors and training vectors is carried out using Euclidian distance during matching process. KNN Classifier is used during decision making. Performance evaluation of these techniques are also carried out using performance measures such as Accuracy, False Acceptance Rate (FAR), False Rejection Rate (FRR) and ROC curves.

Ensemble neural network approach detecting pain intensity from facial expressions.

This paper reports on research to design an ensemble deep learning framework that integrates fine-tuned, three-stream hybrid deep neural network (i.e., Ensemble Deep Learning Model, EDLM), employing Convolutional Neural Network (CNN) to extract facial image features, detect and accurately classify the pain. To develop the approach, the VGGFace is fine-tuned and integrated with Principal Component Analysis and employed to extract features in images from the Multimodal Intensity Pain database at the early phase of the model fusion. Subsequently, a late fusion, three layers hybrid CNN and recurrent neural network algorithm is developed with their outputs merged to produce image-classified features to classify pain levels. The EDLM model is then benchmarked by means of a single-stream deep learning model including several competing models based on deep learning methods. The results obtained indicate that the proposed framework is able to outperform the competing methods, applied in a multi-level pain detection database to produce a feature classification accuracy that exceeds 89 %, with a receiver operating characteristic of 93 %. To evaluate the generalization of the proposed EDLM model, the UNBC-McMaster Shoulder Pain dataset is used as a test dataset for all of the modelling experiments, which reveals the efficacy of the proposed method for pain classification from facial images. The study concludes that the proposed EDLM model can accurately classify pain and generate multi-class pain levels for potential applications in the medical informatics area, and should therefore, be explored further in expert systems for detecting and classifying the pain intensity of patients, and automatically evaluating the patients' pain level accurately.

A novel deep learning system for facial feature extraction by fusing CNN and MB-LBP and using enhanced loss function

Extensive use of surveillance cameras for human tracking and observation have been fostering the research on face recognition technique for individual identification in an unconstrained environment. However, face recognition is a challenging task in an unconstrained environment, where the captured images are affected by illumination effect, varying poses, noise and occlusion. The main objective of this research is to improve the accuracy and processing time in extracting facial features by using the fusion of deep learning and handcrafted architecture for recognizing individuals in unconstrained conditions, thereby providing accurate information about the individuals to security systems. The proposed system consists of Multi-Block Local Binary Pattern (MB-LBP) modules for extracting the handcrafted features and Convolutional Neural Network (CNN) for extracting the high-level distinctive features. The features from both modules are fused and passed through fully connected layer with Softmax classifier to identify individuals. The results show that the enhanced algorithm based on Softmax loss function aided classifier with regularization improves the accuracy and processing time for face recognition. The proposed model improves accuracy by 94.37% against 90.01% for the state-of-the-art solution. In addition to that, it improves the processing time of 307 ms against 357 ms. The proposed system focuses on fusing hand-crafted and deep learned features to extract face features accurately and thus improving the accuracy and overall performance of the proposed system in an unconstrained environment.

Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood.

Visual and auditory signs of patient functioning have long been used for clinical diagnosis, treatment selection, and prognosis. Direct measurement and quantification of these signals can aim to improve the consistency, sensitivity, and scalability of clinical assessment. Currently, we investigate if machine learning-based computer vision (CV), semantic, and acoustic analysis can capture clinical features from free speech responses to a brief interview 1 month post-trauma that accurately classify major depressive disorder (MDD) and posttraumatic stress disorder (PTSD). N = 81 patients admitted to an emergency department (ED) of a Level-1 Trauma Unit following a life-threatening traumatic event participated in an open-ended qualitative interview with a para-professional about their experience 1 month following admission. A deep neural network was utilized to extract facial features of emotion and their intensity, movement parameters, speech prosody, and natural language content. These features were utilized as inputs to classify PTSD and MDD cross-sectionally. Both video- and audio-based markers contributed to good discriminatory classification accuracy. The algorithm discriminates PTSD status at 1 month after ED admission with an AUC of 0.90 (weighted average precision = 0.83, recall = 0.84, and f1-score = 0.83) as well as depression status at 1 month after ED admission with an AUC of 0.86 (weighted average precision = 0.83, recall = 0.82, and f1-score = 0.82). Direct clinical observation during post-trauma free speech using deep learning identifies digital markers that can be utilized to classify MDD and PTSD status.

Constrained Local Models (CLM) For Facial Feature Extraction using CLNF and SVR as Patch Experts

Methods for detection of facial characteristics have again developed greatly in recent times. However, they also argue in the presence of poor lighting conditions for amazing pose or occlusions. A well-established group of strategies for facial feature extraction is the Constrained Local Model (CLM). Recently, they are bringing cascaded regression-built methodologies out of favor. This is because the failure of presenting nearby CLM detectors to model the highly complex special signature look affected to a small degree by voice, illumination, facial hair and make-up. This paper keeps tabs on execution to collect facial features for the Constrained Local Model (CLM). CLM model relies on patch model to collect facial image demand features. In this paper patch model built using Support Vector Regression (SVR) and Constrained Local Neural Field (CLNF). We show that the CLNF model exceeds SVR by a large margin on the LFPW database to identify facial landmarks.

AutisMitr: Emotion Recognition Assistive Tool for Autistic Children

AbstractAssistive technology has proven to be one of the most significant inventions to aid people with Autism to improve the quality of their lives. In this study, a real-time emotion recognition system for autistic children has been developed. Emotion recognition is implemented by executing three stages: Face identification, Facial Feature extraction, and feature classification. The objective is to frame a system that includes all three stages of emotion recognition activity that executes expeditiously in real time. Thus, Affectiva SDK is implemented in the application. The propound system detects at most 7 facial emotions: anger, disgust, fear, joy, sadness, contempt, and surprise. The purpose for performing this study is to teach emotions to individuals suffering from autism, as they lack the ability to respond appropriately to others emotions. The proposed application was tested with a group of typical children aged 6–14 years, and positive outcomes were achieved.

Down Syndrome Face Recognition: A Review

One of the most pertinent applications of image analysis is face recognition and one of the most common genetic disorders is Down syndrome (DS), which is caused by chromosome abnormalities in humans. It is currently a challenge in computer vision in the domain of DS face recognition to build an automated system that equals the human ability to recognize face as one of the symmetrical structures in the body. Consequently, the use of machine learning methods has facilitated the recognition of facial dysmorphic features associated with DS. This paper aims to present a concise review of DS face recognition using the currently published literature by following the generic face recognition pipeline (face detection, feature extraction, and classification) and to identify critical knowledge gaps and directions for future research. The technologies underlying facial analysis presented in recent studies have helped expert clinicians in general genetic disorders and DS prediction.

DTFA-Net: Dynamic and Texture Features Fusion Attention Network for Face Antispoofing

For face recognition systems, liveness detection can effectively avoid illegal fraud and improve the safety of face recognition systems. Common face attacks include photo printing and video replay attacks. This paper studied the differences between photos, videos, and real faces in static texture and motion information and proposed a living detection structure based on feature fusion and attention mechanism, Dynamic and Texture Fusion Attention Network (DTFA-Net). We proposed a dynamic information fusion structure of an interchannel attention block to fuse the magnitude and direction of optical flow to extract facial motion features. In addition, for the face detection failure of HOG algorithm under complex illumination, we proposed an improved Gamma image preprocessing algorithm, which effectively improved the face detection ability. We conducted experiments on the CASIA-MFSD and Replay Attack Databases. According to experiments, the DTFA-Net proposed in this paper achieved 6.9% EER on CASIA and 2.2% HTER on Replay Attack that was comparable to other methods.

Diagnosing Parkinson Disease Through Facial Expression Recognition: Video Analysis.

BackgroundThe number of patients with neurological diseases is currently increasing annually, which presents tremendous challenges for both patients and doctors. With the advent of advanced information technology, digital medical care is gradually changing the medical ecology. Numerous people are exploring new ways to receive a consultation, track their diseases, and receive rehabilitation training in more convenient and efficient ways. In this paper, we explore the use of facial expression recognition via artificial intelligence to diagnose a typical neurological system disease, Parkinson disease (PD).ObjectiveThis study proposes methods to diagnose PD through facial expression recognition.MethodsWe collected videos of facial expressions of people with PD and matched controls. We used relative coordinates and positional jitter to extract facial expression features (facial expression amplitude and shaking of small facial muscle groups) from the key points returned by Face++. Algorithms from traditional machine learning and advanced deep learning were utilized to diagnose PD.ResultsThe experimental results showed our models can achieve outstanding facial expression recognition ability for PD diagnosis. Applying a long short-term model neural network to the positions of the key features, precision and F1 values of 86% and 75%, respectively, can be reached. Further, utilizing a support vector machine algorithm for the facial expression amplitude features and shaking of the small facial muscle groups, an F1 value of 99% can be achieved.ConclusionsThis study contributes to the digital diagnosis of PD based on facial expression recognition. The disease diagnosis model was validated through our experiment. The results can help doctors understand the real-time dynamics of the disease and even conduct remote diagnosis.

User Authentication System Based on Speech and Cascade Hybrid Facial Feature

With the increasing demand for security in many fastest growing applications, biometric recognition is the most prominent authentication system. User authentication through speech and face recognition is the important biometric technique to enhance the security. This paper proposes a speech and facial feature-based multi-modal biometric recognition technique to improve the authentication of any system. Mel Frequency Cepstral Coefficients (MFCC) is extracted from audio as speech features. In visual recognition, this paper proposes cascade hybrid facial (visual) feature extraction method based on static, dynamic and key-point salient features of the face and it proves that the proposed feature extraction method is more efficient than the existing method. In this proposed method, Viola–Jones algorithm is used to detect static and dynamic features of eye, nose, lip, Scale Invariant Feature Transform (SIFT) algorithm is used to detect some stable key-point features of face. In this paper, a research on the audio-visual integration method using AND logic is also made. Furthermore, all the experiments are carried out using Artificial Neural Network (ANN) and Support Vector Machine (SVM). An accuracy of 94.90% is achieved using proposed feature extraction method. The main objective of this work is to improve the authenticity of any application using multi-modal biometric features. Adding facial features to the speech recognition improve system security because biometric features are unique and combining evidence from two modalities increases the authenticity as well as integrity of the system.

Face Tracking Performance in Head Gesture Recognition System

This paper describes the comparative analysis of different face tracking methods in the head gesture recognition system. The major constraints of head gesture recognition system, i.e. face detection, feature extraction, tracking, and recognition are explained. We used adaboost algorithm for detection, and Camshift algorithm for tracking with different feature extraction methods. We performed extensive experimentations and presented a comparative analysis of tracking performance of head gesture recognition system under cluttered backgrounds, shadow and sunshine conditions. Experimental results show the robustness in face detection, tracking and direction recognition of the proposed method.

Manifold feature integration for micro-expression recognition

Recognition of micro-expressions depends on the key features provided in the form of the temporal information. It needs considerable effort, however, to manually design useful characteristics. Subtle or micro-facial expressions are much difficult than regular facial expressions rich in emotional expressions in a true environment to be identified. An easy solution is discussed in this paper to recognise facial micro-expressions that utilizes an algorithm mix for facial identification, feature extraction and classification. The technique proposed is a framework which incorporates handcrafted features and deep features. Local Binary Pattern-Three Orthogonal Planes (LBP-TOP) is the handcraft feature which combines spatial and time analysis to encapsulate regional facet movements. The deep feature model is a micro-expression fine-tuned model based on Convolutional Neural Network (CNN). Two classifiers, i.e. SVM and Softmax are trained with combined feature vectors produced by LBP-TOP and CNN functionalities. All seven widely-used micro-expression databases are evaluated in an experiment. Our research can be claimed as the first extensive experimental study on a big amount of the datasets to train and test the suggested model. The findings in the document show that the method proposed, although simple and straightforward, achieves a substantial increase in precision relative to other commonly recognized micro-expression techniques, which are trained and tested with just a few datasets.

RETRACTED ARTICLE: Investigation of rotated local Gabor features in face recognition using fusion techniques

One of the major challenges related to face recognition is the effective face representation. In this paper, a new rotation algorithm is proposed to make the slanting face to upright position. Gabor features have been identified to be effective for face recognition. To investigate the prospective of Gabor phase and its combination with magnitude for face recognition, Rotated Local Gabor XOR patterns (RLGXP), which encodes the Rotated Gabor phase by using the local XOR pattern (LXP) operator is first proposed, which increases the effectiveness of face representation. Then, Cell-based Fishers linear discriminant (CFLD) to reduce the dimensionality of the proposed descriptor is introduced. Finally, by using CFLD, local patterns of Rotated Gabor magnitude and phase for face recognition are combined. This proposed approach is evaluated on MIT, GTAVE, PIE, CMU and Home databases. Moreover, to enhance the effectiveness of face representation through feature design (RLGXP), the Gabor filter parameters like mask size, scale and orientation are very important. The effectiveness of the rotating local Gabor model + CFLD method was verified for upslopes and uprights of different data sets. The analysis of face feature representation using the RLGXP + CFLD method basically depends on the phase range, the appropriate choice of the number of sub-cells per pattern map and size of neasret region. Here, the sensitivity analysis used for the above three parameters is best selected and executed. This paper also investigates the effect of different Gabor filter parameters on face recognition accuracy. Parameter Selection Investigation result proves that Gabor wavelets comprising 5 scales and 8 orientations have been chosen to extract Facial Feature Points (FFP). The final result shows that Rotated Local Gabor Features provides the best face recognition accuracy.

Automated access control system using face recognition

Security to every firm or campus is an essential one nowadays, as the crime rates in recent days are increasing day by day. As the world is transforming into an advanced technological trend, automated systems are implemented in many sectors. Here in this sector, it will be much more effective. Thus many automated access control systems are being implemented such as biometric methods and fingerprint-based access systems. But these existing systems undergo many complications that lead to time consumption in long. Thus face recognition based access systems are introduced which provides us the better solution. There are many face recognition access systems with different algorithms. This paper deals with an effective and modern algorithm called the FaceNet algorithm. The access system with this algorithm uses Face encoding to detect the face and eye which works effectively on light/illumination changes. Later on the facial feature extraction is done by using the Histogram Object Gradient (HOG). Compare face function, which contains Support Vector Machine classifier to classify the face encoding and produces the output. RFID sensor and IR sensor is implemented to make the system function effectively. Also in addition a webpage is created to provide the access authority to the management of the campus or firm where they can also handle the access system manually if needed.

Design and Implementation of a Computerized User Authentication System for E-Learning

The system based on formalized teaching with the help of electronic resources is called as E-learning. Using E-Learning in the university education systems has been growing rapidly based on the application of Information Technology (IT) designed for the substitute of traditional based learning approach. In terms of the E-learning based examinations, security constraints such as, identification of the user, misuse the passwords, unauthorized user instead of the authenticated one are available. Because of these disadvantages, the need of an authentication system is very important. The main challenge facing the security of E-learning environment is how to identify the unauthorized users from accessing and managing any platform. Therefore solving the problem of authenticating the valid user is very important. So, an intelligent examination system along with user authentication based on facial expression recognition is proposed in this paper. The proposed system can be classified in to three phases. The first phase is the design of facial database from the captured images. The second phase is the design of Artificial Intelligence (AI) based system for facial feature extraction and its classification using intelligent Agents. The final phase is the authentication of respective users by automatic identification of facial images. The results obtained shows that the proposed methodology has managed to validate the authenticated users and to provide an effective security in the E-learning platform.

Face Recognition System for Complex Surveillance Scenarios

In recent years, with the continuous development of the Internet and artificial intelligence, face recognition technology has also been widely used in many application scenarios. Facing complex surveillance scenarios, face recognition technology still faces great challenges. This paper focuses on implementing real-time and efficient face recognition systems in complex surveillance scenarios, such as insufficient lighting, small faces, dense crowds, and sides at 45 ° environment. The system is mainly based on RetinaFace for face detection and face alignment, and uses lightweight mobilenet (0.25) as the backbone network of RetinaFace. Facial feature extraction is based on deep residual neural network combined with ArcFace loss, and feature matching is performed by Euclidean distance. The experimental results show that the face recognition system has good real-time performance, accuracy and robustness.

Application analysis of image recognition based on multiple intelligence algorithm

Image recognition technology based on artificial intelligence algorithm has entered a rapid development stage. According to the development and research status at home and abroad, we can see that PCA, ANN, SVN and other technologies can effectively realize the recognition and generation of images, especially the extraction and prediction of face features, classification and recognition. The analysis and application of the generation countermeasure network (GAN) technology is mainly embodied in the realization of the generation and analysis of handwritten numbers, and the accurate prediction of the promotion and application of SVM technology, as well as the comprehensive and accurate analysis of the generation effect. This paper mainly studies the comparative analysis and application of human face database (ORL) through the relevant data. By extracting 50 face samples, 500 face images are produced for recognition analysis and application.