Visual Face Recognition Research Articles

Face Recognition based on Convoluted Neural Networks: Technical Review

Human beings recognize and classify objects with biological senses and brain that processes the input into meaningful information. Other than that humans have come to recognize each other in multiple ways one of which is visual recognition of faces. As a biological trait human faces are certainly a biometric such they are universal, distinctive, mostly permanent and collectable. With that a computerized face recognition system can constructed relying on visual information present on each face uniquely. Generally a face recognition system consists of two main phases, face detection phase where presence of a human face is verified on visual input and face recognition phase where detected face is processed for identification. One of the most sought after methods in field image processing for face recognition is CNN (Convoluted Neural Networks). CNNs have proved its effectiveness and accuracy in many CNN based face detection and face recognition systems. As such in this paper the architecture of CNN is presented. Then different techniques for face detection and face recognition based on CNNs are reviewed. In reviewed papers CNNs have repeatedly demonstrated effectiveness and accuracy on multiple benchmarks for face recognition application.

Brain-like functional specialization emerges spontaneously in deep neural networks.

The human brain contains multiple regions with distinct, often highly specialized functions, from recognizing faces to understanding language to thinking about what others are thinking. However, it remains unclear why the cortex exhibits this high degree of functional specialization in the first place. Here, we consider the case of face perception using artificial neural networks to test the hypothesis that functional segregation of face recognition in the brain reflects a computational optimization for the broader problem of visual recognition of faces and other visual categories. We find that networks trained on object recognition perform poorly on face recognition and vice versa and that networks optimized for both tasks spontaneously segregate themselves into separate systems for faces and objects. We then show functional segregation to varying degrees for other visual categories, revealing a widespread tendency for optimization (without built-in task-specific inductive biases) to lead to functional specialization in machines and, we conjecture, also brains.

Visual scanning and recognition of ambiguous faces in children with autism: The effects of morphing levels and facial identities

Background and aimPrior work on face processing in children with autism spectrum disorder (ASD) has mainly focused on the investigation of unfamiliar faces. The present study aimed to extend earlier research by the inclusion of the different faces with varying levels of perceptual difficulty. MethodsWe employed eye-tracking and morphing techniques to measure face recognition involving identification of face morphs and to record eye movements during the task. Three groups of children participated in the study, comprising 24 with ASD (aged from 4 to 7 years), 25 verbal IQ (VQ)-matched typical developing (TD) controls (aged from 3 to 4 years), and 25 chronological age (CA)-matched TD controls (aged from 4 to 6 years). ResultsThe group-difference was specific for morphed unknown faces – ASD children had a decreased judgement of unknown faces and showed a lower specificity. Concurrent eye-tracking further provided mechanistic insights: the ASD group exhibited a significant reduction in eye-region fixation when recognizing ambiguous unknown faces, relative to both TD groups. ConclusionThe current study provides evidence of a selective difficulty in the identification of unknown faces in ASD when the perceptual demand increased, without atypicality in self and familiar face processing. Reduced attention and social interest may be responsible for difficulties in response to stranger faces in young ASD children, rather than a consequence of inability.

Prior sleep timing and visual recognition of emotional faces in 6-month-old infants

Face recognition is an important mnemonic ability for infants when navigating the social world. While age-related changes in face processing abilities are relatively well documented, less is known about short-term intra-individual fluctuations in this ability. Given that sleep deprivation in adults leads to impairments in information processing, we assessed the role of prior sleep on 6-month-old infants’ (N = 17) visual recognition of faces showing three emotional expressions (neutral, sad, angry). Visual recognition was inferred by assessing novelty preferences for unfamiliar relative to familiarized faces in a visual recognition memory paradigm. In a within-subject design, infants participated once after they had recently woken up from a nap (nap condition) and once after they had been awake for an extended period of time (awake condition). Infants failed to show visual recognition for the neutral faces in either condition. Infants showed recognition for the sad and angry faces when tested in the awake condition, but not in the nap condition. This suggests that timing of prior sleep shapes how effectively infants process emotionally relevant information in their environment.

Dual Face Alignment Learning Network for NIR-VIS Face Recognition

As the most important topic in Heterogeneous Face Recognition (HFR), Near-InfraRed and VISual (NIR-VIS) face recognition has attracted increasing research attentions owing to its potential application in the field of criminal detective cases and multimedia information retrieval. However, due to its dramatic intra-class variations, including modality, pose, occlusion, blurry, lighting, distance, expression, etc, it is very challenging to retain inherent identity information. To address the above issue, we propose a novel Dual Face Alignment Learning (DFAL) algorithm to explore the potential domain-invariant neutral face representations of the cross-modal images. Our model contains three effective components including Feature-level Face Alignment (FFA), Image-level Face Alignment (IFA) and Cross-domain compact Representation (CdR). Firstly, Teacher-Encoder CNNs (TeEn-CNNs) and Student-Encoder CNNs (StEn-CNNs) are designed to encode features for VIS neutral face images and non-neutral face images, and the FFA is introduced to learn neutral face representations by performing feature-level alignment between non-neutral face and VIS neutral face. Secondly, Student-Decoder CNNs (StDe-CNNs) is developed to decode features to restore face images, and the IFA is designed to reconstruct neutral face image by imposing image-level alignment. Notably, the FFA acts as the primary target to learn VIS neutral face representations for cross-view data, while the IFA plays a role in the icing on the cake, i.e., further disentangling domain and residual information through the synthesis process. Finally, the CdR dispels modality features and distills identity features by mining inter-class information, inter-domain information and inter-semantic relationship. The joint scheme enables the elimination of intra-class variations and the purification of identity information. We carry out comprehensive experiments to illustrate the effectiveness of the DFAL approach on three challenging NIR-VIS databases.

Robust Human Face Authentication Leveraging Acoustic Sensing on Smartphones

User authentication on smartphones is the key to many applications, which must satisfy both security and convenience. We propose a novel user authentication system <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EchoPrint</i> , which leverages acoustics and vision for secure and convenient user authentication, without requiring any special hardware. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EchoPrint</i> actively emits almost inaudible acoustic signals from the earpiece speaker to “illuminate” the user's face and authenticates the user by the unique features extracted from the echoes bouncing off the 3D facial contour. To combat changes in phone-holding poses thus echoes, a convolutional neural network (CNN) is trained to extract reliable acoustic features, which are further combined with visual facial features extracted from state-of-the-art face recognition deep models to feed a binary support vector machine (SVM) classifier for final authentication. Because the echo features depend on 3D facial geometries, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EchoPrint</i> is not easily spoofed by images or videos like 2D visual face recognition systems. It needs only commodity hardware, thus avoiding the extra costs of special sensors in solutions like FaceID. Experiments with 62 volunteers and non-human objects such as images, photos, and sculptures show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EchoPrint</i> achieves 93.75 percent balanced accuracy and 93.50 percent F-score, while the average precision is 98.05 percent using acoustic features and basic facial landmarks. The precision is further improved to 99.96 percent with sophisticated visual features.

Profiles on the Orientation Discrimination Processing of Human Faces.

Face recognition is a crucial subject for public health, as socialization is one of the main characteristics for full citizenship. However, good recognizers would be distinguished, not only by the number of faces they discriminate but also by the number of rejected stimuli as unfamiliar. When it comes to face recognition, it is important to remember that position, to some extent, would not entail a high cognitive cost, unlike other processes in similar areas of the brain. The aim of this paper was to examine participant’s recognition profiles according to face position. For this reason, a recognition task was carried out by employing the Karolinska Directed Emotional Faces. Reaction times and accuracy were employed as dependent variables and a cluster analysis was carried out. A total of two profiles were identified in participants’ performance, which differ in position in terms of reaction times but not accuracy. The results can be described as follows: first, it is possible to identify performance profiles in visual recognition of faces that differ in position in terms of reaction times, not accuracy; secondly, results suggest a bias towards the left. At the applied level, this could be of interest with a view to conducting training programs in face recognition.

Temporo-parietal contribution to the mental representations of self/other face

Face recognition requires comparing the current visual input with stored mental representations of faces. Based on its role in visual recognition of faces and mental representation of the body, we hypothesized that the right temporo-parietal junction (rTPJ) could be implicated also in processing mental representation of faces. To test this hypothesis, we asked 30 neurotypical participants to perform mental rotation (laterality judgment of rotated pictures) of self- and other-face images, before and after the inhibition of rTPJ through repetitive transcranial magnetic stimulation. After inhibition of rTPJ the mental rotation of self-face was slower than other-face. In the control condition the mental rotation of self/other faces was not significantly different. This supports that the role of rTPJ extends to mental representation of faces, specifically for the self. Since the experimental task did not require to explicitly recognize identity, we propose that unconscious identity attribution affects also the mental representation of faces. The present study offers insights on the involvement rTPJ in mental representation of faces and proposes that the neural substrate dedicated to mental representation of faces goes beyond the traditional visual and memory areas.

Dual Adversarial Disentanglement and Deep Representation Decorrelation for NIR-VIS Face Recognition

The task of near-infrared and visual (NIR-VIS) face recognition refers to matching face data from different modalities, which has broad application prospects in areas such as multimedia information retrieval and criminal investigation. However, it remains a challenging task due to high intra-class variations and small-scale NIR-VIS dataset. In this paper, we propose a novel approach called Dual Adversarial Disentanglement and deep Representation Decorrelation (DADRD) to solve the NIR-VIS matching problem. In order to reduce the gap between NIR-VIS images, three key components are designed for DADRD model, including Cross-modal Margin (CmM) loss, Dual Adversarial Disentangled Variations (DADV) and Deep Representation Decorrelation (DRD). Firstly, the CmM loss captures within- and between-class information of the data, and it further reduces modality difference by a center-variation item. Secondly, the Mixed Facial Representation (MFR) layer of the backbone network is divided into three parts: the identity-related layer, the modality-related layer and the residual-related layer. The DADV is designed to reduce the intra-class variations, which consists of Adversarial Disentangled Modality Variations (ADMV) and Adversarial Disentangled Residual Variations (ADRV). Specifically, the ADMV and ADRV aim at eliminating spectrum variations and residual variations (i.e., lighting, pose, expression, occlusion, etc) respectively via an adversarial mechanism. Finally, we impose a DRD on the three decomposed features to make them irrelevant to each other, which can more effectively separate the three component information and enhance feature representations. In particular, we develop a Joint Three-stage Optimization (JTsO) strategy to effectively optimize the network. The joint formulation leads to the purification of identity information and the disentanglement of within-class variation information. Extensive experiments have been carried out on three challenging datasets, and the results demonstrate the effectiveness of our method.

Unsupervised Phoneme and Word Discovery From Multiple Speakers Using Double Articulation Analyzer and Neural Network With Parametric Bias.

This paper describes a new unsupervised machine-learning method for simultaneous phoneme and word discovery from multiple speakers. Phoneme and word discovery from multiple speakers is a more challenging problem than that from one speaker, because the speech signals from different speakers exhibit different acoustic features. The existing method, a nonparametric Bayesian double articulation analyzer (NPB-DAA) with deep sparse autoencoder (DSAE) only performed phoneme and word discovery from a single speaker. Extending NPB-DAA with DSAE to a multi-speaker scenario is, therefore, the research problem of this paper.This paper proposes the employment of a DSAE with parametric bias in the hidden layer (DSAE-PBHL) as a feature extractor for unsupervised phoneme and word discovery. DSAE-PBHL is designed to subtract speaker-dependent acoustic features and speaker-independent features by introducing parametric bias input to the DSAE hidden layer. An experiment demonstrated that DSAE-PBHL could subtract distributed representations of acoustic signals, enabling extraction based on the types of phonemes rather than the speakers. Another experiment demonstrated that a combination of NPB-DAA and DSAE-PBHL outperformed other available methods accomplishing phoneme and word discovery tasks involving speech signals with Japanese vowel sequences from multiple speakers.

Location of Eyes in Images of Human Faces through Analysis Variance Shine Intensity

Extraction of facial features is an important step in automatic visual interpretation and recognition of human faces. Among the facial features, the eyes play a major role in the recognition process. In this article, we present an approach to detect and locate the eyes in frontal face images. Eye regions are identified using the technique of voucher detection based on mathematical morphology. After this identification is made a comparison between the variances of three different portions of each candidate region to eye (set of pixels belonging to the candidate region as a whole, set of pixels contained in a minimum rectangle circumscribed to the candidate region and set of pixels of the candidate region belonging to a horizontal band that crosses the center of mass of this region). The calculation of these variances also considers the R, G, and B channels, as well as the gray version of the input image.

Anti-Cheating Presence System Based on 3WPCA- Dual Vision Face Recognition

To prevent counterfeit face image on face presence system, we can use dual vision camera in face recognition system. Dual vision camera is used to produce detectable face images from two positions of the left lens and the right lens. Image retrieval at the two corners of the left lens and the right lens can produce a merged face image database of left lens face image and right lens face image. The use of two sides of the face angle taking is used to avoid falsification of facial data such as the use of a face photo of a person or an image similar to a person's face. This research uses a dual-vision face recognition method on its preprocessing and uses 3WPCA (Three Level Wavelet Decomposition — Principal Component Analysis) as its feature extraction model. In dual-vision face recognition, we use half-join method to combine a half of the left image and a half of the right image into an image that is ready to be extracted using 3WPCA. This research can produce a presence system based on good face recognition and can be used to anticipate falsification of face data with recognition accuracy up to 98%.

Multiple face tracking and recognition with identity-specific localized metric learning

While face recognition has been intensively studied in the literature, there are only a few attempts on visual recognition of multiple faces simultaneously in videos, which has potential applications in practical video surveillance. In this paper, we address the problem of visual recognition and tracking of multiple faces in real-world videos involving large pose variation and occlusion. Instead of recognizing individual face independently, we introduce the constraints of inter-frame temporal smoothness and within-frame identity exclusivity on multiple faces in videos, and model the tasks of multiple face recognition (MFR) and multiple face tracking (MFT) jointly in an alternative optimization framework. We show this joint formulation for two different tasks leads to significantly improved MFR accuracy. Specifically, as appearance matching for face instances over consecutive frames plays a critical role in MFT, we propose an identity-specific metric learning method with a part-based object representation to learn a localized transformation for each face subject in an online manner, under which face instances of the same subject over consecutive frames are pulled as close as possible, while those of different subjects are pushed far away. Empirically, we evaluate our method on several MFR sequences against baselines, and the results demonstrate that our method can achieve improved accuracy performance in various challenging recognition scenarios.

Brain pathway differences between Parkinson's disease patients with and without depressive symptoms

IntroductionDepression occurs frequently in patients suffering from Parkinson's disease (PD). However, the neural basis of depression in PD remains unclear. Diffusion magnetic resonance imaging (DMRI) connectometry is based on the spin distribution function (SDF), which quantifies the density of diffusing water.AimThe aim of this study was to assess the microstructural changes in the brain connectivity of PD patients with and without depressive symptoms.MethodsDMRI was used to assess microstructural abnormalities in the brains of 16 PD patients with depressive symptoms compared to 11 PD patients without depressive symptoms. Data used in the preparation of this paper were obtained from the Parkinson's progression markers initiative (PPMI) database (http://www.ppmi-info.org/data/). This dataset was acquired on a 3-Tesla scanner (Siemens), producing 64 DWI at b = 1000 s/mm2 and one b0 image. Diffusion MRI data were corrected for subject motion, eddy current distortions, and susceptibility artefacts due to magnetic field inhomogeneity. DMRI connectometry was conducted in a total of 27 patients using percentage measurement.ResultsPD Patients with depressive symptoms showed decreased anisotropy (FDR &lt; 0.05) in the fornix bilaterally, left inferior longitudinal fasciculus (ILF) and corticospinal tract bilaterally compared to PD patients without depressive symptoms.ConclusionsLesser WM integrity of the left ILF fibers, which connect visual face recognition areas to the amygdala and hippocampus, seems to be associated with depressive symptoms in PD patients. Our study supports the hypothesis that neurodegenerative processes in projections from the somatosensory, cingulate, and insular cortices may be related to depression in PD.Disclosure of interestThe authors have not supplied their declaration of competing interest.

The visual white matter: The application of diffusion MRI and fiber tractography to vision science.

Visual neuroscience has traditionally focused much of its attention on understanding the response properties of single neurons or neuronal ensembles. The visual white matter and the long-range neuronal connections it supports are fundamental in establishing such neuronal response properties and visual function. This review article provides an introduction to measurements and methods to study the human visual white matter using diffusion MRI. These methods allow us to measure the microstructural and macrostructural properties of the white matter in living human individuals; they allow us to trace long-range connections between neurons in different parts of the visual system and to measure the biophysical properties of these connections. We also review a range of findings from recent studies on connections between different visual field maps, the effects of visual impairment on the white matter, and the properties underlying networks that process visual information supporting visual face recognition. Finally, we discuss a few promising directions for future studies. These include new methods for analysis of MRI data, open datasets that are becoming available to study brain connectivity and white matter properties, and open source software for the analysis of these data.

The cognitive and neural basis of developmental prosopagnosia.

Developmental prosopagnosia (DP) is a severe impairment of visual face recognition in the absence of any apparent brain damage. The factors responsible for DP have not yet been fully identified. This article provides a selective review of recent studies investigating cognitive and neural processes that may contribute to the face recognition deficits in DP, focusing primarily on event-related brain potential (ERP) measures of face perception and recognition. Studies that measured the face-sensitive N170 component as a marker of perceptual face processing have shown that the perceptual discrimination between faces and non-face objects is intact in DP. Other N170 studies suggest that faces are not represented in the typical fashion in DP. Individuals with DP appear to have specific difficulties in processing spatial and contrast deviations from canonical upright visual-perceptual face templates. The rapid detection of emotional facial expressions appears to be unaffected in DP. ERP studies of the activation of visual memory for individual faces and of the explicit identification of particular individuals have revealed differences between DPs and controls in the timing of these processes and in the links between visual face memory and explicit face recognition. These observations suggest that the speed and efficiency of information propagation through the cortical face network is altered in DP. The nature of the perceptual impairments in DP suggests that atypical visual experience with the eye region of faces over development may be an important contributing factor to DP.

Retrieval of TV Talk-Show Speakers by Associating Audio Transcript to Visual Clusters

Retrieval of TV talk-show speakers based on solely visual face recognition is hard because of the significant visual variation caused by illumination, pose, size, and expression, which can exceed those due to identity. Fortunately, TV talk-shows often exhibit specific visual production styles and are accompanied with other modalities, such as audio transcript. Hence, this paper presents a speaker retrieval framework which associates the who and when information provided by the audio transcript to a set of visual clusters. First, to obtain the visual clusters, an unsupervised speaker identity clustering strategy is proposed, by which the same speakers are grouped together but without knowing who exactly he/she is. Then, to further identify the specific speaker for each group, we propose an association strategy, by which the search are initially limited to those corresponding to when the queried speaker speaking, followed by a graph-based densest sub-graph refinement. Comprehensive experiments on 3 h French TV talk-show “Le Grand Echiquier” provided by $K$ -space project show satisfactory results. Moreover, evaluation of the proposed association strategy on more challenging MediaEval 2015 task with just the provided speaker diarization module and face tracking module could provide state-of-the-art performances, demonstrating the effect of the proposed association strategy.

Acquired prosopagnosia without word recognition deficits

It has long been suggested that face recognition relies on specialized mechanisms that are not involved in visual recognition of other object categories, including those that require expert, fine-grained discrimination at the exemplar level such as written words. But according to the recently proposed many-to-many theory of object recognition (MTMT), visual recognition of faces and words are carried out by common mechanisms [Behrmann, M., & Plaut, D. C. (2013). Distributed circuits, not circumscribed centers, mediate visual recognition. Trends in Cognitive Sciences, 17, 210–219]. MTMT acknowledges that face and word recognition are lateralized, but posits that the mechanisms that predominantly carry out face recognition still contribute to word recognition and vice versa. MTMT makes a key prediction, namely that acquired prosopagnosics should exhibit some measure of word recognition deficits. We tested this prediction by assessing written word recognition in five acquired prosopagnosic patients. Four patients had lesions limited to the right hemisphere while one had bilateral lesions with more pronounced lesions in the right hemisphere. The patients completed a total of seven word recognition tasks: two lexical decision tasks and five reading aloud tasks totalling more than 1200 trials. The performances of the four older patients (3 female, age range 50–64 years) were compared to those of 12 older controls (8 female, age range 56–66 years), while the performances of the younger prosopagnosic (male, 31 years) were compared to those of 14 younger controls (9 female, age range 20–33 years). We analysed all results at the single-patient level using Crawford's t-test. Across seven tasks, four prosopagnosics performed as quickly and accurately as controls. Our results demonstrate that acquired prosopagnosia can exist without word recognition deficits. These findings are inconsistent with a key prediction of MTMT. They instead support the hypothesis that face recognition is carried out by specialized mechanisms that do not contribute to recognition of written words.

Awake fMRI reveals a specialized region in dog temporal cortex for face processing.

Recent behavioral evidence suggests that dogs, like humans and monkeys, are capable of visual face recognition. But do dogs also exhibit specialized cortical face regions similar to humans and monkeys? Using functional magnetic resonance imaging (fMRI) in six dogs trained to remain motionless during scanning without restraint or sedation, we found a region in the canine temporal lobe that responded significantly more to movies of human faces than to movies of everyday objects. Next, using a new stimulus set to investigate face selectivity in this predefined candidate dog face area, we found that this region responded similarly to images of human faces and dog faces, yet significantly more to both human and dog faces than to images of objects. Such face selectivity was not found in dog primary visual cortex. Taken together, these findings: (1) provide the first evidence for a face-selective region in the temporal cortex of dogs, which cannot be explained by simple low-level visual feature extraction; (2) reveal that neural machinery dedicated to face processing is not unique to primates; and (3) may help explain dogs’ exquisite sensitivity to human social cues.

Voluntary attention in Asperger's syndrome: Brain electrical oscillation and phase-synchronization during facial emotion recognition

This study investigated electroencephalography (EEG) oscillatory activity and phase-synchronization in patients with Asperger's syndrome (AS) during visual recognition of emotional faces. In the experiment, 10 AS adults (2 females, age 19.6±1.96) and 10 IQ-matched controls (3 females, age 24.4±3.24) participated in tasks involving emotionality evaluation of either photograph or line-drawing faces. Emotional faces elicited comparable reaction times and evaluation scores between the two groups. In the photograph task, the AS group had no visible N400 component and lower delta/theta synchronization (350–450ms post-stimulus onset) in the temporal and occipital–parietal regions, and much weaker phase synchronization between distant scalp regions (200–500ms post-stimulus onset) compared with the control group. In the line-drawing task, the two groups had the same degree of delta/theta synchronization in the central and occipital–parietal regions and comparable phase synchronization between scalp regions. We conclude by hypothesizing that AS patients might have structural deficits in the amygdala and its related limbic structures, a site critical for recognition of emotional faces beyond conscious awareness, but that they preserve the intact function in the cognitive pathway to keep up comparable behavioral performances with the healthy controls through voluntary control of attention.