Heterogeneous Face Research Articles

Syncretic Space Learning Network for NIR-VIS Face Recognition

To overcome the technical bottleneck of face recognition in low-light scenarios, Near-InfraRed and VISible (NIR-VIS) heterogeneous face recognition is proposed for matching well-lit VIS faces with poorly lit NIR faces. Current cross-modal synthesis methods visually convert the NIR modality to the VIS modality and then perform face matching in the VIS modality. However, using a heavyweight GAN network on unpaired NIR-VIS faces may lead to high synthesis difficulty, low inference efficiency, and other problems. To alleviate the above problems, we simultaneously synthesize NIR and VIS images into modality-independent syncretic images and propose a novel syncretic space learning (SSL) model to eliminate the modal gap. First, Syncretic Modality Generator (SMG) synthesizes NIR and VIS images into syncretic images using channel-level convolution with a shallow CNN. In particular, the discriminative structural information is well preserved and the face quality can be further improved with small modal variations in a self-supervised learning manner. Second, Modality-adversarial Syncretic space Learning (MSL) projects NIR and VIS images into the syncretic space by a syncretic-modality adversarial learning strategy with syncretic pattern guided objective, so the modal gap of NIR-VIS faces can be effectively reduced. Finally, the Syncretic Distribution Consistency (SDC) constructed by NIR-syncretic, syncretic-syncretic, and VIS-syncretic consistency can enhance the intra-class compactness and learn discriminative representations. Extensive experiments on three challenging datasets demonstrate the effectiveness of the SSL method.

Analyzing the Factor Structure of the Toronto Empathy Questionnaire: Dimensionality, Reliability, Validity, Measurement Invariance and One-Year Stability of the German Version

In the face of heterogeneity in the measurement of empathy, the Toronto Empathy Questionnaire (TEQ; Spreng et al., Journal of Personality Assessment, 91(1), 62–71 (2009)) was developed as a brief unidimensional tool by statistically forming a consensus from existing measures of the construct. The present study aimed to (1) validate a German version of the TEQ, and (2) contribute empirical evidence to the ongoing debate regarding a singular versus multidimensional factor structure of the TEQ. One cross-sectional and two longitudinal studies were performed, with a total of 1,075 participants. Our initial exploratory factor analyses suggested either a one- or a two-factor structure (with the two-factors clustering straight and reverse-scored items); the two-factor model outperformed the one-factor model using confirmatory factor analyses. However, after negated items were replaced by positively reworded alternatives, both models fit the data equally well. A comparison of the correlation patterns with numerous external measures indicated that a second factor of the TEQ is a methodological artifact of item wording. Finally, a unidimensional TEQ scale showed sufficient internal consistency, two-week test-retest reliability, one-year stability, as well as convergent and discriminant validity with measures of empathy, emotion recognition, emotion regulation, altruism, social desirability, and the Big Five personality traits.

One-way multimodal image-to-image translation for heterogeneous face recognition

Cross-domain face matching is known as heterogeneous face recognition (HFR), which is more valuable than traditional face recognition and plays a key role in public safety. It remains a challenging problem due to insufficient heterogeneous data and large domain discrepancies. Recent work has focused on learning common features from different visual domains and synthesizing face images from other domains into the visible domain to reduce domain discrepancy. The former performs domain transformation during face recognition, which requires additional runtime. The latter has poorer performance limited by insufficient heterogeneous data. We propose a one-way multimodal image-to-image translation (OMIT) to tackle the problem of insufficient heterogeneous data. Specifically, we learn the generation of heterogeneous data from small-scale paired heterogeneous training data, and an identity-preserving loss is imposed on the generated heterogeneous images to ensure their identity consistency. Then, we transform large-scale visible data of abundant identity information into paired heterogeneous data, which addresses the lack of identity diversity due to small-scale heterogeneous data. The generated pairwise heterogeneous data can be directly used to train the HFR network to improve recognition accuracy and precision. Finally, our method exhibits generated quality improvements on BUAA, CASIA NIR–VIS 2.0, Oulu NIR–VIS, and Tufts face database. Heterogeneous datasets generated by OMIT improve the performance of HFR networks.

CoConGAN: Cooperative contrastive learning for few-shot cross-domain heterogeneous face translation

CoConGAN: Cooperative contrastive learning for few-shot cross-domain heterogeneous face translation

A database of heterogeneous faces for studying naturalistic expressions

Facial expressions are thought to be complex visual signals, critical for communication between social agents. Most prior work aimed at understanding how facial expressions are recognized has relied on stimulus databases featuring posed facial expressions, designed to represent putative emotional categories (such as ‘happy’ and ‘angry’). Here we use an alternative selection strategy to develop the Wild Faces Database (WFD); a set of one thousand images capturing a diverse range of ambient facial behaviors from outside of the laboratory. We characterized the perceived emotional content in these images using a standard categorization task in which participants were asked to classify the apparent facial expression in each image. In addition, participants were asked to indicate the intensity and genuineness of each expression. While modal scores indicate that the WFD captures a range of different emotional expressions, in comparing the WFD to images taken from other, more conventional databases, we found that participants responded more variably and less specifically to the wild-type faces, perhaps indicating that natural expressions are more multiplexed than a categorical model would predict. We argue that this variability can be employed to explore latent dimensions in our mental representation of facial expressions. Further, images in the WFD were rated as less intense and more genuine than images taken from other databases, suggesting a greater degree of authenticity among WFD images. The strong positive correlation between intensity and genuineness scores demonstrating that even the high arousal states captured in the WFD were perceived as authentic. Collectively, these findings highlight the potential utility of the WFD as a new resource for bridging the gap between the laboratory and real world in studies of expression recognition.

Facial feature embedded CycleGAN for VIS–NIR translation

Visible and near-infrared (VIS–NIR) heterogeneous face recognition remains a challenging task due to distinctions between spectral components of two modalities and insufficiently pairwise VIS–NIR data. Inspired by the cycle-consistent generative adversarial network (CycleGAN), this paper proposes a facial feature embedded CycleGAN to translate between VIS and NIR face images, aiming to enable the distributions of translated (fake) images to be similar as those of true images. To learn the particular feature of NIR or VIS domain while preserving common facial representation between VIS and NIR domains, a facial feature extractor (FFE), tailored specifically for extracting effective feature from face images, is embedded in the generator of original CycleGAN. For implementing the FFE, we use the MobileFaceNet which is pre-trained on a VIS face database. The domain-invariant feature learning is enhanced by proposing a new pixel consistency loss. Additionally, we establish a new WHU VIS–NIR database including varies in face rotation and expressions to enrich the insufficient training data. Moreover, experiments on the well-known Oulu-CASIA NIR–VIS database and our WHU VIS–NIR database validate the potential benefit of the proposed FFE-based CycleGAN (FFE-CycleGAN). In particular, we achieve 96.5% accuracy on Oulu-CASIA and 98.9% accuracy on WHU VIS–NIR.

An evaluation of multistate occupancy models for estimating relative abundance and population trends

Detecting spatiotemporal changes in the abundances of organisms is key to effectively conserving species. While indices of abundance have long been used, there has been a shift toward model-based estimators that account for the detection process. Popular approaches including traditional occupancy models and N-mixture models entail tradeoffs. The traditional occupancy approach requires the researcher coarsen the characterization of abundance to the probability that a site is occupied or unoccupied. Conversely, N-mixture models make use of variation in counts, but perform poorly when individuals have low detectability or move into or out of sites between visits. Multistate occupancy models that differentiate relatively abundant from non-abundant states have the potential to fill this gap but have been underexplored. We conducted a simulation study to test whether multistate occupancy models could capture spatial abundance patterns and detect population declines in the face of low individual detection probability (p ≤ 0.3) and unmodeled heterogeneity (e.g., that arising from individual movement). We considered 10,773 scenarios to examine the effects of differing amounts of heterogeneity as well as alternative study designs, population parameters, and modeling choices. We tracked bias in the proportion of sites estimated to be in the abundant state for single-season models, and power to detect a declining trend across multiple years. We also evaluated data diagnostic metrics to provide guidance to users. Multistate occupancy models were able to differentiate sites with higher abundances from sites with lower abundances when there were at least medium levels of spatial heterogeneity in true abundances. If different sites were randomly selected each year, power to detect even large population declines (65%) was poor (power < 0.8). However, if the same sites were surveyed each year, and a dynamic multistate occupancy was used, multistate occupancy models could detect (power ≥ 0.8) relatively small declines (5-40%) in 20% of scenarios, and frequently detect large declines of 45-60% (mean power = 0.92). Conservation decisions rely on detecting change reliably, rarely needing absolute abundance information. Multistate occupancy models can improve our ability to detect changing abundance while accommodating low individual detection probability and heterogeneity in count monitoring data.

Seasonal plasticity in morphology and metabolism differs between migratory North American and resident Costa Rican monarch butterflies.

Environmental heterogeneity in temperate latitudes is expected to maintain seasonally plastic life-history strategies that include the tuning of morphologies and metabolism that support overwintering. For species that have expanded their ranges into tropical latitudes, it is unclear the extent to which the capacity for plasticity will be maintained or will erode with disuse. The migratory generations of the North American (NA) monarch butterfly Danaus plexippus lead distinctly different lives from their summer generation NA parents and their tropical descendants living in Costa Rica (CR). NA migratory monarchs postpone reproduction, travel thousands of kilometers south to overwinter in Mexico, and subsist on little food for months. Whether recently dispersed populations of monarchs such as those in Costa Rica, which are no longer subject to selection imposed by migration, retain ancestral seasonal plasticity is unclear. To investigate the differences in seasonal plasticity, we reared the NA and CR monarchs in summer and autumn in Illinois, USA, and measured the seasonal reaction norms for aspects of morphology and metabolism related to flight. NA monarchs were seasonally plastic in forewing and thorax size, increasing wing area and thorax to body mass ratio in autumn. While CR monarchs increased thorax mass in autumn, they did not increase the area of the forewing. NA monarchs maintained similar resting and maximal flight metabolic rates across seasons. However, CR monarchs had elevated metabolic rates in autumn. Our findings suggest that the recent expansion of monarchs into habitats that support year-round breeding may be accompanied by (1) the loss of some aspects of morphological plasticity as well as (2) the underlying physiological mechanisms that maintain metabolic homeostasis in the face of temperature heterogeneity.

Past history shapes evolution of reproductive success in a global warming scenario

Adaptive evolution is critical for animal populations to thrive in the fast-changing natural environments. Ectotherms are particularly vulnerable to global warming and, although their limited coping ability has been suggested, few real-time evolution experiments have directly accessed their evolutionary potential. Here, we report a long-term experimental evolution study addressing the evolution of Drosophila thermal reaction norms, after ∼30 generations under different dynamic thermal regimes: fluctuating (daily variation between 15 and 21 °C) or warming (daily fluctuation with increases in both thermal mean and variance across generations). We analyzed the evolutionary dynamics of Drosophila subobscura populations as a function of the thermally variable environments in which they evolved and their distinct background. Our results showed clear differences between the historically differentiated populations: high latitude D. subobscura populations responded to selection, improving their reproductive success at higher temperatures whereas their low latitude counterparts did not. This suggests population variation in the amount of genetic variation available for thermal adaptation, an aspect that needs to be considered to allow for better predictions of future climate change responses. Our results highlight the complex nature of thermal responses in face of environmental heterogeneity and emphasize the importance of considering inter-population variation in thermal evolution studies.

Prepended Domain Transformer: Heterogeneous Face Recognition Without Bells and Whistles

Heterogeneous Face Recognition (HFR) refers to matching face images captured in different domains, such as thermal to visible images (VIS), sketches to visible images, near-infrared to visible, and so on. This is particularly useful in matching visible spectrum images to images captured from other modalities. Though highly useful, HFR is challenging because of the domain gap between the source and target domain. Often, large-scale paired heterogeneous face image datasets are absent, preventing training models specifically for the heterogeneous task. In this work, we propose a surprisingly simple, yet, very effective method for matching face images across different sensing modalities. The core idea of the proposed approach is to add a novel neural network block called Prepended Domain Transformer (PDT) in front of a pre-trained face recognition (FR) model to address the domain gap. Retraining this new block with few paired samples in a contrastive learning setup was enough to achieve state-of-the-art performance in many HFR benchmarks. The PDT blocks can be retrained for several source-target combinations using the proposed general framework. The proposed approach is architecture agnostic, meaning they can be added to any pre-trained FR models. Further, the approach is modular and the new block can be trained with a minimal set of paired samples, making it much easier for practical deployment. The source code and protocols will be made available publicly.

Robust Cross-domain Pseudo-labeling and Contrastive Learning for Unsupervised Domain Adaptation NIR-VIS Face Recognition.

Near-infrared and visible face recognition (NIR-VIS) is attracting increasing attention because of the need to achieve face recognition in low-light conditions to enable 24-hour secure retrieval. However, annotating identity labels for a large number of heterogeneous face images is time-consuming and expensive, which limits the application of the NIR-VIS face recognition system to larger scale real-world scenarios. In this paper, we attempt to achieve NIR-VIS face recognition in an unsupervised domain adaptation manner. To get rid of the reliance on manual annotations, we propose a novel Robust cross-domain Pseudo-labeling and Contrastive learning (RPC) network which consists of three key components, i.e., NIR cluster-based Pseudo labels Sharing (NPS), Domain-specific cluster Contrastive Learning (DCL) and Inter-domain cluster Contrastive Learning (ICL). Firstly, NPS is presented to generate pseudo labels by exploring robust NIR clusters and sharing reliable label knowledge with VIS domain. Secondly, DCL is designed to learn intra-domain compact yet discriminative representations. Finally, ICL dynamically combines and refines intrinsic identity relationships to guide the instance-level features to learn robust and domain-independent representations. Extensive experiments are conducted to verify an accuracy of over 99% in pseudo label assignment and the advanced performance of RPC network on four mainstream NIR-VIS datasets.

Holistic Feature Reconstruction-Based 3-D Attention Mechanism for Cross-Spectral Periocular Recognition

Recognition of individuals in cross-spectral environments involves situations where probe and gallery images are captured in distinct wavelength ranges and hence differ significantly in terms of appearance and illumination. In case of heterogeneous periocular images, alleviating this wide appearance gap and learning to extract illumination-invariant features from such local regions become cumbersome, giving rise to a challenging research problem. In this work, we design a novel holistic feature reconstruction-based attention module (H-FRAM) to refine and generate discriminative convolutional features. In contrast to existing spatial and channel attention mechanisms that compute 2-D or 1-D attention weights respectively, H-FRAM calculates the importance of each feature location by performing multi-linear principal component analysis on full 3-D tensor space. In H-FRAM, we claim that the feature reconstruction error, computed in a holistic manner, plays a crucial role in determining the relevance of feature locations. This reconstruction error is found by projecting the input feature map onto a multi-dimensional eigenspace that captures most of the important variations. To our knowledge, this is the first work which explores subspace learning approach in the context of 3-D attention mechanism to capture discriminative feature information. We have created an in-house cross-spectral periocular dataset containing visible and near-infrared images from 200 classes. The images are captured in unconstrained acquisition setup involving unsupervised eye and head movements as well as accessory variations (face masks and eyeglasses). Extensive experiments and ablation studies show that the proposed network achieves state-of-the-art recognition performances on the existing and in-house datasets for heterogeneous and homogeneous periocular recognition as well as heterogeneous face recognition.

Disguised heterogeneous face recognition using deep neighborhood difference relational network

Heterogeneous face recognition (HFR) aims to match a probe face image to a gallery of images in a different modality, such as visible to thermal. The complexity of the problem is further increased by the presence of facial disguise, one of the emerging problems in application of face biometrics. While several works have been proposed for HFR, disguised HFR remains one of the unexplored challenges of face recognition. Most of the proposed methods apply constraints on the network output for cross-modality image matching, consequently ignoring the image relationships within the network. The deep features within a network contain additional information which can be exploited for improved performance. With this motivation, we propose a Deep Neighborhood Difference Relational (DNDR) network and Joint Discrimination Loss for disguised HFR. The network calculates the difference relationships between cross-modality images in deep feature space. It implicitly learns to ignore the real-to-disguise feature relationships while focusing on real-to-real facial features for HFR. The proposed method learns the relationships between images of the same identity rather than optimizing the network on a hand-crafted metric loss function. For an input image pair, the DNDR network outputs identity representative embeddings of each image and a match probability of the image pair. The embedding distance and the match probability are then fused for more robust classification accuracy. Extensive experiments on publicly available heterogeneous face databases show the effectiveness of the proposed method for visible to infrared, visible to thermal, and visible to sketch face recognition.

Heterogeneous Face Interpretable Disentangled Representation for Joint Face Recognition and Synthesis.

Heterogeneous faces are acquired with different sensors, which are closer to real-world scenarios and play an important role in the biometric security field. However, heterogeneous face analysis is still a challenging problem due to the large discrepancy between different modalities. Recent works either focus on designing a novel loss function or network architecture to directly extract modality-invariant features or synthesizing the same modality faces initially to decrease the modality gap. Yet, the former always lacks explicit interpretability, and the latter strategy inherently brings in synthesis bias. In this article, we explore to learn the plain interpretable representation for complex heterogeneous faces and simultaneously perform face recognition and synthesis tasks. We propose the heterogeneous face interpretable disentangled representation (HFIDR) that could explicitly interpret dimensions of face representation rather than simple mapping. Benefited from the interpretable structure, we further could extract latent identity information for cross-modality recognition and convert the modality factor to synthesize cross-modality faces. Moreover, we propose a multimodality heterogeneous face interpretable disentangled representation (M-HFIDR) to extend the basic approach suitable for the multimodality face recognition and synthesis. To evaluate the ability of generalization, we construct a novel large-scale face sketch data set. Experimental results on multiple heterogeneous face databases demonstrate the effectiveness of the proposed method.

Clustering of cognitive phenotypes identifies susceptible and resilient offspring in a rat model of maternal immune activation and early-life stress

Schizophrenia and other neurodevelopmental disorders often have very heterogeneous symptoms, especially regarding cognition: while some individuals may exhibit deficient cognition, others are relatively unaffected. Studies using developmental animal models often ignore phenotypic heterogeneity in favour of traditional treatment/control comparisons. This may result in resilient or unaffected individuals masking the effects of susceptible individuals if grouped together. Here, we used maternal immune activation and limited bedding and nesting, respectively, as a two-hit neurodevelopmental model for schizophrenia. Both factors reduced cognitive function in a novel object recognition (NOR) task. While we found treatment group effects on cognitive phenotypes, behavioural clustering identified three subpopulations exposed to either insult: those exhibiting ‘typical’ cognitive performance on the NOR, an intermediate phenotype, or a marked deficit. These clusters included offspring from each treatment group, although both intermediate and marked deficit clusters were composed primarily of offspring from treated groups. Clustering allowed stratification within treatment groups into ‘susceptible’ and ‘resilient’ individuals, while also identifying conserved phenotypes across treatment groups. Using unbiased cluster analyses in preclinical models can better characterize phenotypes and enables a better understanding of both face and construct validity of phenotypic heterogeneity. The use of unbiased clustering techniques may help identify potential markers associated with individual susceptibility and resilience in neurodevelopmental disorder models.

Fibroblast growth factor 23: translating analytical improvement into clinical effectiveness for tertiary prevention in chronic kidney disease.

Fibroblast growth factor 23 (FGF23) plays a key role in the pathophysiology of chronic kidney disease (CKD) and of the associated cardiovascular diseases, ranking on the crossroads of several evolving areas with a relevant impact on the health-care system (ageing, treatment of CKD and prevention from cardiovascular and renal events). In this review, we will critically appraise the overall issues concerning the clinical usefulness of FGF23 determination in CKD, focusing on the analytical performances of the methods, aiming to assess whether and how the clinical introduction of FGF23 may promote cost-effective health care policies in these patients. Our comprehensive critical appraisal of the literature revealed that we are currently unable to establish the clinical usefulness of FGF23 measured by ELISA in CKD, as stability issues and suboptimal analytical performances are the major responsible for the release of misleading results. The meta-analytical approach has failed to report unambiguous evidence in face of the wide heterogeneity of the results from single studies. Our review has largely demonstrated that the clinical usefulness depends on a thorough analytical validation of the assay. The recent introduction of chemiluminescent intact-FGF23 (iFGF23) assays licensed for clinical use, after passing a robust analytical validation, has allowed the actual assessment of preliminary risk thresholds for cardiovascular and renal events and is promising to capture the iFGF23 clinically relevant changes as a result of a therapeutic modulation. In this perspective, the analytical optimization of FGF23 determination may allow a marriage between physiology and epidemiology and a merging towards clinical outcomes.

Perception of dead conspecifics increases reproductive investment in fruit flies

Abstract Adaptive plasticity in life‐history traits is often critical to maximize fitness in the face of environmental heterogeneity. For example, many organisms respond to a threat to their survival (and hence residual reproductive value) by adaptively increasing their investment into current reproduction (i.e. ‘terminal investment’). A key to successful terminal investment is the use of adequate environmental cues to extrinsic mortality sources. In species that live at high population densities, the presence of dead conspecifics could potentially act as a reliable cue to extrinsic mortality sources (e.g. via pathogens). We experimentally tested this idea using the model organism Drosophila melanogaster, a pest species that experiences marked demographic shifts. We monitored the reproductive output of young mated females during repeated bouts of egg laying in the presence (i.e. dead exposed) or absence (i.e. controls) of dead conspecifics, interspersed by mating periods where dead conspecifics were always absent. Dead‐exposed females produced more offspring than controls when in the presence of dead conspecifics but fewer offspring than controls when dead conspecifics were subsequently removed (i.e. during mating periods). These changes were repeatable across time, and are thus indicative of high plasticity in reproductive behaviour induced by perception of dead conspecifics. Flies responded equally to starvation‐induced and flash‐frozen dead conspecifics, indicating that the cause of death is irrelevant to trigger the plastic response observed, and that females respond to the presence of dead conspecifics per se. Finally, we found that dead‐exposed females produced heavier daughters (but not sons) than controls when in the presence of dead conspecifics, but not during mating periods (when dead conspecifics were absent). In this species, body size correlates more strongly with female (than male) fitness, so higher investment in daughter weight is congruent with terminal investment. Altogether, these results show that perception of dead conspecifics leads to highly plastic life‐history adjustments in the form of increased investment in both the quantity and the quality of offspring. We discuss these results in the context of terminal investment and other alternative hypotheses. Read the free Plain Language Summary for this article on the Journal blog.

Rethinking Our Concepts and Assumptions About Autism.

Autism is a clinical consensus diagnosis made based on behavioral symptoms of early developmental difficulties in domains of social-communication (SC) and restricted repetitive behaviors (RRB). Many readily assume that alongside being optimal for separating individuals based on SC and RRB behavioral domains, that the label should also be highly useful for explaining differential biology, outcomes, and treatment (BOT) responses. However, we also now take for granted the fact that the autism population is vastly heterogeneous at multiple scales, from genome to phenome. In the face of such multi-scale heterogeneity, here we argue that the concept of autism along with the assumptions that surround it require some rethinking. While we should retain the diagnosis for all the good it can do in real-world circumstances, we also call for the allowance of multiple other possible definitions that are better tailored to be highly useful for other translational end goals, such as explaining differential BOT responses.

Orthogonal Modality Disentanglement and Representation Alignment Network for NIR-VIS Face Recognition

Near-infrared and visual (NIR-VIS) face matching, as the most typical task in Heterogeneous Face Recognition (HFR), has attracted increasing attention in recent years. However, due to the large within-class discrepancies, including domain differences and residual discrepancies (i.e., lighting, expressions, occlusion, blurry, pose, etc), this is still a difficult task. Conventional NIR-VIS FR methods only focus on reducing the modality gap between cross-domain images, while neglecting to eliminate the residual variations. To better solve the above problems, this paper proposes a novel Orthogonal Modality Disentanglement and Representation Alignment (OMDRA) approach, which consists of three key components, including Modality-Invariant (MI) loss, Orthogonal Modality Disentanglement (OMD) and Deep Representation Alignment (DRA). Firstly, the MI loss is designed to learn modality-invariant and identity-discriminative representation, by increasing between-class separability and within-class compactness between NIR and VIS heterogeneous data. Secondly, the high-level Hybrid Facial Feature (HFF) layer of the backbone network is projected into two subspaces: the modality-related and identity-related subspaces. The OMD is designed to decouple modal information via an adversarial process, and we further impose Orthogonal Representation Decorrelation (ORD) to the OMD to decrease the correlation between identity representations and domain representations, as well as enhancing their representation capabilities. Finally, the DRA aims to eliminate the residual variations by performing a high-level representation alignment between non-neutral face and neutral face, which can effectively guides the network to learn discriminative and residual-invariant face representation. The joint scheme enables the disentanglement of modality variations, elimination of residual discrepancies, and the purification of identity information. Extensive experiments on challenging cross-domain databases indicate that our OMDRA method is superior to the state-of-the-art methods.

M3SPCANet: A simple and effective ConvNets with unsupervised predefined filters for face recognition

The principal component analysis network (PCANet) with predefined filters has recognized as a lightweight convolutional neural networks (CNN) baseline. However, there are three shortages in PCANet: (i) single-scale convolution leads to insufficient feature learning, (ii) the spatial layout information and relationship between channel features are neglected, (iii) lack of nonlinearity because there is no activation function. Therefore, we devise a PCANet alternative dubbed Multi-Scale Spatial pyramid Second-order pooling Principle Component Analysis Network (M3SPCANet) in the paper. First, multi-scale PCA filters are learned and applied to obtain feature maps via convolution. Second, the feature maps are activated via nonlinear function Tanh and stacked together. Third, second-order pooling is exploited to extract correlation information between any two channels of the feature maps, moreover, spatial pyramid strategy is employed to learn local and holistic features simultaneously. Finally, whitening PCA based fully connected layer is constructed to remove redundancy information. We utilize cosine similarity based nearest neighbor classifier for face matching. Extensive experiments on four homogeneous and one heterogeneous face databases indicate that our method can achieve superior identification rates, and yield competitive verification performance. Cross database evaluations demonstrate that the multi-scale filters of our method have strong generalization ability. Our implementation code will be available at https://github.com/dxtyut/facerecognition.