Pair Of Face Images Research Articles

Multi-View Knowledge Ensemble With Frequency Consistency for Cross-Domain Face Translation.

Cross-domain face translation aims to transfer face images from one domain to another. It can be widely used in practical applications, such as photos/sketches in law enforcement, photos/drawings in digital entertainment, and near-infrared (NIR)/visible (VIS) images in security access control. Restricted by limited cross-domain face image pairs, the existing methods usually yield structural deformation or identity ambiguity, which leads to poor perceptual appearance. To address this challenge, we propose a multi-view knowledge (structural knowledge and identity knowledge) ensemble framework with frequency consistency (MvKE-FC) for cross-domain face translation. Due to the structural consistency of facial components, the multi-view knowledge learned from large-scale data can be appropriately transferred to limited cross-domain image pairs and significantly improve the generative performance. To better fuse multi-view knowledge, we further design an attention-based knowledge aggregation module that integrates useful information, and we also develop a frequency-consistent (FC) loss that constrains the generated images in the frequency domain. The designed FC loss consists of a multidirection Prewitt (mPrewitt) loss for high-frequency consistency and a Gaussian blur loss for low-frequency consistency. Furthermore, our FC loss can be flexibly applied to other generative models to enhance their overall performance. Extensive experiments on multiple cross-domain face datasets demonstrate the superiority of our method over state-of-the-art methods both qualitatively and quantitatively.

CoReFace: Sample-guided Contrastive Regularization for Deep Face Recognition

The discriminability of the feature representation is crucial for face recognition. However, previous methods rely solely on the learnable weights of the classification layer, which represent the identities. This reliance could be problematic as the evaluation process depends on the similarity between pairs of face images and requires minimal identity information learned during training. As a result, there is an inconsistency between the training and evaluation processes, which can confuse the feature encoder and hinder the effectiveness of identity-based methods. To address this problem, we propose a novel approach namely Contrastive Regularization for Face Recognition (CoReFace), which applies sample-level regularization in feature representation learning. Specifically, we employ sample-guided contrastive learning to directly regularize the training based on the sample-sample relationship and thus align it with the evaluation process. To avoid image quality degradation, we augment the embeddings instead of the images in order to integrate contrastive learning into face recognition. Additionally, we introduce a new contrastive loss function for the regularization of representation distribution. This function incorporates an adaptive margin and a supervised contrastive mask to ensure stable loss values and prevent interference with the identity supervision signals. Finally, we explore new pair-coupling protocols in order to overcome the problem of semantically repetitive signals in contrastive learning. Extensive experiments demonstrate the efficacy and efficiency of our CoReFace approach, which achieves competitive results compared to state-of-the-art methods. Code could be found https://github.com/IsidoreSong/CoreFace here.

The Influence of the Other-Race Effect on Susceptibility to Face Morphing Attacks.

Facial morphs created between two identities resemble both of the faces used to create the morph. Consequently, humans and machines are prone to mistake morphs made from two identities for either of the faces used to create the morph. This vulnerability has been exploited in "morph attacks" in security scenarios. Here, we asked whether the "other-race effect" (ORE)-the human advantage for identifying own- vs. other-race faces-exacerbates morph attack susceptibility for humans. We also asked whether face-identification performance in a deep convolutional neural network (DCNN) is affected by the race of morphed faces. Caucasian (CA) and East-Asian (EA) participants performed a face-identity matching task on pairs of CA and EA face images in two conditions. In the morph condition, different-identity pairs consisted of an image of identity "A" and a 50/50 morph between images of identity "A" and "B". In the baseline condition, morphs of different identities never appeared. As expected, morphs were identified mistakenly more often than original face images. Of primary interest, morph identification was substantially worse for cross-race faces than for own-race faces. Similar to humans, the DCNN performed more accurately for original face images than for morphed image pairs. Notably, the deep network proved substantially more accurate than humans in both cases. The results point to the possibility that DCNNs might be useful for improving face identification accuracy when morphed faces are presented. They also indicate the significance of the race of a face in morph attack susceptibility in applied settings.

ENHANCING FACIAL RECOGNITION THROUGH CONTRASTIVE CONVOLUTION: A COMPREHENSIVE METHODOLOGY

This study presents an innovative approach to enhance facial recognition technology using contrastive convolutional neural networks (CNNs). The primary focus is on improving the accuracy and efficiency of face recognition systems under varying conditions. Key elements of this approach include meticulous data preparation and preprocessing, where images undergo normalization and diverse augmentation techniques to ensure quality inputs. The network architecture is designed to process pairs of face images, utilizing a common feature extractor and cascaded convolution layers for detailed feature representation. A specialized kernel generator further refines the process, emphasizing unique facial characteristics. The core of the training regimen is a contrastive loss function, optimized through gradient descent to enhance the network's discriminatory capabilities. Results from the study demonstrate a significant improvement in recognition accuracy, particularly highlighted by the superior performance of the proposed model in comparison to standard facial recognition algorithms. This research provides a comprehensive methodology that could revolutionize face recognition technology, offering more reliable and efficient solutions for various applications.

Twin Identification over Viewpoint Change: A Deep Convolutional Neural Network Surpasses Humans.

Deep convolutional neural networks (DCNNs) have achieved human-level accuracy in face identification (Phillips et al., 2018), though it is unclear how accurately they discriminate highly-similar faces. Here, humans and a DCNN performed a challenging face-identity matching task that included identical twins. Participants (N = 87) viewed pairs of face images of three types: same-identity, general imposters (different identities from similar demographic groups), and twin imposters (identical twin siblings). The task was to determine whether the pairs showed the same person or different people. Identity comparisons were tested in three viewpoint-disparity conditions: frontal to frontal, frontal to 45° profile, and frontal to 90°profile. Accuracy for discriminating matched-identity pairs from twin-imposter pairs and general-imposter pairs was assessed in each viewpoint-disparity condition. Humans were more accurate for general-imposter pairs than twin-imposter pairs, and accuracy declined with increased viewpoint disparity between the images in a pair. A DCNN trained for face identification (Ranjan et al., 2018) was tested on the same image pairs presented to humans. Machine performance mirrored the pattern of human accuracy, but with performance at or above all humans in all but one condition. Human and machine similarity scores were compared across all image-pair types. This item-level analysis showed that human and machine similarity ratings correlated significantly in six of nine image-pair types [range r = 0.38 to r = 0.63], suggesting general accord between the perception of face similarity by humans and the DCNN. These findings also contribute to our understanding of DCNN performance for discriminating high-resemblance faces, demonstrate that the DCNN performs at a level at or above humans, and suggest a degree of parity between the features used by humans and the DCNN.

Decoding neural patterns for the processing of fearful faces under different visual awareness conditions: A multivariate pattern analysis.

Previous studies have provided mixed findings regarding the nonconscious processing of fearful faces. Here, we used multivariate pattern analysis on electroencephalography data from three backward masking experiments to examine the processing of fearful faces under different visual awareness conditions. Three groups of participants were shown pairs of face images presented very briefly (for 16 ms) or for sufficiently long (for 266 ms), and completed tasks where the faces were either relevant to the experimental task (Experiment 1) or not (Experiments 2 and 3). Three main decoding analyses were performed. First, in the visual awareness decoding, the visibility of the faces, and hence participants' awareness of them, was maximally decodable in three time windows: 158-168 ms, 235-260 ms and 400-600 ms where the earlier neural patterns were generalized to the later stage activity. Second, we found that the spatial location of a fearful face in the face pairs was decodable, however only when the faces were consciously seen and task-relevant. Finally, we successfully decoded distinct neural patterns associated with the fearful-face-present conditions, compared to the fearful-face-absent conditions, and these patterns were decodable during both short and long presentations of the faces. Together, our results suggest that, while the processing of the spatial location of fearful faces requires awareness and task-relevancy, the mere presence of fearful faces can be processed even when visual awareness is highly restricted.

Face-to-trait inferences in Japanese children and adults based on Caucasian faces.

Recently, many studies have indicated that humans make social evaluations from facial appearances instantaneously and automatically. Furthermore, such judgments play an important role in several social contexts. However, the mechanisms involved in the ability to form impressions from faces are still unknown, as is the extent to which these can be regarded as universal in perceiving impressions. In the current study, computer-generated Caucasian faces were used to assess the universality or cultural differences in impression formation among Japanese children and adults. This study hypothesized that impressions of trustworthiness and dominance may be more fundamental and universal, whereas the impression of competence may be more complex and culture-dependent. In Experiment 1a, 42 children aged 3-6 years were presented with 10 pairs of face images and asked which image in each pair was more trustworthy, dominant, or competent. Overall, it was found that as age increased, the rate of agreement of Japanese participants with the judgment of American participants, obtained in a previous study, increased. However, the agreement rate for competence was relatively low. Experiment 1b, conducted with 46 children, was a replication of Experiment 1a, and the results showed the same tendency. In Experiment 2, 45 Japanese adults made impression judgments on 19 pairs of face images identical to those used in Experiment 1b. The results suggested that while dominance was a dimension not easily influenced by developmental changes or culture, trustworthiness could be influenced by cultural differences in facial expression recognition. Therefore, different judgment criteria are used for children and adults. For competence, the agreement rate with Americans was relatively stable and low among the different age groups. This suggests that depending on the dimension of the trait, certain judgments are influenced by cultural context and, therefore, change criteria, while others are based on more universal criteria.

Propagating Facial Prior Knowledge for Multitask Learning in Face Super-Resolution

Existing face hallucination methods always achieve improved performance through regularizing the model with facial prior. Most of them always estimate facial prior information first and then leverage it to help the prediction of the target high-resolution face image. However, the accuracy of prior estimation is difficult to guarantee, especially for the low-resolution face image. Once the estimated prior is inaccurate or wrong, the following face super-resolution performance is unavoidably influenced. A natural question that arises: how to incorporate facial prior effectively and efficiently without prior estimation? To achieve this goal, we propose to learn facial prior knowledge at training stage, but test only with low-resolution face image, which can overcome the difficulty of estimating accurate prior. In addition, instead of estimating facial prior, we directly explore the potential of high-quality facial prior in the training phase and progressively propagate the facial prior knowledge from the teacher network (trained with the low-resolution face/high-quality facial prior and high-resolution face image pairs) to the student network (trained with the low-resolution face and high-resolution face image pairs). Quantitative and qualitative comparisons on benchmark face datasets demonstrate that our method outperforms the state-of-the-art face super-resolution methods. The source codes of the proposed method will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/wcy-cs/KDFSRNet</uri> .

Verifying unfamiliar identities: Effects of processing name and face information in the same identity-matching task

Matching the identity of unfamiliar faces is important in applied identity verification tasks, for example when verifying photo ID at border crossings, in secure access areas, or when issuing identity credentials. In these settings, other biographical details—such as name or date of birth on an identity document—are also often compared to existing records, but the impact of these concurrent checks on decisions has not been examined. Here, we asked participants to sequentially compare name, then face information between an ID card and digital records to detect errors. Across four experiments (combined n = 274), despite being told that mismatches between written name pairs and face image pairs were independent, participants were more likely to say that face images matched when names also matched. Across all experiments, we found that this bias was unaffected by the image quality, suggesting that the source of the bias is somewhat independent of perceptual processes. In a final experiment, we show that this decisional bias was found only for name checks, but not when participants were asked to check ID card expiration dates or unrelated object names. We conclude that the bias arises from processing identity information and propose that it operates at the level of unfamiliar person identity representations. Results are interpreted in the context of theoretical models of face processing, and we discuss applied implications.

Self-discriminative cycle generative adversarial networks for face image translation

Face cross-domain translation aims at mapping face images from one image domain to another. Common face image translation tasks include face photo-sketch and face photo-APDrawing cross-domain translation, which can be widely applied in real-world scenarios, such as criminal investigation, movie production, and digital entertainment. However, due to the limited face image pairs and the great gap of color and texture between different domains, face image cross-domain translation still faces many challenges. Existing methods usually produce blurring, artifacts, and structural distortion, leading to poor visualization quality. To tackle this problem, we propose a self-discriminative cycle generative adversarial network, in which the generator adopts an encoder-decoder structure and the corresponding discriminator is the encoder of the other generator in the reverse direction.In the self-discriminative manner, the encoder (i.e., discriminator) cleverly incorporates “True/False" semantic information and the sensitivity to pixel-level information, thereby enhancing the robustness and generalization ability of the generative model.Besides, we propose a novel omni-directional pixel-gradient loss. Thedesigned convolution kernel calculates the gradients of all directions around each pixel to extract the gradient information.Our model is motivated to effectively learn the continuous inter-pixel variation pattern by constraining the gradient information of generated images and ground-truth images to be consistent. The omni-directional pixel-gradient loss can be flexibly applied to other generative models and improve their performance.Extensive experiments show that the proposed framework can produce advanced results on the paired face photo-sketch datasets (CUFS, CUFSF) and the photo-APDrawing dataset (APDrawing). We further demonstrate the strong generalization ability of our model on real-world data and the excellent performance on unpaired face datasets.

Testing links between pain-related biases in visual attention and recognition memory: An eye-tracking study based on an impending pain paradigm.

Although separate lines of research have evaluated pain-related biases in attention or memory, laboratory studies examining links between attention and memory for pain-related information have received little consideration. In this eye-tracking experiment, we assessed relations between pain-related attention biases (ABs) and recognition memory biases (MBs) among 122 pain-free adults randomly assigned to impending pain (n = 59) versus impending touch (n = 63) conditions, wherein offsets of trials that included pain images were followed by subsequent possibly painful and non-painful somatosensory stimulation, respectively. Gaze biases of participants were assessed during presentations of pain-neutral (P-N) and happy-neutral (H-N) face image pairs within these conditions. Subsequently, condition differences in recognition accuracy for previously viewed versus novel pained and happy face images were examined. Overall gaze durations were significantly longer for pain (vs. neutral) faces that signalled impending pain than impending non-painful touch, particularly among the less resilient in the former condition. Impending pain cohorts also exhibited comparatively better recognition accuracy for both pained and happy face images. Finally, longer gaze durations on pain faces that signalled potential pain, but not potential touch, were related to more accurate recognition of previously viewed pain faces. In sum, pain cues that signal potential personal discomfort maintain visual attention more fully and are subsequently recognised more accuracy than are pain cues that signal non-painful touch stimulation.

Reliable detection of doppelgängers based on deep face representations

Doppelgängers (or lookalikes) usually yield an increased probability of false matches in a facial recognition system, as opposed to random face image pairs selected for non-mated comparison trials. In this work, the impact of doppelgängers on the HDA Doppelgänger and Disguised Faces in The Wild databases is assessed using a state-of-the-art face recognition system. It is found that doppelgänger image pairs yield very high similarity scores resulting in a significant increase of false match rates. Further, a doppelgänger detection method is proposed, which distinguishes doppelgängers from mated comparison trials by analysing differences in deep representations obtained from face image pairs. The proposed detection system employs a machine learning-based classifier, which is trained with generated doppelgänger image pairs utilising face morphing techniques. Experimental evaluations conducted on the HDA Doppelgänger and Look-Alike Face databases reveal a detection equal error rate of approximately 2.7% for the task of separating mated authentication attempts from doppelgängers.

RAMT-GAN: Realistic and accurate makeup transfer with generative adversarial network

Automatic makeup transfer aims to precisely transfer the makeup style from a given reference makeup face image to a source image while preserving face identity and background information. Current works have achieved promising results in makeup transfer by using deep learning techniques. However, existing methods are focused on only one or two requirements, thus cannot simultaneously achieve face identity preservation, background retention, and accurate makeup transfer. Besides, it is also difficult to acquire a pair of well-aligned face images with different makeup styles. Aimed at these problems, we propose RAMT-GAN (Realistic and Accurate Makeup Transfer Generative Adversarial Network), a GAN-based image transformation framework for achieving realistic and accurate makeup style transfer. Specifically, we utilize a dual input/output network that builds on the BeautyGAN architecture to achieve cross-domain image transformation. Then, identity preservation loss and background invariant loss are introduced in RAMT-GAN to help synthesize realistic and accurate face makeup images. Extensive experiments demonstrate that the proposed makeup transfer model can synthesize makeup faces with accurate reference style as well as maintaining the identity information and the background information.

A registration strategy from scale adjustment to fine for thermal-visible face images

The analysis of multispectral facial features is rising as one of the new emerging tools in medical diagnosis. The dynamic infrared thermography and visible image data fusion of a human face can manifest several physiological behaviours used in medical diagnosis. However, an essential but challenging step is the registration of thermal and visible images. This paper presents a two-step strategy to register visible face image into the same coordinate of the thermal image. First, to adjust the large-scale difference, a pre-transformation is performed, in which the thermal and visible images are coarsely aligned by automatic selection of common keypoints in the image pair using a calibration rig. Then, the fine registration is performed by extracting phase-congruent feature points in the image pair for more precise registration. The particle swarm optimization (PSO) based fast sample consensus (PSOFSC) algorithm is also introduced for selecting correct inlier correspondence pairs. The proposed method was validated using several thermal-visible face image pairs with various resolutions from different cameras, acquired in different scenarios. The results are encouraging when compared to many state-of-the-art registration methods for thermal-visible image pairs.

Component-based metric learning for fully automatic kinship verification

This paper introduces a fully automatic method for kinship verification from facial images. Recently, a number of methods have been proposed to verify kinship from facial images, however, most of these methods are needed to exactly align face images before feature extraction in a manual manner. Unlike these methods, our method does not depend on face alignment. Firstly, we localize several facial feature points by utilizing a facial feature detector to extract SIFT descriptor around each feature point of a face image. Lastly, two ways, feature combination and distance metric learning, are used to verify the kinship of a pair of face images. For feature combination, three simple strategies of feature combination and support vector machine classifier are used for kinship verification. For metric learning, we propose a component-based metric learning (CML) method to measure the distance of each face pair, which jointly learns multiple local distance metrics, and one specific distance metric for each facial feature point. Experimental results show the effectiveness of our proposed approach on two popular kinship datasets.

Image reconstruction reveals the impact of aging on face perception.

Extensive work has demonstrated an age-related decline in face recognition, but the nature and the extent of aging-related alterations in face representations remain unclear. Here, we address these issues using an image reconstruction approach to reveal the content of visual representations. Healthy young and older adults provided similarity judgments for pairs of face images. Facial shape and surface features were subsequently derived and combined into image reconstructions of facial appearance. Both objective and experimental evaluations revealed that reconstructions were successful for every participant. Critically, shape and surface properties, such as eye shape and skin tone, were less accurately represented in older than young individuals. Yet, age-related differences in face representations, though significant, were less pronounced than those due to individual variability. Our results provide novel insights into age-related changes in visual perception and demonstrate the utility of image reconstruction to uncovering internal representations across a variety of populations. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Ring-Regularized Cosine Similarity Learning for Fine-Grained Face Verification

Face verification aims to determine whether a pair of face images belong to the same person. Different from the traditional face verification, the negative sample pairs in fine-grained face verification are composed of similar face images, e.g., facial images of twins, which makes it still very challenging. In this paper, we investigate the fine-grained face verification problem via metric learning techniques, and propose a ring-regularized cosine similarity learning (RRCSL) method to distinguish the negative face pairs. The proposed RRCSL method seeks a linear transformation to enlarge the cosine similarity of intra-class and reduce the cosine similarity of inter-class as much as possible, and adaptively learns the norm of samples to the scaled circle by exploiting the ring regularization term simultaneously. Experimental results on three face datasets demonstrate the effectiveness of RRCSL for fine-grained face verification.

Small data assisting face image illumination normalization

Image transfer based on deep learning methods can achieve good results in face illumination processing. However, data constraints and generalization ability restrict the further development of these methods in this field. In this paper, we propose small data assisting face image illumination normalization. For data constraints, we train the network model on a small number of image pairs. In terms of generalization ability, the proposed normalization network parameters are different for processing different face images. Small data learning can provide prior knowledge, and the reconstruction process can guide detail generation. Therefore, the small data learning network and the reconstruction network are complementary to each other in image generating mode when only a small quantity of data is available. We use this network mode to normalize the illumination and reconstruct the super-resolution of face images. After illumination normalization, super-resolution reconstruction can obtain more precise face information and further improve the face recognition rate. Experiments show that the proposed method has good normalization performance when only 500 face image pairs are used for training.

Weighted Feature Histogram of Multi-Scale Local Patch Using Multi-Bit Binary Descriptor for Face Recognition.

Most face recognition methods employ single-bit binary descriptors for face representation. The information from these methods is lost in the process of quantization from real-valued descriptors to binary descriptors, which greatly limits their robustness for face recognition. In this study, we propose a novel weighted feature histogram (WFH) method of multi-scale local patches using multi-bit binary descriptors for face recognition. First, to obtain multi-scale information of the face image, the local patches are extracted using a multi-scale local patch generation (MSLPG) method. Second, with the goal of reducing the quantization information loss of binary descriptors, a novel multi-bit local binary descriptor learning (MBLBDL) method is proposed to extract multi-bit local binary descriptors (MBLBDs). In MBLBDL, a learned mapping matrix and novel multi-bit coding rules are employed to project pixel difference vectors (PDVs) into the MBLBDs in each local patch. Finally, a novel robust weight learning (RWL) method is proposed to learn a set of robust weights for each patch to integrate the MBLBDs into the final face representation. In RWL, a codebook is first constructed by clustering MBLBDs on each local patch to extract a feature histogram. Then, considering that different parts of the face have different degrees of robustness to local changes, a set of weights is learned to concatenate the feature histograms of all local patches into the final representation of a face image. In addition, to further improve the performance for heterogeneous face recognition, a coupled WFH (C-WFH) method is proposed. C-WFH maintains the similarity of the corresponding MBLBDs and feature histograms for a pair of heterogeneous face images by means of a novel coupled feature learning (CFL) method to reduce the modality gap. A series of experiments are conducted on widely used face datasets to analyze the performance of WFH and C-WFH. Extensive experimental results show that WFH and C-WFH outperform state-of-the-art face recognition methods.

Towards NIR-VIS Masked Face Recognition

Near-infrared to visible (NIR-VIS) face recognition is the most common case in heterogeneous face recognition, which aims to match a pair of face images captured from two different modalities. Existing deep learning based methods have made remarkable progress in NIR-VIS face recognition, while it encounters certain newly-emerged difficulties during the pandemic of COVID-19, since people are supposed to wear facial masks to cut off the spread of the virus. We define this task as NIR-VIS masked face recognition, and find it problematic with the masked face in the NIR probe image. First, the lack of masked face data is a challenging issue for the network training. Second, most of the facial parts (cheeks, mouth, nose etc.) are fully occluded by the mask, which leads to a large amount of loss of information. Third, the domain gap still exists in the remaining facial parts. In such scenario, the existing methods suffer from significant performance degradation caused by the above issues. In this paper, we aim to address the challenge of NIR-VIS masked face recognition from the perspectives of training data and training method. Specifically, we propose a novel heterogeneous training method to maximize the mutual information shared by the face representation of two domains with the help of semi-siamese networks. In addition, a 3D face reconstruction based approach is employed to synthesize masked face from the existing NIR image. Resorting to these practices, our solution provides the domain-invariant face representation which is also robust to the mask occlusion. Extensive experiments on three NIR-VIS face datasets demonstrate the effectiveness and cross-dataset-generalization capacity of our method.