Morphed Faces Research Articles

Preventing face morphing attacks by using legacy face images

Countries allow citizens to upload a face image or provide printed copies to authorities to issue their passport. This allows prior image manipulation with criminal intent. A composite image can be created by blending the images of two individuals before submitting the composite image to the authorities. Depending on several factors, the submitted morphed face image can fool the issuing officer to issue a legitimate document. The document can then be successfully used by either contributor to attack the automatic Face Recognition Systems (FRS) operating, for example, at Automatic Border Control (ABC) airport gates. This is known as a Morphing Attack (MA), an identity sharing scheme with serious consequences. Here, the security vulnerabilities due to MAs are identified and analysed, and an additional security measure that allows mitigating the risk or preventing MAs in certain scenarios is proposed. The measure introduces more comparisons by keeping the old passport or ID card image in the chip, in passport renewal applications or first time passport applications, respectively. This approach is implemented with two FRSs on a challenging dataset and the dramatic decrease in the vulnerability is shown. Finally, their performance is compared with a state-of-the-art MA detection algorithm on the same dataset.

Improving Detection of Manipulated Passport Photos - Training Course for Border Control Inspectors to Detect Morphed Facial Passport Photos - Part I: Introduction, State-of-the-Art and Preparatory Tests and Experiments

In recent years, ID controllers have observed an increase in the use of fraudulently obtained ID documents [1]. This often involves deception during the application process to get a genuine document with a manipulated passport photo. One of the methods used by fraudsters is the presentation of a morphed facial image. Face morphing is used to assign multiple identities to a biometric passport photo. It is possible to modify the photo so that two or more persons, usually the known applicant and one or more unknown companions, can use the passport to pass through a border control [2]. In this way, persons prohibited from crossing a border can cross it unnoticed using a face morphing attack and thus acquire a different identity. The face morphing attack aims to weaken the application for an identity card and issue a genuine identity document with a morphed facial image. A survey among experts at the Security Printers Conference revealed that a relevant number of at least 1,000 passports with morphed facial images had been detected in the last five years in Germany alone [1]. Furthermore, there are indications of a high number of unreported cases. This high presumed number of unreported cases can also be explained by the lack of morphed photographs’ detection capabilities. Such identity cards would be recognized if the controllers could recognize the morphed facial images. Various studies have shown that the human eye has a minimal ability to recognize morphed faces as such [2], [3], [4], [5], [6]. This work consists of two parts. Both parts are based on the complete development of a training course for passport control officers to detect morphed facial images. Part one contains the conception and the first test trials of how the training course has to be structured to achieve the desired goals and thus improve the detection of morphed facial images for passport inspectors. The second part of this thesis will include the complete training course and the evaluation of its effectiveness.

Learning to distinguish: shared perceptual features and discrimination practice tune behavioural pattern separation

ABSTRACT Pattern separation is a computational mechanism performed by the hippocampus allowing the reduction of overlap between sensory inputs with similar perceptual features. Our first aim was to develop a new paradigm sensitive to the behavioural consequences of pattern separation (mnemonic discrimination). For this purpose, we constructed morphed face stimuli with parametrically changing levels of similarity. After encoding participants saw studied items and similar lure faces. Perceptual similarity affected false recognition and there was a gradual reduction in discrimination accuracy with the increment of similarity between the stimuli. However, confidence ratings were sensitive to smaller changes (Experiment 1) than the other test type with “old”/“similar”/“new” response options (Experiment 2). Mnemonic discrimination relies strongly on retrieving details of the original stimulus. Therefore, we investigated whether pattern separation can be tuned by retrieval in the form of a discrimination task (Experiment 3). Our findings suggest that repeatedly encountering the stimuli within a two-alternative forced-choice task (in comparison with the repeated presentation of the material) increased both the correct identification and the false recognition of similar stimuli two days after encoding. We conclude that basic computational mechanisms of the hippocampus can be tuned by a task that requires discrimination between studied and new stimuli.

Log-Polar Facial Recognition Using Minimum Feature Band

There is an increasing demand for biometric security systems in several fields. This study presents a highly accurate facial recognition method that uses high-speed transformation and facial morphing using region-limited log-polar transformation based on a center point calculated from the coordinates of both eyes and corners of the mouth. Log-polar transformation is limited to the region, so that the region including the feature can be suppressed to the minimum range, thereby facilitating high-speed transformation. Additionally, after facial morphing, the shapes of the eyes and mouth are altered based on the outline of the face, enabling high-precision facial recognition. The efficacy of the proposed method is verified experimentally. Therefore, we can confirm that 91.72% of images using the color FERET database and 96% of images using the FEI face database can be recognized using our method.

Enhanced N170 to outgroup faces: Perceptual novelty or prejudice?

ABSTRACT Habituation to ethnic ingroup members has been reported to be greater than to ethnic outgroup members. This pattern could be due to the lack of perceptive experience (familiarity) with outgroup facial morphs or, alternatively, to the prejudice held toward that outgroup. We explored this disjunctive in 71 participants, all Spanish, who were experimentally habituated to faces from their Ingroup and to faces from two unfamiliar outgroups, one for which there is low probability of prejudice in this population (Non-prejudiced Outgroup), and one for which the probability of prejudice is higher (Prejudiced Outgroup). We indexed habituation through event-related potentials, concretely as the differential amplitude of the face-sensitive N170 component from Initial to Final trials of each group. Afterward, participants completed several prejudice measures. N170 showed significant habituation to all faces, though it did not differ among groups. However, a regression analysis revealed that individual habituation to the Outgroup faces was inversely related to implicit prejudice scores. Importantly, N170 amplitudes were maximal for the Prejudiced Outgroup in both Initial and Final trials. We conclude that these effects are explained by the prejudice held toward a specific outgroup rather than perceptive experience.

Titrating the Smell of Fear: Initial Evidence for Dose-Invariant Behavioral, Physiological, and Neural Responses

It is well accepted that emotional intensity scales with stimulus strength. Here, we used physiological and neuroimaging techniques to ask whether human body odor—which can convey salient social information—also induces dose-dependent effects on behavior, physiology, and neural responses. To test this, we first collected sweat from 36 males classified as low-, medium-, and high-fear responders. Next, in a double-blind within-subjects functional-MRI design, 31 women were exposed to three doses of fear-associated human chemosignals and neutral sweat while viewing face morphs varying between expressions of fear and disgust. Behaviorally, we found that all doses of fear-sweat volatiles biased participants toward perceiving fear in ambiguous morphs, a dose-invariant effect generally repeated across physiological and neural measures. Bayesian dose-response analysis indicated moderate evidence for the null hypothesis (except for the left amygdala), tentatively suggesting that the human olfactory system engages an all-or-none mechanism for tagging fear above a minimal threshold.

Instructed threat enhances threat perception in faces.

Anxiety can boost the detection of potential threats in many ways. There is evidence that one and the same facial expression can be perceived differently depending on whether it is seen in a neutral or in a threatening situation. The present study investigated how aversive anticipation influences the accuracy of facial emotion recognition and the perceived emotional intensity of faces that had their objective emotional intensity manipulated. Forty-three participants categorized and rated the intensity of morphed faces (20%, 40%, 60%, and 80%) of fearful, angry, and happy expressions. Differently colored picture frames indicated either threat of electric shock or safety. Threat of shock enhanced the categorization accuracy specifically for fearful faces. During threat, 80% fearful and happy faces, and all levels of angry faces (20%-80%) were rated as more intense. In addition, we found that more trait-anxious individuals more frequently erroneously categorized neutral faces as fearful. Thus, state anxiety enhanced accurate fear categorization and boosted the perceived intensity of emotional faces, whereas trait anxiety led to a biased threat perception in nonthreatening faces. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Are Morphs a Valid Substitute for Real Multiracial Faces in Race Categorization Research?

The rise of the multiracial population has been met with a growing body of research examining multiracial face perception. A common method for creating multiracial face stimuli in past research has been mathematically averaging two monoracial "parent" faces of different races to create computer-generated multiracial morphs, but conclusions from research using morphs will only be accurate to the extent that morphs yield perceptual decisions similar to those that would be made with real multiracial faces. The current studies compared race classifications of real and morphed multiracial face stimuli. We found that oval-masked morphed faces were classified as multiracial significantly more often than oval-masked real multiracial faces (Studies 1 and 2), but at comparable levels to unmasked real multiracial faces (Study 2). Study 3 examined factors that could explain differences in how morphs and real multiracial faces are categorized and pointed to the potential role that unusualness/distinctiveness might play.

Self-esteem predicts sensitivity to dynamic changes in emotional expression

Previous studies found that people with low self-esteem (SE) exhibit heightened sensitivity and reactivity to facial expressions displaying rejection compared with people with high SE. The present study focuses more on dynamic rather than static expressions. Using a face morph task, in which a neutral expression gradually morphs into a full-intensity angry, disgusted, fearful, or happy expression, this study examined whether SE influences sensitivity to dynamic changes in emotional expression. After controlling for depression and anxiety among 160 college students, participants with low SE recognized angry and disgusted expressions at lower degrees of emotional intensity than participants with high SE. However, recognition of fearful and happy expressions was not found to be related to SE. These findings suggest a specific rejection sensitivity in the identification of subtle emotional expressions in people with low SE.

Automated Artifact Retouching in Morphed Images With Attention Maps

Morphing attack is an important security threat for automatic face recognition systems. High-quality morphed images, i.e. images without significant visual artifacts such as ghosts, noise, and blurring, exhibit higher chances of success, being able to fool both human examiners and commercial face verification algorithms. Therefore, the availability of large sets of high-quality morphs is fundamental for training and testing robust morphing attack detection algorithms. However, producing a high-quality morphed image is an expensive and time-consuming task since manual post-processing is generally required to remove the typical artifacts generated by landmark-based morphing techniques. This work describes an approach based on the Conditional Generative Adversarial Network paradigm for automated morphing artifact retouching and the use of Attention Maps to guide the generation process and limit the retouch to specific areas. In order to work with high-resolution images, the framework is applied on different facial crops, which, once processed and retouched, are accurately blended to reconstruct the whole morphed face. Specifically, we focus on four different squared face regions, i.e. the right and left eyes, the nose, and the mouth, that are frequently affected by artifacts. Several qualitative and quantitative experimental evaluations have been conducted to confirm the effectiveness of the proposal in terms of, among the others, pixel-wise metrics, identity preservation, and human observer analysis. Results confirm the feasibility and the accuracy of the proposed framework.

Single Morphing Attack Detection Using Feature Selection and Visualization Based on Mutual Information

Face morphing attack detection is a challenging task. Automatic classification methods and manual inspection are realised in automatic border control gates to detect morphing attacks. Understanding how a machine learning system can detect morphed faces and the most relevant facial areas is crucial. Those relevant areas contain texture signals that allow us to separate the bona fide and the morph images. Also, it helps in the manual examination to detect a passport generated with morphed images. This paper explores features extracted from intensity, shape, texture, and proposes a feature selection stage based on the Mutual Information filter to select the most relevant and less redundant features. This selection allows us to reduce the workload and know the exact localisation of such areas to understand the morphing impact and create a robust classifier. The best results were obtained for the method based on Conditional Mutual Information and Shape features using only 500 features for FERET images and 800 features for FRGCv2 images from 1,048 features available. The eyes and nose are identified as the most critical areas to be analysed.

Self-face perception in 12-month-old infants: A study using the morphing technique

The present study investigated self-face perception in 12-month-old infants using the morphing technique. Twenty-four 12-month-old infants participated in both the main and control experiments. In the main experiment, we used the participant’s own face, an unfamiliar infant’s face (age- and gender-matched), and a morphed face comprising 50 % each of the self and unfamiliar faces as stimuli. The control experiment followed the same procedure, except that the self-face was replaced with another unfamiliar face. In both experiments, two of these stimuli were presented side by side on a monitor in each trial, and infants’ fixation duration was measured. Results showed that shorter fixation durations were found for the morphed face compared with the self-face and the unfamiliar face in the main experiment, but there were no significant preferences for any comparisons in the control experiment. The results suggest that 12-month-old infants could detect subtle differences in facial features between the self-face and the other faces, and infants might show less preference for the self-resembling morphed face due to increased processing costs, which can be interpreted using the uncanny valley hypothesis. Overall, representations of the self-face seem to a certain extent to be formed by the end of the first year of life through daily visual experience.

The Influence of Natural and Induced Emotional States on the Recognition of Emotional Facial Expressions

There are a number of factors that influence emotion recognition, one of which is the perceiver’s emotional state. This study verifies the predictions of two theories concerning the influence of mood on emotion recognition. According to the affect-as-information theory, people in a positive mood are prone to a more global processing style and perceive emotional facial expressions more easily compared to those in a negative mood. The emotion congruence theory claims that people in a positive mood are more sensitive to positive expressions and people in a negative mood are more sensitive to negative expressions. These predictions were tested with the experimental paradigm using morphed faces developed by Jackson and Arlegui-Prieto. Study 1 used participants’ natural moods; its findings failed to replicate the main results of the original study. Study 2 used laboratory mood induction and showed that participants in a negative mood are more sensitive to negative emotions compared to those in positive mood. These findings support the emotion congruence theory. However, this result was obtained only for the participants with the most effective mood induction. The observed effects of mood are weak and fragile. For more persuasive results, a study with greater statistical power using more effective mood induction procedures is needed.

How We Perceive Others Resembling Us.

Eye contact is essential for social cognition, acting as an important tool for social communication. While differences in face scanning patterns concerning familiarity have been thoroughly investigated, the impact of facial similarity on gaze behavior has not been examined yet. We addressed this topic by recording subjects’ eye-directed gazing while looking at faces that were individually created systematically varying in terms of similarity to the self-face and familiarity. Subjects’ self-faces were morphed into three other faces including a close friend of the same sex. Afterwards, they rated similarity to their self-face of those morphed face stimuli in a separate rating task. Our results show a general preference for the eyes’ area as well as differences regarding fixation patterns depending on similarity to the self-face. The lower the similarity to the self-face, the more fixations on the eyes’ area. Subjects’ ratings followed a linear line, indicating well-pronounced face perception. Nevertheless, other faces were rated faster than the self-face independent of familiarity, while morphed faces got the slowest ratings. Our results mirror the importance of similarity to the self-face as a factor shaping the way we look at the eyes of others explaining variance apart from familiarity.

Detecting Morphing Attacks through Face Geometry Features.

Face-morphing operations allow for the generation of digital faces that simultaneously carry the characteristics of two different subjects. It has been demonstrated that morphed faces strongly challenge face-verification systems, as they typically match two different identities. This poses serious security issues in machine-assisted border control applications and calls for techniques to automatically detect whether morphing operations have been previously applied on passport photos. While many proposed approaches analyze the suspect passport photo only, our work operates in a differential scenario, i.e., when the passport photo is analyzed in conjunction with the probe image of the subject acquired at border control to verify that they correspond to the same identity. To this purpose, in this study, we analyze the locations of biologically meaningful facial landmarks identified in the two images, with the goal of capturing inconsistencies in the facial geometry introduced by the morphing process. We report the results of extensive experiments performed on images of various sources and under different experimental settings showing that landmark locations detected through automated algorithms contain discriminative information for identifying pairs with morphed passport photos. Sensitivity of supervised classifiers to different compositions on the training and testing sets are also explored, together with the performance of different derived feature transformations.

Colour matters more than shape for chimpanzees\u2019 recognition of developmental face changes

Social primates must recognise developmental stages of other conspecifics in order to behave appropriately. Infant faces have peculiar morphological characteristics—relatively large eyes, a small nose, and small mouth—known as baby schema. In addition, the infant faces of many primate species have unique skin coloration. However, it is unclear which features serve as critical cues for chimpanzees to recognise developmental changes in their faces. The present study aimed to investigate the relative contributions of facial shape and colour to age categorisation in chimpanzees. We used a symbolic matching-to-sample task in which chimpanzees were trained to discriminate between adult and infant faces. Then, we tested how their age category judgments transferred to a series of morphed faces which systematically differed in facial shape and colour. Statistical image quantification analysis revealed significant differences both in shape and colour between adult and infant faces. However, we found that facial coloration contributed to age categorisation in chimpanzees more than facial shape. Our results showed that chimpanzees use unique infantile facial coloration as a salient cue when discriminating between adult and infant faces. The display of their developmental stages through facial colour may help chimpanzees to induce appropriate behaviour from other individuals.

Can We Detect Face Morphing to Prevent Identity Theft?

A relatively new type of identity theft uses morphed facial images in identification documents in which images of two individuals are digitally blended to create an image that maintains a likeness to each of the original identities. We examined people’s ability to detect facial morphing. We collected 3,500 passport-format facial images. This dataset consists of a diverse set of people across gender, age, and race. Convolutional neural network descriptors are used to extract a low-dimensional, perceptually meaningful, representation of each face. For each of 54 faces, these representations are used to find the most similar face in the dataset. A mid-way morph is generated between each pair of different individuals; another mid-way morph is generated between two different photos of the same individual. The morphs are manually edited to remove obvious artifacts. In Experiment 1a (all experiments, N=100), on each trial, participants viewed two images—an original image alongside a morph (from the same or different individual)—and indicated if they are of the same individual or not. Participants struggled to perform this task accurately and were biased to respond “same” (d’=0.68; B=1.81). In Experiment 1b we focused participants’ attention on the eye/nose/mouth regions and provided feedback on each trial. This did not improve sensitivity but led to a reduced bias (d’=0.58; B=1.09). In Experiment 2a, participants saw a single image—a morph or an original—and indicated if it was a morphed face or not. Participants performed only slightly above chance (d’=0.21; B=0.98). In Experiment 2b, when participants were informed of morphing artifacts to look out for and received feedback, performance improved slightly (d’=0.53; B=0.92). Preliminary results suggest that computational methods for face recognition may outperform humans but remain imperfect. Combined, these results suggest that face morphing might be a worryingly effective technique for committing identity theft.

Tackling heterogeneity: Individual variability of emotion decoding deficits in severe alcohol use disorder

BackgroundSevere alcohol use disorder (SAUD) is associated with social cognition deficits. Patients with SAUD are impaired for the recognition of emotional facial expressions, particularly at early stages of abstinence. These deficits damage interpersonal relations and increase relapse risk. However, uncertainties still abound on their variation across emotions and on the heterogeneity of emotional impairments across patients. We addressed these questions by exploring how the deficit varies according to emotions’ type/intensity and patients’ heterogeneity. MethodsSixty-five recently detoxified patients with SAUD and 65 matched healthy controls performed the Facial Emotion Recognition Test, assessing the ability to identify six emotions (anger, contempt, disgust, fear, happiness, sadness) displayed by morphed faces with various intensities. Accuracy scores and detection thresholds were collected for each emotion. Beyond group comparisons, multiple single-case analyses determined the percentage of patients presenting decoding deficits for each emotion. ResultsWhen current depression and anxiety symptoms were controlled for, patients did not present a general emotion decoding deficit, but were rather characterized by specific deficits for disgust/contempt in accuracy, and for disgust in detection threshold scores. Single-case analyses showed that only a third of patients presented a clinically significant emotional deficit. ConclusionsPatients with SAUD only present emotional decoding deficits for specific interpersonal emotions (disgust/contempt) when subclinical psychopathological states are controlled for, and show no general emotional impairment. This goes against the proposal of a generalized social cognition deficit in this population. This group effect moreover masks a massive heterogeneity across patients, which has implications at experimental and clinical levels.

Face morph detection for unknown morphing algorithms and image sources: a multi‐scale block local binary pattern fusion approach

The vulnerability of face recognition systems against so-called morphing attacks has been revealed in the past years. Recently, different kinds of morphing attack detection approaches have been proposed. However, the vast majority of published results has been obtained from rather constrained experimental setups. In particular, most investigations do not consider variations in morphing techniques, image sources, and image post-processing. Hence, reported performance rates can not be maintained in realistic scenarios, as the NIST FRVT MORPH performance evaluation showed. In this work, existing algorithms are benchmarked on a new, more realistic database. This database consists of two different data sets, from which morphs were created using four different morphing algorithms. In addition, the database contains four different post-processings (including print-scan transformation and JPEG2000 compression). Further, a new morphing attack detection method based on a fusion of different configurations of Multi-scale Block Local Binary Patterns (MB-LBP) on an image divided into multiple cells is presented. The proposed score-level fusion of a maximum number of 18 different configurations is shown to significantly improve the robustness of the resulting morphing attack detection scheme, yielding an average performance between 2.26% and 8.52% in terms of Detection Equal Error Rate (D-EER), depending on the applied post-processing.

Personally familiar faces: Higher precision of memory for idiosyncratic than for categorical information.

Many studies have demonstrated that we can identify a familiar face on an image much better than an unfamiliar one, especially when various degradations or changes (e.g., image distortions or blurring, new illuminations) have been applied, but few have asked how different types of facial information from familiar faces are stored in memory. Here we investigated how well we remember personally familiar faces in terms of their identity, gender, and race. In 3 experiments, based on the faces personally familiar to our participants, we created sets of face morphs that parametrically varied the faces in terms of identity, sex, or race using a 3-dimensional morphable face model. For each familiar face, we presented those face morphs together with the original face and asked participants to pick the correct "real" face among morph distracters in each set. They were instructed to pick the face that most closely resembled their memory of that familiar person. We found that participants excelled in retrieving the correct familiar faces among the distracters when the faces were manipulated in terms of their idiosyncratic features (their identity information), but they were less sensitive to changes that occurred along the gender and race continuum. Image similarity analyses indicate that the observed difference cannot be attributed to different levels of image similarity between manipulations. These findings demonstrate that idiosyncratic and categorical face information is represented differently in memory, even for the faces of people we are very familiar with. Implications to current models of face recognition are discussed. (PsycInfo Database Record (c) 2020 APA, all rights reserved).