Pairs Of Faces Research Articles

Shuffle Algebras and Non-Commutative Probability for Pairs of Faces

One can build an operatorial model for freeness by considering either the right-handed or the left-handed representation of algebras of operators acting on the free product of the underlying pointed Hilbert spaces. Considering both at the same time, that is, computing distributions of operators in the algebra generated by the left- and right-handed representations, led Voiculescu in 2013 to define and study bifreeness and, in the sequel, triggered the development of an extension of noncommutative probability now frequently referred to as multi-faced (two-faced in the example given above). Many examples of two-faced independences emerged these past years. Of great interest to us are biBoolean, bifree and type I bimonotone independences. In this paper, we extend the preLie calculus pertaining to free, Boolean, and monotone moment-cumulant relations initiated by K. Ebrahimi-Fard and F. Patras to their above-mentioned two-faced equivalents.

FSGANv2: Improved Subject Agnostic Face Swapping and Reenactment.

We present Face Swapping GAN (FSGAN) for face swapping and reenactment. Unlike previous work, we offer a subject agnostic swapping scheme that can be applied to pairs of faces without requiring training on those faces. We derive a novel iterative deep learning-based approach for face reenactment which adjusts significant pose and expression variations that can be applied to a single image or a video sequence. For video sequences, we introduce a continuous interpolation of the face views based on reenactment, Delaunay Triangulation, and barycentric coordinates. Occluded face regions are handled by a face completion network. Finally, we use a face blending network for seamless blending of the two faces while preserving the target skin color and lighting conditions. This network uses a novel Poisson blending loss combining Poisson optimization with a perceptual loss. We compare our approach to existing state-of-the-art systems and show our results to be both qualitatively and quantitatively superior. This work describes extensions of the FSGAN method, proposed in an earlier conference version of our work (Nirkin et al. 2019), as well as additional experiments and results.

Young children’s attributes are better conveyed by voices than by faces

The purpose of this study was to explore how young children’s vocal and facial cues contribute to conveying to adults important information about children’s attributes when presented together. In particular, the study aimed to disentangle whether children’s vocal or facial cues, if either, are more dominant when both types of cues are displayed in a contradictory mode. To do this, we assigned 127 college students to one of three between-participants conditions. In the Voices-Only condition, participants listened to four pairs of synthetized voices simulating the voices of 4-5-year-old and 9-10-year-old children verbalizing a neutral-content sentence. Participants needed to indicate which voice was better associated with a series of 14 attributes organized into four trait dimensions (Positive Affect, Negative Affect, Intelligence, and Helpless), potentially meaningful in young child–adult interactions. In the Consistent condition, the same four pairs of voices delivered in the Voices-Only condition were presented jointly with morphed photographs of children’s faces of equivalent age. In the Inconsistent condition, the four pairs of voices and faces were paired in a contradictory manner (immature voices with mature faces vs. mature voices with immature faces). Results revealed that vocal cues were more effective than facial cues in conveying young children’s attributes to adults and that women were more efficient (i.e., faster) than men in responding to children’s cues. These results confirm and extend previous evidence on the relevance of children’s vocal cues to signaling important information about children’s attributes and needs during their first 6 years of life.

Stable individual differences in unfamiliar face identification: Evidence from simultaneous and sequential matching tasks

Matching identity in images of unfamiliar faces is difficult: Images of the same person can look different and images of different people can look similar. Recent studies have capitalized on individual differences in the ability to distinguish match (same ID) vs. mismatch (different IDs) face pairs to inform models of face recognition. We addressed two significant gaps in the literature by examining the stability of individual differences in both sensitivity to identity and response bias. In Study 1, 210 participants completed a battery of four tasks in each of two sessions separated by one week. Tasks varied in protocol (same/different, lineup, sorting) and stimulus characteristics (low vs. high within-person variability in appearance). In Study 2, 148 participants completed a battery of three tasks in a single session. Stimuli were presented simultaneously on some trials and sequentially on others, introducing short-term memory demands. Principal components analysis revealed two components that were stable across time and tasks: sensitivity to identity and bias. Analyses of response times suggest that individual differences in bias reflect decision-making processes. We discuss the implications of our findings in applied settings and for models of face recognition.

Depth map guided triplet network for deepfake face detection

The widespread dissemination of facial forgery technology has brought many ethical issues and aroused widespread concern in society. Most research today treats deepfake detection as a fine grained classification task, which however makes it difficult to enable the feature extractor to express the features related to the real and fake attributes. This paper proposes a depth map guided triplet network, which mainly consists of a depth prediction network and a triplet feature extraction network. The depth map predicted by the depth prediction network can effectively reflect the differences between real and fake faces in discontinuity, inconsistent illumination, and blurring, thus in favor of deepfake detection. Regardless of the facial appearance changes induced by deepfake, we argue that real and fake faces should correspond to their respective latent feature spaces. Particularly, the pair of real faces (original–target) remain close in the latent feature space, while the two pairs of real–fake faces (original–fake, target–fake) instead keep faraway. Following this paradigm, we suggest a triplet loss supervision network to extract the sufficiently discriminative deep features, which minimizes the distance of the original–target pair and maximize the distance of the original–fake (also target–fake) pair. The extensive results on public FaceForensics++ and Celeb-DF datasets validate the superiority of our method over competitors.

An Exploration of Relationships Between Perceptual and Cognitive Racial Biases

People are often biased in how they evaluate characteristics of individuals of different races. Some of these biases are perceptual: for example, the “race-lightness effect” demonstrates that for grayscale photos at equivalent luminance values, the faces of black individuals are frequently judged to be darker than the faces of white individuals. Other biases are cognitive: the Implicit Association Test (IAT) and direct assessments of racial attitudes demonstrate that people hold both negative implicit and explicit racial prejudices. Despite extensive literature on both topics, no study has explored the relationship between perceptual and cognitive racial biases in the same individuals in a within-subjects design. In this experiment, these relationships were explored in 28 individuals using three tasks: in one task, deemed the Race-Lightness Task (RLT), participants completed a 2-interval forced-choice procedure where they were shown pairs of faces (some with European features, some with African features) and asked to judge whether the second face was lighter or darker than the first. In a second task, individuals completed the IAT to measure their associations between race and positive/negative adjectives. Lastly, subjects answered a series of questions to measure explicit attitudes about different racial groups. While results from the RLT and IAT were uncorrelated, a significant correlation was shown between the IAT and a survey question about systemic racism. These results provide preliminary support for the independence of perception and cognition for racially based tasks, and provide insight into the pervasive nature of implicit and explicit racial prejudice.

COVID-19 masks increase the influence of face recognition algorithm decisions on human decisions in unfamiliar face matching.

Face masks, recently adopted to reduce the spread of COVID-19, have had the unintended consequence of increasing the difficulty of face recognition. In security applications, face recognition algorithms are used to identify individuals and present results for human review. This combination of human and algorithm capabilities, known as human-algorithm teaming, is intended to improve total system performance. However, prior work has shown that human judgments of face pair similarity-confidence can be biased by an algorithm's decision even in the case of an error by that algorithm. This can reduce team effectiveness, particularly for difficult face pairs. We conducted two studies to examine whether face masks, now routinely present in security applications, impact the degree to which this cognitive bias is experienced by humans. We first compared the influence of algorithm's decisions on human similarity-confidence ratings in the presence and absence of face masks and found that face masks more than doubled the influence of algorithm decisions on human similarity-confidence ratings. We then investigated if this increase in cognitive bias was dependent on perceived algorithm accuracy by also presenting algorithm accuracy rates in the presence of face masks. We found that making humans aware of the potential for algorithm errors mitigated the increase in cognitive bias due to face masks. Our findings suggest that humans reviewing face recognition algorithm decisions should be made aware of the potential for algorithm errors to improve human-algorithm team performance.

Electrophysiological correlates of attentional bias towards threat in children with high levels of social anxiety.

Pediatric social anxiety is characterized by attentional biases (AB) towards social threats. This study used a new response-based calculation method to assess AB from response times (RT) in a visual dot-probe task and electroencephalography (EEG) to explore its electrophysiological correlates. Twenty, high socially anxious children (HSA) (mean [M ] = 10.1 years; standard deviation [SD] = 1.01) were compared with 22 healthy control children (HC) (M = 10.20 years; SD = 1.30) matched in age and gender. Participants had to identify targets preceded by disgust-neutral, happy-neutral, or neutral-neutral pairs of faces. RT and electroencephalograms were recorded throughout the task. While no significant group difference was found at the behavioral level, principal component analyses performed on EEG data revealed that event-related potentials for threat-related stimuli were impacted by social anxiety. Analyses indicated a larger N170 amplitude in response to all facial stimuli in HC when compared to the HSA. However, we found increased P2 amplitudes for disgust-neutral pairs compared with happy-neutral pairs in has only. Then, thasHSA group showed increased P2 amplitudes for targets following disgusted faces on the opposite side of the screen compared with targets appearing on the same side of the screen. These results suggest that HSA may display an increased anchorage of attention on threatening stimuli and need more effort to disengage their attentional focus from threats and to perform the task correctly. Taken together, our data confirmed the presence of AB in children with high levels of social anxiety, which are reflected by increased neural processing during the confrontation to faces depicting a potential threatening expression.

Let's talk about faces: Identifying faces from verbal descriptions.

Face descriptions inform real-world identification decisions, for example when eyewitnesses describe criminal perpetrators. However, it is unclear how effective face descriptions are for identification. Here, we examined the accuracy of face identification from verbal descriptions, and how individual differences in face perception relate to producing and using descriptions for identification. In Study 1, participants completed a face communication task in pairs. Each participant saw a single face, and via verbal communication only, the pair decided if they were viewing the same person or different people. Dyads achieved 72% accuracy, compared to 81% when participants completed the task individually by matching face pairs side-by-side. Performance on the face communication and perceptual matching tasks were uncorrelated, perhaps due to low measurement reliability of the face communication task. In subsequent studies, we examined the abilities of face 'describers' (Study 2) and 'identifiers' separately (Study 3). We found that 'super-recognizers' - people with extremely high perceptual face identification abilities - outperformed controls in both studies. Overall, these results show that people can successfully describe faces for identification. Preliminary evidence suggests that this ability - and the ability use facial descriptions for identification - has some association with perceptual face identification skill.

Perverse Sheaves over Real Hyperplane Arrangements II

Let \mathcal{H} be an arrangement of hyperplanes in \mathbb{R}^n and \operatorname{Perv}(\mathbb{C}Cn,\mathcal{H}) be the category of perverse sheaves on \mathbb{C}^n smooth with respect to the stratification given by complexified flats of \mathcal{H} . We give a description of \operatorname{Perv}(\mathbb{C}^n, \mathcal{H}) in terms of “matrix diagrams”, i.e., diagrams formed by vector spaces E_{A,B} labeled by pairs (A,B) of real faces of \mathcal{H} (of all dimensions) or, equivalently, by the cells iA+B of a natural cell decomposition of \mathbb{C} . A matrix diagram is formally similar to a datum describing a constructible (nonperverse) sheaf but with the direction of one-half of the arrows reversed.

Multisensory encoding of names via name tags facilitates remembering

AbstractAssociating names to faces can be challenging, in part because this task lacks an inherent semantic relationship between a face and name. The current study seeks to understand whether bolstering names with cross‐modal cues—specifically, name tags—may aid memory for face and name pairings. In a series of five experiments, we investigated whether the presentation of congruent vocalized and written names at encoding might benefit subsequent cued recall and recognition memory tasks. The results showed that participants, cued with a picture of a face, were more likely to recall the associated name when those names were encoded with a name tag (a congruent visual cue) compared to when no supporting cross‐modal cue was available. The findings were consistent with a benefit of multisensory encoding, above any effect from the availability of independent unisensory traces, extending previous findings of multisensory learning and memory benefits to a naturalistic associative memory task.

Temporal dynamics of face adaptation.

The appearance of a face can be strongly affected by adaptation to faces seen previously. A number of studies have examined the time course of these aftereffects, but the integration time over which adaptation pools signals to control the adaptation state remains uncertain. Here we examined the effects of temporal frequency on face gender aftereffects induced by a pair of faces alternating between the two genders to assess when the aftereffects were pooled over successive faces versus driven by the last face seen. In the first experiment, we found that temporal frequencies between 0.25 and 2.00 Hz all failed to produce an aftereffect, suggesting a fairly long integration time. In the second experiment, we therefore probed slower alternation rates of 0.03 to 0.25 Hz. A rate of 0.0625 Hz (i.e., 8 seconds per face) was required to generate significant aftereffects from the last presented face and was consistent with an average time constant of 15 to 20 seconds for an exponentially decaying integration window. This integration time is substantially longer than found previously for analogous effects for alternating colors, and thus points to a potentially slower mechanism of adaptation for faces compared with chromatic adaptation.

Diminished Visual Attention to Emotional Faces Is Associated with Poor Emotional Valence Perception in Frontotemporal Dementia

Aim: The current study examined whether visual attention to emotional facial expressions is lower in individuals with frontotemporal dementia (FTD) compared to healthy controls, and whether visual attention to emotional facial expressions is associated with the ability to perceive others’ emotional valence accurately. Methods: Participants with FTD (n = 17) and healthy controls (n = 23) passively viewed pairs of emotional and neutral faces while their visual attention was measured using eye-tracking. A subsample of participants (n = 28) also completed an emotional valence perception task. Results: Individuals with FTD spent less time looking at emotional faces than healthy controls. However, there was no difference in the amount of time individuals with FTD spent looking at neutral faces as compared to healthy controls. In the subsample, less time spent looking at emotional faces (but not neutral faces) was associated with a less accurate perception of others’ emotional valence. Conclusion: Individuals with FTD displayed diminished visual attention to emotional facial expressions compared to healthy controls. Reduced attention towards emotional faces was associated with poorer emotional valence perception. Findings point toward diminished visual attention as potentially relevant for understanding oft-observed impairments in socioemotional functioning in FTD.

PointFace: Point Cloud Encoder-Based Feature Embedding for 3-D Face Recognition

The accuracy of 2D face recognition (FR) has progressed significantly due to the availability of large-scale training data. However, the research of deep learning based 3D FR is still in the early stage. Most of available 3D FR generate 2D maps from 3D data and apply existing 2D CNNs to the generated 2D maps for feature extraction. We propose in this paper a light-weight framework, named PointFace, to directly process point set data for 3D FR. In this framework, two weight-shared encoders are designed to directly extract features from a pair of 3D faces and the distances between embeddings of the same person and different person are minimized and maximized, respectively. The framework also use a feature similarity loss to guide the encoders to obtain discriminative face representations. A pair selection strategy is proposed to generate positive and negative face pairs to further improve the FR performance. Extensive experiments on Lock3DFace and Bosphorus show that the proposed PointFace outperforms state-of-the-art 2D CNN based FR methods.

Benchmarking human face similarity using identical twins

The problem of distinguishing identical twins and non-twin look-alikes in automated facial recognition (FR) applications has become increasingly important with the widespread adoption of facial biometrics. Due to the high facial similarity of both identical twins and look-alikes, these face pairs represent the hardest cases presented to facial recognition tools. This work presents an application of one of the largest twin data sets compiled to date to address two FR challenges: (1) determining a baseline measure of facial similarity between identical twins and (2) applying this similarity measure to determine the impact of doppelgangers, or look-alikes, on FR performance for large face data sets. The facial similarity measure is determined via a deep convolutional neural network. This network is trained on a tailored verification task designed to encourage the network to group together highly similar face pairs in the embedding space and achieves a test AUC of 0.9799. The proposed network provides a quantitative similarity score for any two given faces and has been applied to large-scale face data sets to identify similar face pairs. An additional analysis that correlates the comparison score returned by a facial recognition tool and the similarity score returned by the proposed network has also been performed.

A-287 Attention Shifting Individual Differences Moderate Attention Bias Modification Training Effects Among Formerly Depressed Persons

Abstract Objective: Biased information processing towards negative content is associated with depressive disorders and may be reduced by training attention away from dysphoric stimuli (attention bias modification training, ABM). Empirical findings are mixed concerning ABM outcomes within depressed clinical and analog samples, which may reflect methodological differences (e.g., reaction time- vs. eye-tracking-based measurement), and individual attention-shifting differences. This study tests attention-shifting as a moderator of ABM outcomes among adults with depression histories via reaction-time and eye-tracking-based paradigms. Method: Adults (N=197, Mage=26.83, 46% female, n=28 depressed) completed psychiatric interviews and received dot-probe-based ABM (n=126) or sham training (SHAM-ABM) (n=71) in 3 laboratory sessions. Across training conditions, respondents viewed sad-neutral same-actor face pairs with incongruent trials (probe following the neutral face) comprising 85% of the 288 session trials in ABM vs. 50% in SHAM-ABM. Training effects reflect pre-to-post-training change in average reaction times to incongruent (vs. congruent) image-probe trials via another dot-probe task and times to first visual fixation across same-actor sad-neutral face-pairs in response to visual prompts (i.e., a shape framing the target face) collected through E-prime 3.0 and the Tobii x2-60 system. Attention-shifting indexes time-to-first-fixation across neutral-neutral face-pair trials. Results. Pre-training attention-shifting differences moderated training effects when shifting attention away (but not towards) sad faces (F=3.98, p=.047): slow attention-shifting predicted rapid sad-face disengagement for those in SHAM-ABM (ΔM=-61ms, F=7.96, p=.005), but not other conditions. No other training effects emerged for eye-tracking and reaction time indices. Conclusion: Findings support the utility of measuring attention processes via eye-tracking methods, and the need to account for individual attention-shifting differences.

A-288 Valenced Attention Shifting Deficits Evident via Eye-tracking and not Reaction Time Measures in a Psychiatric Sample

Abstract Objective: Inflexible attention shifting from dysphoric information is a key component of contemporary depression risk models. Yet, the literature is mixed concerning such deficits in depressed samples, perhaps partly due to construct-irrelevant variance (e.g., motor speed) that saturates some behavioral attention shifting measures. This study evaluates the clinical validity of an eye-tracking-based measure of motivated attention shifting in response to neutral and valenced stimuli. Method: Hungarian adults with depression histories (N=198; Mage=26.83, 45% female; n=28 depressed) completed psychiatric interviews (SCID), depression measures (BDI-II, PANAS-X), and a visual attention shifting task via E-prime 3.0 and Tobii x2-60 eye-tracking system. Participants viewed 72 same-actor face pairs (angry-neutral, sad-neutral, happy-neutral, neutral-neutral) drawn from the KDEF database; attention shifting indices (outcomes of interest) reflected visual attention shifting speed away from a sad- or happy-valenced face towards a neutral face (disengagement), in reverse order (engagement), and across neutral face pairs (valence-free shifting) in response to a visual prompt (i.e., a shape framing the target face). Parallel behavioral indices reflect reaction times to said visual prompts. Depression status, severity, and affects served as the validators. Results: Independent of demographics, valence-free shifting, and valid task trial frequency, current depression (βDep=.29, p=.016), low trait positive affect (βPANAS-X,PA=-.14, p=.02), and depression severity at a trend level (βBDI-II=.06, p=.059) predicted slow sad-face disengagement; no other effect reached significance. Conclusion: Results suggest that shifting visual attention away from sad, but not positively-valenced information may be linked to depressive states among formerly depressed persons and provide some evidence for the eye-tracking-based task’s incremental validity to behavioral parallels.

A-193 Median- and Robust Statistics-based Attention Indices Outperform Mean-based Counterparts in Predicting Anxiety

Abstract Objectives: Eye-tracking paradigms may measure attentional process with greater precision than parallel reaction-time-based methods by reducing construct-irrelevant systematic error (e.g., motor activity). However, the benefits of increased sensitivity offered through eye-tracking are offset by their potential susceptibility for eye-movement outliers due to instrumentation and related effects. Arithmetic means are commonly used to index attention in multi-trial tasks but are known to be biased in the presence of outliers. Median-based indices and robust mean estimators reduce outliers’ influence but are not commonly used in the literature. This study compares the value of mean-, median-, and robust-mean based eye-tracking attention shifting indices in predicting social and generalized anxiety symptoms among community-dwelling adults with various psychiatric histories. Methods: Participants (N=148) completed self-report measures (SIAS & PSWQ) and a visual attention shifting task via E-prime 3.0 and Tobii x3-120 eye-tracking system. Participants viewed 72 same-actor face pairs (sad-neutral & happy-neutral), and motivated attention shifting speed away from sad- or happy-valenced faces towards neutral faces served as predictors of interest. Arithmetic means, medians, and Tukey’s bisquare function-based robust means summarized attention shifting trials. Results. Arithmetic-mean-based indices did not predict anxiety, while median-based and robust-mean indices did: slow disengagement from sad faces predicted elevated social anxiety (b[Mdn]=.02, p=.02; b[robust-M] = .021, p =.004), and robust-mean-based indices reached trend-levels when predicting worry (ps=.055-.076), while arithmetic-mean- and median-based indices did not (ps=.43-98). Conclusion: Findings support the utility of quantifying attention processes via robust statistical approaches that attenuate outlier influence.

Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models

Human vision is attuned to the subtle differences between individual faces. Yet we lack a quantitative way of predicting how similar two face images look and whether they appear to show the same person. Principal component-based three-dimensional (3D) morphable models are widely used to generate stimuli in face perception research. These models capture the distribution of real human faces in terms of dimensions of physical shape and texture. How well does a "face space" based on these dimensions capture the similarity relationships humans perceive among faces? To answer this, we designed a behavioral task to collect dissimilarity and same/different identity judgments for 232 pairs of realistic faces. Stimuli sampled geometric relationships in a face space derived from principal components of 3D shape and texture (Basel face model [BFM]). We then compared a wide range of models in their ability to predict the data, including the BFM from which faces were generated, an active appearance model derived from face photographs, and image-computable models of visual perception. Euclidean distance in the BFM explained both dissimilarity and identity judgments surprisingly well. In a comparison against 16 diverse models, BFM distance was competitive with representational distances in state-of-the-art deep neural networks (DNNs), including novel DNNs trained on BFM synthetic identities or BFM latents. Models capturing the distribution of face shape and texture across individuals are not only useful tools for stimulus generation. They also capture important information about how faces are perceived, suggesting that human face representations are tuned to the statistical distribution of faces.

Nonverbal emotion perception and vocabulary in late infancy

Language has been proposed as a potential mechanism for young children’s developing understanding of emotion. However, much remains unknown about this relation at an individual difference level. The present study investigated 15- to 18-month-old infants’ perception of emotions across multiple pairs of faces. Parents reported their child’s productive vocabulary, and infants participated in a non-linguistic emotion perception task via an eye tracker. Infant vocabulary did not predict nonverbal emotion perception when accounting for infant age, gender, and general object perception ability (β = −0.15, p = .300). However, we observed a gender difference: Only girls’ vocabulary scores related to nonverbal emotion perception when controlling for age and general object perception ability (β = 0.42, p = .024). Further, boys showed a stronger preference for the novel emotion face vs. girls (t(48) = 2.35, p = .023, d= 0.67). These data suggest that pathways of processing emotional information (e.g., using language vs visual information) may differ for girls and boys in late infancy.