Chimeric Faces Research Articles

Mutual variation of information on transfer-CNN for face recognition with degraded probe samples

Abstract Learning based on convolutional neural networks (CNNs) or deep learning has been a major research area with applications in face recognition (FR). Under degraded conditions, performance of FR algorithms severely degrade. The work presented in this paper has three contributions. First, it proposes a transfer-CNN architecture of deep learning tailor-made for domain adaptation (DA), to overcome the difference in feature distributions between the gallery and probe samples. The proposed architecture consists of three units: base convolution (BCM), transfer (TM) and linear (LM) modules. Secondly, a novel 3-stage algorithm for Mutually Exclusive Training (3-MET) based on stochastic gradient descent, has been proposed. The initial stage of 3-MET involves updating the parameters of the BCM and LM units using samples from gallery. The second stage involves updating the parameters of TM, to bridge the disparity between the source and target distributions, based on mutual variation of information (MVI). The final stage of training in 3-MET freezes the layers of the BCM and TM, for updating (fine-tuning) only the parameters of the LM using a few probe (as target) samples. This helps the proposed transfer-CNN to provide enhanced domain-invariant representation for efficient deep-DA learning and classification. The third contribution comes from rigorous experimentations performed on three benchmark real-world degraded face datasets captured using surveillance cameras, one real-world dataset with non-uniform motion blur and three synthetically degraded benchmark face datasets. This exhibits superior performance of the proposed transfer-CNN architecture with 3-MET training, using Rank-1 recognition rates and ROC and CMC metrics, over many recent state-of-the-art techniques of CNN and DA. Experiments also include performance analysis under unbiased training with two large-scale chimeric face datasets.

Are only infants held more often on the left? If so, why? Testing the attention-emotion hypothesis with an infant, a vase, and two chimeric tests, one “emotional,” one not

ABSTRACTMost adults, especially women, hold infants and dolls but not books or packages on the left side. One reason may be that attention is more often leftward in response to infants, unlike emotionally neutral objects like books and packages. Women's stronger bias may reflect greater responsiveness to infants. Previously, we tested the attention hypothesis by comparing women's side-of-hold of a doll, book, and package with direction-of-attention on the Chimeric Faces Test (CFT) [Harris, L. J., Cárdenas, R. A., Spradlin, Jr., M. P., & Almerigi, J. B. (2010). Why are infants held on the left? A test of the attention hypothesis with a doll, a book, and a bag. Laterality: Asymmetries of Body, Brain and Cognition, 15(5), 548–571. doi:10.1080/13576500903064018]. Only the doll was held more often to the left, and only for the doll were side-of-hold and CFT scores related, with left-holders showing a stronger left-attention bias than right-holders. In the current study, we tested men and women with a doll and the CFT along with a vase as a neutral object and a “non-emotional” chimeric test. Again, only the doll was held more often to the left, but now, although both chimeric tests showed left-attention biases, scores were unrelated to side-of-hold. Nor were there sex differences. The results support left-hold selectivity but not the attention hypothesis, with or without the element of emotion. They also raise questions about the contribution of sex-of-holder. We conclude with suggestions for addressing these issues.

Prenatal and pubertal testosterone affect brain lateralization

After decades of research, the influence of prenatal testosterone on brain lateralization is still elusive, whereas the influence of pubertal testosterone on functional brain lateralization has not been investigated, although there is increasing evidence that testosterone affects the brain in puberty. We performed a longitudinal study, investigating the relationship between prenatal testosterone concentrations in amniotic fluid, pubertal testosterone concentrations in saliva, and brain lateralization (measured with functional Transcranial Doppler ultrasonography (fTCD)) of the Mental Rotation, Chimeric Faces and Word Generation tasks. Thirty boys and 30 girls participated in this study at the age of 15 years. For boys, we found a significant interaction effect between prenatal and pubertal testosterone on lateralization of Mental Rotation and Chimeric Faces. In the boys with low prenatal testosterone levels, pubertal testosterone was positively related to the strength of lateralization in the right hemisphere, while in the boys with high prenatal testosterone levels, pubertal testosterone was negatively related to the strength of lateralization. For Word Generation, pubertal testosterone was negatively related to the strength of lateralization in the left hemisphere in boys. For girls, we did not find any significant effects, possibly because their pubertal testosterone levels were in many cases below quantification limit. To conclude, prenatal and pubertal testosterone affect lateralization in a task-specific way. Our findings cannot be explained by simple models of prenatal testosterone affecting brain lateralization in a similar way for all tasks. We discuss alternative models involving age dependent effects of testosterone, with a role for androgen receptor distribution and efficiency.

The Presentation Location of the Reference Stimuli Affects the Left-Side Bias in the Processing of Faces and Chinese Characters.

For faces and Chinese characters, a left-side processing bias, in which observers rely more heavily on information conveyed by the left side of stimuli than the right side of stimuli, has been frequently reported in previous studies. However, it remains unclear whether this left-side bias effect is modulated by the reference stimuli's location. The present study adopted the chimeric stimuli task to investigate the influence of the presentation location of the reference stimuli on the left-side bias in face and Chinese character processing. The results demonstrated that when a reference face was presented in the left visual field of its chimeric images, which are centrally presented, the participants showed a preference higher than the no-bias threshold for the left chimeric face; this effect, however, was not observed in the right visual field. This finding indicates that the left-side bias effect in face processing is stronger when the reference face is in the left visual field. In contrast, the left-side bias was observed in Chinese character processing when the reference Chinese character was presented in either the left or right visual field. Together, these findings suggest that although faces and Chinese characters both have a left-side processing bias, the underlying neural mechanisms of this left-side bias might be different.

Neurophysiological evidence (ERPs) for hemispheric processing of facial expressions of emotions: Evidence from whole face and chimeric face stimuli

ABSTRACTThis study was designed to investigate the patterns of electrophysiological responses of early emotional processing at frontocentral sites in adults and to explore whether adults’ activation patterns show hemispheric lateralization for facial emotion processing. Thirty-five adults viewed full face and chimeric face stimuli. After viewing two faces, sequentially, participants were asked to decide which of the two faces was more emotive. The findings from the standard faces and the chimeric faces suggest that emotion processing is present during the early phases of face processing in the frontocentral sites. In particular, sad emotional faces are processed differently than neutral and happy (including happy chimeras) faces in these early phases of processing. Further, there were differences in the electrode amplitudes over the left and right hemisphere, particularly in the early temporal window. This research provides supporting evidence that the chimeric face test is a test of emotion processing that elicits right hemispheric processing.

An fMRI half-field repetition suppression study of chimeric faces

An fMRI half-field repetition suppression study of chimeric faces

Emotion processing in persons who respond vicariously towards others in pain: Disinhibited left-lateralized neural activity for threatening expressions

ABSTRACTWe investigated emotional processing in vicarious pain (VP) responders. VP responders report an explicit sensory and emotional feeling of pain when they witness another in pain, which is greater in magnitude than the empathic processing of pain in the general population. In Study 1, 31 participants completed a chimeric faces task, judging whether emotional chimera in the left, or right, visual field was more intense. VP responders took longer to judge emotionality than non-responders, and fixated more on the angry hemiface in the right visual field, whereas non-responder controls had no lateralized fixation bias. In Study 2, blood-oxygen level-dependent signals were recorded during an emotional face matching task. VP intensity was correlated with increased insula activity and reduced middle frontal gyrus activity for angry faces, and with reduced activity in the inferior and middle frontal gyri for sad faces. Together, these findings suggest that VP responders are more reactive to negative emotional expressions. Specifically, emotional judgements involved altered left-hemisphere activity in VP responders, and reduced engagement of regions involved in emotion regulation.

Lower Activation in Frontal Cortex and Posterior Cingulate Cortex Observed during Sex Determination Test in Early-Stage Dementia of the Alzheimer Type.

Face-labeling refers to the ability to classify faces into social categories. This plays a critical role in human interaction as it serves to define concepts of socially acceptable interpersonal behavior. The purpose of the current study was to characterize, what, if any, impairments in face-labeling are detectable in participants with early-stage clinically diagnosed dementia of the Alzheimer type (CDDAT) through the use of the sex determination test (SDT). In the current study, four (1 female, 3 males) CDDAT and nine (4 females, 5 males) age-matched neurotypicals (NT) completed the SDT using chimeric faces while undergoing BOLD fMRI. It was expected that CDDAT participants would have poor verbal fluency, which would correspond to poor performance on the SDT. This could be explained by decreased activation and connectivity patterns within the fusiform face area (FFA) and anterior cingulate cortex (ACC). DTI was also performed to test the association of pathological deterioration of connectivity in the uncinate fasciculus (UF) and verbally-mediated performance. CDDAT showed lower verbal fluency test (VFT) performance, but VFT was not significantly correlated to SDT and no significant difference was seen between CDDAT and NT for SDT performance as half of the CDDAT performed substantially worse than NT while the other half performed similarly. BOLD fMRI of SDT displayed differences in the left superior frontal gyrus and posterior cingulate cortex (PCC), but not the FFA or ACC. Furthermore, although DTI showed deterioration of the right inferior and superior longitudinal fasciculi, as well as the PCC, it did not demonstrate significant deterioration of UF tracts. Taken together, early-stage CDDAT may represent a common emerging point for the loss of face labeling ability.

Modifications of Visual Field Asymmetries for Face Categorization in Early Deaf Adults: A Study With Chimeric Faces.

Right hemisphere lateralization for face processing is well documented in typical populations. At the behavioral level, this right hemisphere bias is often related to a left visual field (LVF) bias. A conventional mean to study this phenomenon consists of using chimeric faces that are composed of the left and right parts of two faces. In this paradigm, participants generally use the left part of the chimeric face, mostly processed through the right optic tract, to determine its identity, gender or age. To assess the impact of early auditory deprivation on face processing abilities, we tested the LVF bias in a group of early deaf participants and hearing controls. In two experiments, deaf and hearing participants performed a gender categorization task with chimeric and normal average faces. Over the two experiments the results confirmed the presence of a LVF bias in participants, which was less frequent in deaf participants. This result suggested modifications of hemispheric lateralization for face processing in deaf participants. In Experiment 2 we also recorded eye movements to examine whether the LVF bias could be related to face scanning behavior. In this second study, participants performed a similar task while we recorded eye movements using an eye tracking system. Using areas of interest analysis we observed that the proportion of fixations on the mouth relatively to the other areas was increased in deaf participants in comparison with the hearing group. This was associated with a decrease of the proportion of fixations on the eyes. In addition these measures were correlated to the LVF bias suggesting a relationship between the LVF bias and the patterns of facial exploration. Taken together, these results suggest that early auditory deprivation results in plasticity phenomenon affecting the perception of static faces through modifications of hemispheric lateralization and of gaze behavior.

Music-induced changes in functional cerebral asymmetries

After decades of research, it remains unclear whether emotion lateralization occurs because one hemisphere is dominant for processing the emotional content of the stimuli, or whether emotional stimuli activate lateralised networks associated with the subjective emotional experience. By using emotion-induction procedures, we investigated the effect of listening to happy and sad music on three well-established lateralization tasks. In a prestudy, Mozart’s piano sonata (K. 448) and Beethoven’s Moonlight Sonata were rated as the most happy and sad excerpts, respectively. Participants listened to either one emotional excerpt, or sat in silence before completing an emotional chimeric faces task (Experiment 1), visual line bisection task (Experiment 2) and a dichotic listening task (Experiment 3 and 4). Listening to happy music resulted in a reduced right hemispheric bias in facial emotion recognition (Experiment 1) and visuospatial attention (Experiment 2) and increased left hemispheric bias in language lateralization (Experiments 3 and 4). Although Experiments 1–3 revealed an increased positive emotional state after listening to happy music, mediation analyses revealed that the effect on hemispheric asymmetries was not mediated by music-induced emotional changes. The direct effect of music listening on lateralization was investigated in Experiment 4 in which tempo of the happy excerpt was manipulated by controlling for other acoustic features. However, the results of Experiment 4 made it rather unlikely that tempo is the critical cue accounting for the effects. We conclude that listening to music can affect functional cerebral asymmetries in well-established emotional and cognitive laterality tasks, independent of music-induced changes in the emotion state.

Effect of expressional chimeric faces on the attentional bias in neglect patients

Objective To investigate the effect of expressional chimeric faces on the attentional bias in unilateral spatial neglect (USN) patients. Methods 12 USN patients and 30 matched healthy controls (HC) were involved in this study. All of them were performed with happy-neutral and sad-neutral expressional chimeric faces (ECF) and the attentional deviation scores were compared between and within groups. Results There were statistically significant differences of attentional deviation scores (happy-neutral ECF: t=15.550, P<0.01; sad-neutral ECF: t=12.192, P<0.01) between the USN group (happy-neutral ECF: 0.78±0.22; sad-neutral ECF: 0.50±0.20) and the HC group (happy-neutral ECF: -0.31±0.20; sad-neutral ECF: -0.25±0.17), with rightward bias in the USN group and small leftward bias in the HC group. For the USN group, the magnitude of rightward bias in the sad-neutral ECF condition was reduced than in the happy-neutral ECF condition (t=3.954, P=0.002). There were no statistically differences under happy-and sad-neutral ECF conditions within the HC group (t=-1.302, P=0.203). Conclusions USN patients show rightward attentional bias under both happy-and sad-neutral ECF conditions. However, the magnitude of rightward bias under the sad-neutral ECF condition are reduced significantly. This might be attributed to the unconscious processing of stimuli in the neglected side and emotion cognition of the brain hemispheres. Key words: Emotions; Attention; Cognition disorder; Neuropsychology

Perceptual bias, more than age, impacts on eye movements during face processing

Consistent with the right hemispheric dominance for face processing, a left perceptual bias (LPB) is typically demonstrated by younger adults viewing faces and a left eye movement bias has also been revealed. Hemispheric asymmetry is predicted to reduce with age and older adults have demonstrated a weaker LPB, particularly when viewing time is restricted. What is currently unclear is whether age also weakens the left eye movement bias. Additionally, a right perceptual bias (RPB) for facial judgments has less frequently been demonstrated, but whether this is accompanied by a right eye movement bias has not been investigated. To address these issues older and younger adults' eye movements and gender judgments of chimeric faces were recorded in two time conditions. Age did not significantly weaken the LPB or eye movement bias; both groups looked initially to the left side of the face and made more fixations when the gender judgment was based on the left side. A positive association was found between LPB and initial saccades in the freeview condition and with all eye movements (initial saccades, number and duration of fixations) when time was restricted. The accompanying eye movement bias revealed by LPB participants contrasted with RPB participants who demonstrated no eye movement bias in either time condition. Consequently, increased age is not clearly associated with weakened perceptual and eye movement biases. Instead an eye movement bias accompanies an LPB (particularly under restricted viewing time conditions) but not an RPB.

A leftward bias however you look at it: Revisiting the emotional chimeric face task as a tool for measuring emotion lateralization

ABSTRACTLeft hemiface biases observed within the Emotional Chimeric Face Task (ECFT) support emotional face perception models whereby all expressions are preferentially processed by the right hemisphere. However, previous research using this task has not considered that the visible midline between hemifaces might engage atypical facial emotion processing strategies in upright or inverted conditions, nor controlled for left visual field (thus right hemispheric) visuospatial attention biases. This study used novel emotional chimeric faces (blended at the midline) to examine laterality biases for all basic emotions. Left hemiface biases were demonstrated across all emotional expressions and were reduced, but not reversed, for inverted faces. The ECFT bias in upright faces was significantly increased in participants with a large attention bias. These results support the theory that left hemiface biases reflect a genuine bias in emotional face processing, and this bias can interact with attention processes similarly localized in the right hemisphere.

Right hemisphere or valence hypothesis, or both? The processing of hybrid faces in the intact and callosotomized brain

The valence hypothesis and the right hemisphere hypothesis in emotion processing have been alternatively supported. To better disentangle the two accounts, we carried out two studies, presenting healthy participants and an anterior callosotomized patient with ‘hybrid faces’, stimuli created by superimposing the low spatial frequencies of an emotional face to the high spatial frequencies of the same face in a neutral expression. In both studies we asked participants to judge the friendliness level of stimuli, which is an indirect measure of the processing of emotional information, despite this remaining “invisible”. In Experiment 1 we presented hybrid faces in a divided visual field paradigm using different tachistoscopic presentation times; in Experiment 2 we presented hybrid chimeric faces in canonical view and upside-down. In Experiments 3 and 4 we tested a callosotomized patient, with spared splenium, in similar paradigms as those used in Experiments 1 and 2. Results from Experiments 1 and 3 were consistent with the valence hypothesis, whereas results of Experiments 2 and 4 were consistent with the right hemisphere hypothesis. This study confirms that the low spatial frequencies of emotional faces influence the social judgments of observers, even when seen for 28ms (Experiment 1), possibly by means of configural analysis (Experiment 2). The possible roles of the cortical and subcortical emotional routes in these tasks are discussed in the light of the results obtained in the callosotomized patient. We propose that the right hemisphere and the valence accounts are not mutually exclusive, at least in the case of subliminal emotion processing.

Relationship status and sex differences in emotion lateralisation: An examination contrasting the processing of emotional infant and adult faces

When processing facial emotion, most individuals are right hemisphere dominant; however there is variability in this pattern with males typically being more strongly lateralised than females. Relationship status has been found to influence the processing of facial stimuli in women, and therefore, in this research the lateralised processing of facial emotion is considered whilst taking into account the participant’s relationship status and sex. Using the chimeric faces test, with both infant and adult facial stimuli, it was shown that partnered participants, but not single participants, were more strongly lateralised for the processing of adult stimuli than infant stimuli, and that partnered women did not show any hemispheric bias. These findings suggest that the neuropsychological processing of emotion may change dependent on an individual’s relationship status, and are discussed in terms of the possible evolutionary significance of infant faces for individuals who are in a relationship and who wish to have children.

The processing of chimeric and dichotic emotional stimuli by connected and disconnected cerebral hemispheres

Hemispheric asymmetries have been widely explored in both the visual and the auditory domain, but little is known about hemispheric asymmetries in audio–visual integration. We compared the performance of a partially callosotomized patient, a total split-brain patient and a control group during the evaluation of the emotional valence of chimeric faces and dichotic syllables (an emotional syllable in one ear and white noise in the other ear) presented unimodally (only faces or only syllables) or bimodally (faces and syllables presented simultaneously). Stimuli could convey happy and sad expressions and participants were asked to evaluate the emotional content of each presentation, using a 5-point Likert scale (from very sad to very happy). In unimodal presentations, the partially callosotomized patient's judgments depended on the emotional valence of the stimuli processed by the right hemisphere, whereas those of the total split-brain patient showed the opposite lateralization; in these conditions, the control group did not show asymmetries. Moreover, in bimodal presentations, results provided support for the valence hypothesis (i.e., left asymmetry for positive emotions and vice versa) in both the control group and the partially callosotomized patient, whereas the total split-brain patient showed a tendency to evaluate the emotional content of the right hemiface even when asked to focus on the acoustic modality. We conclude that partial and total hemispheric disconnections reveal opposite patterns of hemispheric asymmetry in auditory, visual and audio–visual emotion processing. These results are discussed in the light of the right-hemisphere hypothesis and the valence hypothesis.

Individual differences in emotion lateralisation and the processing of emotional information arising from social interactions

Previous research examining the possible association between emotion lateralisation and social anxiety has found conflicting results. In this paper two studies are presented to assess two aspects related to different features of social anxiety: fear of negative evaluation (FNE) and emotion regulation. Lateralisation for the processing of facial emotion was measured using the chimeric faces test. Individuals with greater FNE were more strongly lateralised to the right hemisphere for the processing of anger, happiness and sadness; and, for the processing of fearful faces the relationship was found for females only. Emotion regulation strategies were reduced to two factors: positive strategies and negative strategies. For males, but not females, greater reported use of negative emotion strategies is associated with stronger right hemisphere lateralisation for processing negative emotions. The implications for further understanding the neuropsychological processing of emotion in individuals with social anxiety are discussed.

Facial transplants in Xenopus laevis embryos.

Craniofacial birth defects occur in 1 out of every 700 live births, but etiology is rarely known due to limited understanding of craniofacial development. To identify where signaling pathways and tissues act during patterning of the developing face, a 'face transplant' technique has been developed in embryos of the frog Xenopus laevis. A region of presumptive facial tissue (the "Extreme Anterior Domain" (EAD)) is removed from a donor embryo at tailbud stage, and transplanted to a host embryo of the same stage, from which the equivalent region has been removed. This can be used to generate a chimeric face where the host or donor tissue has a loss or gain of function in a gene, and/or includes a lineage label. After healing, the outcome of development is monitored, and indicates roles of the signaling pathway within the donor or surrounding host tissues. Xenopus is a valuable model for face development, as the facial region is large and readily accessible for micromanipulation. Many embryos can be assayed, over a short time period since development occurs rapidly. Findings in the frog are relevant to human development, since craniofacial processes appear conserved between Xenopus and mammals.

Task demands modulate decision and eye movement responses in the chimeric face test: examining the right hemisphere processing account

A large and growing body of work, conducted in both brain-intact and brain-damaged populations, has used the free viewing chimeric face test as a measure of hemispheric dominance for the extraction of emotional information from faces. These studies generally show that normal right-handed individuals tend to perceive chimeric faces as more emotional if the emotional expression is presented on the half of the face to the viewer's left (“left hemiface”). However, the mechanisms underlying this lateralized bias remain unclear. Here, we examine the extent to which this bias is driven by right hemisphere processing advantages vs. default scanning biases in a unique way—by changing task demands. In particular, we compare the original task with one in which right-hemisphere-biased processing cannot provide a decision advantage. Our behavioral and eye movement data are inconsistent with the predictions of a default scanning bias account and support the idea that the left hemiface bias found in the chimeric face test is largely due to strategic use of right hemisphere processing mechanisms.

Perceptual and Gaze Biases during Face Processing: Related or Not?

Previous studies have demonstrated a left perceptual bias while looking at faces, due to the fact that observers mainly use information from the left side of a face (from the observer's point of view) to perform a judgment task. Such a bias is consistent with the right hemisphere dominance for face processing and has sometimes been linked to a left gaze bias, i.e. more and/or longer fixations on the left side of the face. Here, we recorded eye-movements, in two different experiments during a gender judgment task, using normal and chimeric faces which were presented above, below, right or left to the central fixation point or on it (central position). Participants performed the judgment task by remaining fixated on the fixation point or after executing several saccades (up to three). A left perceptual bias was not systematically found as it depended on the number of allowed saccades and face position. Moreover, the gaze bias clearly depended on the face position as the initial fixation was guided by face position and landed on the closest half-face, toward the center of gravity of the face. The analysis of the subsequent fixations revealed that observers move their eyes from one side to the other. More importantly, no apparent link between gaze and perceptual biases was found here. This implies that we do not look necessarily toward the side of the face that we use to make a gender judgment task. Despite the fact that these results may be limited by the absence of perceptual and gaze biases in some conditions, we emphasized the inter-individual differences observed in terms of perceptual bias, hinting at the importance of performing individual analysis and drawing attention to the influence of the method used to study this bias.