Identity-specific Information Research Articles

The neural dynamics of familiarity-dependent face identity representation.

Recognizing a face as belonging to a given identity is essential in our everyday life. Clearly, the correct identification of a face is only possible for familiar people, but 'familiarity' covers a wide range-from people we see every day to those we barely know. Although several studies have shown that the processing of familiar and unfamiliar faces is substantially different, little is known about how the degree of familiarity affects the neural dynamics of face identity processing. Here, we report the results of a multivariate EEG analysis, examining the representational dynamics of face identity across several familiarity levels. Participants viewed highly variable face images of 20 identities, including the participants' own face, personally familiar (PF), celebrity and unfamiliar faces. Linear discriminant classifiers were trained and tested on EEG patterns to discriminate pairs of identities of the same familiarity level. Time-resolved classification revealed that the neural representations of identity discrimination emerge around 100 ms post-stimulus onset, relatively independently of familiarity level. In contrast, identity decoding between 200 and 400 ms is determined to a large extent by familiarity: it can be recovered with higher accuracy and for a longer duration in the case of more familiar faces. In addition, we found no increased discriminability for faces of PF persons compared to those of highly familiar celebrities. One's own face benefits from processing advantages only in a relatively late time-window. Our findings provide new insights into how the brain represents face identity with various degrees of familiarity and show that the degree of familiarity modulates the available identity-specific information at a relatively early time window.

SFANet: A Spectrum-Aware Feature Augmentation Network for Visible-Infrared Person Reidentification.

Visible-Infrared person reidentification (VI-ReID) is a challenging matching problem due to large modality variations between visible and infrared images. Existing approaches usually bridge the modality gap with only feature-level constraints, ignoring pixel-level variations. Some methods employ a generative adversarial network (GAN) to generate style-consistent images, but it destroys the structure information and incurs a considerable level of noise. In this article, we explicitly consider these challenges and formulate a novel spectrum-aware feature augmentation network named SFANet for cross-modality matching problem. Specifically, we put forward to employ grayscale-spectrum images to fully replace RGB images for feature learning. Learning with the grayscale-spectrum images, our model can apparently reduce modality discrepancy and detect inner structure relations across the different modalities, making it robust to color variations. At feature level, we improve the conventional two-stream network by balancing the number of specific and sharable convolutional blocks, which preserve the spatial structure information of features. Additionally, a bidirectional tri-constrained top-push ranking loss (BTTR) is embedded in the proposed network to improve the discriminability, which efficiently further boosts the matching accuracy. Meanwhile, we further introduce an effective dual-linear with batch normalization identification (ID) embedding method to model the identity-specific information and assist BTTR loss in magnitude stabilizing. On SYSU-MM01 and RegDB datasets, we conducted extensively experiments to demonstrate that our proposed framework contributes indispensably and achieves a very competitive VI-ReID performance.

Siamese decoupling network for speaker-independent lipreading

Lipreading aims to decode the speech content from a moving mouth. It is a very challenging task because lip appearance variations and speech contents are coupled together in the subtle movements of lip region. Especially in the speaker-independent recognition scenario, training and testing data are totally different in distribution due to the diverse speaker identities, making the learned model generalize poorly in the testing task. We propose a Siamese decoupling lipreading network (SDLipNet) to address this problem. Specially, we exploit an encoder–decoder framework to establish a collaborative representation of speaker identities and speech contents, and utilize the identity-specific information to regularize the content feature space. The identity features are derived from a Siamese identity encoder trained with paired visual speech data from different speakers. In addition, we align the content representation with a prior Gaussian distribution by imposing a Kullback–Leibler divergence constraint between the two outputs of the Siamese content encoder. In this way, the learned content feature space is supposed to be universal to the target speaker domain. Extensive experiments on two lipreading benchmarks demonstrate that our proposed SDLipNet can achieve better performance in the speaker-independent recognition task compared with the state-of-the-art lipreading methods.

Detecting a viewer's familiarity with a face: Evidence from event-related brain potentials and classifier analyses.

Human observers recognize the faces of people they know efficiently and without apparent effort. Consequently, recognizing a familiar face is often assumed to be an automatic process beyond voluntary control. However, there are circumstances in which a person might seek to hide their recognition of a particular face. The present study therefore used event-related potentials (ERPs) and a classifier based on logistic regression to determine if it is possible to detect whether a viewer is familiar with a particular face, regardless of whether the participant is willing to acknowledge it or not. In three experiments, participants were presented with highly variable "ambient" images of personally familiar and unfamiliar faces, while performing an incidental butterfly detection task (Experiment 1), an explicit familiarity judgment task (Experiment 2), and a concealed familiarity task in which they were asked to deny familiarity with one truly known facial identity while acknowledging familiarity with a second known identity (Experiment 3). In all three experiments, we observed substantially more negative ERP amplitudes at occipito-temporal electrodes for familiar relative to unfamiliar faces starting approximately 200ms after stimulus onset. Both the earlier N250 familiarity effect, reflecting visual recognition of a known face, and the later sustained familiarity effect, reflecting the integration of visual with additional identity-specific information, were similar across experiments and thus independent of task demands. These results were further supported by the classifier analysis. We conclude that ERP correlates of familiar face recognition are largely independent of voluntary control and discuss potential applications in forensic settings.

Face Familiarity and Image-Specific Memory.

Face familiarity produces advantages for both memory and matching. By developing an internal representation through repeated experience, viewers extract identity-specific information that aids subsequent recognition. However, researchers have recently argued that this process may also result in a familiarity disadvantage, whereby specific instances of the face are more difficult to remember, perhaps due to this process of prioritising identity- over image-specific information. Although previous experiments found no evidence of this disadvantage in working memory, initial research has demonstrated an effect in longer term storage. Here, we attempted to replicate this finding by focussing on the ability to learn images of a single (un)familiar identity. Our results failed to demonstrate a familiarity disadvantage while replicating the finding that familiarity influences response bias. As researchers continue to investigate how familiarity alters both internal representations and associated processes, it is important to establish which processes may or may not be affected.

The occipital face area is causally involved in identity-related visual-semantic associations

Faces are processed in a network of areas within regions of the ventral visual stream. However, familiar faces typically are characterized by additional associated information, such as episodic memories or semantic biographical information as well. The acquisition of such non-sensory, identity-specific knowledge plays a crucial role in our ability to recognize and identify someone we know. The occipital face area (OFA), an early part of the core face-processing network, is recently found to be involved in the formation of identity-specific memory traces but it is currently unclear if this role is limited to unimodal visual information. The current experiments used transcranial magnetic stimulation (TMS) to test whether the OFA is involved in the association of a face with identity-specific semantic information, such as the name or job title of a person. We applied an identity-learning task where unfamiliar faces were presented together with a name and a job title in the first encoding phase. Simultaneously, TMS pulses were applied either to the left or right OFA or to Cz, as a control. In the subsequent retrieval phase, the previously seen faces were presented either with two names or with two job titles and the task of the participants was to select the semantic information previously learned. We found that the stimulation of the right or left OFA reduced subsequent retrieval performance for the face-associated job titles. This suggests a causal role of the OFA in the association of faces and related semantic information. Furthermore, in contrast to prior findings, we did not observe hemispherical differences of the TMS intervention, suggesting a similar role of the left and right OFAs in the formation of the visual-semantic associations. Our results suggest the necessity to reconsider the hierarchical face-perception models and support the distributed and recurrent models.

Neurophysiological evidence for crossmodal (face-name) person-identity representation in the human left ventral temporal cortex.

Putting a name to a face is a highly common activity in our daily life that greatly enriches social interactions. Although this specific person-identity association becomes automatic with learning, it remains difficult and can easily be disrupted in normal circumstances or neurological conditions. To shed light on the neural basis of this important and yet poorly understood association between different input modalities in the human brain, we designed a crossmodal frequency-tagging paradigm coupled to brain activity recording via scalp and intracerebral electroencephalography. In Experiment 1, 12 participants were presented with variable pictures of faces and written names of a single famous identity at a 4-Hz frequency rate while performing an orthogonal task. Every 7 items, another famous identity appeared, either as a face or a name. Robust electrophysiological responses were found exactly at the frequency of identity change (i.e., 4 Hz / 7 = 0.571 Hz), suggesting a crossmodal neural response to person identity. In Experiment 2 with twenty participants, two control conditions with periodic changes of identity for faces or names only were added to estimate the contribution of unimodal neural activity to the putative crossmodal face-name responses. About 30% of the response occurring at the frequency of crossmodal identity change over the left occipito-temporal cortex could not be accounted for by the linear sum of unimodal responses. Finally, intracerebral recordings in the left ventral anterior temporal lobe (ATL) in 7 epileptic patients tested with this paradigm revealed a small number of "pure" crossmodal responses, i.e., with no response to changes of identity for faces or names only. Altogether, these observations provide evidence for integration of verbal and nonverbal person identity-specific information in the human brain, highlighting the contribution of the left ventral ATL in the automatic retrieval of face-name identity associations.

Deep Latent Low-Rank Representation for Face Sketch Synthesis.

Face sketch synthesis is useful and profitable in digital entertainment. Most existing face sketch synthesis methods rely on the assumption that facial photographs/sketches form a low-dimensional manifold. Once the training data are insufficient, the manifold could not characterize the identity-specific information that is included in a test photograph but excluded in the training data. Thus, the synthesized sketch would lose this information, such as glasses, earrings, hairstyles, and hairpins. To provide the sufficient data and satisfy the assumption on manifold, we propose a novel face sketch synthesis framework based on deep latent low-rank representation (DLLRR) in this paper. The DLLRR induces the hidden training sketches with the identity-specific information as the hidden data to the insufficient original training sketches as the observed data. And it searches the lowest rank representation on the candidates of a test photograph from the both hidden and observed data. For the strong representational capability of the coupled autoencoder, we leverage it to reveal the hidden data. Experiment results on face photograph-sketch database illustrate that the proposed method can successfully provide the sufficient training data with the identity-specific information. And compared to the state of the arts, the proposed method synthesizes more clean and vivid face sketches.

Cross-Domain Face Sketch Synthesis

Synthesizing sketches from facial photos is of great significance to digital entertainment. Along with higher demands on sketch quality in a complex environment, however, it has been an urgent issue on how to synthesize realistic sketches with the limited training data. The existing face sketch methods pay less attention to the insufficient problem of the training data, leading to the synthesized sketches with some noise or without some identity-specific information in real-world applications. Target on providing sufficient photo-sketch pairs to meet the demand of users in digital entertainment, we present a cross-domain face sketch synthesis framework in this paper. In the photo-sketch mixed domain, we leverage the generative adversarial network to construct a cross-domain mapping function and generate identity-preserving face sketches as the hidden training data. Combined it with the insufficient original training data, we provide sufficient training data to recover the underlying structures and learn the cross-domain transfer of the high-level qualitative knowledge from the photo domain to the sketch domain by the latent low-rank representation. The qualitative and quantitative evaluations on the public facial photo-sketch database demonstrate that the proposed cross-domain face sketch synthesis method can solve the insufficient problem of the training data successfully. And it outperforms other state-of-the-art works and generates more vivid and cleaner facial sketches.

Feature Encoder Guided Generative Adversarial Network for Face Photo-Sketch Synthesis

Face photo-sketch synthesis often suffers from many problems, such as low clarity, facial distortion, contents loss, texture missing and color inconsistency in the synthesized images. To alleviate these problems, we propose a feature Encoder Guided Generative Adversarial Network (EGGAN) for face photo-sketch synthesis. We adopt the cycle-consistent generative adversarial network with skipped connections as the general framework, which can train the models for both sketch synthesis and photo synthesis simultaneously. The two generators can constrain each other. In addition, a feature auto-encoder is introduced to refine the synthetic results. The feature encoder is trained to explore a latent space between the photo domain and sketch domains, assuming that there exists a uniform feature representation for a photo-sketch pair. Instead of participating in the generation process, the feature encoder is only utilized to guide the training process. Meanwhile, the feature loss and the feature consistency loss between the fake images and real images from the latent space are calculated to prevent the important identity-specific information from missing and reduce the artifacts in the synthesized images. Extensive experiments demonstrate that our method can achieve state-of-the-art performance on public databases both in terms of perceptual quality and quantitative assessments.

A feasibility randomized trial of an identity-based physical activity intervention among university students

ABSTRACTINTRODUCTION: Exercise identity has considerable evidence as a correlate of physical activity (PA), but almost no research has focused on intervention. Theory suggests identity may be formed through indirect means of motivated behaviour change over time or through direct targeting of identity related antecedents. Using a parallel, single blind design, the purpose of this study was to explore the feasibility (recruitment, retention, and satisfaction) of these two types of interventions (indirect, direct) to increase exercise identity and subsequent PA. METHODS: Participants between the ages of 18–25 who were not meeting PA guidelines were recruited from the University of Victoria, and randomized at a 1:1 ratio to an indirect or direct intervention group. The indirect intervention group received information on the benefits of PA and behaviour change techniques such as planning. The direct intervention group received the same information, with the addition of identity-specific information. Intervention materials were delivered bi-weekly for 6 weeks. Feasibility and participant satisfaction at the study end-point were assessed using mixed methods, and both PA change and exercise identity change were assessed via self-report. RESULTS: Twenty participants were randomized to the direct or indirect intervention group (10 each), with 18 participants completing full study protocol. The recruitment rate was 26% and retention was 90%. Mean scores from the satisfaction survey (five-point scale) were high for both groups (indirect M = 2.69, SD = 0.62; direct M = 2.83, SD = 0.40). Both intervention groups increased their PA (η2 = 0.25), and exercise identity levels (η2 = 0.43) across six weeks. DISCUSSION: High feasibility ratings, both through retention, and survey and interview data show that the study could be extended to a full-scale RCT. Modifications to recruitment including oversampling to account for low recruitment rates may be useful. No adverse events were reported.

Dual-Transfer Face Sketch-Photo Synthesis.

Recognizing the identity of a sketched face from a face photograph dataset is a critical yet challenging task in many applications, not least law enforcement and criminal investigations. An intelligent sketched face identification system would rely on automatic face sketch synthesis from photographs, thereby avoiding the cost of artists manually drawing sketches. However, conventional face sketch-photo synthesis methods tend to generate sketches that are consistent with the artists'drawing styles. Identity-specific information is often overlooked, leading to unsatisfactory identity verification and recognition performance. In this paper, we discuss the reasons why conventional methods fail to recover identity-specific information. Then, we propose a novel dual-transfer face sketch-photo synthesis framework composed of an inter-domain transfer process and an intra-domain transfer process. In the inter-domain transfer, a regressor of the test photograph with respect to the training photographs is learned and transferred to the sketch domain, ensuring the recovery of common facial structures during synthesis. In the intra-domain transfer, a mapping characterizing the relationship between photographs and sketches is learned and transferred across different identities, such that the loss of identity-specific information is suppressed during synthesis. The fusion of information recovered by the two processes is straightforward by virtue of an ad hoc information splitting strategy. We employ both linear and nonlinear formulations to instantiate the proposed framework. Experiments on The Chinese University of Hong Kong face sketch database demonstrate that compared to the current state-of-the-art the proposed framework produces more identifiable facial structures and yields higher face recognition performance in both the photo and sketch domains.

Am I seeing myself, my friend or a stranger? The role of personal familiarity in visual distinction of body identities in the human brain

Several brain regions appear to play a role in representing different body identities. The specific contribution of each of these regions is still unclear, however. Here we investigated which brain areas enable the visual distinction between self and other bodies of different familiarity, and between familiar and unfamiliar other individuals, and moreover, where identity-specific information on the three individuals was encoded. Participants were confronted with standardized headless human body stimuli either showing the participant's own, a personally familiar or an unfamiliar other body, while performing a luminance discrimination task. Employing multivariate pattern analysis, we were able to identify areas that allowed for the distinction of self from personal familiar other bodies within the medial prefrontal cortex (mPFC) and posterior cingulate cortex/precuneus. Successful distinction of self from unfamiliar others was possible in the left middle frontal gyrus, the right inferior frontal gyrus, the left pre-supplementary motor area and the right putamen. Personally familiar others could be distinguished from unfamiliar others in the right temporoparietal junction (TPJ). An analysis of identity-specific information revealed a spatial gradient ranging from inferior posterior to superior anterior portions of the mPFC that was associated with encoding identity-related information for self via familiar to unfamiliar other bodies, respectively. Furthermore, several midline and frontal regions encoded information on more than one identity. The TPJ's role in deviance detection was underlined, as only identity-specific information on unfamiliar others was encoded here. Together, our findings suggest substantial spatial overlap in neural correlates of self and other body representation and thus, support the hypothesis of a socially-related representation of the self.

The processing difference between person names and common nouns in sentence contexts: an ERP study.

Person names and common nouns differ in how they are stored in the mental lexicon. Using event-related potentials, this study compared the integration of names and nouns into sentence contexts. Both person names and common nouns were highly related in meaning and either congruent or incongruent within the previous contexts. Name incongruence elicited an N400 effect, suggesting that people were able to rapidly retrieve the semantic meaning of names from long-term memory even when this process was mediated by person identification. Conversely, participants showed a "good enough" processing of the nouns due to their low specificity level and, thus, rich semantic associations, leading to a P600 effect. These distinctive ERP effects provide clear evidence for the distinctive semantic representations of these word categories by showing that the activation of a name's meaning is mediated by a single connection between identity-specific information and person identity, whereas multiple connections exist between nouns and their meanings.

Effects of Stimulus Type and Level Repetition on Content-Level Binding in Global/Local Processing

The processing and representation of hierarchical objects not only involves the identification of information at the different levels, but also the binding of the identified content to its respective level. Whereas identification is well understood, little is known about content-level binding (CLB). In a recent study, however, it has been shown that attentional priming of certain spatial frequencies is advantageous for this binding. Therefore, the present study investigated effects of related factors on the binding process, namely stimulus type (filled or outlined hierarchical letters), stimulus-type repetition, and target-level repetition. The results show that CLB was improved for outlined stimuli and after target-level repetition, whereas stimulus-type repetition had no effect. The data suggest that hierarchical stimuli are mentally represented by abstract level categories and that content is linked to these categories by means of level-specific and identity-specific spatial-frequency information.

Fixation patterns during recognition of personally familiar and unfamiliar faces

Previous studies recording eye gaze during face perception have rendered somewhat inconclusive findings with respect to fixation differences between familiar and unfamiliar faces. This can be attributed to a number of factors that differ across studies: the type and extent of familiarity with the faces presented, the definition of areas of interest subject to analyses, as well as a lack of consideration for the time course of scan patterns. Here we sought to address these issues by recording fixations in a recognition task with personally familiar and unfamiliar faces. After a first common fixation on a central superior location of the face in between features, suggesting initial holistic encoding, and a subsequent left eye bias, local features were focused and explored more for familiar than unfamiliar faces. Although the number of fixations did not differ for un-/familiar faces, the locations of fixations began to differ before familiarity decisions were provided. This suggests that in the context of familiarity decisions without time constraints, differences in processing familiar and unfamiliar faces arise relatively early – immediately upon initiation of the first fixation to identity-specific information – and that the local features of familiar faces are processed more than those of unfamiliar faces.

Adaptation effects of highly familiar faces: Immediate and long lasting

A central problem of face identification is forming stable representations from entities that vary--both in a rigid and nonrigid manner--over time, under different viewing conditions, and with altering appearances. Three experiments investigated the underlying mechanism that is more flexible than has often been supposed. The experiments used highly familiar faces that were first inspected as configurally manipulated versions. When participants had to select the veridical version (known from TV/media/movies) out of a series of gradually altered versions, their selections were biased toward the previously inspected manipulated versions. This adaptation effect (face identity aftereffect, Leopold, Rhodes, Müller, & Jeffery, 2005) was demonstrated even for a delay of 24 h between inspection and test phase. Moreover, the inspection of a specific image version of a famous person not only changed the veridicality decision of the same image, but also transferred to other images of this person as well. Thus, this adaptation effect is apparently not based on simple pictorial grounds, but appears to have a rather structural basis. Importantly, as indicated by Experiment 3, the adaptation effect was not based on a simple averaging mechanism or an episodic memory effect, but on identity-specific information.

Putting names to faces

It is well established that retrieval of names is harder than the retrieval of other identity specific information. This paper offers a review of the more influential accounts put forward as explanations of why names are so difficult to retrieve. A series of five experiments tests a number of these accounts. Experiments One to Three examine the claims that names are hard to recall because they are typically meaningless (Cohen 1990), or unique (Burton and Bruce 1992; Brédart, Valentine, Calder, and Gassi 1995). Participants are shown photographs of unfamiliar people (Experiments One and Two) or familiar people (Experiment Three) and given three pieces of information about each: a name, a unique piece of information, and a shared piece of information. Learning follows an incidental procedure, and participants are given a surprise recall test. In each experiment shared information is recalled most often, followed by unique information, followed by name. Experiment Four tests both the ‘uniqueness’ account and an account based on the specificity of the naming response (Brédart 1993). Participants are presented with famous faces and asked to categorise them by semantic group (occupation). Results indicate that less time is needed to perform this task when the group is a subset of a larger semantic category. A final experiment examines the claim that names might take longer to access because they are less often retrieved than other classes of information. Latencies show that participants remain more efficient when categorising faces by their occupation than by their name even when they have received extra practice of naming the faces. We conclude that the explanation best able to account for the data is that names are stored separately from other semantic information and can only be accessed after other identity specific information has been retrieved. However, we also argue that the demands we make of these explanations make it likely that no single theory will be able to account for all existing data.

Discussion

Lewis (1999) argued that effects of age of acquisition (AoA) are entirely attributable to cumulative frequency. He reported an instance-based model in which the number of instances of the stimulus stored in memory predicts reaction time. We note four aspects of the literature on AoA that cannot be explained by this instance-based approach. Firstly, an effect of AoA has been observed in the absence of an effect of frequency. Secondly, an effect of AoA has been observed when cumulative frequency has been controlled. Thirdly, the effect of AoA is dependent on task. Fourthly, the effect of word frequency is dependent on stimulus modality. Lewis reported an experiment in which participants make a decision based on identity-specific semantic information to celebrity faces to demonstrate an effect of the number of instances in memory, which he interpreted as an effect of AoA. We note that effects of AoA have been found in lexical and perceptual tasks, but to date all attempts to demonstrate an advantage for early-acquired items in semantic classification tasks have failed. We conclude that the effects of AoA cannot be attributed solely to the effects of cumulative frequency.

Names and Identification: An Access Problem

Abstract information about individuals unless given their names, which, however, he could not produce. It is proposed that CB's previously unreported pattern of symptoms results from multiple functional disconnections isolating the collection of identity-specific semantic information from: (1) the corresponding phonological form, hence proper name anomia; (2) face recognition units, preventing faces being named and matched to semantic information; (3) general semantics, preventing the elements of a definition referring to an individual eliciting further individual specific information. The case is discussed together with similar cases and relevant theories that recently appeared in the literature. The peculiar role of persons' proper names in the semantic system is highlighted. Unlike common names, which denote categories, proper names are thought to just label unique sets of casually clustered attributes constituting a person's identity-specific semantics.