Recognition Of Mental States Research Articles

Social influence on associative learning: Double dissociation in high-functioning autism, early-stage behavioural variant frontotemporal dementia and Alzheimer's disease

IntroductionMost of our learning activity takes place in a social context. I examined how social interactions influence associative learning in neurodegenerative diseases and atypical neurodevelopmental conditions primarily characterised by social cognitive and memory dysfunctions. MethodsParticipants were individuals with high-functioning autism (HFA, n = 18), early-stage behavioural variant frontotemporal dementia (bvFTD, n = 16) and Alzheimer's disease (AD, n = 20). The leading symptoms in HFA and bvFTD were social and behavioural dysfunctions, whereas AD was characterised by memory deficits. Participants received three versions of a paired associates learning task. In the game with boxes test, objects were hidden in six candy boxes placed in different locations on the computer screen. In the game with faces, each box was labelled by a photo of a person. In the real-life version of the game, participants played with real persons. ResultsIndividuals with HFA and bvFTD performed well in the computer games, but failed on the task including real persons. In contrast, in patients with early-stage AD, social interactions boosted paired associates learning up to the level of healthy control volunteers. Worse performance in the real life game was associated with less successful recognition of complex emotions and mental states in the Reading the Mind in the Eyes Test. Spatial span did not affect the results. ConclusionsWhen social cognition is impaired, but memory systems are less compromised (HFA and bvFTD), real-life interactions disrupt associative learning; when disease process impairs memory systems but social cognition is relatively intact (early-stage AD), social interactions have a beneficial effect on learning and memory.

Disconnection Mechanism and Regional Cortical Atrophy Contribute to Impaired Processing of Facial Expressions and Theory of Mind in Multiple Sclerosis: A Structural MRI Study

Successful socialization requires the ability of understanding of others’ mental states. This ability called as mentalization (Theory of Mind) may become deficient and contribute to everyday life difficulties in multiple sclerosis. We aimed to explore the impact of brain pathology on mentalization performance in multiple sclerosis. Mentalization performance of 49 patients with multiple sclerosis was compared to 24 age- and gender matched healthy controls. T1- and T2-weighted three-dimensional brain MRI images were acquired at 3Tesla from patients with multiple sclerosis and 18 gender- and age matched healthy controls. We assessed overall brain cortical thickness in patients with multiple sclerosis and the scanned healthy controls, and measured the total and regional T1 and T2 white matter lesion volumes in patients with multiple sclerosis. Performances in tests of recognition of mental states and emotions from facial expressions and eye gazes correlated with both total T1-lesion load and regional T1-lesion load of association fiber tracts interconnecting cortical regions related to visual and emotion processing (genu and splenium of corpus callosum, right inferior longitudinal fasciculus, right inferior fronto-occipital fasciculus, uncinate fasciculus). Both of these tests showed correlations with specific cortical areas involved in emotion recognition from facial expressions (right and left fusiform face area, frontal eye filed), processing of emotions (right entorhinal cortex) and socially relevant information (left temporal pole). Thus, both disconnection mechanism due to white matter lesions and cortical thinning of specific brain areas may result in cognitive deficit in multiple sclerosis affecting emotion and mental state processing from facial expressions and contributing to everyday and social life difficulties of these patients.

Looking and thinking: How individuals with Williams syndrome make judgements about mental states

Individuals with the neuro-developmental disorder Williams syndrome (WS) are characterised by a combination of features which makes this group vulnerable socially, including mild-moderate cognitive difficulties, pro-social drive, and indiscriminate trust. The purpose of this study was to explore a key socio-communicative skill in individuals with WS, namely, mental state recognition abilities. We explored this skill in a detailed way by looking at how well individuals with WS recognise complex everyday mental states, and how they allocate their attention while making these judgements. Participants with WS were matched to two typically developing groups for comparison purposes, a verbal ability matched group and a chronological age matched group. While eye movements were recorded, participants were shown displays of eight different mental states in static and dynamic form, and they performed a forced-choice judgement on the mental state. Mental states were easier to recognise in dynamic form rather than static form. Mental state recognition ability for individuals with WS was poorer than expected by their chronological age, and at the level expected by their verbal ability. However, the pattern of mental state recognition for participants with WS varied according to mental state, and we found some interesting links between ease/difficulty recognising some mental states (worried/do not trust) and the classic behavioural profile associated with WS (high anxiety/indiscriminate trust). Furthermore, eye tracking data revealed that participants with WS allocated their attention atypically, with less time spent attending the information from the face regions. This challenges the widely held understanding of WS being associated with prolonged face and eye gaze, and indicates that there is more heterogeneity within this disorder in terms of socio-perception than previous reports would suggest.

Theory of mind performance in women with posttraumatic stress disorder related to childhood abuse

Key questions remain unaddressed concerning the nature of interpersonal functioning in trauma survivors, including the ability to understand and interpret other people's thoughts and feelings. Here, we investigate theory of mind (ToM) performance of women with PTSD related to childhood abuse in comparison to healthy controls. Participants completed two ToM tasks, the Interpersonal Perception Task-15 (IPT-15) and the Reading the Mind in the Eyes Task - Revised (RMET). Relative to controls, women with a history of childhood trauma had difficulty recognizing familial relationships depicted in the IPT-15 (P = 0.005). No other category of the IPT-15 showed significant group differences. In addition, while healthy women displayed faster RMET reaction times to emotionally valenced mental states (positive: P = 0.003; negative: P = 0.016) compared with neutral mental states, the PTSD group showed similar reaction times across all valences. The presence of dissociative symptoms (e.g., disengagement, amnesia, identity dissociation) was strongly associated with hindered accuracy of complex mental state identification and altered perception of kinship interactions. Women with PTSD stemming from childhood trauma show changes in ToM abilities particularly those often involved in the interpretation of family interactions. In addition, individuals with PTSD showed slower reaction times during the recognition of complex mental states from emotionally salient facial/eye expressions in comparison with healthy subjects.

Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design

While recent research on Brain-Computer Interfaces (BCI) has highlighted their potential for many applications, they remain barely used outside laboratories. The main reason is their lack of robustness. Indeed, with current BCI, mental state recognition is usually slow and often incorrect. Spontaneous BCI (i.e., mental imagery-based BCI) often rely on mutual learning efforts by the user and the machine, with BCI users learning to produce stable ElectroEncephaloGraphy (EEG) patterns (spontaneous BCI control being widely acknowledged as a skill) while the computer learns to automatically recognize these EEG patterns, using signal processing. Most research so far was focused on signal processing, mostly neglecting the human in the loop. However, how well the user masters the BCI skill is also a key element explaining BCI robustness. Indeed, if the user is not able to produce stable and distinct EEG patterns, then no signal processing algorithm would be able to recognize them. Unfortunately, despite the importance of BCI training protocols, they have been scarcely studied so far, and used mostly unchanged for years. In this paper, we advocate that current human training approaches for spontaneous BCI are most likely inappropriate. We notably study instructional design literature in order to identify the key requirements and guidelines for a successful training procedure that promotes a good and efficient skill learning. This literature study highlights that current spontaneous BCI user training procedures satisfy very few of these requirements and hence are likely to be suboptimal. We therefore identify the flaws in BCI training protocols according to instructional design principles, at several levels: in the instructions provided to the user, in the tasks he/she has to perform, and in the feedback provided. For each level, we propose new research directions that are theoretically expected to address some of these flaws and to help users learn the BCI skill more efficiently.

Association of cerebral small vessel disease with delusions in patients with Alzheimer's disease

Cerebral small vessel disease (SVD) is frequently observed in patients with Alzheimer's disease (AD). However, the association between SVD and clinical symptoms exhibited by patients with AD remains unclear. This study examined the association of SVD as observed on magnetic resonance imaging (MRI) with behavioural and psychological symptoms of dementia and cognitive function of patients with probable AD. A total of 163 consecutive patients (55 men, 108 women) with probable AD were included in this cross-sectional study of a prospective cohort. Patients were divided into two groups based on the presence or absence of cerebral SVD [white matter hyperintensities (WMH) grade 0/1 (Fazekas scale) and no lacunes: SVD absent, WMH grade 2/3 (Fazekas scale) or the number of lacunes ≥1: SVD present]. Cognitive functions were assessed using the Mini mental state examination, word recall and recognition subtests in the Alzheimer's Disease Assessment Scale-Cognitive Subscale, as well as the letter fluency task and the category fluency task. Psychiatric symptoms were rated according to Neuropsychiatric Inventory. Patients with probable AD with cerebral SVD had significantly more delusions and depression than those without SVD. No significant differences were observed in other neuropsychiatric symptoms, MMSE or word recall and recognition tests between both groups. Our results suggest that cerebral SVD observed on MRI of patients with AD is associated with delusions and depression.

Brief Report: Impaired Differentiation of Vegetative/Affective and Intentional Nonverbal Vocalizations in a Subject with Asperger Syndrome (AS)

The Asperger syndrome (AS) includes impaired recognition of other people's mental states. Since language-based diagnostic procedures may be confounded by cognitive-linguistic compensation strategies, nonverbal test materials were created, including human affective and vegetative sounds. Depending on video context, each sound could be interpreted either as direct expression of an agent's affective/vegetative state or as result of intentional-executive mental operations. "Situational relevance" and "intentionality" ratings by a group of twelve healthy subjects nicely differentiated between context types. By contrast, an AS subject showed a systematic over-interpretation of vegetative/affective signals in terms of planned activities. Such overestimation of intentional motivation, leading to impaired social cognition, might be due to the inability to utilize "affective resonance" mechanisms for the interpretation of an individual's internal state.

Emotion and mental state recognition from speech

As research in speech processing has matured, attention has gradually shifted from linguistic-related applications such as speech recognition towards paralinguistic speech processing problems, in particular the recognition of speaker identity, language, emotion, gender, and age. Determination of a speaker’s emotion or mental state is a particularly challenging problem, in view of the significant variability in its expression posed by linguistic, contextual, and speaker-specific characteristics within speech. In response, a range of signal processing and pattern recognition methods have been developed in recent years. Recognition of emotion and mental state from speech is a fundamentally multidisciplinary field, comprising contributions from psychology, speech science, linguistics, (cooccurring) nonverbal communication, machine learning, artificial intelligence and signal processing, among others. Some of the key research problems addressed to date include isolating sources of emotion-specific information in the speech signal, extracting suitable features, forming reduced-dimension feature sets, developing machine learning methods applicable to the task, reducing feature variability due to speaker and linguistic content, comparing and evaluating diverse methods, robustness, and constructing suitable databases. Studies examining the relationships between the psychological basis of emotion, the effect of emotion on speech production, and the measurable differences in the speech signal due to emotion have helped to shed light on these problems; however, substantial research is still required. Taking a broader view of emotion as a mental state, signal processing researchers have also explored the possibilities of automatically detecting other types of mental state which share some characteristics with emotion, for example stress, depression, cognitive load, and ‘cognitive epistemic’ states such as interest, scepticism, etc. The recent interest in emotion recognition research has seen applications in call centre analytics, human-machine and humanrobot interfaces, multimedia retrieval, surveillance tasks, behavioural health informatics, and improved speech recognition. This special issue comprises nine articles covering a range of topics in signal processing methods for vocal source and acoustic feature extraction, robustness issues, novel applications of pattern recognition techniques, methods for detecting mental states and recognition of non-prototypical spontaneous and naturalistic emotion in speech. These articles were accepted following peer review, and each submission was handled by an editor who was independent from all authors listed in that manuscript. Herein, we briefly introduce the articles comprising this special issue. Trevino, Quatieri and Malyska bring a new level of sophistication to an old problem, detecting signs of depressive disorders in speech. Their measures of depression come from standard psychiatric instruments, Quick Inventory of Depressive Symptomatology and Hamilton Depression rating scales. These are linked to measures of speech timing that are much richer than the traditional global measures of speech rate. Results indicate that different speech sounds and sound types behave differently in depression, and may relate to different aspects of depression. Caponetti, Buscicchio and Castellano propose the use of a more detailed auditory model than that embodied in the widely employed mel frequency cepstral coefficients, for extracting detailed spectral features during emotion recognition. Working from the Lyon cochlear model, the authors demonstrate improvements on a five-class problem from the speech under simulated and actual stress database. Their study also further validates the applicability of long short-term memory recurrent neural networks for classification in emotion and mental state recognition problems. Callejas, Griol and Lopez-Cozar propose a mental state prediction approach that considers both speaker * Correspondence: j.epps@unsw.edu.au School of Electrical Engineering and Telecommunications, The University of New South Wales, Sydney, NSW 2052, Australia Full list of author information is available at the end of the article Epps et al. EURASIP Journal on Advances in Signal Processing 2012, 2012:15 http://asp.eurasipjournals.com/content/2012/1/15

Theory of mind in frontal and temporal lobe epilepsy: Cognitive and neural aspects

Theory of mind (ToM) is an important prerequisite to social behavior. This study evaluated ToM in patients with temporal (TLE) or frontal lobe epilepsy (FLE) aiming to determine the cognitive aspects, severity, and pathophysiologic mechanisms of ToM impairment in focal epilepsy. One hundred thirty-eight patients with TLE (n = 109) or FLE (n = 29) and 69 healthy subjects underwent the Faux Pas task (FPT), which evaluates the recognition and comprehension of others' mental states, and neuropsychological tests for other cognitive functions. Factor analysis of all test scores yielded two ToM factors (Recognizing faux pas, FP; Excluding nonexistent FP) distinct from the Control, Language, Matching, and Praxis factors. With respect to healthy subjects, both TLE and FLE patients showed correct exclusion of nonexistent FPs but significantly lower recognition and comprehension of real FPs. FLE patients were also impaired with respect to TLE patients. In the whole patient group, schooling and group membership predicted ToM impairment. In FLE patients, the comprehension of mental states was predicted by disease duration, whereas TLE patients' comprehension of affects and intentions was associated with early age of seizure onset and medial temporal lobe sclerosis (MTLS). Focal epilepsy impairs advanced ToM abilities. FLE may affect online performances owing to long-lasting dysfunctions of the prefrontal areas. MTLS may provoke selective ToM deficits due to medial temporal damage, prefrontal dysfunctions, or early interference with cognitive development. Future studies are needed to determine the implications of ToM impairment on behavior and quality of life.

Can Gaze Avoidance Explain Why Individuals with Asperger’s Syndrome Can’t Recognise Emotions From Facial Expressions?

Research has shown that individuals with Autism Spectrum Disorders (ASD) have difficulties recognising emotions from facial expressions. Since eye contact is important for accurate emotion recognition, and individuals with ASD tend to avoid eye contact, this tendency for gaze aversion has been proposed as an explanation for the emotion recognition deficit. This explanation was investigated using a newly developed emotion and mental state recognition task. Individuals with Asperger's Syndrome were less accurate at recognising emotions and mental states, but did not show evidence of gaze avoidance compared to individuals without Asperger's Syndrome. This suggests that the way individuals with Asperger's Syndrome look at faces cannot account for the difficulty they have recognising expressions.

Mental state decoding abilities in young adults with borderline personality disorder traits.

Previous studies have demonstrated that patients with borderline personality disorder (BPD) tend to misattribute malevolence to benign social stimuli, including facial expressions. Yet, facial emotion recognition studies examining those with BPD have yielded mixed results, with some studies showing impaired accuracy and others demonstrating enhanced accuracy in the recognition of emotions or mental states. The current study examined the ability to decode mental states from photographs of just the eye region of faces in a nonclinical sample of young adults who exhibited BPD traits (high BPD) compared with those who did not (low BPD). Group differences in mental state decoding ability depended on the valence of the stimuli. The high-BPD group performed better for negative stimuli compared with the low-BPD group, but did not perform significantly different from the low-BPD group for stimuli of neutral or positive valence. The high-BPD group also demonstrated a response bias for attributing negative mental states to facial stimuli. In addition, findings suggested that the group difference in accuracy for negative stimuli could not be explained by response bias, because the group difference in response bias for negative stimuli did not reach significance. These findings suggest that BPD traits may be associated with enhanced ability to detect negative emotions and a bias for attributing negative emotions to nonnegative social stimuli.

Communicating emotions and mental states to robots in a real time parallel framework using Laban movement analysis

This paper presents a parallel real time framework for emotions and mental states extraction and recognition from video fragments of human movements. In the experimental setup human hands are tracked by evaluation of moving skin-colored objects. The tracking analysis demonstrates that acceleration and frequency characteristics of the traced objects are relevant for classification of the emotional expressiveness of human movements. The outcomes of the emotional and mental states recognition are cross-validated with the analysis of two independent certified movement analysts (CMA’s) who use the Laban movement analysis (LMA) method. We argue that LMA based computer analysis can serve as a common language for expressing and interpreting emotional movements between robots and humans, and in that way it resembles the common coding principle between action and perception by humans and primates that is embodied by the mirror neuron system. The solution is part of a larger project on interaction between a human and a humanoid robot with the aim of training social behavioral skills to autistic children with robots acting in a natural environment.

ARE ENDOPHENOTYPES A USEFUL CONCEPT?

perspectives. In this context, imaging studies revealed that crosssectional and longitudinal abnormalities exist on the structural and functional brain dimensions of at-risk individuals and firstepisode patients. These findings have offered important insights into the brain pathology emerging during the transition from the prodromal state to the full-blown clinical picture of schizophrenia. Thus, the accumulating brain alterations described by recent neuroimaging studies may be interpreted within the concept of a late neurodevelopmental process that evolves when higher-order cortical areas are placed “under functional demand” during the critical transition from adolescence to adulthood. Beside these advances in the basic science of psychosis, the recent advent of multivariate pattern recognition techniques in the neuroimaging field has opened up new possibilities to derive valid biomarkers from, up to now, group-level statistical inferences that may allow for the individualized early recognition of the at-risk mental state and the prediction of disease transition. Thus, these findings may stir up the debate whether (a) structural and/or functional brain abnormalities represent reliable intermediate phenotypes, or “endophenotypes” of psychosis and whether (b) these imagingbased endophenotypes are sufficiently sensitive and specific markers of the underlying neurobiology and its behavioral expressions. A positive answer to these questions would allow neuroimaging techniques to cross the bridge between basic science and its potential clinical applications. The symposium aims to bring together leading experts from the field to present the state-of-the-art basic science aspects of neuroimaging research as well as its clinical perspectives regarding the application of individualized neurodiagnostic procedures for the early recognition of the high-risk states and the early phases of psychosis as well as the prediction of disease transition.

The recognition of mental states from dynamic and static facial expressions

The ability to recognize mental states from facial expressions is essential for effective social interaction. However, previous investigations of mental state recognition have used only static faces so the benefit of dynamic information for recognizing mental states remains to be determined. Experiment 1 found that dynamic faces produced higher levels of recognition accuracy than static faces, suggesting that the additional information contained within dynamic faces can facilitate mental state recognition. Experiment 2 explored the facial regions that are important for providing dynamic information in mental state displays. This involved using a new technique to freeze motion in a particular facial region (eyes, nose, mouth) so that this region was static while the remainder of the face was naturally moving. Findings showed that dynamic information in the eyes and the mouth was important and the region of influence depended on the mental state. Processes involved in mental state recognition are discussed.

Anomalous Subjective Experience and Psychosis Risk in Young Depressed Patients

Background: Help-seeking young people often display depressive symptoms. In some patients, these symptoms may co-exist with clinically high-risk mental states for psychosis. The aim of this study was to determine differences in subjective experience and social perception in young depressed patients with and without psychosis risk. Methods: Participants were 68 young persons with major depressive disorder. Twenty-six patients also met the criteria of attenuated or brief limited intermittent psychotic symptoms according to the Comprehensive Assessment of At Risk Mental States (CAARMS) criteria. Subjective experiences were assessed with the Bonn Scale for the Assessment of Basic Symptoms (BSABS). Recognition of complex social emotions and mental states was assessed using the ‘Reading the Mind in the Eyes’ test. Results: Perplexity, self-disorder, and diminished affectivity significantly predicted psychosis risk. Depressed patients without psychosis risk displayed impaired recognition performance for negative social emotions, whereas patients with psychosis risk were also impaired in the recognition of cognitive expressions. In the high-risk group, self-disorder was associated with impaired recognition of facial expressions. Conclusions: These results suggest that anomalous subjective experience and impaired recognition of complex emotions may differentiate between young depressed patients with and without psychosis risk.

Inference Degradation of Active Information Fusion within Bayesian Network Models

Bayesian networks have been extensively used in active information fusion that selects the best sensor based on expected utility calculation. However, inference degradation happens when the same sensors are selected repeatedly over time if the applied strategy is not well designed to consider the history of sensor engagement. This phenomenon decreases fusion accuracy and efficiency, in direct conflict to the objective of information integration with multiple sensors. This paper provides mathematical scrutiny of the inference degradation problem in the popular myopia planning. It examines the generic dynamic Bayesian network models and shows experimentation results for mental state recognition tasks. It also discusses the candidate solutions with initial results. The inference degradation problem is not limited to the discussed fusion tasks and may emerge in variants of sensor planning strategies with more global optimization approach. This study provides common guidelines in information integration applications for information awareness and intelligent decision.

The ‘Reading the Mind in Films’ Task [Child Version]: Complex Emotion and Mental State Recognition in Children with and without Autism Spectrum Conditions

Children with autism spectrum conditions (ASC) have difficulties recognizing others' emotions. Research has mostly focused on basic emotion recognition, devoid of context. This study reports the results of a new task, assessing recognition of complex emotions and mental states in social contexts. An ASC group (n = 23) was compared to a general population control group (n = 24). Children with ASC performed lower than controls on the task. Using task scores, more than 87% of the participants were allocated to their group. This new test quantifies complex emotion and mental state recognition in life-like situations. Our findings reveal that children with ASC have residual difficulties in this aspect of empathy. The use of language-based compensatory strategies for emotion recognition is discussed.

RECOGNITION OF COMPLEX MENTAL STATES IN PATIENTS WITH ALCOHOLISM AFTER LONG-TERM ABSTINENCE

Previous studies demonstrated that patients with alcoholism display impaired emotional facial expression recognition even after long-term abstinence. These studies focused on basic emotions (happiness, anger, sadness, and disgust). In this study, we investigated the recognition of complex social emotions and mental states in patients with alcoholism after long-term abstinence and healthy control subjects. Thirty patients with DSM-IV alcohol dependence and 30 age-matched, gender-matched, education-matched, and IQ-matched healthy control subjects participated. The patients were abstinent for >6 months. For the assessment of the recognition of complex social emotions and mental states, the Baron-Cohen Eyes Test was used. The experimenter presented 29 photographs of the eye-region of faces of actors and actresses on separate cards. Participants were asked to choose which of the four words (one target and three foils) best described the mental state of the actor/actress (for example, interested, doubtful, flirtatious, and insisting). The primary dependent measure was the number of correctly recognized stimuli. Patients with alcoholism correctly identified 22.4 (SD = 3.4) stimuli, whereas control participants identified 22.5 (SD = 2.9) stimuli. The difference was not statistically significant (P = 0.85). There was no significant difference in the proportion of patients and controls who correctly recognized each mental state. These results are against the hypotheses suggesting long-term adverse effects of alcohol on social cognition or supposing an inherent vulnerability of patients that may manifest before the development of alcohol dependence.

The “Reading the Mind in Films” Task: Complex emotion recognition in adults with and without autism spectrum conditions

Background: Individuals with autism spectrum conditions (ASC) have difficulties recognizing mental states in others. Most research has focused on recognition of basic emotions from faces and voices separately. This study reports the results of a new task, assessing recognition of complex emotions and mental states from social scenes taken from feature films. The film format arguably is more challenging and ecologically closer to real social situations. Sample and method: A group of adults with ASC (n=22) were compared to a group of matched controls from the general population (n=22). Participants were tested individually. Results: Overall, individuals with ASC performed significantly lower than controls. There was a positive correlation between verbal IQ and task scores. Using task scores, more than 90% of the participants were correctly allocated to their group. Item analysis showed that the errors individuals with ASC make when judging socioemotional information are subtle. Conclusions: This new test of complex emotion and mental state recognition reveals that adults with ASC have residual difficulties in this aspect of empathy. The use of language-based compensatory strategies for emotion recognition is discussed.

Making Virtual Learning Environment more intelligent: the problem of software agent’s mental state recognition

Intelligent decision making process, which is performed according to a learner in the Virtual Learning Environment (VLE), leads to the problem of solving several rather complex tasks. Two of them are most of interest. First, we need to train a software agent to recognize its mental state; and second, we want agent to apply optimal strategy to teach learners when it is in some mental state. The second issue we have discussed in [2]. We propose the agent’s ability to recognise its mental state could be based on the classification result to a particular learner. Classification used to be done should be based on the discovered groups in the database data and classification rules that prescribe given learner to the one of the available clusters. Conceptual clustering seems to be the suitable technique able to provide a solution for the problem raised.