Cognitive State Assessment Research Articles

Symbiotic Brain-Machine interaction: Beyond control and monitoring

Symbiotic Brain-Machine interaction: Beyond control and monitoring

Analysis of engagement behavior in children during dyadic interactions using prosodic cues

Child engagement is defined as the interaction of a child with his/her environment in a contextually appropriate manner. Engagement behavior in children is linked to socio-emotional and cognitive state assessment with enhanced engagement identified with improved skills. A vast majority of studies however rely solely, and often implicitly, on subjective perceptual measures of engagement. Access to automatic quantification could assist researchers/clinicians to objectively interpret engagement with respect to a target behavior or condition, and furthermore inform mechanisms for improving engagement in various settings. In this paper, we present an engagement prediction system based exclusively on vocal cues observed during structured interaction between a child and a psychologist involving several tasks. Specifically, we derive prosodic cues that capture engagement levels across the various tasks. Our experiments suggest that a child's engagement is reflected not only in the vocalizations, but also in the speech of the interacting psychologist. Moreover, we show that prosodic cues are informative of the engagement phenomena not only as characterized over the entire task (i.e., global cues), but also in short term patterns (i.e., local cues). We perform a classification experiment assigning the engagement of a child into three discrete levels achieving an unweighted average recall of 55.8% (chance is 33.3%). While the systems using global cues and local level cues are each statistically significant in predicting engagement, we obtain the best results after fusing these two components. We perform further analysis of the cues at local and global levels to achieve insights linking specific prosodic patterns to the engagement phenomenon. We observe that while the performance of our model varies with task setting and interacting psychologist, there exist universal prosodic patterns reflective of engagement.

Time Out-of-Home and Cognitive, Physical, and Emotional Wellbeing of Older Adults: A Longitudinal Mixed Effects Model.

BackgroundTime out-of-home has been linked with numerous health outcomes, including cognitive decline, poor physical ability and low emotional state. Comprehensive characterization of this important health metric would potentially enable objective monitoring of key health outcomes. The objective of this study is to determine the relationship between time out-of-home and cognitive status, physical ability and emotional state.Methods and FindingsParticipants included 85 independent older adults, age 65–96 years (M = 86.36; SD = 6.79) who lived alone, from the Intelligent Systems for Assessing Aging Changes (ISAAC) and the ORCATECH Life Laboratory cohorts. Factors hypothesized to affect time out-of-home were assessed on three different temporal levels: yearly (cognitive status, loneliness, clinical walking speed), weekly (pain and mood) or daily (time out-of-home, in-home walking speed, weather, and season). Subject characteristics including age, race, and gender were assessed at baseline. Total daily time out-of-home in hours was assessed objectively and unobtrusively for up to one year using an in-home activity sensor platform. A longitudinal tobit mixed effects regression model was used to relate daily time out-of-home to cognitive status, physical ability and emotional state. More hours spend outside the home was associated with better cognitive function as assessed using the Clinical Dementia Rating (CDR) Scale, where higher scores indicate lower cognitive function (β CDR = -1.69, p<0.001). More hours outside the home was also associated with superior physical ability (β Pain = -0.123, p<0.001) and improved emotional state (β Lonely = -0.046, p<0.001; β Low mood = -0.520, p<0.001). Weather, season, and weekday also affected the daily time out-of-home.ConclusionsThese results suggest that objective longitudinal monitoring of time out-of-home may enable unobtrusive assessment of cognitive, physical and emotional state. In addition, these results indicate that the factors affecting out-of-home behavior are complex, with factors such as living environment, weather and season significantly affecting time out-of-home. Studies investigating the relationship between time out-of-home and health outcomes may be optimized by taking into account the environment and life factors presented here.

R-principal subspace for driver cognitive state classification.

Using EEG signals, a novel technique for driver cognitive state assessment is presented, analyzed and experimentally verified. The proposed technique depends on the singular value decomposition (SVD) in finding the distributed energy of the EEG data matrix A in the direction of the r-principal subspace. This distribution is unique and sensitive to the changes in the cognitive state of the driver due to external stimuli, so it is used as a set of features for classification. The proposed technique is tested with 42 subjects using 128 EEG channels and the results show significant improvements in terms of accuracy, specificity, sensitivity, and false detection in comparison to other recently proposed techniques.

Cognitive State Assessment

This effort investigated the ability of a neurophysiological measure to detect changes in workload during a task which is sensitive to cognitive function. A growing collection of research suggests that physiological measures such as EEG can be used to inform the adaptation of systems. However, it has been proposed that such measures often provide a gross interpretation of cognitive workload during complex tasks and are not sensitive to differences in specific cognitive function. To understand the utility of neurophysiological measures for human-machine interaction, we must know if these measures are sensitive to tasks which are sensitive to changes in cognitive function. To begin to answer this question, we investigated the sensitivity of Advanced Brain Monitoring’s EEG-based measures to changes in workload experienced during a Stroop task. Results indicated that ABM’s workload measure can detect changes associated with the attentional demands and cognitive processes linked to the ability to inhibit word naming during tasks involving semantic interference. This indicates that changes in workload associated with the ability to inhibit competing cognitive processes can be identified using neurophysiological workload measures.

Feasibility and validity of ecological momentary assessment in the investigation of suicide risk

Ecological Momentary Assessment has been used to investigate a wide range of behaviors and psychiatric conditions. Previous investigations have consistently obtained promising results with high acceptance and compliance rates, and with only minor reactive effects for specific variables. Despite the promise of this methodology for the study of severe psychiatric populations, little is known about its feasibility in samples at risk for suicide. In the present study, four samples at varying risk for suicide completed an Ecological Momentary Assessment study by responding to five electronic assessments per day over a one-week period. Samples included healthy controls (n=13), affective controls (n=21), past suicide attempters (n=20), and recent suicide attempters (n=42). The results demonstrate satisfactory participation rates and high compliance with daily life repeated assessments across all groups. Importantly, negative thoughts or suicidal ideation were not reactive to the duration of the study, indicating that the repeated assessment of such cognitions in daily life have little or no effect on their frequency. The findings provide support for the use of Ecological Momentary Assessment in the study of suicidal ideation and suggest that mobile technologies represent new opportunities for the assessment of high-risk cognitive states experienced by patients in daily life.

Brain biomarkers based assessment of cognitive workload in pilots under various task demands.

Cognitive workload is an important element of cognitive-motor performance such as that exhibited during the piloting of an aircraft. Namely, an increase in task demands on the pilot can elevate cognitive information processing and, thus, the risk of human error. As such, there is a need to develop methods that reliably assess mental workload in pilots within operational settings. The present study contributes to this research goal by identifying physiological and brain biomarkers of cognitive workload and attentional reserve during a simulated aircraft piloting task under three progressive levels of challenge. A newly developed experimental method was employed by which electroencephalography (EEG) was acquired via a dry (i.e., gel-free sensors) system using few scalp sites. Self-reported responses to surveys and piloting performance indicators were analyzed. The findings revealed that as the challenge (task demands) increased, the perceived mental load increased, attentional reserve was attenuated, and task performance decreased. Such an increase in task demands was also reflected by changes in heart rate variability (HRV), as well as in the amplitude of the P300 component of event-related potentials to auditory probes, and in the spectral power of specific EEG frequency bands. This work provides a first step towards a long-term goal to develop a composite system of biomarkers for real-time cognitive workload assessment and state assessment of pilots in operational settings.

Neuropsychology in the Real World

Current ambulatory technologies are highly relevant for neuropsychological assessment and treatment as they provide a gateway to real life data. Ambulatory assessment of cognitive complaints, skills and emotional states in natural contexts provides information that has a greater ecological validity than traditional assessment approaches. This issue presents an overview of current technological and methodological innovations, opportunities, problems and limitations of these methods designed for the context-sensitive measurement of cognitive, emotional and behavioral function. The usefulness of selected ambulatory approaches is demonstrated and their relevance for an ecologically valid neuropsychology is highlighted.

Involuntary postural responses of users as input to Attentive Computing Systems: An investigation on head movements

Automatic motor or involuntary postural behaviors of users have been receiving an increasing interest in recent years, as unobtrusive measures of cognitive states. In this paper, we investigate the involuntary postural responses of seated users derived from their cognitive changes as input for Attentive Computing Systems. The paper first introduces seated posture, its advantages for cognitive state assessment and connections with cognitive states and, related studies in order to provide a research background for this emerging area of research. We then focus on head posture of seated users and examine the involuntary head movements correlated with task engagement and changing task difficulty through an experiment conducted using a display-oriented cognitive task with changing difficulties. The experiment included 31 participants. Based on different measures, head response and speed, data gathered from user studies were analyzed. Repeated measures Analysis of Variances revealed that head response and speed could serve as cognitive engagement measures. The results indicated that participants get closer to a computer display and became more stationary when they were engaged in a task. The task difficulty analysis results, conversely, partially fulfilled our initial expectations. Head response and speed exhibited limited sensitive behaviors as task difficulties changed.

Psychophysiological measures of human cognitive states applied in human computer interaction

This paper reviews psychophysiological measures applied in Human Computer Interaction (HCI) by mainly focusing on studies related to human cognitive states. Although they have their own disadvantages, psychophysiological measures provide a promising way of user understanding to areas that seek “a sixth sense” for user psychological changes like HCI. We first briefly present the related literature and the major cognitive state assessment measurements. Then, fundamentals of psychophysiological measures are focused in-detail. We provide detailed information concerning their diagnosticity and sensitivity to human cognitive processes in table form. Finally, the paper discusses latest applications and point on the possible future directions.

Towards a Modular Cognitive State Gauge: Assessing Spatial Ability Utilization with Multiple Physiological Measures

Many current and emerging systems for Cognitive State Assessment have yet to achieve the goals required for application for Augmented Cognition. Numerous efforts are working towards this goal but may lack the ability to be unified into a reliable and generalizable Cognitive State Gauge. The overall purpose of this effort is to determine under what conditions multiple minimally intrusive physiological sensors can be used together and validly applied to areas within a modular cognitive state assessment framework. This study investigated the utility of various measures in detecting changes in cognitive state. We examined the differences within the output from multiple physiological sensing devices under varying levels of task difficulty during a task that was highly dependent on spatial ability (SPA). Supportive evidence for a modular cognitive state gauge (MCSG) was revealed with the finding of significant results for several physiological measures on a novel spatial task.

A Demonstration of a Dry/No Preparation Electrode System for EEG

Psychophysiological assessment using electroencephalography (EEG) has previously been demonstrated to be an effective method for objectively assessing a variety of human factors-relevant states, such as mental workload and fatigue, as well as providing a mechanism for brain-computer interfaces. Wider adoption of EEG methods has been hampered by the extensive preparation and care that must be taken in order to ensure good signal quality. This demonstration will exhibit a new EEG system that is capable of recording data with no preparation, can be worn for long periods of time, and is reasonably robust in mobile environments. While still in development, this system has the potential to enable objective assessment of cognitive state in a wide variety of human factors applications.

Enhancing Mitigation in Augmented Cognition

In augmented cognition (AugCog), mitigation strategies are used as real-time intervention techniques that are triggered by the outcome of cognitive state assessment and context to significantly improve human-systems performance. Yet, no common ground has been established regarding best practices and what aspects to consider during implementation. This paper discusses mitigation strategies currently used in AugCog systems and provides insights into their strengths and weaknesses. An event-based conceptual framework is introduced that aids real-time mitigation strategy selection by linking system events to real-time cognitive state indicators, which together determine when, what, and how to mitigate. Insights from the implementation of this framework in an AugCog system designed to optimize situation awareness are presented, which support the architecture of the framework and identify further challenges to mitigation. Future work should focus on further validating the proposed framework and leveraging techniques from other domains (e.g., film, theater) to create more effective mitigation concepts in AugCog systems.

Evaluation of a Tactile Navigation Cueing System and Real-Time Assessment of Cognitive State

This paper details an evaluation of a tactile cueing system that was created to enhance the navigation of a complex route. The complexity of the task along with simultaneously challenging cognitive tasks also enabled the real-time assessment of cognitive state in various task load conditions. Honeywell has been working with the US Army's Future Force Warrior program to develop adaptive systems that will effectively manage the available cognitive resources used for information processing by the dismounted Soldier in highly dynamic, information rich environments. The appropriate allocation of cognitive resources is key to managing multiple tasks, focusing on the most important ones, and maintaining overall situation awareness. Non-visual navigation support would offload a typically visual task, such as viewing a paper map or computer-based map display, to a sensory channel that is underutilized, tactile sensation. Both benefits and costs to this type of automation support are explored in detail, where the evaluation supports the premise that strong automated support should only be used in high workload situations where the benefits outweigh the costs.

Assessing normative cut points through differential item functioning analysis: An example from the adaptation of the Middlesex Elderly Assessment of Mental State (MEAMS) for use as a cognitive screening test in Turkey

BackgroundThe Middlesex Elderly Assessment of Mental State (MEAMS) was developed as a screening test to detect cognitive impairment in the elderly. It includes 12 subtests, each having a 'pass score'. A series of tasks were undertaken to adapt the measure for use in the adult population in Turkey and to determine the validity of existing cut points for passing subtests, given the wide range of educational level in the Turkish population. This study focuses on identifying and validating the scoring system of the MEAMS for Turkish adult population.MethodsAfter the translation procedure, 350 normal subjects and 158 acquired brain injury patients were assessed by the Turkish version of MEAMS. Initially, appropriate pass scores for the normal population were determined through ANOVA post-hoc tests according to age, gender and education. Rasch analysis was then used to test the internal construct validity of the scale and the validity of the cut points for pass scores on the pooled data by using Differential Item Functioning (DIF) analysis within the framework of the Rasch model.ResultsData with the initially modified pass scores were analyzed. DIF was found for certain subtests by age and education, but not for gender. Following this, pass scores were further adjusted and data re-fitted to the model. All subtests were found to fit the Rasch model (mean item fit 0.184, SD 0.319; person fit -0.224, SD 0.557) and DIF was then found to be absent. Thus the final pass scores for all subtests were determined.ConclusionThe MEAMS offers a valid assessment of cognitive state for the adult Turkish population, and the revised cut points accommodate for age and education. Further studies are required to ascertain the validity in different diagnostic groups.

Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks

A symbiotic relation between the operator and the operational environment can be realized by an advanced computing platform designed to understand and adapt to the cognitive and the physiological state of the user, especially during sensitive and cognitively demanding operations. The success of such a complex system depends not only on the efficacy of the individual components, but also on the efficient and appropriate integration of its parts. Because near infrared technology allows the design of portable, safe, affordable, and negligibly intrusive monitoring systems, the functional near infrared (fNIR) monitoring of brain hemodynamics can be of value in this type of complex system, particularly in helping to understand the cognitive and emotional state of the user during mentally demanding operations. This article presents the deployment and statistical analysis of fNIR spectroscopy for the purpose of cognitive state assessment while the user performs a complex task. This article is based on data collected during the Augmented Cognition-Technical Integration Experiment session. The experimental protocol for this session used a complex task, resembling a video game, called the Warship Commander Task (WCT). The WCT was designed to approximate naval air warfare management. Task difficulty and task load were manipulated by changing the following: (a) the number of airplanes that had to be managed at a given time, (b) the number of unknown (vs. known) airplane identities, and (c) the presence or absence of an auditory memory task. The fNIR data analysis explored the following: (a) the relations among cognitive workload, the participant's performance, and changes in blood oxygenation levels of the dorsolateral prefrontal cortex; and (b) the effect of divided attention as manipulated by the secondary component of the WCT (the auditory task). The primary hypothesis was that blood oxygenation in the prefrontal cortex, as assessed by fNIR, would rise with increasing task load and would demonstrate a positive correlation with performance measures. The results indicated that the rate of change in blood oxygenation was significantly sensitive to task load changes and correlated fairly well with performance variables.

Serial assessment of serum calcium correlates with mental state

AbstractSerial assessment of cognitive state using clinical and psychometric instruments correlated closely with serum calcium in an elderly woman with hyperparathyroidism. ‘Medical parathyroidectomy’ was associated with biochemical and clinical improvement but was not sustained. Following surgical removal of a parathyroid adenoma her serum calcium and her cognitive state returned to normal.

The “Mini-Mental State” Examination in the Syndrome of Self-Induced Water Intoxication and Schizophrenic Disorders (SIWIS): A Pilot Study

A quantitative assessment of cognitive state (Folstein's “mini-mental state” scores) was correlated with serum sodium concentration in four patients with the syndrome of self-induced water intoxication and schizophrenic disorders (SIWIS). Diagnostic studies that demonstrated inappropriate antidiuresis in these patients are reported also. Neither absolute serum sodium concentrations nor their daily changes correlated significantly ( p &gt; .05) with the cognitive state of any patient. The authors conclude that serial laboratory determinations of serum electrolytes remains the only satisfactory method of monitoring body tonicity in SIWIS patients.

Attributions and Performance: An Empirical Test of Kukla's Theory

The purpose of this investigation was to test Kukla's attributional theory of performance and to address the mediating link between causal attribution and subsequent action in a competitive motor task. Two experiments were conducted: the first was designed to determine the effect of perceived task difficulty and attributional instructions on the ball-tossing performance of high achievers while competing against a standard of excellence. Results indicated that high achievers performed with greater intensity when receiving an effort rather than ability-oriented instructional set and when they perceive themselves to be behind a normative score of their classmates. To refine and clarify results found in Experiment 1, low as well as high achievers were added to the second study and were placed in face-to-face competition instead of a competition against a standard of excellence. Results from the performance data in Experiment 2 supported Kukla's theory in that high achievers performed best when they perceived themselves to be behind an opponent's score midway through the experiment, whereas low achievers performed best when they perceived themselves to be ahead. Results from the cognitive assessment procedure, however, failed to support the performance findings. Implications for the viability of Kukla's attributional theory of performance and problems related to the assessment of cognitive states are discussed.

Mental Preparation Strategies, Cognitions, and Strength Performance

Two experiments were conducted to determine if different mental preparation strategies produced differential strength performance and whether arousal was the major mediating variable explicating this relationship. In the first experiment, 15 male and 15 female subjects performed under five different mental preparation conditions in a 2 × 5 (sex by mental preparation strategy) Latin square design. The mental preparation conditions included: attentional focus, imagery, preparatory arousal, a control-rest condition, and a counting backwards cognitive-distraction condition. Immediately after employing each technique, all subjects performed four trials on a leg-strength task, and measures of state anxiety and other cognitions were then obtained. The findings revealed that the preparatory arousal and imagery techniques produced the greatest change in performance, with preparatory arousal subjects also reporting the greatest changes in cognitive states. However, due to the possibility of range effects resulting from the within-subjects design used in Experiment I, a second between-subjects experiment was conducted. Thirty males and 30females performed in a 2 × 3 (sex by mental preparation) design using the preparatory arousal, imagery and control conditions of Experiment 1. Only the preparatory arousal condition was found to facilitate performance. However, no consistent changes in cognitive states were found between experiments, and these inconsistent findings were interpreted as being caused by methodological problems associated with self-report assessment of cognitive states.