Complex Task Performance Research Articles

Understanding the structure of skill through a detailed analysis of Individuals' performance on the Space Fortress game

In this paper we describe a novel approach to the study of individual differences in acquired skilled performance in complex laboratory tasks based on an extension of the methodology of the expert-performance approach (Ericsson & Smith, 1991) to shorter periods of training and practice. In contrast to more traditional approaches that study the average performance of groups of participants, we explored detailed behavioral changes for individual participants across their development on the Space Fortress game. We focused on dramatic individual differences in learning and skill acquisition at the individual level by analyzing the archival game data of several interesting players to uncover the specific structure of their acquired skill. Our analysis revealed that even after maximal values for game-generated subscores were reached, the most skilled participant's behaviors such as his flight path, missile firing, and mine handling continued to be refined and improved (Participant 17 from Boot et al., 2010). We contrasted this participant's behavior with the behavior of several other participants and found striking differences in the structure of their performance, which calls into question the appropriateness of averaging their data. For example, some participants engaged in different control strategies such as “world wrapping” or maintaining a finely-tuned circular flight path around the fortress (in contrast to Participant 17′s angular flight path). In light of these differences, we raise fundamental questions about how skill acquisition for individual participants should be studied and described. Our data suggest that a detailed analysis of individuals' data is an essential step for generating a general theory of skill acquisition that explains improvement at the group and individual levels.

Implicit Coordination Strategies for Effective Team Communication.

We investigated implicit communication strategies for anticipatory information sharing during team performance of tasks with varying degrees of complexity. We compared the strategies used by teams with the highest level of performance to those used by the lowest-performing teams to evaluate the frequency and methods of communications used as a function of task structure. High-performing teams share information by anticipating the needs of their teammates rather than explicitly requesting the exchange of information. As the complexity of a task increases to involve more interdependence among teammates, the impact of coordination on team performance also increases. This observation motivated us to conduct a study of anticipatory information sharing as a function of task complexity. We conducted an experiment in which 13 teams of four people performed collaborative search-and-deliver tasks with varying degrees of complexity in a simulation environment. We elaborated upon prior characterizations of communication as implicit versus explicit by dividing implicit communication into two subtypes: (a) deliberative/goal information and (b) reactive status updates. We then characterized relationships between task structure, implicit communication, and team performance. We found that the five teams with the fastest task completion times and lowest idle times exhibited higher rates of deliberative communication versus reactive communication during high-complexity tasks compared with the five teams with the slowest completion times and longest idle times (p = .039). Teams in which members proactively communicated information about their next goal to teammates exhibited improved team performance. The findings from our work can inform the design of communication strategies for team training to improve performance of complex tasks.

Skills, Tasks and the Scarcity of Talent in a Global Economy

AbstractThe scarcity of talent is a tremendous challenge for firms in the globalized world. This paper investigates the role of labor market imperfection in open economies for the usage of talent in the production process of firms. For this purpose, I set up a heterogeneous firms model, where production consists of a continuum of tasks that differ in complexity. Firms hire low‐skilled and high‐skilled workers to perform these tasks. How firms assign workers to tasks depends on factor prices for the two skill types and the productivity advantage of high‐skilled workers in the performance of complex tasks. I study the firms’ assignment problem under two labor market regimes, which capture the polar cases of fully flexible wages and a binding minimum wage for low‐skilled workers. Since the minimum wage lowers the skill premium, it increases the range of tasks performed by high‐skilled workers, which enhances the stock of knowledge within firms to solve complex tasks and reduces the mass of active firms. In a setting with fully flexible wages trade does not affect the firm‐internal assignment of workers to tasks. On the contrary, if low‐skilled wages are fixed by a minimum wage, trade renders high‐skilled workers a scarce resource and reduces the range of tasks performed by this skill type with negative consequences for the human capital stock within firms. In this case, trade leads to higher per‐capita income for both skill types and thus to higher welfare in the open than in the closed economy, whereas – somewhat counter‐intuitive – inequality between the two skill types decreases, as more low‐skilled workers find employment in the production process.

The effect of action observation/execution on mirror neuron system recruitment: an fMRI study in healthy individuals.

Action observation and execution activate regions that are part of the motor and mirror neuron systems (MNS). Using functional magnetic resonance (fMRI), we defined the presence and extent of MNS activation during three different motor tasks with the dominant, right-upper limb in healthy individuals. The influence of the modality of task administration (execution, observation, observation and execution) was also investigated. fMRI scans during the execution (E) of a motor task, the observation (O) of a video showing the same task performed by another person and the simultaneous observation and execution (OE) of the task were obtained from three groups of healthy subjects (15 subjects per group) randomized to perform: a simple motor (SM) task, a complex motor (CM) task and a finalistic motor (FM) task. Manual dexterity was assessed using the 9-hole peg test and maximum finger tapping frequency. MNS activation was higher during FM than SM or CM tasks, independently from the modality of administration (E, O, or OE). Inferior frontal gyrus recruitment was more significant during SM than CM tasks in the E and O conditions. Compared to SM and FM, CM task resulted in increased recruitment of brain regions involved in complex motor task performance. Compared to O and E, OE resulted in the recruitment of additional, specific, brain areas in the cerebellum, temporal and parietal lobes. The modality of administration and the type of task modulated MNS recruitment during motor acts. This might have practical implications for the set-up of individualized motor rehabilitation strategies.

The Impact of Different Environmental Conditions on Cognitive Function: A Focused Review.

Cognitive function defines performance in objective tasks that require conscious mental effort. Extreme environments, namely heat, hypoxia, and cold can all alter human cognitive function due to a variety of psychological and/or biological processes. The aims of this Focused Review were to discuss; (1) the current state of knowledge on the effects of heat, hypoxic and cold stress on cognitive function, (2) the potential mechanisms underpinning these alterations, and (3) plausible interventions that may maintain cognitive function upon exposure to each of these environmental stressors. The available evidence suggests that the effects of heat, hypoxia, and cold stress on cognitive function are both task and severity dependent. Complex tasks are particularly vulnerable to extreme heat stress, whereas both simple and complex task performance appear to be vulnerable at even at moderate altitudes. Cold stress also appears to negatively impact both simple and complex task performance, however, the research in this area is sparse in comparison to heat and hypoxia. In summary, this focused review provides updated knowledge regarding the effects of extreme environmental stressors on cognitive function and their biological underpinnings. Tyrosine supplementation may help individuals maintain cognitive function in very hot, hypoxic, and/or cold conditions. However, more research is needed to clarify these and other postulated interventions.

Condensation Task as an Experimental Model for Studying Individual Differences in Cognitive Control

Successful performance in complex tasks depends upon the functioning of the cognitive control system involving the maintenance of sustained attention, retention and activation of task rules, as well as the inhibition of preliminary responses. Failure of any of these functions can lead to performance errors. In this study, we investigated behavioral data obtained from participants performing the auditory condensation task, which is highly demanding of the level of cognitive control but does not require participants to inhibit or override any prepotent automatic responses. We identified pre-error speeding and error slowing, while post-error slowing was not evident. Our results suggest that there are three factors contributing to the variability within the behavioral measures obtained. The first factor is related to the overall response latency, the second to the main individual mechanism of performance errors, and the third to the subject’s ability to increase motor threshold in the event of uncertainty and choice ambiguity. The data obtained evidence that the auditory condensation task is a promising model for studying cognitive control.

Toward real-time remote processing of laparoscopic video.

Laparoscopic surgery is a minimally invasive surgical technique where surgeons insert a small video camera into the patient's body to visualize internal organs and use small tools to perform surgical procedures. However, the benefit of small incisions has a drawback of limited visualization of subsurface tissues, which can lead to navigational challenges in the delivering of therapy. Image-guided surgery uses the images to map subsurface structures and can reduce the limitations of laparoscopic surgery. One particular laparoscopic camera system of interest is the vision system of the daVinci-Si robotic surgical system (Intuitive Surgical, Sunnyvale, California). The video streams generate approximately 360MB of data per second, demonstrating a trend toward increased data sizes in medicine, primarily due to higher-resolution video cameras and imaging equipment. Processing this data on a bedside PC has become challenging and a high-performance computing (HPC) environment may not always be available at the point of care. To process this data on remote HPC clusters at the typical 30 frames per second (fps) rate, it is required that each 11.9MB video frame be processed by a server and returned within 1/30th of a second. The ability to acquire, process, and visualize data in real time is essential for the performance of complex tasks as well as minimizing risk to the patient. As a result, utilizing high-speed networks to access computing clusters will lead to real-time medical image processing and improve surgical experiences by providing real-time augmented laparoscopic data. We have performed image processing algorithms on a high-definition head phantom video (1920 × 1080 pixels) and transferred the video using a message passing interface. The total transfer time is around 53ms or 19fps. We will optimize and parallelize these algorithms to reduce the total time to 30ms.

Sequential actions as markers of behavioural and cognitive processes: Extracting empirical pathways from data streams of complex tasks

This paper presents a novel exploratory method to extract information about behavioural and cognitive processes that occur during the performance of complex tasks. The methodology is framed in a structure where process data on interactions between the task performer and the task space environment are captured in a data stream for big data analytics.We begin by describing a reductionist approach that involves deconstructing complex processes into constituent parts. We extend this approach by looking at sequences of these constituent parts with the aim of detecting patterns in a person's action pathways. We propose the exploration of these pathways as indicators of underlying behavioural and cognitive processes. Current sequence-based approaches are presented and a proposed approach in dealing with sequences as empirical pathways is demonstrated using data stream from an interactive and collaborative online-administered complex task. These empirical pathways were visualised as directed graphs and an exploratory network analysis is proposed to examine specific features of the emergent paths.General challenges encountered during the development of such a method are described and specific future challenges that we foresee in transitioning from an exploratory phase to a confirmatory phase are examined. Finally, possibilities for further research as well as future applications are discussed.

Interplay of prior knowledge, self‐regulation and motivation in complex multimedia learning environments

AbstractThis study examined the direct and indirect effects of medical clerkship students' prior knowledge, self‐regulation and motivation on learning performance in complex multimedia learning environments. The data from 386 medical clerkship students from six medical schools were analysed using structural equation modeling. The structural model revealed that medical students' prior knowledge directly positively affected their learning outcome, self‐efficacy and performance approach goal orientation. The learners' self‐regulation had a significant positive direct effect on learning outcome. The learners' mastery goal orientation directly affected their learning outcome. Interestingly, inconsistent with our hypothesis, the learners' performance approach goal orientation showed a significant negative direct effect on learning outcome, and performance avoidance goal orientation had a significant positive effect on learning outcome. These findings help develop a more comprehensive understanding of the role of individual characteristics on learning performance of complex tasks in multimedia learning environments.

Circadian Effects on Simple Components of Complex Task Performance

The goal of this study was to advance understanding and prediction of the impact of circadian rhythm on aspects of complex task performance during unexpected automation failures, and subsequent fault management. Participants trained on two tasks: a process control simulation, featuring automated support; and a multi-tasking platform. Participants then completed one task in a very early morning (circadian night) session, and the other during a late afternoon (circadian day) session. Small effects of time of day were seen on simple components of task performance, but impacts on more demanding components, such as those that occur following an automation failure, were muted relative to previous studies where circadian rhythm was compounded with sleep deprivation and fatigue. Circadian low participants engaged in compensatory strategies, rather than passively monitoring the automation. The findings and implications are discussed in the context of a model that includes the effects of sleep and fatigue factors.

Visual Attention Distribution Based Assessment of User's Skill in Electronic Medical Record Navigation

Currently, the most precise way of reflecting the skills level is an expert’s subjective assessment. In this paper we investigate the possibility of the use of eye tracking data for scalar quantitative and objective assessment of medical staff competency in EMR system navigation. According to the experiment conducted by Yarbus the observation process of particular features is associated with thinking. Moreover, eye tracking is proved to clarify the relationships between eye movements, visual attention and insight to the complex task performance. Using the similarities between the reference attention map and the one recorded for the assessed person, as well as the total time of the performance, we have proposed the Visual Attention Time Product (VATP) to distinguish the level of knowledge and familiarity with EMR systems. We used Spearman Rank to compare the obtained VATP scores with the 5-point scale expert’s assessment about the test participants’ performance. The VATP was also compared with the other methods used for classification of the competency test results (attention, similarities and time). Twenty-five people with completed basic training in the EMR usage participated in the proposed experiment. The proposed VATP, calculated with regards to the reference performance of the exemplary expert, shows very strong correlation (0, 92) with the expert’s assessment of the observed performance in solving EMR related tasks. The VATP was also calculated regarding the average performance of all experts who participated in the experiment. Again the results show strong correlation (0, 90). Moreover, the results show that the eye tracking interface does not influence on the EMR navigation. We demonstrate that the eye tracking data and proposed VATP can be successfully used to differentiate the level of performance in EMR system related tasks. It also appeared that the parameter VATP enables the assessment of the examinee comparable to that performed by expert. The level of complexity of the computation allows the gathering of results directly after the conducted test. Thus, the VATP can be successfully used for EMR end user skills differentiation and competency testing.

Hippocampal Insulin Resistance Impairs Spatial Learning and Synaptic Plasticity.

Insulin receptors (IRs) are expressed in discrete neuronal populations in the central nervous system, including the hippocampus. To elucidate the functional role of hippocampal IRs independent of metabolic function, we generated a model of hippocampal-specific insulin resistance using a lentiviral vector expressing an IR antisense sequence (LV-IRAS). LV-IRAS effectively downregulates IR expression in the rat hippocampus without affecting body weight, adiposity, or peripheral glucose homeostasis. Nevertheless, hippocampal neuroplasticity was impaired in LV-IRAS–treated rats. High-frequency stimulation, which evoked robust long-term potentiation (LTP) in brain slices from LV control rats, failed to evoke LTP in LV-IRAS–treated rats. GluN2B subunit levels, as well as the basal level of phosphorylation of GluA1, were reduced in the hippocampus of LV-IRAS rats. Moreover, these deficits in synaptic transmission were associated with impairments in spatial learning. We suggest that alterations in the expression and phosphorylation of glutamate receptor subunits underlie the alterations in LTP and that these changes are responsible for the impairment in hippocampal-dependent learning. Importantly, these learning deficits are strikingly similar to the impairments in complex task performance observed in patients with diabetes, which strengthens the hypothesis that hippocampal insulin resistance is a key mediator of cognitive deficits independent of glycemic control.

Crowding is proportional to visual acuity in young and aging eyes.

Spatial crowding decreases object recognition and conscious visual perception in clutter. In a previous study we used brief presentation times to reveal the effects of a crowded presentation in the fovea. Here we aimed to test the relationships between varying visual acuity (VA) and crowding in the fovea, under the assumption that in uncorrected presbyopia, the processing is relatively normal, whereas the retinal input is blurred. We tested whether normal participants whose near VA is gradually reduced due to age-related deterioration (presbyopia, or "aging eye") will show an acuity-dependent increase in foveal crowding. We used brief presentations and acuity-threshold letter targets in order to magnify the crowding-effect amplitude in the fovea. A total of 195 participants with an age range of 20-68 years and an average of 44.3 ± 11.7 years (M ± SD) were divided into four age groups, all without any optical correction for the near distance. Our findings show that crowding is proportional to VA. This proportionality is affected by VA-age dependency, with a nonlinear S-shaped pattern: A steep VA reduction begins to develop, which is compatible with the normal onset age of presbyopia symptoms and a saturation in the VA-age dependency in the oldest age group, for which we propose a VA-eccentricity account. Finally, there is a high variance in the crowding amplitude in the young, even before the onset age of presbyopia symptoms, suggesting crowding conditions with limited presentation times as a highly sensitive measure of VA, which predicts visual performance in complex tasks, such as reading.

The reliability and validity of the Complex Task Performance Assessment: A performance-based assessment of executive function

The objective of this study was to evaluate the inter-rater reliability, test-retest reliability, concurrent validity, and discriminant validity of the Complex Task Performance Assessment (CTPA): an ecologically valid performance-based assessment of executive function. Community control participants (n = 20) and individuals with mild stroke (n = 14) participated in this study. All participants completed the CTPA and a battery of cognitive assessments at initial testing. The control participants completed the CTPA at two different times one week apart. The intra-class correlation coefficient (ICC) for inter-rater reliability for the total score on the CTPA was .991. The ICCs for all of the sub-scores of the CTPA were also high (.889–.977). The CTPA total score was significantly correlated to Condition 4 of the DKEFS Color-Word Interference Test (p = −.425), and the Wechsler Test of Adult Reading (p = −.493). Finally, there were significant differences between control subjects and individuals with mild stroke on the total score of the CTPA (p = .007) and all sub-scores except interpretation failures and total items incorrect. These results are also consistent with other current executive function performance-based assessments and indicate that the CTPA is a reliable and valid performance-based measure of executive function.

Self-motion perception in autism is compromised by visual noise but integrated optimally across multiple senses

Perceptual processing in autism spectrum disorder (ASD) is marked by superior low-level task performance and inferior complex-task performance. This observation has led to theories of defective integration in ASD of local parts into a global percept. Despite mixed experimental results, this notion maintains widespread influence and has also motivated recent theories of defective multisensory integration in ASD. Impaired ASD performance in tasks involving classic random dot visual motion stimuli, corrupted by noise as a means to manipulate task difficulty, is frequently interpreted to support this notion of global integration deficits. By manipulating task difficulty independently of visual stimulus noise, here we test the hypothesis that heightened sensitivity to noise, rather than integration deficits, may characterize ASD. We found that although perception of visual motion through a cloud of dots was unimpaired without noise, the addition of stimulus noise significantly affected adolescents with ASD, more than controls. Strikingly, individuals with ASD demonstrated intact multisensory (visual-vestibular) integration, even in the presence of noise. Additionally, when vestibular motion was paired with pure visual noise, individuals with ASD demonstrated a different strategy than controls, marked by reduced flexibility. This result could be simulated by using attenuated (less reliable) and inflexible (not experience-dependent) Bayesian priors in ASD. These findings question widespread theories of impaired global and multisensory integration in ASD. Rather, they implicate increased sensitivity to sensory noise and less use of prior knowledge in ASD, suggesting increased reliance on incoming sensory information.

Task analysis of discrete and continuous skills: a dual methodology approach to human skills capture for automation

There is a growing requirement within the field of intelligent automation for a formal methodology to capture and classify explicit and tacit skills deployed by operators during complex task performance. This paper describes the development of a dual methodology approach which recognises the inherent differences between continuous tasks and discrete tasks and which proposes separate methodologies for each. Both methodologies emphasise capturing operators’ physical, perceptual, and cognitive skills, however, they fundamentally differ in their approach. The continuous task analysis recognises the non-arbitrary nature of operation ordering and that identifying suitable cues for subtask is a vital component of the skill. Discrete task analysis is a more traditional, chronologically ordered methodology and is intended to increase the resolution of skill classification and be practical for assessing complex tasks involving multiple unique subtasks through the use of taxonomy of generic actions for physical, perceptual, and cognitive actions.

Meta-T: TetrisⓇ as an experimental paradigm for cognitive skills research.

Studies of human performance in complex tasks using video games are an attractive prospect, but many existing games lack a comprehensive way to modify the game and track performance beyond basic levels of analysis. Meta-T provides experimenters a tool to study behavior in a dynamic task environment with time-stressed decision-making and strong perceptual-motor elements, offering a host of experimental manipulations with a robust and detailed logging system for all user events, system events, and screen objects. Its experimenter-friendly interface provides control over detailed parameters of the task environment without need for programming expertise. Support for eye-tracking and computational cognitive modeling extend the paradigm's scope.

Wearable functional near infrared spectroscopy (fNIRS) and transcranial direct current stimulation (tDCS): expanding vistas for neurocognitive augmentation.

Contemporary studies with transcranial direct current stimulation (tDCS) provide a growing base of evidence for enhancing cognition through the non-invasive delivery of weak electric currents to the brain. The main effect of tDCS is to modulate cortical excitability depending on the polarity of the applied current. However, the underlying mechanism of neuromodulation is not well understood. A new generation of functional near infrared spectroscopy (fNIRS) systems is described that are miniaturized, portable, and include wearable sensors. These developments provide an opportunity to couple fNIRS with tDCS, consistent with a neuroergonomics approach for joint neuroimaging and neurostimulation investigations of cognition in complex tasks and in naturalistic conditions. The effects of tDCS on complex task performance and the use of fNIRS for monitoring cognitive workload during task performance are described. Also explained is how fNIRS + tDCS can be used simultaneously for assessing spatial working memory. Mobile optical brain imaging is a promising neuroimaging tool that has the potential to complement tDCS for realistic applications in natural settings.

Working alone or in the presence of others: exploring social facilitation in baggage X-ray security screening tasks

This study investigated whether the mere presence of a human audience would evoke a social facilitation effect in baggage X-ray security screening tasks. A 2 (target presence: present vs. absent) × 2 (task complexity: simple vs. complex) × 2 (social presence: alone vs. human audience) within-subject experiment simulating a real baggage screening task was conducted. This experiment included 20 male participants. The participants' search performance in this task was recorded. The results showed that the presence of a human audience speeded up responses in simple tasks and slowed down responses in complex tasks. However, the social facilitation effect produced by the presence of a human audience had no effect on response accuracy. These findings suggested that the complexity of screening tasks should be considered when designing work organisation modes for security screening tasks.Practitioner summary: This study investigated whether the presence of a human audience could evoke a social facilitation effect in baggage X-ray security screening tasks. An experimental simulation was conducted. The results showed that the presence of a human audience facilitated the search performance of simple tasks and inhibited the performance of complex tasks.

Automation Complacency

Automation complacency has been considered by some to result from a state of mindlessness. An attentional framework offered by Parasuraman and Manzey (2010) has suggested that automation bias and complacency may be overlapping functions of the same central cognitive process, with attentional resources as the shared mechanism. Non-invasive brain stimulation methods present novel ways to manipulate attentional resources and influence operator allocation of attention. Study results indicate that positive stimulation affects complex task performance, although not necessarily evenly across task function. Stimulation effects on complacency potential may be modulated by an operator’s automation bias. This has broad implications for interface design, as well cognitive theory of human-automation interaction.