Cognitive Workload Assessment Research Articles

Commentator Discussion: A novel, multimodal, intraoperative cognitive workload assessment of cardiac surgery team members

Commentator Discussion: A novel, multimodal, intraoperative cognitive workload assessment of cardiac surgery team members

Assessment of cognitive workload using simultaneous EEG and fNIRS: A comparison of feature combinations

The assessment of cognitive workload is a critical component in evaluating mental activity. It is essential in psychology, especially in professions with heightened cognitive demands, such as aviation and surgery. Even marginal improvements in workload assessment can substantially enhance safety and overall performance. This study aimed to improve the accuracy of cognitive workload assessment by employing a combined EEG+fNIRS approach with a minimal number of features. The Tunable Q-factor Wavelet Transform (TQWT) was applied to signals from EEG and fNIRS to extract relevant features. TQWT was chosen for its ability to adapt to the oscillatory behavior of the signal, and the features extracted from the transform facilitated the differentiation between different signal classes. This study introduced six proposed approaches, with the first focusing on EEG signals, the second on fNIRS signals, and the subsequent three to six approaches involving combinations of these two signals. Features evaluation highlighted the pivotal role of channels related to the frontal region and sub-bands associated with alpha and theta rhythms in EEG signals. Additionally, the very low-frequency range and the primary undecomposed signal in fNIRS signals contributed significantly to cognitive workload classification. The classification results demonstrated the efficacy of combining EEG and fNIRS signals in cognitive workload detection, providing higher accuracy than single-signal approaches. Compared to other feature combination approaches, the proposed approach based on Canonical Correlation Analysis (CCA) with a linear SVM classifier achieved superior accuracy across all investigated cognitive workload levels, with accuracies of 99.6 %, 99.4 %, 98.9 %, and 97.5 % for the 0-back vs. 3-back, 0-back vs. 2-back, 2-back vs. 3-back, and 0-back vs. 2-back vs. 3-back comparisons, respectively.

Exploring Cognitive Workload Changes in Teleoperators: Collaboration with On-Site Workers and Robots in Assembly Tasks

Teleoperation is commonly employed to perform industrial tasks in remote or inaccessible areas. However, there is a noticeable gap in evaluating cognitive workload of teleoperators in collaboration environments. This study compared the variation of cognitive workload for teleoperators guiding an on-site participant to complete a wire assembly task in two scenarios: one aided by a robot arm (tRH) and another without any robot assistance (HH). Additionally, the task demands for on-site participants were manipulated to measure its impact on teleoperator’s workload. NASA-TLX and EEG activity were utilized to assess workload. The results indicated that EEG theta activity was significantly higher for the HH group than tRH group, potentially showing lower workload for teleoperators in the scenario with robot assistance. Task difficulty did not affect any of the workload measures. The study highlights the importance of cognitive workload assessment in human-robot collaborations to optimize human cognitive demands in complex settings.

A novel multimodal, intraoperative cognitive workload assessment of cardiac surgery team members

ObjectiveTo characterize cognitive workload (CWL) of cardiac surgery team members in a real-world setting during coronary artery bypass grafting (CABG) surgery using providers’ heart rate variability (HRV) data as a surrogate measure of CWL. MethodsHRV was collected from the surgeon, anesthesiologist, perfusionist, and scrub nurse, and audio/video recordings were made during isolated, nonemergency CABG surgeries (n = 27). Eight surgical phases were annotated by trained researchers, and HRV was calculated for each phase. ResultsSignificant differences in CWL were observed within a given role across surgical phases. Results are reported as predicted probability (95% confidence interval [CI]). CWL was significantly higher for anesthesiologists during “preparation and induction” (0.57; 95% CI, 0.42-0.71) and “anastomoses” (0.44; 95% CI, 0.30-0.58) compared to other phases, and the same held for nurses during the “opening” (0.51; 95% CI, 0.37-0.65) and “postoperative” (0.68; 95% CI, 0.42-0.86) phases. Additional significant differences were observed between roles within a given surgical phase. For example, surgeons had significantly higher CWL during “anastomoses” (0.81; 95% CI, 0.69-0.89) compared to all other phases, and the same was true of perfusionists during the “opening” (0.79; 95% CI, 0.66-0.88) and “prebypass preparation” (0.50; 95% CI, 0.36-0.64) phases. ConclusionsOur innovative analysis demonstrates that CWL fluctuates across surgical procedures by role and phase, which may reflect the distribution of primary tasks. This corroborates earlier findings from self-report measures. The data suggest that team-wide, peak CWL during a phase decreases from early phases of surgery through initiation of cardiopumonary bypass (CPB), rises during anastomosis, and decreases after termination of CPB. Knowledge of these trends could encourage the adoption of behaviors to enhance team dynamics and performance.

Objective Assessment of Cognitive Workload in Surgery

Objective Assessment of Cognitive Workload in Surgery

Comparison of cognitive workload and surgical outcomes between a three-dimensional and conventional microscope macular hole surgery

BackgroundPerforming a surgical task subjects the surgeon to multitudinal stressors, especially with the newer 3D technology. The quantum of cognitive workload using this modern surgical system in comparison to the Conventional microscope system remains unexplored. We evaluate the surgeon’s cognitive workload and the surgical outcomes of macular hole(MH) surgery performed on a 3D versus a Conventional microscope operating system.Methods50 eyes of 50 patients with MH undergoing surgery using the 3D or Conventional microscope visualization system. Cognitive workload assessment was done by real-time tools(Surgeons’ heart rate [HR] and oxygen saturation[SPO2]) and self-report tool(Surgery Task Load Index[SURG-TLX] questionnaire) of three Vitreoretinal surgeons. Based on the SURG-TLX questionnaire, an assessment of the workload was performed.ResultsOf the 50 eyes, 30 eyes and 20 eyes underwent surgery with the Conventional microscope and the 3D system, respectively. No difference was noted in the MH basal-diameter(p = 0.128), total surgical-duration(p = 0.299), internal-limiting membrane(ILM) peel time(p = 0.682), and the final visual acuity (VA; p = 0.515) between the two groups. Both groups showed significant improvement in VA(p < 0.001) with a 90% closure rate at one-month post-surgery. Cognitive workload comparison, the intraoperative HR(p = 0.024), total workload score(P = 0.005), and temporal-demand dimension(p = 0.004) were significantly more in Conventional microscope group as compared to 3D group. In both the groups, the HR increased significantly from the baseline while performing ILM peeling and at the end.ConclusionThe surgeon’s cognitive workload is markedly reduced while performing macular hole surgery with a 3D viewing system. Moreover, duration of surgery including ILM peel time, MH closure rates, and visual outcomes remains unaffected irrespective of the operating microscope system.

Adopting Stimulus Detection Tasks for Cognitive Workload Assessment: Some Considerations.

This article tackles the issue of correct data interpretation when using stimulus detection tasks for determining the operator's workload. Stimulus detection tasks are a relative simple and inexpensive means of measuring the operator's state. While stimulus detection tasks may be better geared to measure conditions of high workload, adopting this approach for the assessment of low workload may be more problematic. This mini-review details the use of common stimulus detection tasks and their contributions to the Human Factors practice. It also borrows from the conceptual framework of the inverted-U shape model to discuss the issue of data interpretation. The evidence being discussed here highlights a clear limitation of stimulus detection task paradigms. There is an inherent risk in using a unidimensional tool like stimulus detection tasks as the primary source of information for determining the operator's psychophysiological state. Two recommendations are put forward to Human Factors researchers and practitioners dealing with the interpretation conundrum of dealing with stimulus detection tasks.

Safe physical interaction with cobots: a multi-modal fusion approach for health monitoring.

Health monitoring is a critical aspect of personalized healthcare, enabling early detection, and intervention for various medical conditions. The emergence of cloud-based robot-assisted systems has opened new possibilities for efficient and remote health monitoring. In this paper, we present a Transformer-based Multi-modal Fusion approach for health monitoring, focusing on the effects of cognitive workload, assessment of cognitive workload in human-machine collaboration, and acceptability in human-machine interactions. Additionally, we investigate biomechanical strain measurement and evaluation, utilizing wearable devices to assess biomechanical risks in working environments. Furthermore, we study muscle fatigue assessment during collaborative tasks and propose methods for improving safe physical interaction with cobots. Our approach integrates multi-modal data, including visual, audio, and sensor- based inputs, enabling a holistic assessment of an individual's health status. The core of our method lies in leveraging the powerful Transformer model, known for its ability to capture complex relationships in sequential data. Through effective fusion and representation learning, our approach extracts meaningful features for accurate health monitoring. Experimental results on diverse datasets demonstrate the superiority of our Transformer-based multi- modal fusion approach, outperforming existing methods in capturing intricate patterns and predicting health conditions. The significance of our research lies in revolutionizing remote health monitoring, providing more accurate, and personalized healthcare services.

Multimodal Assessment of Cognitive Workload Using Neural, Subjective and Behavioural Measures in Smart Factory Settings.

Collaborative robots (cobots) have largely replaced conventional industrial robots in today's workplaces, particularly in manufacturing setups, due to their improved performance and intelligent design. In the framework of Industry 5.0, humans are working alongside cobots to accomplish the required level of automation. However, human-robot interaction has brought up concerns regarding human factors (HF) and ergonomics. A human worker may experience cognitive stress as a result of cobots' irresponsive nature in unpredictably occurring situations, which adversely affects productivity. Therefore, there is a necessity to measure stress to enhance a human worker's performance in a human-robot collaborative environment. In this study, factory workers' mental workload was assessed using physiological, behavioural, and subjective measures. Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals were collected to acquire brain signals and track hemodynamic activity, respectively. The effect of task complexity, cobot movement speed, and cobot payload capacity on the mental stress of a human worker were observed for a task designed in the context of a smart factory. Task complexity and cobot speed proved to be more impactful. As physiological measures are unbiased and more authentic means to estimate stress, eventually they may replace the other conventional measures if they prove to correlate with the results of traditional ones. Here, regression and artificial neural networks (ANN) were utilised to determine the correlation between physiological data and subjective and behavioural measures. Regression performed better for most of the targets and the best correlation (rsq-adj = 0.654146) was achieved for predicting missed beeps, a behavioural measure, using a combination of multiple EEG and fNIRS predictors. The k-nearest neighbours (KNN) algorithm was used to evaluate the accuracy of correlation between traditional measures and physiological variables, with the highest accuracy of 77.8% achieved for missed beeps as the target. Results show that physiological measures can be more insightful and have the tendency to replace other biased parameters.

In-Ear SpO₂ for Classification of Cognitive Workload

Classification of cognitive workload promises immense benefit in diverse areas ranging from driver safety to augmenting human capability through closed loop brain computer interface. The brain is the most metabolically active organ in the body and increases its metabolic activity and thus oxygen consumption with increasing cognitive demand. In this study, we explore the feasibility of in-ear SpO2 cognitive workload tracking. To this end, we preform cognitive workload assessment in 8 subjects, based on an N-back task, whereby the subjects are asked to count and remember the number of odd numbers displayed on a screen in 5 second windows. The 2 and 3-back tasks lead to either the lowest median absolute SpO2 or largest median decrease in SpO2 in all of the subjects, indicating a robust and measurable decrease in blood oxygen in response to increased cognitive workload. Using features derived from in-ear pulse oximetry, including SpO2, pulse rate and respiration rate, we were able to classify the 4 N-back task categories, over 5 second epochs, with a mean accuracy of 94.2%. Moreover, out of 21 total features, the 9 most important features for classification accuracy were all SpO2 related features. The findings suggest that in-ear SpO2 measurements provide valuable information for classification of cognitive workload over short time windows, which together with the small form factor promises a new avenue for real time cognitive workload tracking.

Utilizing Grasp Monitoring to Predict Microsurgical Expertise

IntroductionMost microsurgical procedures require the surgeon to use tools to grasp and hold fragile objects in the surgical site. Prior research on grasping in surgery has mostly either been in other surgical techniques or used grasping as an auxiliary metric. We focus on microsurgery and investigate what grasping can tell about microsurgical skill and suturing performance. This study lays groundwork for using automatic detection of grasps to evaluate surgical skill. MethodsFive expert surgeons and six novices completed sutures on a microsurgical training board. Video recordings of the performance were annotated for the number of grasps, while an eye tracker recorded the participants’ pupil dilations for cognitive workload assessment. Performance was measured with suturing duration and the University of Western Ontario Microsurgical Skills Assessment instrument (UWOMSA). Differences in skill, suturing performance and cognitive workload were compared with grasping behavior. ResultsNovices needed significantly more grasps to complete sutures and failed to grasp more often than the experts. The number of grasps affected the suturing duration more in novices. Decreasing suturing efficiency as measured by UWOMSA instrument was associated with increase in grasps, even when we controlled for overall skill differences. Novices displayed larger pupil dilations when averaged over a sufficiently large sample, and the difference increased after the grasp. ConclusionsGrasping action during microsurgical procedures can be used as a conceptually simple yet objective proxy in microsurgical performance assessment. If the grasps could be detected automatically, they could be used to aid in computational evaluation of surgical trainees’ performance.

Mask-wearing: effects on cognitive task performance and cardiac workload indices

Consistent with CDC recommendations and government mandates during COVID, masks are commonly required/suggested in public gathering places like workplaces and schools. Considering the implications for human factors research, the extent to which mask-wearing impacts cognitive abilities and/or workload assessment from cardiac indicators are unknown. This study investigated these effects by engaging subjects in cognitively-loading tasks (math or memory) while not wearing masks, and additionally by wearing one of three types of masks (surgical, N95, and cloth). Results suggest that mask-wearing does not have significant impact on task performance data (accuracy, total question answered) or cardiac indices (heart rate). This study can foster a deeper understanding of the influence of mask-wearing on actual and perceived cognitive workload, and how mask-wearing may mitigate common workload assessment methods. These findings are valuable for informing human subjects research during COVID, and in identifying implications of mask-wearing on domains that impose cognitive work demands.

The Measurement of Cognitive Workload in Surgery Using Pupil Metrics: A Systematic Review and Narrative Analysis

IntroductionIncreased cognitive workload (CWL) is a well-established entity that can impair surgical performance and increase the likelihood of surgical error. The use of pupil and gaze tracking data is increasingly being used to measure CWL objectively in surgery. The aim of this review is to summarize and synthesize the existing evidence that surrounds this. MethodsA systematic review was undertaken in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. A search of OVID MEDLINE, IEEE Xplore, Web of Science, Google Scholar, APA PsychINFO, and EMBASE was conducted for articles published in English between 1990 and January 2021. In total, 6791 articles were screened and 32 full-text articles were selected based on the inclusion criteria. A narrative analysis was undertaken in view of the heterogeneity of studies. ResultsSeventy-eight percent of selected studies were deemed high quality. The most frequent surgical environment and task studied was surgical simulation (75%) and performance of laparoscopic skills (56%) respectively. The results demonstrated that the current literature can be broadly categorized into pupil, blink, and gaze metrics used in the assessment of CWL. These can be further categorized according to their use in the context of CWL: (1) direct measurement of CWL (n = 16), (2) determination of expertise level (n = 14), and (3) predictors of performance (n = 2). ConclusionsEye-tracking data provide a wealth of information; however, there is marked study heterogeneity. Pupil diameter and gaze entropy demonstrate promise in CWL assessment. Future work will entail the use of artificial intelligence in the form of deep learning and the use of a multisensor platform to accurately measure CWL.

Anthropometrics and body composition predict performance during a simulated direct-fire engagement

ABSTRACT This study determined anthropometric and body composition predictors of performance during a simulated direct-fire engagement. Healthy subjects (N = 33, age =25.7 ± 7.0yr) underwent anthropometric and body composition assessments before completing a simulated direct-fire engagement – consisting of marksmanship with cognitive workload assessment and a fire-and-move drill (16x6-m sprints) while wearing combat load. Susceptibility to enemy fire was modeled on sprint duration. Partial correlations and multiple linear regressions established relationships between predictors and performance outcomes, controlling for age and sex. Significance was p ≤ 0.05. Higher percent body fat, fat mass, fight load index predicted greater susceptibility to enemy fire (r = 0.40 to 0.42) and lower cognitive performance (r=-0.45 to -0.49). Higher BMI predicted lower cognitive performance (r=-0.49). Shorter stature/hand length predicted higher marksmanship accuracy (r=-0.40), while higher fat-free mass/fat-free mass index predicted slower reaction times (r = 0.36 to 0.41). These data suggest anthropometric and body composition measures modulate combat effectiveness and reinforce body composition standards in military organizations.

Cognitive Workload Assessment of Prosthetic Devices: A Review of Literature and Meta-Analysis

Limb amputation can cause severe functional disability for the performance of activities of daily living. Previous studies have found differences in cognitive demands imposed by prosthetic devices due to variations in their design. The objectives of this article were to 1) identify the range of cognitive workload (CW) assessment techniques used in prior studies comparing different prosthetic devices, 2) identify the device configurations or features that reduced CW of users, and 3) provide guidelines for designing future prosthetic devices to reduce CW. A literature search was conducted using Compendex, Inspec, Web of Science, Proquest, IEEE, Engineering Research Database, PubMed, Cochrane, and Google Scholar. Forty-three studies met the inclusion criteria. Findings suggested that CW of prosthetic devices was assessed using physiological, task performance, and subjective measures. However, due to the limitations of these methods, there is a need for more theoretical and model-based approaches to quantify CW. Device configurations such as hybrid input signals and use of multimodal feedback can reduce CW of prosthetic devices. Furthermore, to evaluate the effectiveness of a training strategy for reducing CW and improving device usability, both task performance and subjective measures should be considered. Based on the literature review, a set of guidelines was provided to improve the usability of future prosthetic devices and reduce CW.

A neurotechnological aid for semi-autonomous suction in robotic-assisted surgery

Adoption of robotic-assisted surgery has steadily increased as it improves the surgeon’s dexterity and visualization. Despite these advantages, the success of a robotic procedure is highly dependent on the availability of a proficient surgical assistant that can collaborate with the surgeon. With the introduction of novel medical devices, the surgeon has taken over some of the surgical assistant’s tasks to increase their independence. This, however, has also resulted in surgeons experiencing higher levels of cognitive demands that can lead to reduced performance. In this work, we proposed a neurotechnology-based semi-autonomous assistant to release the main surgeon of the additional cognitive demands of a critical support task: blood suction. To create a more synergistic collaboration between the surgeon and the robotic assistant, a real-time cognitive workload assessment system based on EEG signals and eye-tracking was introduced. A computational experiment demonstrates that cognitive workload can be effectively detected with an 80% accuracy. Then, we show how the surgical performance can be improved by using the neurotechnological autonomous assistant as a close feedback loop to prevent states of high cognitive demands. Our findings highlight the potential of utilizing real-time cognitive workload assessments to improve the collaboration between an autonomous algorithm and the surgeon.

Lower-body muscular power and exercise tolerance predict susceptibility to enemy fire during a tactical combat movement simulation

This study examined if field-expedient physical fitness/performance assessments predicted performance during a simulated direct-fire engagement. Healthy subjects (n = 33, age = 25.7 ± 7.0 years) completed upper- and lower-body strength and power assessments and a 3-min all-out running test to determine critical velocity. Subjects completed a simulated direct-fire engagement that consisted of marksmanship with cognitive workload assessment and a fire-and-move drill (16 × 6-m sprints) while wearing a combat load. Susceptibility to enemy fire was modelled on average sprint duration during the fire-and-move drill. Stepwise linear regression identified predictors for the performance during the simulated direct-fire engagement. Critical velocity (β = −0.30, p < 0.01) and standing broad jump (β = −0.67, p < 0.001) predicted susceptibility to enemy fire (R 2 = 0.74, p < 0.001). All predictors demonstrated poor relationships with marksmanship accuracy and cognitive performance. These data demonstrate the importance of exercise tolerance and lower-body power during simulated direct-fire engagements and provide potential targets for interventions to monitor and enhance performance and support soldier survivability. Practitioner Summary: This study identified field-expedient physical fitness/performance predictors of a simulated direct-fire engagement which evaluated susceptibility to enemy fire, marksmanship, and cognitive performance. Our findings suggest that high-intensity exercise tolerance and lower-body power are key determinants of performance that predicted susceptibility to enemy fire.

An Assessment of the Driver's Cognitive Workload Associated with Various Cyclist Accommodations in Work Zones

An Assessment of the Driver's Cognitive Workload Associated with Various Cyclist Accommodations in Work Zones

Classification of cross task cognitive workload using deep recurrent network with modelling of temporal dynamics

Cognitive workload assessment is important for optimizing multitasking abilities in the current information-rich world. A few good research studies assess cross task classification where a model trained for one task can efficiently handle other tasks. Building a reliable model for the estimation of cross task cognitive workload using electroencephalogram (EEG) signal is challenging as brain dynamics differ for various activities. The present study explores the wide visual aspects for inducing two levels of cognitive workload in the two self-design tasks. The present study addresses temporal dynamics of EEG signal using time windowing, time segment smoothing, and formation of variable duration time frames. The present study computes features from the statistical, morphological and non-linear domain for variable duration segments considering four clinical sub-bands. The neighbourhood component feature selection algorithm identifies the topmost common features of both tasks. Binary classification of cross task utilizes deep structure employing bidirectional long short-term memory (BLSTM) and LSTM. The result shows that the deep recurrent neural network (RNN) achieves the highest classification accuracy of 92.8% for the cross task in comparison to the traditional classifiers. The deep RNN also outperforms the existing literature studies for cross task classification. Result also reveals a significant improvement in classification accuracy with the usage of handcrafted features when supported by proper modelling of temporal dynamics of EEG signal. The present study exploits the capability of deep recurrent structure in handling long term dependencies of non-stationary signal and proves to be very efficient.

Cognitive and metabolic workload assessment techniques: A review in automotive manufacturing context

AbstractErgonomics assessment in the automotive industry has, to date, focused mainly on physical ergonomics, for example, manual handling and posture. However, workload and, in particular, metabolic and cognitive workload, contributes to worker efficiency but has not received sufficient attention to yield practical guidance for industry. Successful workload assessment requires in‐depth understanding of the context in which it will be conducted and of the various assessment techniques which will be applied, with consideration given to factors such as feasibility, resources, and skill of the assessor. These requirements are met with challenges within large and complex organizations and are often dealt with in a piecemeal and isolated matter (i.e., reactive workload assessment). The present paper explores these challenges within the automotive manufacturing industry and aims to develop a decision matrix to guide effective selection of workload assessment techniques focused on metabolic and cognitive demands. It also presents the requirements for time, equipment, and knowledge to implement these techniques as part of a participatory ergonomics approach. Early findings suggest that most assessment techniques reviewed require further development, for example, to establish the acceptance criteria for the specific workload scenario. However, five methods (Garg, Borg RPE, IPAQ, SWAT, and NASA‐TLX) are ready to use in certain applications. Ultimately, the findings suggest that it is possible to implement a participatory workload evaluation program within large and complex manufacturing plants.