Cognitive Task Analysis Research Articles

Development and optimisation of a mobile app (iMPAKT) for improving person-centred practice in healthcare settings: A multi-methods evaluation study.

To develop and optimise an app (iMPAKT) for improving implementation and measurement of person-centred practice in healthcare settings. Two iterative rounds of testing were carried out based on cognitive task analysis and qualitative interview methods. The System Usability Scale (SUS) was also used to evaluate the app. Quantitative data on task completion and SUS scores were evaluated descriptively, with thematic analysis performed on qualitative data. The MoSCoW prioritisation system was used to identify key modifications to improve the app. Twelve participants took part (eight health professionals and four patient and public involvement representatives). Views on design and structure of the app were positive. The majority of the 16 tasks undertaken during the cognitive task analysis were easy to complete. Mean SUS scores were 73.5/100 (SD: 7.9; range = 60-92.5), suggesting good overall usability. For one section of the app that transcribes patients speaking about their experience of care, a non-intuitive user interface and lack of transcription accuracy were identified as key issues influencing usability and acceptability. Findings from the evaluation were used to inform iterative modifications to further develop and optimise the iMPAKT App. These included improved navigational flow, and implementation of an updated artificial intelligence (AI) based Speech-To-Text software; allowing for more accurate, real-time transcription. Use of such AI-based software represents an interesting area that requires further evaluation. This is particularly apparent in relation to potential for large-scale collection of data on person-centred measures using the iMPAKT App, and for assessing initiatives designed to improve patient experience.

Cognitive task analysis of clinicians’ drug–drug interaction management during patient care and implications for alert design

BackgroundDrug–drug interactions (DDIs) are common and can result in patient harm. Electronic health records warn clinicians about DDIs via alerts, but the clinical decision support they provide is inadequate. Little...

Using the Allen Cognitive Levels to determine level of supervision needed for those with mild to moderate cognitive decline

AbstractBackgroundA comparison was made of the common screening tools for Alzheimer’s disease (AD).MethodThe Mini‐Mental State Examination (MMSE) is one of the most commonly used screening tools but has an emphasis on verbal skills and because it is so widely used, some patients with minimal cognitive decline are able to remember many of the answers from one assessment to the next. The benefits of using the Allen Cognitive Levels (ACL) became apparent as it is a functional based assessment and can be administered without the patient having to say anything.ResultUsing the Allen Cognitive Level screening tool as well as the ACL task activities to narrow a patients level to a number between 0‐6 to the nearest .2 of a point. For example, if a patient scores between 4.6‐4.8, it is recommended that the patient may live alone in the community, provided they have a safe environment with well‐established routines and rehearsed safety plans including daily and weekly supervision. Specific recommendations are available to educate family on safety equipment needs, and executive functioning strategies. For example in the 4.6‐4.8 level, verbal skills are often stronger than a patient’s ability to complete novel tasks, something that is not as easily picked up on a verbal or memory related screener.ConclusionThe Allen Cognitive Level screener and task analysis kits are very helpful for healthcare providers including doctors or therapists to obtain specific information about a clients current cognitive level and what can be expected and how to modify the environment for the client safety and greatest level of independence. Because the levels are structured sequentially, caregivers can compare the current cognitive level to the subsequent level in order to know what to watch out for in the future and clinicians can complete the lacing screener or the task activities as the disease progresses to provide the most accurate information about a clients current cognitive level.

Cognitive Task Analysis of Escharotomy.

With the Army's emerging doctrine of prolonged field care, and with burns being a common injury among soldiers, non-expert providers must be trained to perform escharotomy when indicated. However, the existing physical simulators and training protocols are not sufficient for training non-experts for performing effective escharotomy. Hence, to provide guidance in developing realistic escharotomy simulators and effective training protocols suitable for prolonged field care, a cognitive task analysis (CTA) is needed. This work aims to obtain educative information from expert burn surgeons regarding escharotomy procedures via the CTA. The CTA was done by interviewing five subject matter experts with experience in performing escharotomy ranging from 20 to over 100 procedures and analyzing their responses. Interview questions were developed to obtain educative information from expert burn surgeons regarding the escharotomy procedure. A "gold standard protocol" was developed based on the CTA of each of the subject matter experts. The CTA helped identify general themes, including objectives, conditions that mandate escharotomy, signs of successful escharotomy, precautions, challenges, decisions, and performance standards, and specific learning goals such as the use of equipment, vital signs, performing the procedure, and preoperative and postoperative care. A unique aspect of this CTA is that it identifies the background information and preparations that could be useful to the practitioners at various levels of expertise. The CTA enabled us to compile a "gold standard protocol" for escharotomy that may serve as a guide for practitioners at various levels of expertise. This information will provide a framework for escharotomy training systems and simulators.

Developing and evaluating an interprofessional shared decision-making care model for patients with perinatal depression in maternal care in urban China: a study protocol

BackgroundThe majority of patients with perinatal depression (PND) in China do not receive adequate treatment. As forming a therapeutic alliance with patients is crucial for depression treatment, shared decision-making (SDM) shows promise in promoting patients’ uptake of evidence-based mental health services, but its impact on patient outcomes and implementation in real-world maternal care remain uncertain. Therefore, this study aims to develop and evaluate an interprofessional shared decision-making (IP-SDM) model for PND to enhance maternal mental health services.MethodsThis study contains four research phases: feasibility testing (Phase 1), toolkit development (Phase 2), usability evaluation (Phase 3), and effectiveness evaluation (Phase 4). During the development stage, focus group interviews will be conducted with expectant and new mothers, as well as maternal care providers for feasibility testing. A toolkit, including a patient decision aid along with its user guide and training materials, will be developed based on the findings of Phase 1 and syntheses of up-to-date evidence and appraised by the Delphi method. Additionally, a cognitive task analysis will be used for assessing the usability of the toolkit. During the evaluation stage, a prospective randomized controlled trial embedded in a mixed methods design will be used to evaluate the effectiveness and cost-effectiveness of the IP-SDM care model. The study targets to recruit 410 expectant and new mothers who screen positive for depression. They will be randomly assigned to either an intervention group or a control group in a 1:1 ratio. Participants in the intervention group will receive decision aid, decision coaching, and clinical consultation, in addition to usual services, while the control group will receive usual services. The primary outcome is the quality of decision-making process, and the secondary outcomes include SDM, mental health service utilization and costs, depressive symptoms, and health-related quality of life. In-depth interviews will be used to explore the facilitating and hindering factors of SDM.DiscussionThis study will develop an IP-SDM care model for PND that can be implemented in maternal care settings in China. This study will contribute to the understanding of how SDM impacts mental health outcomes and facilitate the integration of mental health services into maternal care.Trial registrationChiCTR2300072559. Registered on 16 June 2023.

Estimating Cognitive Load in a Mobile Personal Health Record Application: A Cognitive Task Analysis Approach.

Mobile health applications that are designed without considering usability criteria can lead to cognitive overload, resulting in the rejection of these apps. To avoid this problem, the user interface of mobile health applications should be evaluated for cognitive load. This evaluation can contribute to the improvement of the user interface and help prevent cognitive overload for the user. In this study, we evaluated a mobile personal health records application using the cognitive task analysis method, specifically the goals, operators, methods, and selection rules (GOMS) approach, along with the related updated GOMS model and gesture-level model techniques. The GOMS method allowed us to determine the steps of the tasks and categorize them as physical or cognitive tasks. We then estimated the completion times of these tasks using the updated GOMS model and gesture-level model. All 10 identified tasks were split into 398 steps consisting of mental and physical operators. The time to complete all the tasks was 5.70 minutes and 5.45 minutes according to the updated GOMS model and gesture-level model, respectively. Mental operators covered 73% of the total fulfillment time of the tasks according to the updated GOMS model and 76% according to the gesture-level model. The inter-rater reliability analysis yielded an average of 0.80, indicating good reliability for the evaluation method. The majority of the task execution times comprised mental operators, suggesting that the cognitive load on users is high. To enhance the application's implementation, the number of mental operators should be reduced.

A cognitive task analysis of the teacher skills and knowledge required for differentiated instruction in secondary education

Providing differentiated instruction (DI) is a complex teacher task that many secondary school teachers do not master well. In the current study, a cognitive task analysis of this teacher task was conducted by analyzing how expert teachers do this and why, resulting in an inventory of the necessary teacher skills and knowledge for providing DI, and a description of the factors that influence the complexity of DI. The results of this analysis show what providing DI in secondary education entails, which is valuable for designing teacher professional development programs for DI at that level.

P076 Comparing Four Different Simulated Shiftwork Schedules on Higher order Cognitive Function

Abstract Introduction Changes in cognitive performance during shiftwork due to fatigue and circadian misalignment are well-documented. However experimental evaluations of shiftwork schedules are often limited to basic tasks such as vigilance and attention. The current study evaluated several higher order cognitive functions such as memory and task switching during four simulated shiftwork schedules. Methods N=27 completed a 10-day laboratory shiftwork simulation. They were randomly assigned to one of four shiftwork conditions: Condition A, 4h-on/4h-off/8h-on/8h-off schedule; Condition B, 6h-on/6h-off schedule; Conditions C and D represented to different sections of a 4h-on/4h-on call/4h-off/4h-on/4h-on call/4h-off schedule. The Automated Neuropsychological Assessment Metrics (ANAM) was used to assess cognitive function. The tasks included in this analysis were Code Substitution (CDS/CDD), Pursuit Tracking (PUR), Matching to Sample (M2S), and Switching Task (SWT). Stability of performance during waking was compared between the conditions using Kruskal-Wallis tests. Results The analysis of cognitive performance tasks yielded mixed findings. Performance on all tasks was ranked from best to worst: Condition D (>Conditions A, B, C; all p < .05), Condition A (>Conditions B, C; p < .05 and p <.001, respectively), Condition B, Condition C. Regarding stability of higher order cognitive function across the waking day, Condition A exhibited the worst stability, and was the only condition whereby performance degraded across the wake period. Discussion These findings highlight the variability in cognitive capacities during different shiftwork schedules. It is important to examine higher order cognitive function, and other cognitive abilities during shiftwork, when evaluating different shiftwork schedules.

Assessing the training effect of knee arthroscopy cognitive-task surgical simulations on the theoretical and technical knowledge acquisition in novices

Background The COVID-19 pandemic had drastic consequences on medical education, especially in surgical exposure and procedural learning. Cognitive task analysis (CTA) simulators, including Touch Surgery, are gaining prominence as a learning tool for surgical training in a low-risk environment virtually. The study objective was to investigate the training effect of Touch Surgery’s knee arthroscopy module in both theoretical and technical knowledge acquisition for novices, and qualitative feedback on CTA simulators for surgical education. Methods Thirteen medical students were recruited in an Institutional Review Board-approved study to complete the knee arthroscopy simulation module on Touch Surgery, a clinically validated, cognitive task mobile-app based surgical simulator. Participants were given an expert-validated questionnaire evaluating surgical theory and technique on knee arthroscopy before and after at least three consecutive attempts on the module, along with qualitative questions assessing attitudes towards using the platform. A 2-tailed paired Student t-test was used to assess for statistically significant changes between pre- and post-study scores, with p-value < 0.05 considered significant. Results Mean scores for pre- and post-simulator use demonstrated statistically significant improvements for technical and overall knowledge acquisition (p < 0.05). Improvement in theoretical knowledge acquisition was also observed, but the difference was not statistically significant (p = 0.052). Participants generally found the app “easy to use”, “fun to use” and “more interactive compared to other study material”. Conclusion CTA simulators provide an effective means of technical knowledge acquisition for novices such as medical students, and serve as an engaging and useful training adjunct for surgical education.

SAPERO: a new tool for safety analyses in advanced radiotherapy

Improving safety standards in advanced radiotherapy technologies, where historical experience is not sufficient due to innovation aspects, raises the need to perform studies using new tools that follow a holistic view of the process chain. Such studies are useful to identify additional critical elements compared to safety analyses carried out by classic tools. In this field, SAPERO (La SicurezzA del PazientE: tecniche avanzate ed innovative per la valutazione del rischio di eventi indesiderati all'interno del percorso assistenziale nel settore RadioterapicO) is a new assessment tool that allows to use integrated of Hierarchical Task Analysis (HTA); Failure Mode Effects and Criticality Analysis (FMECA); Cognitive Task Analysis (CTA); Human Error Assessment Reduction Technique (HEART). The methodological approaches have been suitably modified to address their operating in the medical sector under study. Some improvements have concerned the application of the fuzzy theory in FMECA and the use of a linguistic approach in HEART. In this paper, SAPERO application on a case study related to treatment procedures in helical tomotherapy performed at the radiotherapy department of ARNAS Civico hospital, Italy, is described. The results have allowed to provide suggestions aimed to improve the examined process.

Gastroscopy for dyspepsia: Understanding primary care and gastroenterologist mental models of practice: A cognitive task analysis approach.

Gastroscopy to investigate dyspepsia without alarm symptoms rarely results in clinically actionable findings or sustained health-related quality-of-life improvements among patients aged 18-60 years and is, therefore, not recommended. Despite this, referrals for and performance of gastroscopy among this patient population remain high. The purpose of this study was to understand family physicians' and gastroenterologists' mental models of dyspepsia and the drivers behind referring or performing gastroscopy. Cognitive task analysis routine critical decision method interviews with family physicians (n = 8) and gastroenterologists (n = 4). Family physicians and gastroenterologists hold rich mental models of dyspepsia that rely on sensemaking; however, gaps in information continuity affect their ability to plan and coordinate patient care. Drivers behind decisions to refer or perform gastroscopy were: eliminating risk for serious pathology, providing reassurance, perceived preference by patients to receive information and reassurance from gastroenterologists, maintaining relationships with patients, and saving costs to the health system. Family physicians refer for dyspepsia when they are seeking support from gastroenterologists, they believe that alternative factors may be impacting the patient's health or view it as a cost-saving measure. Likewise, gastroenterologists perform gastroscopy for dyspepsia when they perceive it as a cost-saving measure, they want to support their primary care colleagues and provide their colleagues and patients with reassurance. An improved degree of communication between speciality and primary care could allow for continuity in the transfer of information about patients and reduce referrals for dyspepsia.

A cognitive task analysis of expert surgeons performing the robotic roux-en-y gastric bypass.

The safe and effective performance of a robotic roux-en-y gastric bypass (RRNY) requires the application of a complex body of knowledge and skills. This qualitative study aims to: (1)define the tasks, subtasks, decision points, and pitfalls in a RRNY; (2)create a framework upon which training and objective evaluation of a RRNY can be based. Hierarchical and cognitive task analyses for a RRNY were performed using semi-structured interviews of expert bariatric surgeons to describe the thoughts and behaviors that exemplify optimal performance. Verbal data was recorded, transcribed verbatim, supplemented with literary and video resources, coded, and thematically analyzed. A conceptual framework was synthesized based on three book chapters, three articles, eight online videos, nine field observations, and interviews of four subject matter experts (SME). At the time of the interview, SME had practiced a median of 12.5years and had completed a median of 424 RRNY cases. They estimated the number of RRNY to achieve competence and expertise were 25 cases and 237.5 cases, respectively. After four rounds of inductive analysis, 83 subtasks, 75 potential errors, 60 technical tips, and 15 decision points were identified and categorized into eight major procedural steps (pre-procedure preparation, abdominal entry & port placement, gastric pouch creation, omega loop creation, gastrojejunal anastomosis, jejunojejunal anastomosis, closure of mesenteric defects, leak test & port closure). Nine cognitive behaviors were elucidated (respect for patient-specific factors, tactical modification, adherence to core surgical principles, task completion,judicious technique & instrument selection, visuospatial awareness, team-based communication, anticipation & forward planning, finessed tissue handling). This study defines the key elements that formed the basis of a conceptual framework used by expert bariatric surgeons to perform the RRNY safely and effectively. This framework has the potential to serve as foundational tool for training novices.

Cognitive Fitness of Post-Retirement Employees in Control Room Settings

The increasing number of post-retirement workers in the workforce has become a global trend due to efforts to extend the working period after retirement, which is implemented by many countries considering the rapid aging of their populations. While the presence of post-retirement workers can have positive impacts on personal, company, and government levels, they face challenges in interacting with equipment through interfaces that require heavy reliance on cognitive abilities. Cognitive abilities become more important than physical abilities in multitasking jobs that require a lot of interaction with technology, such as control room jobs. These jobs require a high level of attention, concentration, and decision-making ability, making them highly demanding and challenging. This study aims to determine how post-retirement workers meet job demands in the control room, based on identification of critical cognitive abilities. The method employed in this study is divided into two stages. The first stage involves identifying tasks in the control room based on Cognitive Task Analysis (CTA) and identifying critical tasks using risk assessment and Focus Group Discussion (FGD). FGD was conducted on four different types of companies (telecommunication, broadcasting, process industry, and electrical power) that still employ retired workers in the control room jobs. The second stage involves measuring cognitive abilities in 116 individuals aged 56-60 years. The results of CTA indicate there are critical cognitive abilities in control room jobs, such as reasoning, attention, memory, and perception ability. According to the measurement, post-retirement workers (56 - 60 years old) had the problem of improving reasoning, attention, and perceptive ability, despite cognitive capacity measurements remaining in the medium range. In general, post-retirement workers have high memory abilities (above 500), while their abilities in recognition, perception, coordination, and attention are only at 200-400 (a medium level), indicating a decline in cognitive abilities for post-retirement workers. The high memory ability is positive for supporting work in complex systems, while improvement of cognitive abilities is needed for the declining cognitive aspects. Implication of this study were discuss to this paper to consider factors that affect critical cognitive abilities in post-retirement workers when recruiting and training them to work in the control room. 
 
 Received: 7 May 2023 / Accepted: 28 July 2023 / Published: 5 September 2023

An innovative conceptual model for education and training on hybrid warfare

Abstract This study aims to present an innovative conceptual model of learning, teaching, and training that appears to be applicable to the field of hybrid warfare. It is based on integrating three critical elements such as reference curriculum, teaching methodology, and online support capabilities. By using qualitative research through correlating critical analysis of relevant sources, logical scheme, and cognitive task analysis, each descriptive element is analysed in order to provide an eloquent image of the proposed conceptual model. Also, to gain real feedback on the proposed model as well as on its constitutive elements, the quantitative research using statistical data grouping is performed. The analysis and interpretation of the collected data demonstrate not only a high interest in the designed conceptual model, but also a positive impact at the level of each descriptive element on the academic staff and students from the military and civilian university environment.

Towards an approach to define transparency requirements for maritime collision avoidance

This study discusses an approach to support human supervision of autonomous maritime collision avoidance systems by disclosing the system’s perceived information, internal reasoning, decisions, and planned actions as layers of transparency. Information requirements, identified through a cognitive task analysis, were structured using the information processing model by Parasuraman, Sheridan, and Wickens (2000). This model was contextualized to the maritime collision avoidance setting such that the information from the analysis could be structured into unique and distinct layers. A set of minimum information requirements was identified depicting the system’s decisions and planned action, supported by additional layers to reveal its internal reasoning. This approach aims at supporting humans in effectively supervising autonomous collision avoidance systems in their operational context by providing understandability and predictability about what the system is doing, why it is doing it, and what it will do next, i.e., transparency.

Designing Visual Displays for Deep Space Habitats: Challenges and Opportunities from a Cognitive Task Analysis with ISS Mission Control

This paper explores some challenges and opportunities in designing visual interfaces for deep space habitats with communication delays to Earth. The study first examines how Mission Control for the International Space Station (ISS) operates in detecting and responding to anomalous events. Interviews with participants who have held roles as flight controllers, backroom engineers, astronauts, or flight instructors for ISS and shuttle missions were conducted to gain insights into the responses to anomalous events. The collected anomalous scenarios were then re-considered under the context of deep space missions, where communication delays and other environmental factors are substantially different. Findings and insights on data analysis of the anomalous scenarios were then presented subsequently. Based on the findings, four areas of challenges and opportunities for visual interfaces design were identified. The paper suggests that these design challenges and opportunities be considered in more detail in future research.

Development of a Real-Time Trust/Distrust Metric Using Interactive Hybrid Cognitive Task Analysis

While there is increased interest in how trust spreads in Human Autonomy Teams (HATs), most trust measurements are subjective and do not examine real-time changes in trust. To develop a trust metric that consists of objective variables influenced by trust/distrust manipulations, we conducted an Interactive hybrid Cognitive Task Analysis (IhCTA) for a Remotely Piloted Aerial System (RPAS) HAT. The IhCTA adapted parts of the hybrid Cognitive Task Analysis (hCTA) framework. In this paper, we present the four steps of the IhCTA approach, including 1) generating a scenario task overview, 2) generating teammate-specific event flow diagrams, 3) identifying interactions and interdependencies impacted by trust/distrust manipulations, and 4) processing RPAS variables based on the IhCTA to create a metric. We demonstrate the application of the metric through a case study that examines how the influence of specific interactions on team state changes before and after the spread of distrust.

Experienced Users’ Strategies to Address Challenges with Internet Protocol Caption Telephone Services

Internet Protocol Caption Telephone Services provide access to telecommunication for persons who are d/Deaf or Hard of Hearing as users can listen to and read captions of what the other speaker says. This paper presents the results of Applied Cognitive Task Analyses with d/Deaf or Hard of Hearing users to uncover the strategies experienced caption phone users deploy to address communication challenges. Results indicate that users learn strategies that involve adapting their communication behaviors, obtaining the collaboration of the other speaker, selecting the best technology for specific settings, and disclosing hearing loss. Some situations are particularly challenging, e.g., higher caption error when the other speaker has an “accent” or the conversation includes technical language; having to disclose hearing loss and thus risking negative consequences due to biases against disability. Findings can help accelerate novice users’ learning of strategies and inform the development of sociotechnical solutions for caption phones’ current shortcomings.

Validation of an Automated Speech Analysis of Cognitive Tasks within a Semiautomated Phone Assessment

Introduction: We studied the accuracy of the automatic speech recognition (ASR) software by comparing ASR scores with manual scores from a verbal learning test (VLT) and a semantic verbal fluency (SVF) task in a semiautomated phone assessment in a memory clinic population. Furthermore, we examined the differentiating value of these tests between participants with subjective cognitive decline (SCD) and mild cognitive impairment (MCI). We also investigated whether the automatically calculated speech and linguistic features had an additional value compared to the commonly used total scores in a semiautomated phone assessment. Methods: We included 94 participants from the memory clinic of the Maastricht University Medical Center+ (SCD N = 56 and MCI N = 38). The test leader guided the participant through a semiautomated phone assessment. The VLT and SVF were audio recorded and processed via a mobile application. The recall count and speech and linguistic features were automatically extracted. The diagnostic groups were classified by training machine learning classifiers to differentiate SCD and MCI participants. Results: The intraclass correlation for inter-rater reliability between the manual and the ASR total word count was 0.89 (95% CI 0.09–0.97) for the VLT immediate recall, 0.94 (95% CI 0.68–0.98) for the VLT delayed recall, and 0.93 (95% CI 0.56–0.97) for the SVF. The full model including the total word count and speech and linguistic features had an area under the curve of 0.81 and 0.77 for the VLT immediate and delayed recall, respectively, and 0.61 for the SVF. Conclusion: There was a high agreement between the ASR and manual scores, keeping the broad confidence intervals in mind. The phone-based VLT was able to differentiate between SCD and MCI and can have opportunities for clinical trial screening.

Adapting Cognitive Task Analysis Methods for Use in a Large Sample Simulation Study of High-Risk Healthcare Events.

Cognitive task analysis (CTA) methods are traditionally used to conduct small-sample, in-depth studies. In this case study, CTA methods were adapted for a large multi-site study in which 102 anesthesiologists worked through four different high-fidelity simulated high-consequence incidents. Cognitive interviews were used to elicit decision processes following each simulated incident. In this paper, we highlight three practical challenges that arose: (1) standardizing the interview techniques for use across a large, distributed team of diverse backgrounds; (2) developing effective training; and (3) developing a strategy to analyze the resulting large amount of qualitative data. We reflect on how we addressed these challenges by increasing standardization, developing focused training, overcoming social norms that hindered interview effectiveness, and conducting a staged analysis. We share findings from a preliminary analysis that provides early validation of the strategy employed. Analysis of a subset of 64 interview transcripts using a decompositional analysis approach suggests that interviewers successfully elicited descriptions of decision processes that varied due to the different challenges presented by the four simulated incidents. A holistic analysis of the same 64 transcripts revealed individual differences in how anesthesiologists interpreted and managed the same case.