Goal-directed Task Analysis Research Articles

The awareness of operators: a goal-directed task analysis in SOCs for critical infrastructure

Security operation centers (SOCs) are increasingly established to meet the growing threat against cyber security. The operators of SOCs respond to complex incidents under time constraints. Within critical infrastructure, the consequences of human error or low performance in SOCs may be detrimental. In other domains, situation awareness (SA) has proven useful to understand and measure how operators use information and decide the correct actions. Until now, SA research in SOCs has been restricted by a lack of in-depth studies of SA mechanisms. Therefore, this study is the first to conduct a goal-directed task analysis in a SOC for critical infrastructure. The study was conducted through a targeted series of unstructured and semi-structured interviews with SOC operators and their leaders complemented by a review of documents, incident reports, and in situ observation of work within the SOC and real incidents. Among the presented findings is a goal hierarchy alongside a complete overview of the decisions the operators make during escalated incidents. How the operators gain and use SA in these decisions is presented as a complete set of SA requirements. The findings are accompanied by an analysis of contextual differences in how the operators prioritize goals and use information in network incidents and security incidents. This enables a discussion of what SA processes might be automated and which would benefit from different SA models. The study provides a unique insight into the SA of SOC operators and is thus a steppingstone for bridging the knowledge gap of Cyber SA.

Situation awareness information requirement for marine engine room monitoring at the future shore control center

Information integration for explicit comprehension can be highly challenging in a large-scale system with interconnected equipment under the influence of the environment. This presents a great challenge in acquiring and maintaining situation awareness during the transition from traditional to remote and, eventually, to autonomous operation. However, the maritime industry appears to approach situation awareness in a highly restricted manner that focuses on visual and navigational awareness only, resulting in no support system for engine room monitoring. This research conducted a goal-directed task analysis with thirty-one subject matter experts, with an average sea experience of 9.78 years, to investigate situation awareness in the engine room. As a result, the situation awareness information requirements for nine machineries and twelve systems are developed. The findings reveal that numerous perception elements, and their complex combinations, for higher-level awareness comprise situation awareness, and situation awareness deterioration is highly likely, due to the replication or reproduction of ship sense and expanded system coverage to compensate for the lack of marine engineers on ships. This research proposes that presenting higher-level situation awareness information in consideration of schema instantiation could be a promising alternative to developing and optimising the situation awareness support system for engine room monitoring at the future shore control center. Highlights The replication of conventional monitoring systems for remote operation results in a loss of situation awareness information as the conventional systems are designed to collaborate with human operators onsite The relocation of humans from ships to shore must incorporate the transfer of information that human operators generate Situation awareness in the engine room comprises numerous information elements and requires a high level of information integration skills for comprehension and projection Situation awareness support systems must facilitate information integration while maintaining a balance between system explainability and simplicity Cue strength differences in comprising comprehension and projection might effectively address system complexity and situation awareness

Situation awareness in Sweden’s emergency medical services: a goal-directed task analysis

Situation awareness is knowing what is going on in the situation. Clinicians working in the emergency medical services (EMS) encounter numerous situations in various conditions, and to be able to provide efficient and patient safe care they need to understand what is going on and possible projections of the current situation. The design of this study encompassed a Goal-Directed Task analysis where situation awareness information requirements were mapped in relation to goals related to various aspects of the EMS mission. A group of 30 EMS subject matter experts were recruited and answered a web-based survey in three rounds related to what they though themselves or a colleague might need to achieve situation awareness related to the specific goals of various situations. The answers were analysed using content analysis and descriptive statistics. Answers reached consensus at a predetermined level of 75%. Those who reached consensus were entered into the final goal-directed task analysis protocol. The findings presented that EMS clinicians must rely on their own, or their colleagues prior experience or knowledge to achieve situation awareness. This suggests that individual expertise plays a crucial role in developing situation awareness. There also seems to be limited support for situation awareness from organizational guidelines. Furthermore, achieving situation awareness also involves collaborative efforts from the individuals involved in the situation. These findings could add to the foundation for further investigation in this area which could contribute to the development of strategies and tools to enhance situation awareness among EMS clinicians, ultimately improving patient care and safety.

Supporting human supervision in autonomous collision avoidance through agent transparency

Ongoing trends in society point towards the adoption of intelligent agents across safety critical industries. In the maritime domain, artificially intelligent agents may soon be capable of autonomously performing collision and grounding avoidance (CAGA); a task traditionally performed by humans. Consequently, the role of humans is anticipated to change from those performing collision avoidance to those supervising an agent performing collision avoidance. One of the key concerns with regards to human factors is avoiding the out-of-the-loop performance problem where humans lose situation awareness (SA) and become susceptible to misinterpreting the agent’s decisions and planned actions. Despite previous research addressing human factors in autonomous shipping and remote control, few studies have focused on how to support the humans’ mental processes in this new role. Therefore, this study performed a goal-directed task analysis addressing goals, decisions, and SA requirements for human-supervised collision avoidance. Data was obtained from in situ observations and interviews with nine navigators onboard passenger ferries, an appraisal of the collision regulations, and of relevant company documentation. The task analysis identified specific SA requirements to make agents, capable of collision and grounding avoidance, transparent to their users. The results further indicate a change towards increased cognitive activities required to verify agent performance. Therefore, providing insight into the agents’ internal reasoning and actions becomes a key consideration in supporting future supervisors. Given the trends towards the application of artificially intelligent agents capable of autonomous behaviour, this study anticipates that transparency becomes an essential prerequisite for safe and effective human-autonomy system oversight.

Identifying firefighters' situation awareness requirements for fire and non-fire emergencies using a goal-directed task analysis

A firefighter's situation awareness (SA) is considered crucial to making effective tactical decisions and actions at the scene. Despite the importance of the firefighter's SA, there have been limited research efforts to understand what cues and information firefighters use to assess ongoing situations and predict future conditions. In addition to fire events, contemporary firefighters respond to an increasing volume of non-fire incidents. Thus, this study aims to identify firefighters' SA during three fire incidents (single house fire, vehicle fire, and passenger aircraft fire) and three non-fire incidents (medical emergency, hazardous materials, and urban search and rescue). A goal-directed task analysis was conducted via focus group discussions with eight career firefighters. Findings indicate that firefighters build their SA by processing various cues from hazards (e.g., fire, ignition source), humans (e.g., occupants, bystanders, drivers, passengers), spatial elements (e.g., building structure, location of hazards), and surrounding conditions (e.g., traffic, weather). Our findings provide insights into SA measurement, SA-oriented work processes, training for SA, and designing technologies to support firefighters' SA during all-hazard responses.

Modern Firefighters’ Three-Level Situation Awareness in Fire and Non-Fire Incidents

This study aimed to identify professional firefighters’ situation awareness (SA) requirements for six high-risk emergencies: i) a single-house fire, ii) an unconscious person as a medical emergency scenario, iii) a vehicle fire, iv) a gas leak incident as a hazardous material (HazMat) case, v) a passenger aircraft fire, vi) and a search and rescue operation for a collapsed building. A Goal-Directed Task Analysis (GDTA) was employed to elicit major goals, sub-goals, decisions, and three-level SA information requirements for firefighters in six different types of an incident. Findings indicate that firefighters have common goals of ensuring life safety and incident stabilization. Sub-goals were concerned with the safety of residents as well as responders’ safety. Incident stabilization meant extinguishing fire for fire events and providing medical treatment in case of a medical emergency. To build SA for different situations, firefighters indicated that they use characteristics of fire and smoke, the location and condition of residents, a person’s appearance and behavior, and building structure. These findings can inform the design of work processes and emergency response technologies for fire and non-fire operations.

Formalizing the information requirements for decision-making of field managers during indoor construction activities

PurposeInformation technology provides important support for on-site decision-making of field personnel. Most literature focuses on the technological aspects of decision-support systems, without fully understanding the information required for effective decision-making. This study aimed to formalize decision-makers’ requirements in terms of the major goals, decisions and information.Design/methodology/approachThe situation awareness (SA) approach was applied through the goal-directed task analysis (GDTA) method, narrowing the scope to field managers’ decision-making during indoor construction activities. This method was based on a series of interviews to define, revise and validate the decision-making requirements for the given scope.FindingsThe study yielded 1,056 highly interrelated elements. The results indicate that the field manager’s overall goal is to execute and handoff work within the established deadlines, with the required quality, maximizing profits, within a safe work environment. The overall goal construes into five main goals regarding work progress, quality, costs, safety and communication. These goals include subgoals, decisions, and the information necessary to attain them, depicted in diagrams.Practical implicationsThe findings allow enhancing the design of decision-support solutions by identifying information required for future developments and showing the interrelations between goals and information requirements that need to be addressed to present interfaces for effectively assisting on-site decision-making. Moreover, the results allow for the assessment of solutions regarding the sufficiency of information.Originality/valueThis is the first effort to fully understand the information required by field managers for on-site decision-making during indoor construction activities.

Fire emergency response systems information requirements' data model for situational awareness of responders: A goal-directed task analysis

Building fire emergencies are considered a high-risk domain for significant loss of lives and property. Accurate information and Situational Awareness (SA) enable fire responders to make timely decisions and perform safe operations during fire emergencies. Although information technologies such as Building Information Modeling (BIM) and the Internet of Things (IoT) have the potential to supply information and support the decision-making processes of fire emergency response, the provided data are not necessarily the needed information at the operation level. This paper uses Goal-Directed Task Analysis (GDTA) to define fire emergency responders' SA Goals, Decisions, and Information (GDI) requirements. The semi-structured scenario-based interviews were conducted with ten Municipal fire department experts to determine GDI requirements. Based on the identified requirements, the SA-oriented use-case and sequence data models for decision-support Fire Emergency Information Systems (FERS) were developed to reveal the dynamic aspect of the FERS and to describe how the system should behave to enhance the SA of responders. The use-case data model visualizes the possible interactions of fire responders with the system, and the sequence data models visualize the order of the interactions and the call/return messages between the lifelines of the FERS. The validation of the GDI requirements and data models were performed by obtaining experts' feedback, and based on the feedback; the models were finalized. The SA GDI requirements and data models can eventually be helpful for FERS's user-friendly graphical interface design to support responders with higher SA. The FERS, which has comprehension and projection levels of information, will enhance effective decision-making, especially in cases where the skills and training of the firefighters are not sufficient to direct the emergency response.

A task analysis of central line-associated bloodstream infection (CLABSI) surveillance in home infusion therapy

Barriers for home infusion therapy central line associated bloodstream infection (CLABSI) surveillance have not been elucidated and are needed to identify how to support home infusion CLABSI surveillance. We aimed to (1) perform a goal-directed task analysis of home infusion CLABSI surveillance, and (2) describe barriers to, facilitators for, and suggested strategies for successful home infusion CLABSI surveillance. We conducted semi-structured interviews with team members involved in CLABSI surveillance at 5 large home infusion agencies to explore work systems used by members of the agency for home infusion CLABSI surveillance. We analyzed the transcribed interviews qualitatively for themes. Twenty-one interviews revealed 8 steps for performing CLABSI surveillance in home infusion therapy. Major barriers identified included the need for training of the surveillance staff, lack of a standardized definition, inadequate information technology support, struggles communicating with hospitals, inadequate time, and insufficient clinician engagement and leadership support. Staff performing home infusion CLABSI surveillance need health system resources, particularly leadership and front-line engagement, access to data, information technology support, training, dedicated time, and reports to perform tasks. Building home infusion CLABSI surveillance programs will require support from home infusion leadership.

Does Explicit Categorization Taxonomy Facilitate Performing Goal-Directed Task Analysis?

Situation awareness (SA) is an important factor that affects the performance of operators who work in complex environments. SA is defined as the perception of elements in an environment (level 1), understanding their meaning (level 2), and the projection of their state into the future (level 3). The first step to assess SA is identifying its requirements, for which goal-directed task analysis (GDTA) is the recommended technique. GDTA is a type of cognitive task analysis that focuses on the goals that a human operator must achieve, and the information required to accomplish them. The result of GDTA is a list of information (SA elements) that are categorized into the three SA levels. However, GDTA-based studies typically categorize SA elements into the three SA levels without stating their categorization rules. Therefore, this article proposes a taxonomy to categorize SA elements obtained via GDTA into SA levels. First, we present the results of a systematic literature review (N = 87) to gain insight into how analysts apply their classification criteria. Then, we propose a categorization taxonomy based on the ISO 15939 standard. To validate the proposed taxonomy, we analyze a subset of GDTA results in two cases, and the results of the analysis show that the proposed categorization rules are applicable to the two analyzed cases.

Social Media Data in an Augmented Reality System for Situation Awareness Support in Emergency Control Rooms

During crisis situations, emergency operators require fast information access to achieve situation awareness and make the best possible decisions. Augmented reality could be used to visualize the wealth of user-generated content available on social media and enable context-adaptive functions for emergency operators. Although emergency operators agree that social media analytics will be important for their future work, it poses a challenge to filter and visualize large amounts of social media data. We conducted a goal-directed task analysis to identify the situation awareness requirements of emergency operators. By collecting tweets during two storms in Germany we evaluated the usefulness of Twitter data for achieving situation awareness and conducted interviews with emergency operators to derive filter strategies for social media data. We synthesized the results by discussing how the unique interface of augmented reality can be used to integrate social media data into emergency control rooms for situation awareness support.

Multi-Dimensional, Short-Timescale Quantification of Parkinson's Disease and Essential Tremor Motor Dysfunction.

Introduction: Parkinson's disease (PD) is a progressive movement disorder characterized by heterogenous motor dysfunction with fluctuations in severity. Objective, short-timescale characterization of this dysfunction is necessary as therapies become increasingly adaptive.Objectives: This study aims to characterize a novel, naturalistic, and goal-directed tablet-based task and complementary analysis protocol designed to characterize the motor features of PD.Methods: A total of 26 patients with PD and without deep brain stimulation (DBS), 20 control subjects, and eight patients with PD and with DBS completed the task. Eight metrics, each designed to capture an aspect of motor dysfunction in PD, were calculated from 1-second, non-overlapping epochs of the raw positional and pressure data captured during task completion. These metrics were used to generate a classifier using a support vector machine (SVM) model to produce a unifying, scalar “motor error score” (MES). The data generated from these patients with PD were compared to same-day standard clinical assessments. Additionally, these data were compared to analogous data generated from a separate group of 12 patients with essential tremor (ET) to assess the task's specificity for different movement disorders. Finally, an SVM model was generated for each of the eight patients with PD and with DBS to differentiate between their motor dysfunction in the “DBS On” and “DBS Off” stimulation states.Results: The eight metrics calculated from the raw positional and force data captured during task completion were non-redundant. MES generated by the SVM analysis protocol showed a strong correlation with MDS-UPDRS-III scores assigned by movement disorder specialists. Analysis of the relative contributions of each of the eight metrics showed a significant difference between the motor dysfunction of PD and ET. Much of this difference was attributable to the homogenous, tremor-dominant phenotype of ET motor dysfunction. Finally, in individual patients with PD with DBS, task performance and subsequent SVM classification effectively differentiated between the “DBS On” and “DBS Off” stimulation states.Conclusion: This tablet-based task and analysis protocol correlated strongly with expert clinical assessments of PD motor dysfunction. Additionally, the task showed specificity for PD when compared to ET, another common movement disorder. This specificity was driven by the relative heterogeneity of motor dysfunction of PD compared to ET. Finally, the task was able to distinguish between the “DBS On” and “DBS Off” states within single patients with PD. This task provides temporally-precise and specific information about motor dysfunction in at least two movement disorders that could feasibly correlate to neural activity.

Situation awareness information requirements for maritime navigation: A goal directed task analysis

Maritime shipping transports about 90% of the international trade and is considered a high-risk industry. Among all daily operations carried out by merchant ships, navigation is deemed the most critical. With the ever-increasing dimensions of ocean-going ships and the threat possessed by dangerous cargoes in the event of an accident to individual lives and environment alike, the human factors which affect the navigation of merchant ships require paramount consideration to enhance safety in maritime operations. In this study, we explore the concept of Situation Awareness (SA) within the maritime domain, identifying the SA information requirements of navigators and factors affecting their SA. A total of 7 experienced navigators were interviewed in this exploratory study to determine the SA information requirements by a Goal-Directed Task Analysis (GDTA). Three Subject Matter Experts (SMEs) possessing Captain’s license for merchant ships were used for validation of initial findings. The findings reveal the information navigators use during the pilotage phase of navigation and further classify them in three levels. The findings shed light on the factors that affect SA of navigators and are discussed with their potential implications for the procedures and practices which better support SA in maritime navigation.

It's Complicated: Patient and Informal Caregiver Performance of Outpatient Parenteral Antimicrobial Therapy-Related Tasks.

Outpatient parenteral antimicrobial therapy (OPAT) requires patients and caregivers to infuse antimicrobials through venous catheters (VCs) in the home. The objective of this study was to perform a patient-centered goal-directed task analysis to identify what is required for successful completion of OPAT. The authors performed 40 semi-structured patient interviews and 20 observations of patients and caregivers performing OPAT-related tasks. Six overall goals were identified: (1) understanding and developing skills in OPAT, (2) receiving supplies, (3) medication administration and VC maintenance, (4) preventing VC harm while performing activities of daily living, (5) managing when hazards lead to failures, and (6) monitoring status. The authors suggest that patients and caregivers use teach-back, take formal OPAT classes, receive visual and verbal instructions, use cognitive aids, learn how to troubleshoot, and receive clear instructions to address areas of uncertainty. Addressing these goals is essential to ensuring the safety of and positive experiences for our patients.

Finding Common Ground

The article “Situation Awareness Misconceptions and Misunderstandings” (Endsley, 2015) discusses common fallacies noted in the literature in the interpretation of Endsley’s 1995 model of situation awareness (SA; Endsley, 1995). The clarifications presented in the article provide a more complete and comprehensible explanation of the model. As in the complex domains that we study, our attention suffers from limitations as we simplify each other’s work in our eager and well-intentioned quest to attack new and exciting problems. Dr. Endsley is to be thanked for giving us pause to rebuild our own SA of SA in her thoughtful article. SA has, for many years, been a powerful and influential construct. In our own work in cognitive work analysis (CWA), we have viewed SA as a complementary framework that challenges and drives CWA. Without doubt, the output of a CWA-based design process should be the design of a system that promotes better SA and performance (Burns et al., 2008). CWA and goaldirected task analysis may organize the world in slightly different dimensions, but the overall intent is the same: to create systems that support human decision making as well as we can. Indeed, it is this common intent that unites us in our field. To advance cognitive engineering, there are times when we must challenge each other, critique each other’s models, hunt for flaws, and identify promising new directions. Assuredly, this helps us progress, strengthen our methods, and deepen our understanding. This article clearly identifies such activity and responds to it. We are all better for this exercise, as it challenges both SA and all our approaches to grow and deepen. Acknowledging this, we would like to change our perspective to a larger one and discuss challenges facing cognitive engineering as a whole. In these challenges, our existing methods, SA, CWA, and other approaches must adapt and grow. Advancements in intelligent systems and automation have increased in the amount of data produced by information systems and yet placed the human in new roles. In many cases, these roles are partially in the loop and partially out of the loop and may involve supervisory control or may have the human working in systems with very little supervisory control at all because the automation is largely nontransparent. We outline three core areas of challenges: self-awareness and self-regulation, memory failures or performance with incorrect SA, and design for unstructured environments.

GDIA: Eliciting information requirements in emergency first response

Good information is vital for first responders in an emergency. However, although information systems can provide vast amounts of data, the information requirements of emergency first responders in complex, dynamic, ad hoc, and stressful environments cannot be systematically captured by existing requirement engineering approaches. This paper outlines the unique features of emergency response operations first and then drives an emergency-response-specific method to suit those features. The method is named as goal-directed information analysis (GDIA), which is based on but easier to use than an approach, goal-directed task analysis. We argue that goals are implicit and thus difficult to be captured from first responders because of the features of the emergency operations. GDIA starts from scenarios and has seven clearly defined and repeatable steps, including task analysis, which then leads to a simpler and more accurate analysis of the goal structure, before the rest of the hierarchy, including decisions and information requirements, is completed. A case study is presented using this GDIA approach to retrieve information requirements for four types of key fire fighters responding to a fire in a high-risk building environment. This work led to several real applications in the emergency service area, indicating the success of the approach.

Using the situation present assessment method to measure situation awareness in simulated submarine track management

Extant query-based measures of situation awareness that require interruption of dynamic tasks can be inappropriate in certain work contexts. A candidate real-time query-based method, the situation present assessment method (SPAM), was used to examine participants’ situation awareness in simulated submarine track management. The simulation and situation awareness queries were based upon a goal-directed task analysis conducted with expert submariners. Participants decided whether to engage enemy vessels using rules based on clear firing corridors for submarine torpedoes, and responded to SPAM queries that assessed their awareness of the current and future display situation. Participants were less accurate and slower to respond to SPAM queries that assessed awareness of the future display situation compared to the current display situation. Consistent with theoretical frameworks of visual sampling and attention-situation awareness integration, the relative predictive utility of current- and future- SPAM queries depended on the nature of the display information relevant to performance goals.

Cognitive information flow analysis

The cognitive information flow analysis (CIFA) is introduced as a method to integrate results from cognitive task and work analyses in order to provide a focus on the necessary system information flow, which includes how information is produced, consumed, and transformed by the various system functions and users. CIFA can be used as a tool to augment cognitive task and work analyses. This paper presents the CIFA technique, provides a case study that applies the CIFA method to existing goal-directed task analysis and modified cognitive work analysis results, and provides insight into CIFA's use for informing the design of a human-robot system. CIFA augments the results provided by cognitive task and work analyses and can guide system design and development. CIFA differs from existing information flow techniques in that it allows representation of systems containing large numbers of users for highly complex and uncertain domains. Existing cognitive task and work analyses integration methods rely heavily on relational tables. CIFA specifically expresses the interconnectivity of the various system subcomponents, including partial ordering and parallelism, by fundamentally focusing on the information flow. CIFA also identifies both existing and potential, information bottlenecks and highlights teamwork.

Using fuzzy cognitive mapping techniques to model situation awareness for army infantry platoon leaders

This paper describes work on the development of an actionable model of situation awareness for Army infantry platoon leaders using fuzzy cognitive mapping techniques. Developing this model based on the formal representation of the platoon leader provided by the Goal-Directed Task Analysis (GDTA) methodology advances current cognitive models because it provides valuable insight on how to effectively support human cognition within the decision-making process. We describe the modeling design approach and discuss validating the model using the VBS2 simulation environment.

Team and Shared Situation Awareness in Disaster Action Teams

Disaster Action Teams assist with American Red Cross disaster relief operations by providing support and services to people who are affected by local emergencies, such as fire or water damage to a home. Ensuring the successful performance of these teams requires promoting not only the situation awareness (SA) of individual team members, but also the SA of the team as a whole as well as their shared SA. In this paper, we demonstrate how the Goal-Directed Task Analysis methodology can be utilized to identify the critical information requirements of Disaster Action Teams, at both the individual and team level. We also illustrate how SA-Oriented Design Principles can be applied to meet their information demands by effectively presenting these information requirements.