Individual Situation Awareness Research Articles

Team Performance in a Simulated Maritime Control Room: The Effect of Information Integration and Operator Configuration

How team cognition is conceptualized has evolved rapidly in the last decade with the emerging use of a systems approach, moving the focus from the cognition residing in the heads of individuals, to that distributed across the team. This is referred to as ‘distributed cognition’. Increasingly, network approaches are being explored in attempts to model team distributed cognition. The specific domain of interest in the present study is the sociotechnical system within a maritime control room. This comprises human, machine and software agents interacting to interpret sensor data in order to develop a timely and accurate picture of surrounding contacts at sea. To achieve the goal, information is shared or integrated across the maritime control room consoles. The aim of this study was to develop and apply a suite of workload, situation awareness and team performance measures, including network analysis techniques, to examine how the distributed cognition of a team might change as a function of console configuration and information integration within a control room, and how these changes, if any, impact overall team performance. Sixteen teams of six novices conducted two one-hour scenarios operating generic maritime control room positions. Each team completed a one-hour simulation in each of two console configuration layouts with the order counter-balanced (within-subject design). Half the teams conducted the two scenarios in a high integration condition, and half in a low integration condition (between-subjects). The human machine interface (HMI) designs for the high integration condition emerged from a series of task analyses and user-centered design workshops. The emergent cognitively –oriented HMI designs are based on the assumption that each console can freely share information with other consoles. To create an analogue of current, less-integrated, and more stove-piped systems, a low integration condition was created where not all information was shared across consoles, but instead was shared verbally by console operators. Contacts detected at sea were introduced into the simulation and the team’s task was to assess, report and derive a solution (location, course, and speed) for each detected contact. Individual situation awareness was measured through the Situation Present Assessment Method (SPAM) and individual workload through the Air Traffic Workload Indicator Task (ATWIT). Team interaction from the scenarios were video recorded and we applied the Event Analysis of Systemic Teamwork (EAST) approach to examine the task, social and information networks which emerged. Team performance was measured as the accuracy and timeliness of the solutions We found higher information integration lowered average team workload, and improved average team situation awareness and team performance (faster solutions and a more accurate tactical picture). We found no impact of console configuration on team performance or any other dependent measure. The EAST method uncovered patterns in the network analysis that are potentially explanatory for the team workload, situation awareness and performance findings as a function of the information integration manipulation. This experiment showed that there can be reductions in workload, and improvements to situation awareness and performance when information is shared between consoles in a considered design. This has implications for HMI design within a team setting. The set of diagnostic metrics developed were largely effective in examining teamwork and team performance. Acknowledgements. The authors would like to thank Justin Hill (Royal Australian Navy) for his subject matter expertise, Graeme Muller (elmTEK) for his software, technical and infrastructure support, David Munro-Ford (Total Technology Partners) for his simulation programming, Dr Aaron Roberts for his advice on general aspects of the experiment, and Professor Paul Salmon for his advice on EAST.

Conceptualization and Measurement of Individual Situation Awareness (SA) in Expert Populations across Operational Domains: a systematic review of the literature with a practical purpose on our minds

From the time when the first formal SA theories were introduced (Endsley 1995; Smith & Hancock, 1995), an underlying ontological debate concerning the nature of human perception and cognition became evident. Indeed, despite more than two decades have passed since their publication, SA epistemological status and methodology are still object of dispute. For that reason, and bearing in mind the ultimate practice-oriented goal of developing/adapting SA measurement methods for elite soccer, a systematic review of the literature was performed regarding the definition and the methods used for SA measurement in expert populations. Fifty-four studies (N=54) met the established inclusion criteria and revealed important differences concerning SA definition and assessment across different operational domains. The results are discussed regarding both the state of the art of SA literature and the adaption/development of SA assessment methods for elite soccer.

Workarounds to Intended Use of Health Information Technology: A Narrative Review of the Human Factors Engineering Literature.

Objective To integrate and synthesize insights from recent studies of workarounds to the intended use of health information technology (HIT) by health care professionals. Background Systems are safest when the documentation of how work is done in policies and procedures closely matches what people actually do when they are working. Proactively identifying and managing workarounds to the intended use of technology, including deviations from expected workflows, can improve system safety. Method A narrative review of studies of workarounds with HIT was conducted to identify themes in the literature. Results Three themes were identified: (1) Users circumvented new additional steps in the workflow when using HIT, (2) interdisciplinary team members communicated via HIT in text fields that were intended for other purposes, and (3) locally developed paper-based and manual whiteboard systems were used instead of HIT to support situation awareness of individuals and groups; an example of a locally developed system was handwritten notes about a patient on a piece of paper folded up and carried in a nurse's pocket. Conclusion Workarounds were employed to avoid changes to workflow, enable interdisciplinary communication, coordinate activities, and have real-time portable access to summarized and synthesized information. Application Implications for practice include providing summary overview displays, explicitly supporting role-based communication and coordination through HIT, and reducing the risk to reputation due to electronic monitoring of individual performance.

The Low-Event Task Subjective Situation Awareness (LETSSA) technique: Development and evaluation of a new subjective measure of situation awareness

Situation awareness (SA) is an important component of an individual's ability to function in a complex environment. As such, it is essential to have effective measures of an individual's SA. The most widely used subjective measure of SA is the Situation Awareness Rating Technique [SART]. However, SART has been criticised for not predicting performance or objective SA, and being highly correlated with workload. This paper describes the development and testing of a new subjective measure of SA, the Low-Event Task Subjective Situation Awareness (LETSSA) measure. To evaluate LETSSA a train simulator study was conducted with 23 novice and 26 expert freight train drivers. LETSSA was able to detect differences in manipulated SA and was comparable to an established objective SA measure (SAGAT). LETSSA was significantly associated with performance but not significantly associated with workload. While further validation is required, LETSSA shows promise as an effective subjective measure of SA.

Design and Evaluation of a Team Mutual Awareness Toolkit for Digital Interfaces of Nuclear Power Plant Context

ABSTRACTIn the teamwork of nuclear power plants (NPPs), the maintenance of mutual awareness enables the operators to have an up-to-the-moment understanding of each other’s work and makes the collaboration more efficient. Providing interface support for mutual awareness is proven to be an effective way to enhance the operators’ mutual awareness in digital systems. What mutual awareness-relevant information to provide and how to present the information on the display are two questions worth studying. This research focused on the above two questions and provided a newly designed mutual awareness toolkit on the context of digital interfaces in NPPs. The usability of the designed toolkit was evaluated through a laboratory experiment. The influence of the mutual awareness toolkit on team performance and its interaction effect with task complexity was further examined under incident scenarios. The results showed that the designed mutual awareness toolkit improved the operators’ mutual awareness, while it did not decrease their individual situation awareness (SA) or impose extra mental workloads. In team diagnosis tasks, the teams using the mutual awareness toolkit more thoroughly discussed the incident scenarios and identified more key points of the incidents. The diagnosis correctness, perhaps moderated by other factors besides mutual awareness, was not significantly improved.

Mission Command in the Age of Network-Enabled Operations: Social Network Analysis of Information Sharing and Situation Awareness.

A common assumption in organizations is that information sharing improves situation awareness and ultimately organizational effectiveness. The sheer volume and rapid pace of information and communications received and readily accessible through computer networks, however, can overwhelm individuals, resulting in data overload from a combination of diverse data sources, multiple data formats, and large data volumes. The current conceptual framework of network enabled operations (NEO) posits that robust networking and information sharing act as a positive feedback loop resulting in greater situation awareness and mission effectiveness in military operations (Alberts and Garstka, 2004). We test this assumption in a large-scale, 2-week military training exercise. We conducted a social network analysis of email communications among the multi-echelon Mission Command staff (one Division and two sub-ordinate Brigades) and assessed the situational awareness of every individual. Results from our exponential random graph models challenge the aforementioned assumption, as increased email output was associated with lower individual situation awareness. It emerged that higher situation awareness was associated with a lower probability of out-ties, so that broadly sending many messages decreased the likelihood of attaining situation awareness. This challenges the hypothesis that increased information sharing improves situation awareness, at least for those doing the bulk of the sharing. In addition, we observed two trends that reflect a compartmentalizing of networked information sharing as email links were more commonly formed among members of the command staff with both similar functions and levels of situation awareness, than between two individuals with dissimilar functions and levels of situation awareness; both those findings can be interpreted to reflect effects of homophily. Our results have major implications that challenge the current conceptual framework of NEO. In addition, the information sharing network was largely imbalanced and dominated by a few key individuals so that most individuals in the network have very few email connections, but a small number of individuals have very many connections. These results highlight several major growing pains for networked organizations and military organizations in particular.

Mutual awareness: Enhanced by interface design and improving team performance in incident diagnosis under computerized working environment

Mutual Awareness, which measures understanding of the behaviors and status of other members in a team is supposed to influence team decision-making process in safety-critical tasks/systems. This study aims to explore enhancement of mutual awareness by adding a Mutual Awareness Tool (MAT) on a user interface and examining its effects on team diagnosis performance in emergency situations of a simulated nuclear power plant system. According to the experimental results, the embedded MAT on the operation interface enhanced team mutual awareness significantly, and improved incident diagnosis performance. The results also showed that the increase in mutual awareness led to a reduction of individual situation awareness, possibly due to the limited mental resources.

Outbreak column 17: Situational Awareness for healthcare outbreaks.

Outbreak column 17 introduces the utility of Situation Awareness (SA) for outbreak management. For any given time period, an individual or team's SA involves a perception of what is going on, meaning derived from the perception and a prediction of what is likely to happen next. The individual or team's SA informs, but is separate to, both the decisions and actions that follow. The accuracy and completeness of an individual or team's SA will therefore impact on the effectiveness of decisions and actions taken. SA was developed by the aviation industry and is utilised in situations which, like outbreaks, have dynamic, i.e. continuously changing problem spaces, and wherein a loss of SA is likely to lead to both poor decision-making and actions with potentially fatal consequences. The potential benefits of using SA for outbreaks are discussed and include: (1) retrospectively to identify if poor decision-making was a result of a poor SA; (2) prospectively to identify where the system is weakest; and (3) as a teaching tool to improve the skills of individuals and teams in developing a shared understanding of the here and now.

Managing information overload in virtual teams: Effects of a structured online team adaptation on cognition and performance

Information overload (IO) indicates the exchange of too much low-quality information in virtual teams. When being overloaded with information, teams need to adapt and to change communication behaviour. This study introduces and tests a structured online team adaptation (STROTA) procedure that enables virtual teams to reduce IO by improving their team mental model quality. STROTA, built from team adaptation models, is a moderated intervention consisting of three stages: (1) individual situation awareness, (2) team situation awareness, and (3) plan formulation. STROTA was tested in the context of an experimental problem-solving task. Participants (N = 363) worked in virtual teams of three and were randomly assigned to one of four experimental conditions: no STROTA, incomplete STROTA (step 1, steps 1–2), and complete STROTA (steps 1–2–3). We found teams that followed a complete STROTA procedure experienced lessened IO over time. Teams with complete STROTA showed the largest development of TMM immediately after STROTA. Finally, multilevel mediation analyses showed that TMM are mediators that explain the influence of STROTA on IO.

Measuring situation awareness of operation teams in NPPs using a verbal protocol analysis

Situation awareness (SA) continues to receive a considerable amount of attention from the ergonomics community given that need for operators to maintain SA is frequently cited as a key to effective and efficient performance. Although complex and dynamic environments such as that of a main control room (MCR) in a nuclear power plant (NPP) are operated by operation teams, and while team situation awareness (TSA) is also cited as an important factor, research is limited to individual SA. However, understanding TSA can provide a window onto the characteristics of team acquisition as well as the performance of a complex skill. Therefore, such knowledge can be valuable in diagnosing team performance successes and failures. Moreover, training and design interventions can target the cognitive underpinnings of team performance, with implications for the design of technological aids to improve team performance. Despite these advantages and the importance of understanding TSA, measures and methods targeting TSA are sparse and fail to address it properly. In this study, an objective TSA measurement method is developed in an effort to understand TSA. First, key considerations for developing a method are derived. Based on these considerations, the proposed method is developed while mainly focusing on the creation of logical connections between team communications and TSA. A speech act coding scheme is also implemented to analyze team communications. The TSA measurement method developed in this study provides a measure for each level of TSA. It was revealed from a preliminary study that this TSA measurement method is feasible for measuring TSA to a fair extent. Useful insight into TSA is also derived.

Mechanisms for the acquisition of situation awareness in situated agents

Situation awareness (SA) is the understanding required to operate a complex system in a highly dynamic environment. We evaluate theories of individual SA and the processes by which individuals maintain their understanding of a situation. We support a situated approach, which holds that individual operators make use of limited internal representation, and rely extensively on interactions with external props and tools to achieve and maintain SA. We also propose a synthesis of Durso et al. (2007) [Durso, F., Rawson, K., and Girotto, S., 2007. Comprehension and situation awareness. In: F. Durso, et al., eds. Handbook of applied cognition. 2nd ed. Hoboken, NJ: Wiley, 163–194.] Construction-Integration model of scene analysis with Sperber and Wilson's (1995) [Sperber, D. and Wilson, D., 1995. Relevance: communication and cognition. 2nd ed. Oxford: Blackwell.] Relevance Theory of comprehension. We show that the combination of the two theories can provide a more complete account of the mechanisms and processes underlying situation assessment, or sense-making, and is consistent with the situated approach to SA.

Influence of Team Leaders' Situation Awareness on their Team's Situation Awareness and Performance

In complex operational environments, such as found in the military, it is necessary to consider not only the situation awareness (SA) of individual team members, but also the SA of the team as a whole. Given their important role in military combat operations, we hypothesized that the SA of team leaders (e.g., unit commanders) may have a significant influence on the SA and subsequent performance of those under their command. Accordingly, the overall objective of this research was to evaluate how much of the variance in a team's SA can be accounted for by the individual SA of the team's leader. To test our hypothesis, we performed an analysis of data collected at three experiments involving complex military operations. Results highlight the significant role of leaders' SA on their team's SA.

Is situation awareness all in the mind?

This paper addresses the fundamental discipline theoretic question of whether situation awareness is a phenomenon best described by psychology, engineering or systems ergonomics. Each of these disciplines places a different emphasis on the notion of what situation awareness is and how it manifests itself. The approach from psychology places situation awareness as something that can only exist in the minds of people in a system. This means that the unit of analysis is the individual and that team situation awareness is the summation of individual situation awareness. The engineering perspective puts situation awareness in the world, represented in the artefacts and objects that people use. This means that the unit of analysis is the things that people interact with. Finally, the systems ergonomics perspective places emphasis on the interaction between people and their artefacts in the world, to propose that situation awareness functions like distributed cognition. This means that the unit of analysis is the whole socio-technical system. Each of these perspectives is presented and compared with reference to studies in aviation and other domains. It is concluded that the distributed cognition perspective of situation awareness offers the most comprehensive explanation of the phenomena observed in socio-technical systems. Socio-technical systems theory allows exploration of the social and technical sub-systems independently, which offers a theoretical framework for aligning the three views of situation awareness.

Team situation awareness measure using semantic utility functions for supporting dynamic decision-making

Team decision-making is a remarkable feature in a complex dynamic decision environment, which can be supported by team situation awareness. In this paper, a team situation awareness measure (TSAM) method using a semantic utility function is proposed. The semantic utility function is used to clarify the semantics of qualitative information expressed in linguistic terms. The individual and team situation awareness are treated as linguistic possibility distributions on the potential decisions in a dynamic decision environment. In the TSAM method, team situation awareness is generated through reasoning and aggregating individual situation awareness based on a multi-level hierarchy mental model of the team. Individual and team mental models are composed of key drivers and significant variables. An illustrative example in telecoms customer churn prediction is given to explain the effectiveness and the main steps of the TSAM method.

Measuring and Predicting Shared Situation Awareness in Teams

In order to improve our understanding of situation awareness (SA) in teams performing in technologically advanced command, control, and communications (C3) operations, researchers need to develop valid approaches to assess both individual and shared SA. We investigated SA in an interdisciplinary military rescue operation training exercise. For this study, we developed procedures to measure the degree of shared SA between two team members and to improve the accuracy of their shared SA scores. We suggest that SA scores that are calculated using many existing methods may be inflated because they often fail to account for error in terms of both the amount of information that is thought to be relevant and in the accuracy of a person's knowledge of it. We calculated true SA scores that account for both of these types of error. The measures were then used to evaluate five potential predictors of shared SA. Our analysis suggested that failure to compensate for error in SA may lead to overestimation of performance in a situation. The results also revealed a significant relationship between shared SA and participants' distance from a central, joint service team, which acted as the organizational hub within the C3 structure. Shared SA was better the further away from the hub people were, which suggests that a person's role and position within an organization affects the level of shared SA that can be achieved with other individuals.

Genotype and phenotype schemata and their role in distributed situation awareness in collaborative systems

Situation awareness (SA) is a critical commodity for teams working in complex systems. This article builds on existing schema theory to postulate an explanation of how teams develop and maintain SA during collaborative activities. The perceptual-action cycle approach and schema theory are used to formulate a model of distributed SA. Extracts from a case study undertaken in the UK energy distribution domain are used to demonstrate the concept of genotype and phenotype schemata as distributed SA. The sub-concepts of compatible and transactive SA are also outlined and explored via the case study. The differences between this perspective and the more commonly cited ‘shared SA’ perspective is articulated. In conclusion, whilst the ideas presented in this article are quite different to those expressed by the dominant models of individual and team SA presented in the literature, it is contended that they are more appropriate for the study of SA in collaborative environments.

Situation awareness measurement: A review of applicability for C4i environments

The construct of situation awareness (SA) has become a core theme within the human factors (HF) research community. Consequently, there have been numerous attempts to develop reliable and valid measures of SA but there is a lack of techniques developed specifically for the assessment of SA in command, control, communication, computers and intelligence (C4i) environments. During the design, development and evaluation of novel systems, technology and procedures, valid and reliable situation awareness measurement techniques are required for the assessment of individual and team SA, in order to determine the improvements (or in some cases decrements) resulting from proposed design and technological interventions. The paper presents a review of existing situation awareness measurement techniques for their suitability for use in the assessment of SA in C4i environments. Seventeen SA measures were evaluated against a set of HF methods criteria. It was concluded that current SA measurement techniques are inadequate by themselves for use in the assessment of SA in C4i environments, and a multiple-measure approach utilising different approaches is recommended.

Using Signal Detection Theory to Measure Situation Awareness in Command and Control

This paper describes a technique for measuring situation awareness (SA) referred to as QUASA™ (Quantitative Analysis of Situation Awareness). QUASA™ uses the principles of signal detection theory to assess situation awareness. The technique is designed to provide objective measures of individual SA, dynamic SA and different components of SA. The paper describes the theoretical basis of QUASA™ and its application in a simulated command and control environment.

Determinants of Perceived Teamwork: Examination of Team Performance and Social Psychophysiology

Team situation awareness (SA) has not received the same amount of research attention as individual SA. Given a lack of established relationships between objective and perceived measures of teamwork, as well as between team psychophysiology and perceived teamwork, the present study examined these potential relationships during a two-person, continuous tracking task. Seventeen two-person teams of undergraduates performed a computer-based, simulated teleoperation task. This investigation tested for relationships between objective performance measures (task completion time, team tracking error, coordination and collision damage), psychophysiological measures (electrodermal activity, heart inter-beat interval and respiration), and three subjective measures (team coordination, task engagement and task difficulty). The results suggest that objective team performance and some social psychophysiological measures (heart cross-correlation) contribute to team member perceptions of teamwork. Theoretical and practical implications of using all three techniques to assess team SA are discussed.

Supporting Situation Awareness of Individuals and Teams Using Group View Displays

This paper describes a project to design and test a large wall-mounted group view display intended to support situation awareness of individuals and multi-person teams in an advanced power plant control room. The design process began with an analysis of the requirements for situation awareness of individuals and teams to establish the design basis. A function-based cognitive task analysis was then used to define the plant state information to be included in the group view display. A phased test program was then initiated to establish the adequacy of the design concept. The first study employed sequences of static displays representing “snapshots” of evolving normal and emergency events. The second study tested dynamic group view displays (three 100 inch wall mounted displays) driven by a high-fidelity plant simulation. The study compared operator performance with the function-based group view display to operator performance using a more traditional physical mimic overview display. Dependent measures included (1) objective measures of performance, (2) operator ratings of the displays, and (3) workload measures. The measures provided statistically significant, converging evidence of the superiority of the function-based group view display. The implication of the results for design of group view displays to support individuals and teams are discussed.