Human Factors Research Research Articles

Ironies of Generative AI: Understanding and Mitigating Productivity Loss in Human-AI Interaction

Generative AI (GenAI) systems offer opportunities to increase user productivity in many tasks, such as programming and writing. However, while they boost productivity in some studies, many others show that users are working ineffectively with GenAI systems and losing productivity. Despite the apparent novelty of these usability challenges, these ‘ironies of automation’ have been observed for over three decades in Human Factors research on the introduction of automation in domains such as aviation, automated driving, and intelligence. We draw on this extensive research alongside recent GenAI user studies to outline four key reasons for productivity loss with GenAI systems: a shift in users’ roles from production to evaluation, unhelpful restructuring of workflows, interruptions, and a tendency for automation to make easy tasks easier and hard tasks harder. We then suggest how Human Factors research can also inform GenAI system design to mitigate productivity loss by using approaches such as continuous feedback, system personalization, ecological interface design, task stabilization, and clear task allocation. Thus, we ground developments in GenAI system usability in decades of Human Factors research, ensuring that the design of human-AI interactions in this rapidly moving field learns from history instead of repeating it.

Cooperation objective evaluation in aviation: validation and comparison of two novel approaches in simulated environment.

In operational environments, human interaction and cooperation between individuals are critical to efficiency and safety. These states are influenced by individuals' cognitive and emotional states. Human factor research aims to objectively quantify these states to prevent human error and maintain constant performances, particularly in high-risk settings such as aviation, where human error and performance account for a significant portion of accidents. Thus, this study aimed to evaluate and validate two novel methods for assessing the degree of cooperation among professional pilots engaged in real-flight simulation tasks. In addition, the study aimed to assess the ability of the proposed metrics to differentiate between the expertise levels of operating crews based on their levels of cooperation. Eight crews were involved in the experiments, consisting of four crews of Unexperienced pilots and four crews of Experienced pilots. An expert trainer, simulating air traffic management communication on one side and acting as a subject matter expert on the other, provided external evaluations of the pilots' mental states during the simulation. The two novel approaches introduced in this study were formulated based on circular correlation and mutual information techniques. The findings demonstrated the possibility of quantifying cooperation levels among pilots during realistic flight simulations. In addition, cooperation time is found to be significantly higher (p < 0.05) among Experienced pilots compared to Unexperienced ones. Furthermore, these preliminary results exhibited significant correlations (p < 0.05) with subjective and behavioral measures collected every 30 s during the task, confirming their reliability.

Assessment of Electric Grid Transmission System Simulator for Human Factors Research

Assessment of Electric Grid Transmission System Simulator for Human Factors Research

A Review of Human Factors Research on People Who Crochet

A Review of Human Factors Research on People Who Crochet

The Status of Human Factors Research in Cancer Care: A Look at the Landscape

The Status of Human Factors Research in Cancer Care: A Look at the Landscape

Using a Critical FMEA Approach to Identify Equity-Related Failures in Usability Evaluations of Health Technologies

Design issues that affect the usability of health technologies for marginalized populations may lead to further exacerbating health inequity. To address this concern, we propose the use of failure modes and effects analysis (FMEA) to systematically identify potential inequities that result from usability evaluation methods. This new application of FMEA grounds the traditional FMEA in critical theory and introduces two key concepts: the equity failure mode and reflexivity. To test our approach, we engaged 13 usability practitioners with expertise in healthcare in a series of four workshops. We found that when participants located themselves reflexively in their practice of usability evaluation, they were able to identify more nuanced equity failure modes. Through reflecting on our experience using this method, we aim to illustrate that the critical FMEA is a viable approach human factors practitioners and researchers can use to anticipate equity failure modes in design and evaluation methods.

Toward User-centered Explainable Displays for Complex Machine Learning Models in Healthcare: A Case Study of Heart Disease Prediction

The current approaches to explaining black box machine learning models have primarily been based on the intuition of model developers, rather than being informed by end-user needs or existing literature. Our goal is to utilize existing cognitive science and human factors research to design explanation displays. To achieve this, we used the Cleveland Heart Disease Data Set to create an eXtreme Gradient Boosting heart disease prediction model. We established an initial context of use to inform the design of a prototype explanation display. Our design choices were based on cognitive chunk organization, and we used SHapley Additive exPlanation to generate instance-level explanations for our model. Model evaluation showed good performance, and we developed four prototype explanation displays. Our work demonstrates that it is feasible to design multiple prototype explanation displays for complex machine learning models by organizing features in a structured manner. We also provide a set of steps that can be followed for designing and evaluating user-centered explanations in healthcare.

Visual Mapping of Operating Theater Team Dynamics and Communication for Reflexive Feedback and Surgical Practice Optimization.

Effective operating theater (OT) communication and teamwork are essential to optimal surgical outcomes. We mapped the OT team dynamics and infection control practices using visual methods to guide reflexive feedback and optimize perioperative practices. Data were gathered from adult gastrointestinal surgical teams at a tertiary hospital in India using observations, sociograms (communication mapping tool), and focus group discussions (FGDs). Our methods aimed to map team communication, roles and responsibilities in infection-related practices, and door openings. Qualitative data were thematically analyzed. Quantitative data were analyzed using descriptive statistics. Data were gathered from 10 surgical procedures (over 51 hours) using 16 sociograms, 15 traffic flow maps, and 3 FGDs. Senior surgeons directly influence team hierarchies, dynamics, and communication. While the surgeons, anesthetic residents, and technicians lead most tasks during procedures, the scrub nurse acts as a mediator coordinating activity among role players across hierarchies. Failing to provide the scrub nurse with complete details of the planned surgery leads to multiple door openings to fetch equipment and disposables. Traffic flow observed in 15-minute intervals corresponds to a mean frequency of 56 door openings per hour (min: 16; max: 108), with implications for infection control. Implementing the World Health Organization surgical safety checklist was inconsistent across pathways and does not match reported compliance data. Human factors research is important in optimizing surgical teamwork. Using visual methods to provide feedback to perioperative teams on their communication patterns and behaviors, provided an opportunity for contextualized enhancement of infection prevention and control practices.

Review of the Impacts of Human Factors on Cycling: Perceptions, Workload, and Behavior

Cycling remains a popular mode of transportation, yet cyclists are vulnerable road users that face numerous safety challenges. Although human factors research typically focuses on motor vehicle drivers, studies addressing active transportation users, like cyclists, are scarce. The unique aspects of cycling, such as physical effort, exposure to the environment, and disconnected infrastructure, can affect cyclists’ mental perception, workload, and behavior, which are argued to influence their safety on the road. Therefore, this scoping literature review identified factors influencing cyclists’ workload and explored different measures used to quantify mental workload. The findings highlighted age, infrastructure, portable devices, and type of bike as factors that could affect workload levels. However, research on cyclists’ workload from their perspective is limited. This paper summarizes three types of workload measure: subjective, performance, and physiological. These have been used to quantify workload in relation to cyclists and in other settings. We reflect on their benefits and challenges were they to be used to quantify cyclists’ workload. Our discussion emphasizes the need for future research to take a comprehensive approach that considers multiple factors simultaneously to gain a more holistic understanding of their collective impact on cyclists’ mental workload. Moreover, we emphasize the importance of supplementing subjective workload measures with psychophysiological ones for better accuracy and reliability. The review revealed a lack of data and guidelines specific to cycling infrastructure, contributing to cyclists’ vulnerability, and underscored the need for previous findings to be translated into actionable recommendations to improve cyclist safety.

How will technology change people’s home care in the next 20 years? A strategic foresight study

The rapid expansion of home health care has raised many unresolved issues and will have far-reaching consequences that can only be overcome with a holistic approach to help build and use collective intelligence in a structured, systemic way to anticipate developments. In this frame, the set of issues covered by the human factors research field will significantly impact the safety, quality, and effectiveness of home health care. However, only with a gaze of strategic foresight will we be capable of exploring, anticipating, and shaping the future. A group of researchers from the Italian Society of Ergonomics and Human Factors (SIE) has developed a road map to help all the stakeholders involved in this process.

Correction to: Assessing human emotional responses to the design of public spaces around subway stations: a human factors research

Correction to: Assessing human emotional responses to the design of public spaces around subway stations: a human factors research

Adopting Stimulus Detection Tasks for Cognitive Workload Assessment: Some Considerations.

This article tackles the issue of correct data interpretation when using stimulus detection tasks for determining the operator's workload. Stimulus detection tasks are a relative simple and inexpensive means of measuring the operator's state. While stimulus detection tasks may be better geared to measure conditions of high workload, adopting this approach for the assessment of low workload may be more problematic. This mini-review details the use of common stimulus detection tasks and their contributions to the Human Factors practice. It also borrows from the conceptual framework of the inverted-U shape model to discuss the issue of data interpretation. The evidence being discussed here highlights a clear limitation of stimulus detection task paradigms. There is an inherent risk in using a unidimensional tool like stimulus detection tasks as the primary source of information for determining the operator's psychophysiological state. Two recommendations are put forward to Human Factors researchers and practitioners dealing with the interpretation conundrum of dealing with stimulus detection tasks.

Using the Operating Room Black Box to Assess Surgical Team Member Adaptation Under Uncertainty: An Observational Study.

Identify how surgical team members uniquely contribute to teamwork and adapt their teamwork skills during instances of uncertainty. The importance of surgical teamwork in preventing patient harm is well documented. Yet, little is known about how key roles (nurse, anesthesiologist, surgeon, and medical trainee) uniquely contribute to teamwork during instances of uncertainty, particularly when adapting to and rectifying an intraoperative adverse event (IAE). Audiovisual data of 23 laparoscopic cases from a large community teaching hospital were prospectively captured using OR Black Box. Human factors researchers retrospectively coded videos for teamwork skills (backup behavior, coordination, psychological safety, situation assessment, team decision-making, and leadership) by team role under 2 conditions of uncertainty: associated with an IAE versus no IAE. Surgeons identified IAEs. In all, 1015 instances of teamwork skills were observed. Nurses adapted to IAEs by expressing more backup behavior skills (5.3× increase; 13.9 instances/hour during an IAE vs 2.2 instances/hour when no IAE) while surgeons and medical trainees expressed more psychological safety skills (surgeons: 3.6× increase; 30.0 instances/hour vs 6.6 instances/hour and trainees: 6.6× increase; 31.2 instances/hour vs 4.1 instances/hour). All roles expressed fewer situation assessment skills during an IAE versus no IAE. OR Black Box enabled the assessment of critically important details about how team members uniquely contribute during instances of uncertainty. Some teamwork skills were amplified, while others dampened when dealing with IAEs. The knowledge of how each role contributes to teamwork and adapts to IAEs should be used to inform the design of tailored interventions to strengthen interprofessional teamwork.

Water, lava, and wind

Abstract Small unmanned aerial systems (sUAS) are used more regularly and widely in disaster response. Like other personnel involved in disaster response, the sUAS pilots work for long periods, experience extreme stress and fatigue. They often arrive at the disaster fatigued (due to long drives to get there). However, unlike other personnel in this domain, there is little research on the effects of fatigue on sUAS pilots. Our experiences with a series of three real-world deployments highlight the challenges of conducting human factors research during disaster response and recovery. We specifically present lessons learned from having participant researchers embedded in three disasters with the sUAS pilot teams. These lessons result in a set of feasible and non-interruptive methods and metrics for conducting human factors research during field events. Preliminary results and recommended next steps are presented.

Holistic Human-Serving Digitization of Health Care Needs Integrated Automated System-Level Assessment Tools.

Digital health tools, platforms, and artificial intelligence- or machine learning-based clinical decision support systems are increasingly part of health delivery approaches, with an ever-greater degree of system interaction. Critical to the successful deployment of these tools is their functional integration into existing clinical routines and workflows. This depends on system interoperability and on intuitive and safe user interface design. The importance of minimizing emergent workflow stress through human factors research and purposeful design for integration cannot be overstated. Usability of tools in practice is as important as algorithm quality. Regulatory and health technology assessment frameworks recognize the importance of these factors to a certain extent, but their focus remains mainly on the individual product rather than on emergent system and workflow effects. The measurement of performance and user experience has so far been performed in ad hoc, nonstandardized ways by individual actors using their own evaluation approaches. We propose that a standard framework for system-level and holistic evaluation could be built into interacting digital systems to enable systematic and standardized system-wide, multiproduct, postmarket surveillance and technology assessment. Such a system could be made available to developers through regulatory or assessment bodies as an application programming interface and could be a requirement for digital tool certification, just as interoperability is. This would enable health systems and tool developers to collect system-level data directly from real device use cases, enabling the controlled and safe delivery of systematic quality assessment or improvement studies suitable for the complexity and interconnectedness of clinical workflows using developing digital health technologies.

The Effect of the Presence of Others in Ergonomics and Human Factors Research

The Article Abstract is not available.

Reviews and prospects of human factors research on curve driving

Reviews and prospects of human factors research on curve driving

From the Flight Deck to the Bedside: Core Aviation Concepts Applied to Acute Care Physical Therapist Practice and Education.

The health care industry has sought to reduce errors and patient harm by drawing upon human factors research developed largely in the aviation industry. However, literature that explicitly applies human factors concepts to physical therapist practice is scarce. Therefore, this clinical perspective highlights the key concepts in aviation safety and training that should be considered for application to physical therapist education and practice in the acute care setting, including the law of primacy, pilot in command, crew resource management, situational awareness, the instrument scan, currency versus proficiency, the advanced qualification program, and the use of checklists. The authors identify, discuss, and apply existing parallel concepts in physical therapist literature. The authors then challenge acute care physical therapists, academic faculty, and clinical instructors on ways to integrate these concepts in practice.

Rethinking Human Factors Methodologies: User Needs &amp; Applied Domain Characteristics

Technologies and systems have become more complex with the advancement of modern digitalization. Human factors practitioners and researchers face challenges in designing products for everyday activities and complex domains. Learning one human factors methodology at a time is the most common approach, and finding complementary methodologies is sometimes difficult. In this paper, we summarize achievements needed in human factors research in three categories: motivation-related needs, task-related needs, and applied domain assumptions and characteristics. Some common methodologies are discussed, and we briefly introduce how to implement them in general resilient and cyber-resilient systems.

Shifting Perspectives: A proposed framework for analyzing head-mounted eye-tracking data with dynamic areas of interest and dynamic scenes.

Eye-tracking is a valuable research method for understanding human cognition and is readily employed in human factors research, including human factors in healthcare. While wearable mobile eye trackers have become more readily available, there are no existing analysis methods for accurately and efficiently mapping dynamic gaze data on dynamic areas of interest (AOIs), which limits their utility in human factors research. The purpose of this paper was to outline a proposed framework for automating the analysis of dynamic areas of interest by integrating computer vision and machine learning (CVML). The framework is then tested using a use-case of a Central Venous Catheterization trainer with six dynamic AOIs. While the results of the validity trial indicate there is room for improvement in the CVML method proposed, the framework provides direction and guidance for human factors researchers using dynamic AOIs.