Human Factors Usability Research Articles

Usability Testing Via Simulation: Optimizing the NEAR4PEM Preintubation Checklist With a Human Factors Approach.

To inform development of a preintubation checklist for pediatric emergency departments via multicenter usability testing of a prototype checklist. This was a prospective, mixed methods study across 7 sites in the National Emergency Airway Registry for Pediatric Emergency Medicine (NEAR4PEM) collaborative. Pediatric emergency medicine attending physicians and senior fellows at each site were first oriented to a checklist prototype, including content previously identified using a modified Delphi approach. Each site used the checklist in 2 simulated cases: an "easy airway" and a "difficult airway" scenario. Facilitators recorded verbalization, completion, and timing of checklist items. After each simulation, participants completed an anonymous usability survey. Structured debriefings were used to gather additional feedback on checklist usability. Comments from the surveys and debriefing were qualitatively analyzed using a framework approach. Responses informed human factors-based optimization of the checklist. Fifty-five pediatric emergency medicine physicians/fellows (4-13 per site) participated. Participants found the prototype checklist to be helpful, easy to use, clear, and of appropriate length. During the simulations, 93% of checklist items were verbalized and more than 80% were completed. Median time to checklist completion was 6.2 minutes (interquartile range, 4.8-7.1) for the first scenario and 4.2 minutes (interquartile range, 2.7-5.8) for the second. Survey and debriefing data identified the following strengths: facilitating a shared mental model, cognitively offloading the team leader, and prompting contingency planning. Suggestions for checklist improvement included clarifying specific items, providing more detailed prompts, and allowing institution-specific customization. Integration of these data with human factors heuristic inspection resulted in a final checklist. Simulation-based, human factors usability testing of the National Emergency Airway Registry for Pediatric Emergency Medicine Preintubation Checklist allowed optimization prior to clinical implementation. Next steps involve integration into real-world settings utilizing rigorous implementation science strategies, with concurrent evaluation of the impact on patient outcomes and safety.

Human Factors Usability Analysis of the ArthroFree Wireless Surgical Camera

Conventional arthroscopy is performed with a narrow tube, the arthroscope, attached to a camera and two cables - one for power and one for fiber-optic light. The cables can be cumbersome, routed across or around the patient, and risk bridging the sterile and nonsterile fields. ArthroFree®, a wireless surgical camera for arthroscopy, features wireless communication and battery power and was designed to eliminate cable-associated burdens. An iterative product development process established the usability of ArthroFree. Feedback on ArthroFree’s design was received from 88 participants (82 surgeons and 6 other medical professionals) on ergonomics, ease of use, image quality, and patient experience following design validation testing (pre-FDA clearance; 76 participants) or clinical evaluation (post-FDA clearance; 12 participants). Participants were trained on the use of ArthroFree and then used the device in a surgical procedure performed on a cadaver, simulated model, or patient. Their experience was evaluated in a 13-statement human factors survey scored using a 5-point Likert scale. ArthroFree scored well in all survey statements, scoring “good” or “excellent” (4.5 mean score), suggesting users’ needs were satisfied in 91.30% of participants. Additionally, one-on-one interviews with surgeons gathered qualitative feedback on their independent experience with the ArthroFree device, providing key support for its introduction to the operating room.

Usability of an At-Home Anterior Nares SARS-CoV-2 RT-PCR Sample Collection Kit: Human Factors Feasibility Study.

BackgroundReadily available testing for SARS-CoV-2 is necessary to mitigate COVID-19 disease outbreaks. At-home collection kits, in which samples are self-collected without requiring a laboratory or clinic visit and sent to an external laboratory for testing, can provide convenient testing to those with barriers to access. They can prevent unnecessary exposure between patient and clinical staff, increase access for patients with disabilities or remote workers, and decrease burdens on health care resources, such as provider time and personal protective equipment. Exact Sciences developed an at-home collection kit for samples to be tested to detect SARS-CoV-2 that includes an Instructions for Use (IFU) document, which guides people without prior experience on collecting a nasal swab sample. Demonstrating successful sample collection and usability is critical to ensure that these samples meet the same high-quality sample collection standards as samples collected in clinics.ObjectiveThe aim of this study was to determine the usability of a SARS-CoV-2 at-home nasal swab sample collection kit.MethodsA human factors usability study was conducted with 30 subjects without prior medical, laboratory, or health care training and without COVID-19 sample self-collection experience. Subjects were observed while they followed the IFU for the at-home sample collection portion of the SARS-CoV-2 test in a setting that simulated a home environment. IFU usability was further evaluated by requiring the subjects to complete a survey, answer comprehension questions, provide written feedback, and respond to questions from the observer about problems during use.ResultsAll 30 subjects successfully completed the sample collection process, and all 30 samples were determined by reverse transcription–polymerase chain reaction (RT-PCR) testing to meet quality standards for SARS-CoV-2 testing. The subjects’ written feedback and comments revealed several recommendations to improve the IFU.ConclusionsThe study demonstrated the overall usability of an at-home SARS-CoV-2 collection kit. Various feedback mechanisms provided opportunities to improve the wording and graphics for some critical tasks, including placing the label correctly on the tube. A modified IFU was prepared based on study outcomes.

Moderation is the Key to Successful Human Factors Studies

Human factors usability studies play an important role in the product development process. To yield valid and useful data from these formative and validation studies there must be effective moderation. In study sessions with participants, moderators must play the role of unbiased facilitator, perceptive observer, and curious researcher. In all studies, positive interactions and effective communication between moderator and participant can establish the rapport necessary to spark meaningful and honest discussions. From the conversational focus group to the passive moderation of a validation study, involvement and interactions with participants can vary greatly. Furthermore, consideration must be given to the stage of device development and the goals of the study. Strong moderation is the cornerstone of any successful usability study, opening the doors to valuable insights and thorough evaluations.

A pen device for injection of recombinant human growth hormone: a European usability engineering study

ABSTRACT Objective The Aluetta™ reusable pen device and instructions for use (IFU) for growth hormone (r-hGH; Saizen®, Merck KGaA, Darmstadt, Germany) administration were tested for Human-Factors Usability, to ensure it could be used safely and effectively by the intended users in the intended use environment. Research design and methods Usability testing was conducted under simulated conditions in three groups of participants: pediatric or adult patients with growth hormone deficiency (GHD), participants without GHD, and healthcare professionals (HCPs). The testing comprised a 45-minute training session, a 2-hour testing session, and a participant-feedback session. Results Twenty-six participants completed the training session and performed all critical tasks related to the pen use across three scenarios. The most difficult tasks were related to the preparation, checking, and maintenance of the device; only 8% of use errors occurred during tasks related to the injection process. Eighty-five percent considered the pen safe and effective to use without further modifications and the training to be clear and effective. Conclusions The pen device and associated materials benefited from Human Factors Engineering throughout the development process. These evaluations show that patients and HCPs could safely and effectively use the pen device, and the IFU and training were clear and effective.

Usability and preference of epinephrine auto-injectors: Auvi-Q and EpiPen Jr

BackgroundDespite the importance of prompt epinephrine auto-injector (EAI) treatment during anaphylaxis, proper administration technique is often lacking among patients and caregivers. ObjectiveTo compare usability and participant preference of Auvi-Q and EpiPen Jr EAIs in a simulated life-threatening allergic emergency-use scenario. MethodsIn this randomized, crossover, human-factors usability study, untrained adults (18-65 years) were tasked with using 0.15 mg Auvi-Q and EpiPen Jr trainers to simulate epinephrine administration to a child-sized manikin. Only written instructions on the device label and/or device voice instructions were available to participants. Endpoints included completing injection tasks per device instructions (primary endpoint), completing key injection tasks, and participant preference/ratings of devices. Completion of injection tasks were evaluated using a McNemar test for paired dichotomous data. ResultsNinety-six adults were included in study analyses. Significantly more participants completed all injection tasks per device instructions with Auvi-Q (85.4%) vs EpiPen Jr (19.8%; P < .001). Significant differences were also observed for completion of key injection tasks (Auvi -Q, 94.8%; EpiPen Jr, 72.9%; P < .001). No digital/hand injection errors were seen with Auvi-Q, whereas 14 participants (14.6%) would have accidentally received a digital/hand injection with EpiPen Jr (P < .001). Overall, significantly more participants preferred Auvi-Q over EpiPen Jr (91.7% vs 6.3%; P < .001 [2.1% no preference]). Median scores for 8 EAI characteristics were also higher for Auvi-Q vs EpiPen Jr. ConclusionIn this study, untrained adults preferred and were more likely to use Auvi-Q correctly vs EpiPen Jr, highlighting the importance of device design for successful epinephrine administration during a life-threatening allergic emergency.

Human factors engineering validation study for a novel 0.1-mg epinephrine auto-injector.

Background: Anaphylaxis in infants and young children is increasing. Historically, epinephrine auto-injectors (EAI) were not available in a dose or platform designed for patients who weighed <15 kg, and, therefore, 0.15-mg EAIs were prescribed for these patients. Results of ultrasound studies indicate that currently marketed 0.15-mg EAIs have needle lengths that may strike bone in ∼29 to 43% of patients who weigh <15 kg and result in possible intraosseous injection. An EAI with a more weight-appropriate 0.1-mg dose and shorter needle length has been developed to potentially help minimize the risk of striking bone during epinephrine injection in patients who weigh 7.5-15 kg. Objective: A human factors usability study was completed to validate the 0.1-mg EAI user interface. Methods: This study was conducted with parents of children who were severely allergic so to evaluate simulated use of the 0.1-mg EAI by the intended user group. Fifteen participants were enrolled and received training on using the 0.1-mg EAI. Approximately 24 hours later, the participants completed a simulated emergency-use scenario by using the 0.1-mg EAI with an infant manikin. The primary end point was successful simulated administration of a meaningful epinephrine dose. Results: All the participants simulated administration of a meaningful epinephrine dose. Fourteen participants successfully used the 0.1-mg EAI per the instructions for use in the simulated emergency-use scenario. One participant did not press the EAI against the thigh for the length of time defined for this critical task; however, the EAI was pressed long enough for complete delivery of the 0.1-mg dose. Conclusion: This study validated the user interface of the 0.1-mg EAI for the intended users, uses, and use environments.

Iterative Implementation of a Remote Cardiac Patient Monitoring Device using Qualitative Analysis and Human Factors Engineering

Remote monitoring has the potential to improve patient care and satisfaction while reducing the cost of care and burden on the healthcare system for Congestive Heart Failure (CHF) and Atrial Fibrillation (AF) patients. However, the device must be convenient and easy for the population to use correctly. In this study, human factors and qualitative researchers evaluated the use of a remote cardiac monitoring device during a clinical trial with 74 CHF and AF patients. Human factors usability concerns centered on device use and the information presented in the manual. The key themes identified during the qualitative analysis were related to patient education. These findings were incorporated into device design, user manual, or education material updates. This study illustrates how these methods can infuse patient experience into device design to inform continued quality improvement and usability refinement specifically in devices to aid CHF and AF patients without added burden.

The Impact of Environmental Design on Doffing Personal Protective Equipment in a Healthcare Environment: A Formative Human Factors Trial.

OBJECTIVE To explore the impact of environmental design on doffing personal protective equipment in a simulated healthcare environment. METHODS A mixed-methods approach was used that included human-factors usability testing and qualitative questionnaire responses. A patient room and connecting anteroom were constructed for testing purposes. This experimental doffing area was designed to overcome the environmental failures identified in a previous study and was not constructed based on any generalizable hospital standard. RESULTS In total, 72 healthcare workers from Ontario, Canada, took part in the study and tested the simulated doffing area. The following environmental design changes were tested and were deemed effective: increasing prominence of color-coded zones; securing disinfectant wipes and hand sanitizer; outlining disposal bins locations; providing mirrors to detect possible contamination; providing hand rails to assist with doffing; and restricting the space to doff. Further experimentation and iterative design are required with regard to several important features: positioning the disposal bins for safety, decreasing the risk of contamination and user accessibility; optimal positioning of mirrors for safety; communication within the team; and positioning the secondary team member for optimal awareness. Additional design suggestions also emerged during this study, and they require future investigation. CONCLUSIONS This study highlights the importance of the environment on doffing personal protective equipment in a healthcare setting. Iterative testing and modification of the design of the environment (doffing area) are important to enhancing healthcare worker safety. Infect Control Hosp Epidemiol 2017;38:712-717.

Usability Evaluation of a Blood Glucose Monitoring System With a Spill-Resistant Vial, Easier Strip Handling, and Connectivity to a Mobile App: Improvement of Patient Convenience and Satisfaction.

Proper management of diabetes mellitus requires regular self-monitoring of blood glucose (SMBG). This research evaluated the usability of the Accu-Chek® Guide Meter that includes a spill-resistant vial, easier strip handling, and wireless connectivity to a mobile app. A total of 197 participants were allowed to experience typical blood glucose testing tasks on the Accu-Chek Guide Meter, review data such as last result, patterns, and target percentage on the meter and on the mobile app, and then evaluate their experience through a human factors usability survey. Participants used a 6-point agree/disagree scale to rate 34 market statement questions. The results of a Pearson chi-square proportions test for each of the 34 market statement questions showed a significant difference (P < .0001) between the disagree responses (1-3) and agree responses (4-6). An overwhelming majority of participants found all aspects of the system, including the test strips, strip vial, and data analysis on the meter and the mobile app, to be a good fit for their lifestyle and to provide a better testing experience. This study found superior usability of the new meter system over the participants' current meters in both the United States and France.

Mental Models and Deliberate Manipulation of Data

Human factors risk analysis and usability tests in telemedicine contexts primarily seek to investigate how to reduce the likelihood of slips, lapses and mistakes. However, to ensure trustworthiness in e-health data, one must also be aware of more intentional patient actions which could potentially compromise patient safety and/or the integrity of the system. The pilot study in this paper set out to explore mental models and deliberate manipulation of data in a Danish telemedicine setting of home monitoring among pregnant women. Results show, that patients construct mental models of the telemedicine system – and that the patient can utilize such mental models in attempts to manipulate their data input to get a desired output from the telemedicine system.

Evaluating the Labeling-User Interface and Device Design-User Interface in Usability Validation Studies for Combination Products: Resolving the Catch-22

The strategy of the Center for Devices and Radiological Health (CDRH) at the Food and Drug Administration (FDA) for usability validation of medical devices is aimed at the priority of evaluating device design as a means to reduce risks through mitigation of use-errors that may present serious harms to end-users and patients. In contrast, the strategy of the Center for Drug Evaluation and Research (CDER) is aimed at evaluating product packaging and labeling to ensure end-user comprehension of drug and delivery device instructions-for-use (IFU) and other labeling as a means to minimize risks for medication errors that may result in harms to the patient or ineffective drug treatment. Although both strategies align to a common purpose, each FDA Center's requirement often presents a Catch-22, particularly when product Sponsors postpone usability evaluations and try to combine each Center's requirements for the purpose of fulfilling usability validation requirements. Although not the ideal strategy, the purpose of this presentation is to discuss and compare past-proven methods that have been successful for satisfying the usability validation testing requirements of both CDRH and CDER for combination products when Sponsors have not conducted usability evaluations throughout a product's development. This in-depth presentation will include a discussion of Centers' requirements that are in direct opposition to each other and alternative methods for usability validation testing to satisfy both Centers. Strengths and limitations of each strategy will be discussed in relation to the overall usability validation testing process. Recognizing and understanding usability validation testing strategies of CDRH and CDER that are in direct opposition of each other in advance is essential for applying a human factors usability strategy that leads to the successful validation of a combination product. Knowledge of alternative testing strategies, each with their own strengths and limitations, designed to meet expectations of both Centers allows HF usability investigators to select and/or tailor the approach that best meets the unique challenges of their investigational combination product in order to efficiently pass validation from both Centers and enter the market.

Human factors studies for injectable combination products: from planning to reporting.

Human factors is the study of the interaction of people and technology to ensure the safety and effectiveness of that interaction and to improve human/device compatibility, including the user interface, instructions, and training programs to avoid use error. The enhanced focus on human factors usability assessments for injectable combination products is an acknowledgement by regulators and industry that the device mechanics are typically quite reliable and that device risk hazards are likely due to device usability. Use errors can occur when the device is not being used as intended or the design features are less than optimal. Human factors testing, analysis, and validation helps to identify and eliminate use errors by informing appropriate mitigation strategies to ensure the device design provides the optimum use of injectable drug/device combination products.

Redesigning Transplant Organ Labeling: An Innovative Approach to Improve Patient Safety.

Background: Over 28,000 organ transplants are performed in the US annually and labels used for procurement are still completed by hand. Organ transplantation lacks a systematic approach to evaluate potential vulnerabilities in the labeling and identification process. A comprehensive review of organ procurement safety incidents identified “labeling” as the most common incident. As part of a larger HHS Innovation Initiative, we conducted a three phase project to improve the organ labeling and identification process: (1) Risk assessment to identify and rank failures and underlying causes (2) Development and testing of a tablet application (“app”) to generate and wirelessly print standardized, barcoded labels and Donor ID bands; and (3) Use of laboratory-based simulations to assess organ procurement workflow with new “app.” Methods: A Failure Mode Effects and Criticality Analysis (FMECA) was conducted for the deceased donor abdominal organ procurement process focused on steps related to labeling and organ identification. UNOS representatives, clinicians, and staff from selected Organ Procurement Organizations (OPOs) and the local Transplant Center (TC) contributed to the creation of a comprehensive process map (42 individuals total). Potential failures for each process step were identified, scored, ranked by criticality, and then used to inform the design of the “app.” We conducted laboratory-based simulations to test integration of the “app” and wireless printer into procurement process, facilitated debriefings with participants to optimize workflow redesign, and human factors usability analysis. Results: The FMECA revealed a total of 146 potential failures from 60 identified process steps. Key “high criticality” risks include: Accuracy of donor information on the label; Identification of the laterality of a kidney; and Validation of receipt of the right donor organ for the right recipient. Significant features of the “app” include a double-entry to verify accuracy of donor information, standardized label format, and ability to wirelessly print labels, Donor ID bands, and shipping manifest for each procured organ at “bedside.” Conclusions: The “app” impacts 65% of the top 10 identified high criticality risks and should mitigate labeling errors that lead to additional testing, wrong donor organ-wrong recipient errors, and discarding of organs due to identification errors. Further validation of new technology must be conducted before implementation nationwide.

Medical Device Usability Analyses

Errors committed during healthcare providers’ use of patient controlled analgesia (PCA) systems in the acute care setting are poorly understood. A PCA is a complex, medical device that delivers a prescribed pain medication primarily through the intravenous route into the patient. The benefits of PCA use are numerous, but PCAs are also a known source of errors resulting in patient injury. Human factors usability analyses are methods to identify and mitigate factors that contribute to errors during user-device interaction. The purpose of this paper is to present the findings from an integrative review of published medical device usability analyses and to provide the foundation for future empirical research to improve our understanding of PCA system errors.

The recombinant human chorionic gonadotropin prefilled pen: results of patient and nurse human factors usability testing

Objectives: The first prefilled pen for administration of recombinant human chorionic gonadotropin (r-hCG) has been developed. Usability testing was undertaken to evaluate the risk of dosing errors versus the existing r-hCG prefilled syringe, and assess function and handling of the pen.Methods: Infertile women who were trying to conceive, and specialist nurses, were recruited in Germany. Usability goals were defined and categorized as critical or functional operational goals. Individual, non-interventional, standardized, usability tests (including ease-of-use assessment) were performed with patients and nurses. Cumulative test scores for critical operations were compared. Non-standardized qualitative analyses of nurse–patient training sessions were performed.Results: The cumulative test score for the r-hCG prefilled pen was better than that of the existing prefilled syringe, so it was concluded that the overall risk of dosing errors was not higher with the pen. The ease of use of the pen was rated favorably by patients and nurses. Both user groups were confident that they could inject the correct dose using the pen.Conclusions: The overall risk of dosing errors was not higher with the r-hCG prefilled pen than the existing prefilled syringe. The ease-of-use of the r-hCG prefilled pen was rated favorably by patients and nurses.

The redesigned follitropin alfa pen injector: results of the patient and nurse human factors usability testing

Objectives:A redesigned pen injector for administration of follitropin alfa (follitropin α) has been developed for use in fertility treatment cycles. Pre-summative and summative usability testing was undertaken to assess the risk of dosing errors compared with the existing follitropin α pen. The study also assessed proper use of and dose selection with the redesigned pen.Methods:Infertile women who were trying to conceive and specialist nurses were recruited from four cities in Germany. Usability goals relating to proper use of the pen device were defined from a risk assessment and further categorized as critical and functional operational goals. Individual, non-interventional, standardized, usability tests were performed with patients and nurses by four experienced research professionals using questionnaires that also included ease-of-use ratings. A non-standardized qualitative analysis of nurse–patient training sessions was performed in the presence of a research professional; reasons for confidence, safety, possible misunderstandings and risks when handling the pen were noted.Results:The overall risk of dosing errors with the redesigned pen was not higher than with the existing pen. No unexpected operational risks and no major concerns regarding the risk of misuse or dosing errors were identified.Conclusions:The study provides useful practical information on the redesigned pen from both patient and nurse perspectives.

Human Factors in Action: Getting "Pumped" at a Nursing Usability Laboratory

We describe our experience with a Nursing Usability Laboratory, where human factors design principles were applied to common nursing procedures. Our first step was to develop a human factors usability checklist. We then used this checklist while observing 11 nurses completing two standardized tasks on a simulated patient: (1) programming an insulin infusion and (2) programming a heparin infusion. We found that a usability checklist can help to uncover systematic error-provoking conditions in nursing tasks, that immediate improvements can be made in nursing training and practice and that participant nurses found the process useful. This paper will be of interest to any hospital seeking to enhance safety by applying human factors design principles.

Effects of the Space Shuttle Cockpit Avionics Upgrade on Crewmember Performance and Situation Awareness

The Space Shuttle Cockpit Avionics Upgrade (CAU) is a proposed cockpit display upgrade designed to address human-factors usability issues of the current suite of cockpit displays, Multifunction Electronic Display System (MEDS). Unlike MEDS, CAU consolidates information in a task-oriented manner, rather than a data-source-oriented manner. CAU also makes greater use of color coding and graphical depictions in systems status presentations. An ascent-phase operation simulation study showed that CAU formats significantly improved the participants' abort-related situation awareness. Participants also performed certain malfunction management procedures more accurately when CAU was used. The Space Shuttles are now scheduled to be retired by 2010 without incorporating CAU; however, the results of the present study suggest that the human-centered design concepts are effective and can be extended to the cockpit interface design of NASA's next generation Crew Exploration Vehicle.

Applying Human Factors to the Procurement of Electrosurgical Medical Devices: A Case Study

Human factors evaluations are currently not conducted as part of the procurement process for medical devices in most hospitals. The complexity of medical devices and interactions between those devices, the working environment and the people who use them can create a high potential for errors. This study reports on the methods used to integrate human factors usability testing into the product evaluation of electrosurgical units (ESU's) prior to procurement. It also comments on the results of the various testing methods and the impact of the results on the final purchasing decision. The results of the human factors evaluations were used to make a purchasing decision for a major metropolitan hospital in Canada. A new purchase was necessary because the manufacturer was no longer supporting the product in use. Surprisingly, the product of choice was the oldest on the market with few new features. It was preferred and chosen based on usability and clinical acceptance by all users.