Technology-induced Error Research Articles

Integrating Human Patterns of Qualitative Coding with Machine Learning: A Pilot Study Involving Technology-Induced Error Incident Reports.

The objective of this research was to develop a reproducible method of integrating human patterns of qualitative coding with machine learning. The application of qualitative codes from the technology-induced error and safety literatures to the analysis of incident reports was done successfully, helping to identify the factors that lead to an error as well as the errors themselves. The method described in this paper may provide additional insights into understanding technology-induced errors.

Using Publicly Available Recall and Safety Alert Reports for Learning from Technology-Induced Error.

Information technologies have the potential to increase the safety of healthcare and advance safety science. However, it is now well known that health information systems may also inadvertently introduce new forms of error known as technology-induced error. Such errors may be difficult to detect as they may only appear under conditions of system use in real healthcare settings. In this paper, the authors explore the use and assessment of recall and safety alerts for both identifying and learning from technology-induced error. Publically available safety and recall reports from Canada were analyzed to identify opportunities to improve organizational learning from technology-induced errors. Although a range of error types were identified, it was found that none of the reports provided detailed information about the underlying technical circumstances that led to the need for a recall. Implications for future reporting systems to support learning from technology-induced error are discussed.

A Safety Maturity Model for Technology-Induced Errors.

In this paper we describe a review of the literature and the development of an initial maturity model for the research literature in the area of technology-induced errors and health technology safety. The capability maturity model provides a way forward for organizations to formalize their health technology safety processes and also compare their efforts in this area to other organizations to support organizational learning. The application of maturity models to technology-induced error is described.

Classification of Electronic Health Record-Related Patient Safety Incidents: Development and Validation Study.

BackgroundIt is assumed that the implementation of health information technology introduces new vulnerabilities within a complex sociotechnical health care system, but no international consensus exists on a standardized format for enhancing the collection, analysis, and interpretation of technology-induced errors.ObjectiveThis study aims to develop a classification for patient safety incident reporting associated with the use of mature electronic health records (EHRs). It also aims to validate the classification by using a data set of incidents during a 6-month period immediately after the implementation of a new EHR system.MethodsThe starting point of the classification development was the Finnish Technology-Induced Error Risk Assessment Scale tool, based on research on commonly recognized error types. A multiprofessional research team used iterative tests on consensus building to develop a classification system. The final classification, with preliminary descriptions of classes, was validated by applying it to analyze EHR-related error incidents (n=428) during the implementation phase of a new EHR system and also to evaluate this classification’s characteristics and applicability for reporting incidents. Interrater agreement was applied.ResultsThe number of EHR-related patient safety incidents during the implementation period (n=501) was five-fold when compared with the preimplementation period (n=82). The literature identified new error types that were added to the emerging classification. Error types were adapted iteratively after several test rounds to develop a classification for reporting patient safety incidents in the clinical use of a high-maturity EHR system. Of the 427 classified patient safety incidents, interface problems accounted for 96 (22.5%) incident reports, usability problems for 73 (17.1%), documentation problems for 60 (14.1%), and clinical workflow problems for 33 (7.7%). Altogether, 20.8% (89/427) of reports were related to medication section problems, and downtime problems were rare (n=8). During the classification work, 14.8% (74/501) of reports of the original sample were rejected because of insufficient information, even though the reports were deemed to be related to EHRs. The interrater agreement during the blinded review was 97.7%.ConclusionsThis study presents a new classification for EHR-related patient safety incidents applicable to mature EHRs. The number of EHR-related patient safety incidents during the implementation period may reflect patient safety challenges during the implementation of a new type of high-maturity EHR system. The results indicate that the types of errors previously identified in the literature change with the EHR development cycle.

Behavioral signs of an unintended error in nursing information sharing with electronic clinical pathways: a mixed research approach

ABSTRACT Electronic clinical pathways (ECPs) strongly encourage the standardization of medical treatment and the sharing of information among medical staff. The goal of this study was to determine the influence of ECPs on information sharing among nurses in a university hospital. Four experienced nurses, selected based on ECP composing and operation experience, were recruited from the department with the most frequent users in the first-round interview, 132 nurses’ questionnaire answers were analyzed, and eight nurses participated in the second-round interview. This study conducted a mixed-method (interview–questionnaire–interview) investigation to extract the behavioral signs of unintended errors in information sharing after the ethical approval was obtained. On the basis of ANOVA and t-test for the questionnaire and constant comparison for interview, this study found that the greater extent of user dependency on convenient ECPs in the frequent-use group led to mistakes under hectic conditions. This study also found evidence of poor management of ECPs when problems occurred. The immature design of ECPs provoked inappropriate behaviors among nurses even though they brought about some benefits such as mitigation of the burden of daily recording tasks. The findings empirically showed the ECP user’s behavioral changes regarding the technology-induced error.

Methods for Addressing Technology-induced Errors: The Current State.

The objectives of this paper are to review and discuss the methods that are being used internationally to report on, mitigate, and eliminate technology-induced errors. The IMIA Working Group for Health Informatics for Patient Safety worked together to review and synthesize some of the main methods and approaches associated with technology- induced error reporting, reduction, and mitigation. The work involved a review of the evidence-based literature as well as guideline publications specific to health informatics. The paper presents a rich overview of current approaches, issues, and methods associated with: (1) safe HIT design, (2) safe HIT implementation, (3) reporting on technology-induced errors, (4) technology-induced error analysis, and (5) health information technology (HIT) risk management. The work is based on research from around the world. Internationally, researchers have been developing methods that can be used to identify, report on, mitigate, and eliminate technology-induced errors. Although there remain issues and challenges associated with the methodologies, they have been shown to improve the quality and safety of HIT. Since the first publications documenting technology-induced errors in healthcare in 2005, we have seen in a short 10 years researchers develop ways of identifying and addressing these types of errors. We have also seen organizations begin to use these approaches. Knowledge has been translated into practice in a short ten years whereas the norm for other research areas is of 20 years.

Reducing Technology-Induced Errors: Organizational and Health Systems Approaches.

Technology-induced errors are a growing concern for health care organizations. Such errors arise from the interaction between healthcare and information technology deployed in complex settings and contexts. As the number of health information technologies that are used to provide patient care rises so will the need to develop ways to improve the quality and safety of the technology that we use. The objective of the panel is to describe varying approaches to improving software safety from and organizational and health systems perspective. We define what a technology-induced error is. Then, we discuss how software design and testing can be used to improve health information technologies. This discussion is followed by work in the area of monitoring and reporting at a health district and national level. Lastly, we draw on the quality, safety and resilience literature. The target audience for this work are nursing and health informatics researchers, practitioners, administrators, policy makers and students.

National efforts to improve health information system safety in Canada, the United States of America and England

ObjectiveIn this paper we review progress as well as challenges encountered in Canada, the United States and England with regard to ensuring safety of health information technology. MethodA review of major programs and initiatives for ensuring safety of health information technology in the three countries was conducted. Published literature and Web resources from national programs were reviewed for relevant information. ResultsIt was found that in all three countries the issue of technology-induced error has been recognized as being of critical importance. The three countries have developed approaches for dealing with the issue that have some commonalities; however, they are at varying different stages of maturity, with England having the longest standing and most well developed safety programs, while Canada and the United States are at earlier stages. The types of approaches employed have included work on developing standards related to usability and interface design, certifications, directives from regulatory bodies, educational initiatives in health information technology (HIT) safety as well as research into safer HIT design and implementation methods. ConclusionsHIT promises to lead to improved patient safety. However, it has become recognized that if not designed and deployed appropriately, such systems can lead to new types of errors. Based on this recognition, a variety of initiatives are being undertaken in Canada, the United States and England to promote the safe design, procurement and deployment of HIT. It is concluded that improved approaches to system design, testing, regulation, error reporting, safety education and cross-country collaboration will be needed to further promote safer HIT.

Trends in health information technology safety: from technology-induced errors to current approaches for ensuring technology safety.

ObjectivesHealth information technology (HIT) research findings suggested that new healthcare technologies could reduce some types of medical errors while at the same time introducing classes of medical errors (i.e., technology-induced errors). Technology-induced errors have their origins in HIT, and/or HIT contribute to their occurrence. The objective of this paper is to review current trends in the published literature on HIT safety.MethodsA review and synthesis of the medical and life sciences literature focusing on the area of technology-induced error was conducted.ResultsThere were four main trends in the literature on technology-induced error. The following areas were addressed in the literature: definitions of technology-induced errors; models, frameworks and evidence for understanding how technology-induced errors occur; a discussion of monitoring; and methods for preventing and learning about technology-induced errors.ConclusionsThe literature focusing on technology-induced errors continues to grow. Research has focused on the defining what an error is, models and frameworks used to understand these new types of errors, monitoring of such errors and methods that can be used to prevent these errors. More research will be needed to better understand and mitigate these types of errors.

Technology-induced Errors

The objective of this paper is to examine the extent, range and scope to which frameworks, models and theories dealing with technology-induced error have arisen in the biomedical and life sciences literature as indexed by Medline®. To better understand the state of work in the area of technology-induced error involving frameworks, models and theories, the authors conducted a search of Medline® using selected key words identified from seminal articles in this research area. Articles were reviewed and those pertaining to frameworks, models or theories dealing with technology-induced error were further reviewed by two researchers. All articles from Medline® from its inception to April of 2011 were searched using the above outlined strategy. 239 citations were returned. Each of the abstracts for the 239 citations were reviewed by two researchers. Eleven articles met the criteria based on abstract review. These 11 articles were downloaded for further in-depth review. The majority of the articles obtained describe frameworks and models with reference to theories developed in other literatures outside of healthcare. The papers were grouped into several areas. It was found that articles drew mainly from three literatures: 1) the human factors literature (including human-computer interaction and cognition), 2) the organizational behavior/sociotechnical literature, and 3) the software engineering literature. A variety of frameworks and models were found in the biomedical and life sciences literatures. These frameworks and models drew upon and extended frameworks, models and theoretical perspectives that have emerged in other literatures. These frameworks and models are informing an emerging line of research in health and biomedical informatics involving technology-induced errors in healthcare.

Exploring the relationship between usability and technology-induced error: unraveling a complex interaction.

The effective evaluation of the usability of health information systems is currently a major challenge. It is essential that the applications we develop are not only usable, but that they are also shown to be safe and do not inadvertently introduce errors. Furthermore, to provide appropriate feedback to designers of systems new methods for evaluation are needed as applications become more complex and distributed. To ensure system usability and safety a variety of methods have emerged from the area of usability engineering that have been adapted to healthcare. The authors have applied and adapted methods of usability engineering, working with hospitals and other healthcare organizations for designing and evaluating a range of health information systems over a number of years. We describe a methodological framework for considering some of these advances and show how a range of usability evaluations can be used to evaluate both the usability and safety of healthcare information systems both in artificial mocked up and real clinical settings using in-situ testing approaches. We conclude with a discussion of recent trends in the area of usability engineering in healthcare that have potential for improving the safety of healthcare information systems.

Towards an Integrative Cognitive-Socio-Technical Approach in Health Informatics: Analyzing Technology-Induced Error Involving Health Information Systems to Improve Patient Safety

The purpose of this paper is to argue for an integration of cognitive and socio-technical approaches to assessing the impact of health information systems. Historically, health informatics research has examined the cognitive and socio-technical aspects of health information systems separately. In this paper we argue that evaluations of health information systems should consider aspects related to cognition as well as socio-technical aspects including impact on workflow (i.e. an integrated view). Using examples from the study of technology-induced error in healthcare, we argue for the use of simulations to evaluate the cognitive-socio-technical impacts of health information technology [36]. Implications of clinical simulations and analysis of cognitive-social-technical impacts are discussed within the context of the system development life cycle to improve health information system design, implementation and evaluation.

Towards an Integrative Cognitive-Socio-Technical Approach in Health Informatics: Analyzing Technology-Induced Error Involving Health Information Systems to Improve Patient Safety~!2009-04-08~!2009-11-11~!2010-09-14~!

Towards an Integrative Cognitive-Socio-Technical Approach in Health Informatics: Analyzing Technology-Induced Error Involving Health Information Systems to Improve Patient Safety~!2009-04-08~!2009-11-11~!2010-09-14~!

Methods to Assess the Safety of Health Information Systems

Research has shown that the introduction of health information systems (HISs) can reduce the likelihood of medical errors. However, there is a growing body of evidence that suggests that if it is not designed or implemented properly, a HIS can actually cause or induce health professionals to make medical errors (i.e., technology-induced errors). In order to maximize the benefits of HISs while decreasing the likelihood of such inadvertent technology-induced error, it is important that we understand the range of methods that can be used to ensure the safety of our systems. In this article, we report the results of a review of the literature related to the methods used in predicting, preventing and evaluating the potential for a HIS to cause technology-induced error. These methods can be classified in terms of their application, including before a HIS is implemented, after a HIS has been implemented and after a technology-induced error has occurred.

Increasing the Safety of Healthcare Information Systems through Improved Procurement: Toward a Framework for Selection of Safe Healthcare Systems

In this article, we describe a framework that we have developed for improving the effectiveness of critical decision-making in selecting information systems. In our framework, we consider system selection in terms of strength of evidence obtained from the testing of candidate systems in order to reduce risk and increase the likelihood of selection and implementation of an effective and safe system. Two case studies, one from a major North American hospital and one from a major European hospital, are presented to illustrate how methods such as usability testing can be applied to improve system selection as well as customization (through early identification of system-organization mismatches and error-prone system features). It is argued that technology-organization fit and consideration of the potential for technology-induced error should be important selection criteria in the procurement process. Here, implications are discussed for the development of improved procurement processes to lead to safer healthcare systems.

Towards an Integrative Cognitive-Socio-Technical Approach in Health Informatics: Analyzing Technology-Induced Error Involving Health Information Systems to Improve Patient Safety

Towards an Integrative Cognitive-Socio-Technical Approach in Health Informatics: Analyzing Technology-Induced Error Involving Health Information Systems to Improve Patient Safety

Theories, models and frameworks for diagnosing technology-induced error.

Information technology has the potential to greatly streamline healthcare and reduce the chance of human error. However, there is a growing literature indicating that if systems are not designed adequately they may actually increase the possibility of error in the complex interaction between clinician and machine in healthcare (i.e. they may lead to technology-induced errors). In other domains the study of error has been guided by a variety of theories, models and frameworks for understanding human error. In this paper we argue for the need to consider and extend this work to the study of technology-induced error in healthcare. Insights from the software engineering, human factors and organizational behaviour literature will be described, including a set of models and frameworks that we have been using to guide our work in detecting and preventing technology-induced error in healthcare.

Toward an Integrated Simulation Approach for Predicting and Preventing Technology-Induced Errors in Healthcare: Implications for Healthcare Decision-Makers

Research has indicated that health information technology has the potential to reduce medical error and the chances of an adverse event occurring. However, research has indicated that poorly designed systems may inadvertently lead to error (technology-induced error). In this paper, we describe our most recent work in developing a new framework for the integration of multiple forms of simulation to ensure that systems are safe by predicting and preventing technology-induced error in healthcare. The approach taken involves the integration of "clinical simulations" with computer-based simulations. In a case study, the combination of clinical simulations (i.e., involving video analysis of health professionals interacting with computerized physician order entry) and the use of computer modelling and simulation tools is described. In our work, we first employ clinical simulation to obtain baseline error rates. Next, we input data from the clinical simulations into a computer-based simulation and modelling tool to assess the impact of specific aspects of system and interface design upon error rates. The practical implications of combining the advantages of clinical simulation with "in the box" computer-based simulation to predict the impact of healthcare information systems (HIS) are discussed. Implications of this work for healthcare institutions and policy decision-making are explored.

Ensuring the Safety of Health Information Systems: Using Heuristics for Patient Safety

Health information systems (HISs) are typically seen as a mechanism for reducing medical errors. However, there is evidence to suggest that technology can facilitate or induce medical errors. Therefore, it is crucial that we fully test systems prior to their implementation in real-world settings. Presently, evidence-based evaluation heuristics that are specific to HISs do not exist for assessing aspects of interface design that may facilitate errors. A three-phase study was conducted to determine the utility of evidence-based heuristics in evaluating a human-technology interface (i.e., the Veterans Affairs Computerized Patient Record System [VA CPRS]). Phase one consisted of a systematic review of the health informatics literature involving technology-facilitated or technology-induced error. Phase two involved reviewing the literature and generating a comprehensive list of 38 heuristics that could be used to evaluate an HIS for technology-induced errors. Lastly, phase three involved conducting a heuristic evaluation of the VA CPRS system using evidence-based heuristics. Results from this work are discussed.

Identifying and Preventing Technology-Induced Error Using Simulations: Application of Usability Engineering Techniques

In this paper, we describe a framework for the analysis of technology-induced errors, extending approaches from the emerging area of usability engineering. The approach involves collection of a rich set of data consisting of audio and video recordings of interactions of healthcare workers with health information systems under simulated conditions. The application of the approach is discussed, along with methodological considerations and issues in conducting such studies. The steps involved in carrying out such studies are described along with a discussion of our current work. It is argued that health care information systems will need to undergo more rigorous evaluation under simulated conditions in order to detect and prevent technology-induced errors before they are deployed in real healthcare settings.