Incident Learning System Research Articles

Successful Adoption of an Incident Learning Systems in a Regionally Expanding Academic Radiation Oncology Department

Successful Adoption of an Incident Learning Systems in a Regionally Expanding Academic Radiation Oncology Department

Let's Talk: Communication Errors in Radiation Oncology

Although communication is well-appreciated as a potential source of error, little is known about the specific nature and causes of communication errors particularly in the radiation oncology (RO) context. This study used an institutional incident learning system (ILS) to develop a RO-specific classification schema for communication errors with the goal of informing interventions to reduce errors in treatment. Departmental ILS events from January 2012 to January 2017 were analyzed and reports specific to communication were identified. Events were classified by communication type (written, verbal, other), and written incidents were further sub-classified (electronic medical record (EMR), email, text paging). The individuals responsible for sending and receiving the communication were identified. The nature of the communication failure was categorized as: miscommunication, communication absent (not sent), communication not received, or poor communication without incident. Events were classified by point of error origination and detection based on a previously established workflow. A total of 4,617 safety-related events were captured over the 5-year period with 1002 (22%) directly involving communication. Of these, 400 events were selected at random for analysis; 20% of these were potentially severe events. Individuals responsible for sending and detecting communication errors are shown in Table 1. Written communication errors comprise 62% of communication events, whereas verbal errors were 32%. Of the written errors, 87% involved the use of the EMR for communication. Absence of communication and a miscommunication were important factors in 50% and 21% of events respectively. Most communication events (54%) originated during patient assessment or treatment planning phases. To our knowledge this is the first study to examine and classify communication errors in RO. Communication is a common contributing factor to safety events, and our results show that written communication errors are most prevalent, most often driven by poor communication through the EMR. The RO physician was the most common source of error. This work lays a foundation for further efforts to improve communication and reduce errors in patient care and treatment.Abstract 3300; Table 1VariableN (%)Person sendingRO physician154 (39%)Therapist102 (26%)Dosimetrist43 (11%)Person receivingTherapist189 (47%)RO attending157 (39%)Dosimetrist65 (16%)Patient36 (9%) Open table in a new tab

Incident Learning for Stereotactic Radiation Therapy from RO-ILS: Radiation Oncology Incident Learning System

Incident Learning for Stereotactic Radiation Therapy from RO-ILS: Radiation Oncology Incident Learning System

A patient safety education program in a medical physics residency.

Education in patient safety and quality of care is a requirement for radiation oncology residency programs according to accrediting agencies. However, recent surveys indicate that most programs lack a formal program to support this learning. The aim of this report was to address this gap and share experiences with a structured educational program on quality and safety designed specifically for medical physics therapy residencies. Five key topic areas were identified, drawn from published recommendations on safety and quality. A didactic component was developed, which includes an extensive reading list supported by a series of lectures. This was coupled with practice‐based learning which includes one project, for example, failure modes and effect analysis exercise, and also continued participation in the departmental incident learning system including a root‐cause analysis exercise. Performance was evaluated through quizzes, presentations, and reports. Over the period of 2014–2016, five medical physics residents successfully completed the program. Evaluations indicated that the residents had a positive experience. In addition to educating physics residents this program may be adapted for medical physics graduate programs or certificate programs, radiation oncology residencies, or as a self‐directed educational project for practicing physicists. Future directions might include a system that coordinates between medical training centers such as a resident exchange program.

015. - Risk analysis based on the occurrence and severity of incidents from an institutional incident learning system

015. - Risk analysis based on the occurrence and severity of incidents from an institutional incident learning system

Evaluation of near-miss and adverse events in radiation oncology using a comprehensive causal factor taxonomy

PurposeIncident learning systems (ILSs) are a popular strategy for improving safety in radiation oncology (RO) clinics, but few reports focus on the causes of errors in RO. The goal of this study was to test a causal factor taxonomy developed in 2012 by the American Association of Physicists in Medicine and adopted for use in the RO: Incident Learning System (RO-ILS). Methods and materialsThree hundred event reports were randomly selected from an institutional ILS database and Safety in Radiation Oncology (SAFRON), an international ILS. The reports were split into 3 groups of 100 events each: low-risk institutional, high-risk institutional, and SAFRON. Three raters retrospectively analyzed each event for contributing factors using the American Association of Physicists in Medicine taxonomy. ResultsNo events were described by a single causal factor (median, 7). The causal factor taxonomy was found to be applicable for all events, but 4 causal factors were not described in the taxonomy: linear accelerator failure (n = 3), hardware/equipment failure (n = 2), failure to follow through with a quality improvement intervention (n = 1), and workflow documentation was misleading (n = 1). The most common causal factor categories contributing to events were similar in all event types. The most common specific causal factor to contribute to events was a “slip causing physical error." Poor human factors engineering was the only causal factor found to contribute more frequently to high-risk institutional versus low-risk institutional events. ConclusionsThe taxonomy in the study was found to be applicable for all events and may be useful in root cause analyses and future studies. Communication and human behaviors were the most common errors affecting all types of events. Poor human factors engineering was found to specifically contribute to high-risk more than low-risk institutional events, and may represent a strategy for reducing errors in all types of events.

Patient safety is improved with an incident learning system—Clinical evidence in brachytherapy

Background and purposeHealth leaders have advocated for incident learning systems (ILSs) to prevent errors, but there is limited evidence demonstrating that ILSs improve cancer patient safety. Herein, we report a long-term retrospective review of ILS reports for the brachytherapy practice at a large academic institution. Material and methodsOver a nine-year period, the brachytherapy practice was encouraged to report all standard operating procedure deviations, including low risk deviations. A multidisciplinary committee assigned root causes and risk scores to all incidents. Evidence based practice changes were made using ILS data, and relevant incidents were communicated to all staff in order to reduce recurrence rates. Results5258 brachytherapy procedures were performed and 2238 incidents were reported from 2007 to 2015. A ramp-up period was observed in ILS participation between 2007 (0.12 submissions/procedures) and 2011 (1.55 submissions/procedures). Participation remained stable between 2011 and 2015, and we achieved a 60% (p<0.001) decrease in the risk of dose error or violation of radiation safety policy and a 70% (p<0.001) decrease in frequency of high composite-risk scores. Significant decreases were also observed in incidents with root causes of poor communication (60% decrease, p<0.001) and poor quality of written procedures (59% decrease, p<0.001). ConclusionsImplementation of an ILS in brachytherapy significantly reduced risk during cancer patient care. Safety improvements have been sustained over several years.

Incident Learning Systems for Radiation Oncology: Development and Value at the Local, National and International Level

AimsTo discuss the background for incident reporting and learning systems, as well as the infrastructure and operational aspects to run them. Materials and methodsInformation from peer-reviewed literature, online resources and the authors' experience synthesised into a concise understanding of the topic. ResultsIncident learning systems can be local, national or international, each having the same basic goals but facilitating different audiences and environments. A key component of any reporting and learning system is timely and effective analysis of near-misses and incidents as well as feedback to the users of the system. It is important for staff to know that reports are acknowledged, analysed and acted upon. There is a need to comply with current European legislation and other national systems, which can be addressed together with the steps required for comprehensive management of an incident. ConclusionReporting and learning from incidents and near-misses is a key component of quality and safety in radiotherapy. A major benefit of the national or international systems is the potential for a larger database of incidents, supporting wider analysis and comparison, and sharing of knowledge across a larger community.

Safety practices, perceptions, and behaviors in radiation oncology: A national survey of radiation therapists

PurposeRadiation therapy is complex and demands high vigilance and precise coordination. Radiation therapists (RTTs) directly deliver radiation and are often the first to discover an error. Yet, few studies have examined the practices of RTTs regarding patient safety. We conducted a national survey to explore the perspectives of RTTs related to quality and safety. Methods and materialsIn 2016, an electronic survey was sent to a random sample of 1500 RTTs in the United States. The survey assessed department safety, error reporting, safety knowledge, and culture. Questions were multiple choice or recorded on a Likert scale. Results were summarized using descriptive statistics and analyzed using multivariate logistic regression. ResultsA total of 702 RTTs from 49 states (47% response rate) completed the survey. Respondents represented a broad distribution across practice settings. Most RTTs rated department patient safety as excellent (61%) or very good (32%), especially if they had an incident learning system (ILS) (odds ratio, 2.0). Only 21% reported using an ILS despite 58% reporting an accessible ILS in their department. RTTs felt errors were most likely to occur with longer shifts and poor multidisciplinary communication; 40% reported that burnout and anxiety negatively affected their ability to deliver care. Workplace bullying was also reported among 17%. Overall, there was interest (62%) in improving knowledge in patient safety. ConclusionsAlthough most RTTs reported excellent safety cultures within their facilities, overall, there was limited access to and utilization of ILSs by RTTs. Workplace issues identified may also represent barriers to delivering quality care. RTTs were also interested in additional resources regarding quality and safety. These results will further enhance safety initiatives and inform future innovative educational efforts in radiation oncology.

Are we making an impact with incident learning systems? Analysis of quality improvement interventions using total body irradiation as a model system

PurposeDespite increasing interest in incident learning systems (ILS) to improve safety and quality in radiation oncology, little is known about interventions developed in response to safety data. We used total body irradiation (TBI) as a model system to study the effectiveness of interventions from our institutional ILS. Methods and materialsNear-miss event reports specific to TBI were identified from a departmental ILS from March 2012 to December 2015. The near-miss risk index was rated at multidisciplinary review from 0 (no potential harm) to 4 (critical potential harm). Interventions were analyzed for effectiveness with a schema adapted from The Joint Commission and other agencies: “most reliable” (eg, forcing functions, automation), “somewhat reliable” (eg, checklists, standardization), and “least reliable” (eg, training, rules, procedures). Causal factors of each event were drawn from the casual factor schema used in radiation oncology ILS. ResultsOf 4007 safety-related reports, 266 reports pertained to TBI. TBI reports had a somewhat higher proportion of high-risk events (near-miss risk index 3-4) compared with non-TBI reports (25% vs 17%, P = .0045). A total of 117 interventions were implemented. The reliability indicators for the interventions were: most reliable (11% of interventions), somewhat reliable (17%), and least reliable (72%). Interventions were more likely to be applied to high-risk events (54% vs 41%, P = .03). There was a pattern of high-reliability interventions with increased risk score of events. Events involving human error (eg, slips) and equipment/information technology lent themselves more often to high-reliability interventions. Events related to communication, standardization, and training were associated with low-reliability interventions. ConclusionsOver a 3.5-year period, 117 quality improvement strategies were developed for TBI based on ILS. Interventions were more likely to be applied to high-risk events and high-risk events were more likely to be associated with high-quality interventions. These results may be useful to institutions seeking to develop interventions based on ILS data.

A Simple Incident Learning System for Radiation Oncology in a Community Hospital

A Simple Incident Learning System for Radiation Oncology in a Community Hospital

Radiation therapist (RT) perceptions and practices related to safety culture (SC) and error reporting in radiation oncology.

42 Background: The delivery of radiotherapy is complex and demands careful management, high vigilance, and precise coordination of clinical personnel. RTs are at the frontline and are often first to discover or report an error. However, few studies have examined RT patient safety practices. We conducted a national survey to explore the attitudes, training, and experiences of RTs related to safety in radiation oncology. Methods: In 2016, an electronic survey was sent in June and July to a random sample of 1,500 RTs (~10% of licensed RTs in the US). The survey assessed department SC, error reporting, knowledge, and practices. Questions were multiple choice or recorded on a 5-point Likert scale. Results were summarized using descriptive statistics and analyzed using multivariate logistic regression. Results: A total of 702 RTs from 48 states (47% response rate) completed the survey. Respondents represented a broad distribution across practice and geographic settings with varying levels of work experience (see table). Most RTs gave their department a “Patient Safety Grade” of Excellent (61%) or Very Good (32%), especially if they had an Incident Learning System (ILS) (OR: 2.0). Most departments (58%) had an ILS; however 59% of RTs had not reported an event in the previous year. When an error occurred but did not result in patient harm, 39% of RTs said they would report it and 37% said they would not. Bullying was reported among 17% of respondents; 40% reported that burnout, stress, and anxiety negatively impacted their ability to effectively treat patients. RTs reported errors were more likely to occur with longer work days (&gt;8hrs) and poor multidisciplinary communication during handoffs and transitions. Most (62%) reported they could benefit from additional patient safety education and training, indicating significant interest in this topic. Conclusions: The majority of RTs reported excellent SC practices within their treatment facilities. Most facilities had systems in place to report errors. The obstacles identified, the results of this study may inform future educational efforts and safety initiatives directed towards RTs and should help increase awareness of patient SC in radiation oncology.

A survey of residents’ experience with patient safety and quality improvement concepts in radiation oncology

PurposeThe safety and quality of radiation therapy have recently garnered increased attention in radiation oncology (RO). Although patient safety guidelines expect physicians and physicists to lead clinical safety and quality improvement (QI) programs, trainees' level of exposure to patient safety concepts during training is unknown. Methods and materialsWe surveyed active medical and physics RO residents in North America in February 2016. Survey questions involved demographics and program characteristics, exposure to patient safety topics, and residents’ attitude regarding their safety education. ResultsResponses were collected from 139 of 690 (20%) medical and 56 of 248 (23%) physics RO residents. More than 60% of residents had no exposure or only informal exposure to incident learning systems (ILS), root cause analysis, failure mode and effects analysis (FMEA), and the concepts of human factors engineering. Medical residents had less exposure to FMEA than physics residents, and fewer medical than physics residents felt confident in leading FMEA in clinic. Only 27% of residents felt that patient safety training was adequate in their program. Experiential learning through practical workshops was the most desired educational modality, preferred over web-based learning. Residents training in departments with ILS had greater exposure to patient safety concepts and felt more confident leading clinical patient safety and QI programs than residents training in departments without an ILS. ConclusionsThe survey results show that most residents have no or only informal exposure to important patient safety and QI concepts and do not feel confident leading clinical safety programs. This represents a gaping need in RO resident education. Educational programs such as these can be naturally developed as part of an incident learning program that focuses on near-miss events. Future research should assess the needs of RO program directors to develop effective RO patient safety and QI training programs.

Quality Improvement Steps From Implementation of Radiation Oncology Incident Learning System (RO-ILS)

Quality Improvement Steps From Implementation of Radiation Oncology Incident Learning System (RO-ILS)

The effectiveness of pretreatment physics plan review for detecting errors in radiation therapy.

The pretreatment physics plan review is a standard tool for ensuring treatment quality. Studies have shown that the majority of errors in radiation oncology originate in treatment planning, which underscores the importance of the pretreatment physics plan review. This quality assurance measure is fundamentally important and central to the safety of patients and the quality of care that they receive. However, little is known about its effectiveness. The purpose of this study was to analyze reported incidents to quantify the effectiveness of the pretreatment physics plan review with the goal of improving it. This study analyzed 522 potentially severe or critical near-miss events within an institutional incident learning system collected over a three-year period. Of these 522 events, 356 originated at a workflow point that was prior to the pretreatment physics plan review. The remaining 166 events originated after the pretreatment physics plan review and were not considered in the study. The applicable 356 events were classified into one of the three categories: (1) events detected by the pretreatment physics plan review, (2) events not detected but "potentially detectable" by the physics review, and (3) events "not detectable" by the physics review. Potentially detectable events were further classified by which specific checks performed during the pretreatment physics plan review detected or could have detected the event. For these events, the associated specific check was also evaluated as to the possibility of automating that check given current data structures. For comparison, a similar analysis was carried out on 81 events from the international SAFRON radiation oncology incident learning system. Of the 356 applicable events from the institutional database, 180/356 (51%) were detected or could have been detected by the pretreatment physics plan review. Of these events, 125 actually passed through the physics review; however, only 38% (47/125) were actually detected at the review. Of the 81 events from the SAFRON database, 66/81 (81%) were potentially detectable by the pretreatment physics plan review. From the institutional database, three specific physics checks were particularly effective at detecting events (combined effectiveness of 38%): verifying the isocenter (39/180), verifying DRRs (17/180), and verifying that the plan matched the prescription (12/180). The most effective checks from the SAFRON database were verifying that the plan matched the prescription (13/66) and verifying the field parameters in the record and verify system against those in the plan (23/66). Software-based plan checking systems, if available, would have potential effectiveness of 29% and 64% at detecting events from the institutional and SAFRON databases, respectively. Pretreatment physics plan review is a key safety measure and can detect a high percentage of errors. However, the majority of errors that potentially could have been detected were not detected in this study, indicating the need to improve the pretreatment physics plan review performance. Suggestions for improvement include the automation of specific physics checks performed during the pretreatment physics plan review and the standardization of the review process.

WE‐B‐BRC‐01: Current Methodologies in Risk Assessment

Prospective quality management techniques, long used by engineering and industry, have become a growing aspect of efforts to improve quality management and safety in healthcare. These techniques are of particular interest to medical physics as scope and complexity of clinical practice continue to grow, thus making the prescriptive methods we have used harder to apply and potentially less effective for our interconnected and highly complex healthcare enterprise, especially in imaging and radiation oncology.An essential part of most prospective methods is the need to assess the various risks associated with problems, failures, errors, and design flaws in our systems. We therefore begin with an overview of risk assessment methodologies used in healthcare and industry and discuss their strengths and weaknesses. The rationale for use of process mapping, failure modes and effects analysis (FMEA) and fault tree analysis (FTA) by TG‐100 will be described, as well as suggestions for the way forward. This is followed by discussion of radiation oncology specific risk assessment strategies and issues, including the TG‐100 effort to evaluate IMRT and other ways to think about risk in the context of radiotherapy. Incident learning systems, local as well as the ASTRO/AAPM ROILS system, can also be useful in the risk assessment process. Finally, risk in the context of medical imaging will be discussed. Radiation (and other) safety considerations, as well as lack of quality and certainty all contribute to the potential risks associated with suboptimal imaging.The goal of this session is to summarize a wide variety of risk analysis methods and issues to give the medical physicist access to tools which can better define risks (and their importance) which we work to mitigate with both prescriptive and prospective risk‐based quality management methods. Learning Objectives: Description of risk assessment methodologies used in healthcare and industry Discussion of radiation oncology‐specific risk assessment strategies and issues Evaluation of risk in the context of medical imaging and image qualityE. Samei: Research grants from Siemens and GE

WE‐B‐BRC‐03: Risk in the Context of Medical Imaging

Prospective quality management techniques, long used by engineering and industry, have become a growing aspect of efforts to improve quality management and safety in healthcare. These techniques are of particular interest to medical physics as scope and complexity of clinical practice continue to grow, thus making the prescriptive methods we have used harder to apply and potentially less effective for our interconnected and highly complex healthcare enterprise, especially in imaging and radiation oncology.An essential part of most prospective methods is the need to assess the various risks associated with problems, failures, errors, and design flaws in our systems. We therefore begin with an overview of risk assessment methodologies used in healthcare and industry and discuss their strengths and weaknesses. The rationale for use of process mapping, failure modes and effects analysis (FMEA) and fault tree analysis (FTA) by TG‐100 will be described, as well as suggestions for the way forward. This is followed by discussion of radiation oncology specific risk assessment strategies and issues, including the TG‐100 effort to evaluate IMRT and other ways to think about risk in the context of radiotherapy. Incident learning systems, local as well as the ASTRO/AAPM ROILS system, can also be useful in the risk assessment process. Finally, risk in the context of medical imaging will be discussed. Radiation (and other) safety considerations, as well as lack of quality and certainty all contribute to the potential risks associated with suboptimal imaging.The goal of this session is to summarize a wide variety of risk analysis methods and issues to give the medical physicist access to tools which can better define risks (and their importance) which we work to mitigate with both prescriptive and prospective risk‐based quality management methods. Learning Objectives: Description of risk assessment methodologies used in healthcare and industry Discussion of radiation oncology‐specific risk assessment strategies and issues Evaluation of risk in the context of medical imaging and image qualityE. Samei: Research grants from Siemens and GE

TU‐D‐201‐04: Veracity of Data Elements in Radiation Oncology Incident Learning Systems

Purpose:Incident learning systems encompass volumes, varieties, values, and velocities of underlying data elements consistent with the V's of big data. Veracity, the 5th V however exists only if there is high inter‐rater reliability (IRR) within the data elements. The purpose of this work was to assess IRR in the nationally deployed RO‐ILS: Radiation Oncology‐Incident Learning System (R) sponsored by the American Society for Radiation Oncology (ASTRO) and the American Association of Physicists in Medicine (AAPM).Methods:Ten incident reports covering a wide range of scenarios were created in standardized narrative and video formats and disseminated to 67 volunteers of multiple disciplines from 26 institutions along with two published narratives from the International Commission of Radiological Protection to assess IRR on a nationally representative level. The volunteers were instructed to independently enter the associated data elements in a test version of RO‐ILS over a 3‐week period. All responses were aggregated into a spreadsheet to assess IRR using free‐marginal kappa metrics.Results:48 volunteers from 21 institutions completed all reports in the study period. The average kappa score for all raters across all critical data elements was 0.659 [range 0.326–1.000]. Statistically significant differences (p &lt;0.05) were noted between reporters of different disciplines and raters with varying levels of experience. Kappa scores were high for event classification (0.781) and contributory factors (0.777) and low for likelihood‐of‐harm (0.326). IRR was highest among AAPM‐ASTRO members (0.672) and lowest among trainees (0.463).Conclusion:A moderate‐to‐substantial level of IRR in RO‐ILS was noted in this study. Although the number of events reviewed in this study was small, opportunities for improving the taxonomy for the lower scoring data elements as well as specific educational targets for training were identified by assessing data veracity quantitatively. This is expected to improve the quality of the data garnered from RO‐ILS.

WE‐B‐BRC‐00: Concepts in Risk‐Based Assessment

Prospective quality management techniques, long used by engineering and industry, have become a growing aspect of efforts to improve quality management and safety in healthcare. These techniques are of particular interest to medical physics as scope and complexity of clinical practice continue to grow, thus making the prescriptive methods we have used harder to apply and potentially less effective for our interconnected and highly complex healthcare enterprise, especially in imaging and radiation oncology.An essential part of most prospective methods is the need to assess the various risks associated with problems, failures, errors, and design flaws in our systems. We therefore begin with an overview of risk assessment methodologies used in healthcare and industry and discuss their strengths and weaknesses. The rationale for use of process mapping, failure modes and effects analysis (FMEA) and fault tree analysis (FTA) by TG‐100 will be described, as well as suggestions for the way forward. This is followed by discussion of radiation oncology specific risk assessment strategies and issues, including the TG‐100 effort to evaluate IMRT and other ways to think about risk in the context of radiotherapy. Incident learning systems, local as well as the ASTRO/AAPM ROILS system, can also be useful in the risk assessment process. Finally, risk in the context of medical imaging will be discussed. Radiation (and other) safety considerations, as well as lack of quality and certainty all contribute to the potential risks associated with suboptimal imaging.The goal of this session is to summarize a wide variety of risk analysis methods and issues to give the medical physicist access to tools which can better define risks (and their importance) which we work to mitigate with both prescriptive and prospective risk‐based quality management methods. Learning Objectives: Description of risk assessment methodologies used in healthcare and industry Discussion of radiation oncology‐specific risk assessment strategies and issues Evaluation of risk in the context of medical imaging and image qualityE. Samei: Research grants from Siemens and GE

WE‐B‐BRC‐02: Risk Analysis and Incident Learning

Prospective quality management techniques, long used by engineering and industry, have become a growing aspect of efforts to improve quality management and safety in healthcare. These techniques are of particular interest to medical physics as scope and complexity of clinical practice continue to grow, thus making the prescriptive methods we have used harder to apply and potentially less effective for our interconnected and highly complex healthcare enterprise, especially in imaging and radiation oncology.An essential part of most prospective methods is the need to assess the various risks associated with problems, failures, errors, and design flaws in our systems. We therefore begin with an overview of risk assessment methodologies used in healthcare and industry and discuss their strengths and weaknesses. The rationale for use of process mapping, failure modes and effects analysis (FMEA) and fault tree analysis (FTA) by TG‐100 will be described, as well as suggestions for the way forward. This is followed by discussion of radiation oncology specific risk assessment strategies and issues, including the TG‐100 effort to evaluate IMRT and other ways to think about risk in the context of radiotherapy. Incident learning systems, local as well as the ASTRO/AAPM ROILS system, can also be useful in the risk assessment process. Finally, risk in the context of medical imaging will be discussed. Radiation (and other) safety considerations, as well as lack of quality and certainty all contribute to the potential risks associated with suboptimal imaging.The goal of this session is to summarize a wide variety of risk analysis methods and issues to give the medical physicist access to tools which can better define risks (and their importance) which we work to mitigate with both prescriptive and prospective risk‐based quality management methods. Learning Objectives: Description of risk assessment methodologies used in healthcare and industry Discussion of radiation oncology‐specific risk assessment strategies and issues Evaluation of risk in the context of medical imaging and image qualityE. Samei: Research grants from Siemens and GE