TU‐G‐BRD‐02: Automated Systematic Quality Assurance Program for Radiation Oncology Information System Upgrades

Abstract

Purpose:To: (1) describe an independent, automated, systematic software‐based protocol for verifying clinical data accuracy/integrity for mitigation of data corruption/loss risks following radiation oncology information system (ROIS) upgrades; and (2) report on application of this approach in an academic/community practice environment.Methods:We propose a robust approach to perform quality assurance on the ROIS after an upgrade, targeting four data sources: (1) ROIS relational database; (2) ROIS DICOM interface; (3) ROIS treatment machine data configuration; and (4) ROIS‐generated clinical reports. We investigated the database schema for differences between pre‐/post‐upgrade states. Paired DICOM data streams for the same object (such as RT‐Plan/Treatment Record) were compared between pre‐/post‐upgrade states for data corruption. We examined machine configuration and related commissioning data files for changes and corruption. ROIS‐generated treatment appointment and treatment parameter reports were compared to ensure patient encounter and treatment plan accuracy. This protocol was supplemented by an end‐to‐end clinical workflow test to verify essential ROI functionality and integrity of components interfaced during patient care chain of activities. We describe the implementation of this protocol during a Varian ARIA system upgrade at our clinic.Results:We verified 1,638 data tables with 2.4 billion data records. For 222 under‐treatment patients, 605 DICOM RT plans and 13,480 DICOM treatment records retrieved from the ROIS DICOM interface were compared, with no differences in fractions, doses delivered, or treatment parameters. We identified 82 new data tables and 78 amended/deleted tables consistent with the upgrade. Reports for 5,073 patient encounters over a 2‐week horizon were compared and were identical to those before the upgrade. Content in 12,237 xml machine files was compared, with no differences identified.Conclusion:An independent QA/validation approach for ROIS upgrades was developed and implemented at our clinic. The success of this approach ensures a robust QA of ROIS upgrades without manual paper/electronic checks and associated intensive labor.