SU‐D‐201‐05: On the Automatic Recognition of Patient Safety Hazards in a Radiotherapy Setup Using a Novel 3D Camera System and a Deep Learning Framework

Abstract

Purpose:Patient safety hazards such as a wrong patient/site getting treated can lead to catastrophic results. The purpose of this project is to automatically detect potential patient safety hazards during the radiotherapy setup and alert the therapist before the treatment is initiated.Methods:We employed a set of co‐located and co‐registered 3D cameras placed inside the treatment room. Each camera provided a point‐cloud of fraxels (fragment pixels with 3D depth information). Each of the cameras were calibrated using a custom‐built calibration target to provide 3D information with less than 2 mm error in the 500 mm neighborhood around the isocenter. To identify potential patient safety hazards, the treatment room components and the patient's body needed to be identified and tracked in real‐time. For feature recognition purposes, we used a graph‐cut based feature recognition with principal component analysis (PCA) based feature‐to‐object correlation to segment the objects in real‐time. Changes in the object's position were tracked using the CamShift algorithm. The 3D object information was then stored for each classified object (e.g. gantry, couch). A deep learning framework was then used to analyze all the classified objects in both 2D and 3D and was then used to fine‐tune a convolutional network for object recognition. The number of network layers were optimized to identify the tracked objects with >95% accuracy.Results:Our systematic analyses showed that, the system was effectively able to recognize wrong patient setups and wrong patient accessories. The combined usage of 2D camera information (color + depth) enabled a topology‐preserving approach to verify patient safety hazards in an automatic manner and even in scenarios where the depth information is partially available.Conclusion:By utilizing the 3D cameras inside the treatment room and a deep learning based image classification, potential patient safety hazards can be effectively avoided.