Introducing the DM-P approach for analysing the performances of real-time clinical decision support systems

Abstract

This paper introduces the DM-P approach for assessing the performance of the alerts produced from a clinical decisions support system when there is no information to indicate a genuine observation and one that is not. The DM-P approach allows the representation of the p-charts, normally used in a statistical process control chart setting, as a defects map which is ordered according to the occurrence of similar behaviour as determined using agglomerative hierarchical clustering. The effectiveness of this new approach is demonstrated using real life data from the VILIAlert clinical decisions support system deployed in real-time in an intensive care unit to monitor patients on the ventilator and used to trigger alerts to clinicians when a patient is at risk of receiving injurious mechanical ventilation. The VILIAlert data was successfully used as a case study for the DM-P approach proving both the usefulness of the technique and its ability to identify the genuine cases of patients at risk. Such an approach also has scale up quality being designed in such a way to be able to deal with large volumes of data and can be extended to include any form of statistical process control chart not just p-charts.