Comparison metrics for multi-step prediction of rare events in vital sign signals

Abstract

Prediction of changes in biomedical signals, such as vital signs, is useful for many clinical applications. Several signal prediction (forecasting) tools were developed, but their evaluation and applicability to a specific clinical use is context dependent. In this work, we propose a novel method to tackle the problem of evaluation and comparison of vital sign predictors for intervention based clinical studies. The proposed prediction quality measures are particularly well-suited for forecasting rare events scenarios. Specifically, using the novel metrics, we measure the prediction statistics and compare nine deep learning and autoregressive forecasting models for multi-step prediction of rare bradycardia events in preterm infants, however the new concepts allow applications to other biomedical signals. We validated the novel metrics with experimental results on testing sets with several days of vital sign recordings. Our results show that simple statistical predictors could outperform state-of-the-art deep learning architectures for low-dimensional signals.