Uncertainty Estimation with Data Augmentation for Active Learning Tasks on Health Data.

Abstract

Supervised machine learning (ML) is revolutionising healthcare, but the acquisition of reliable labels for signals harvested from medical sensors is usually challenging, manual, and costly. Active learning can assist in establishing labels on-the-fly by querying the user only for the most uncertain -and thus informative- samples. However, current approaches rely on naive data selection algorithms, which still require many iterations to achieve the desired accuracy. To this aim, we introduce a novel framework that exploits data augmentation for estimating the uncertainty introduced by sensor signals.Our experiments on classifying medical signals show that our framework selects informative samples up to 50% more diverse. Sample diversity is a key indicator of uncertainty, and our framework can capture this diversity better than previous solutions as it picks unlabelled samples with a higher average point distance during the first queries compared to the baselines, which pick samples that are closer together. Through our experiments, we show that augmentation-based uncertainty makes better decisions, as the more informative signals are labelled first and the learner is able to train on samples with more diverse features earlier on, thus enabling the potential expansion of ML in more real-life healthcare use cases.