Abstract
In many data classification problems, a number of methods will give similar accuracy. However, when working with people who are not experts in data science such as doctors, lawyers, and judges among others, finding interpretable algorithms can be a critical success factor. Practitioners have a deep understanding of the individual input variables but far less insight into how they interact with each other. For example, there may be ranges of an input variable for which the observed outcome is significantly more or less likely. This paper describes an algorithm for automatic detection of such thresholds, called the Univariate Flagging Algorithm (UFA). The algorithm searches for a separation that optimizes the difference between separated areas while obtaining a high level of support. We evaluate its performance using six sample datasets and demonstrate that thresholds identified by the algorithm align well with published results and known physiological boundaries. We also introduce two classification approaches that use UFA and show that the performance attained on unseen test data is comparable to or better than traditional classifiers when confidence intervals are considered. We identify conditions under which UFA performs well, including applications with large amounts of missing or noisy data, applications with a large number of inputs relative to observations, and applications where incidence of the target is low. We argue that ease of explanation of the results, robustness to missing data and noise, and detection of low incidence adverse outcomes are desirable features for clinical applications that can be achieved with relatively simple classifier, like UFA.
Highlights
Classifiers can be evaluated by multiple parameters, including accuracy, robustness, sensitivity to missing data, or ease of interpretability
Donoho and Jin [1] have demonstrated that the use of very simple univariate discriminant analysis, making no use of covariance matrices, led to a similar performance on a standard series of datasets [2] compared to much more sophisticated popular machine learning methods
We present results for datasets that vary greatly in terms of complexity and target/non-target ratio, allowing us to identify conditions for which Univariate Flagging Algorithm (UFA) is well suited
Summary
Classifiers can be evaluated by multiple parameters, including accuracy, robustness, sensitivity to missing data, or ease of interpretability. Good predictive accuracy is often by far the most important evaluation metric. Donoho and Jin [1] have demonstrated that the use of very simple univariate discriminant analysis, making no use of covariance matrices, led to a similar performance on a standard series of datasets [2] compared to much more sophisticated popular machine learning methods (including Boosted decision trees, Random Forests, SVM, KNN, PAM and DLDA). Authors of the Mas-oMenos algorithm [3] compared their simplified approach to more sophisticated algorithms for treatment predictions of bladder, breast, and ovarian cancers, and came to the conclusion that model interpretation and validation were more important than complexity
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call