CPSC: Conformal Prediction With Shrunken Centroids for Efficient Prediction Reliability Quantification and Data Augmentation, a Case in Alternative Herbal Medicine Classification With Electronic Nose

Abstract

In machine learning applications, the reliability of predictions is significant for assisted decision and risk control. As an effective framework to quantify the prediction reliability, conformal prediction (CP) was commonly developed with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -nearest neighbor, known as CPKNN. However, the conventional CPKNN suffers from high variance and bias, and long computational time as the feature dimensionality increases. To address these limitations, a new CP framework-CP with shrunken centroids (CPSC) is proposed. It regularizes the class centroids to attenuate the irrelevant features and shrink the sample space for predictions and reliability quantification. We have also compared the CPSC with CP based on SVM, LightGBM, RF on the herbal medicine dataset, with the electronic nose as a case and assessed them in two tasks: 1) off-line prediction: the training set was fixed, and the accuracy on the testing set was evaluated and 2) online prediction with data augmentation: CPKNN and CPSC filtered unlabeled data to augment the training data based on the prediction reliability, and the final accuracy of the testing set was compared. The result shows that CPSC significantly outperformed CPKNN in both two tasks: 1) CPSC reached a significantly higher accuracy with lower computation cost, and with the same credibility output, CPSC generally achieved higher accuracy and 2) the data augmentation process with CPSC robustly manifested a statistically significant improvement in prediction accuracy with different reliability thresholds, and the augmented data were more balanced in classes. This novel CPSC provides higher prediction accuracy and better reliability quantification, which can assist in reliable decision support.