Confident Learning Research Articles

A multiview deep learning-based prediction pipeline augmented with confident learning can improve performance in determining knee arthroplasty candidates.

Preoperative prudent patient selection plays a crucial role in knee osteoarthritis management but faces challenges in appropriate referrals such as total knee arthroplasty (TKA), unicompartmental knee arthroplasty (UKA)and nonoperative intervention. Deep learning (DL) techniques can build prediction models for treatment decision-making. The aim is to develop and evaluate a knee arthroplasty prediction pipeline using three-view X-rays to determine the suitable candidates for TKA, UKAor are not arthroplasty candidates. A study was conducted using three-view (anterior-posterior, lateraland patellar) X-rays and surgical data of patients undergoing TKA, UKAor nonarthroplasty interventions from sites A and B. Data from site A were used to derive and validate models. Data from site B were used as external test set. A DL pipeline combining YOLOv3 and ResNet-18 with confident learning (CL) was developed. Multiview Convolutional Neural Network, EfficientNet-b4, ResNet-101and the proposed model without CLwere also trained and tested. The models were evaluated using metrics such as area under the receiver operating characteristic curve (AUC), accuracy, precision, specificity, sensitivityand F1 score. The data set comprised a total of 1779 knees. Of which 1645 knees were from site A as a derivation set and an internal validation cohort. The external validation cohort consisted of 134 knees. The internal validation cohort demonstrated superior performance for the proposed model augmented with CL, achieving an AUC of 0.94 and an accuracy of 85.9%. External validation further confirmed the model's generalisation, with an AUC of 0.93 and an accuracy of 82.1%. Comparative analysis with other neural network models showed the proposed model's superiority. The proposed DL pipeline, integrating YOLOv3, ResNet-18and CL, provides accurate predictions for knee arthroplasty candidates based on three-view X-rays. This prediction model could be useful in performing decision making for the type of arthroplasty procedure in an automated fashion. Level III, diagnostic study.

Confident learning-based Gaussian mixture model for leakage detection in water distribution networks

Leakage detection in the water distribution system not only helps to reduce water waste but also decreases the risk of drinking water pollution. To reduce reliance on hardware devices and enable real-time detection, the water utilities are transitioning towards the data-driven based approach that relies on the analysis of the flow and pressure data collected from the supervisory control and data acquisition (SCADA) system. Due to the lack of leakage data, most of these methods are unsupervised methods that rely heavily on assumptions about the distribution of anomalies; whereas, the water utility's repair records contain much valid information about the leakage and normal characteristics. To convert this information into available labels and to address the lack of leakage data, this paper proposed a new leakage detection framework to infer the pressure characteristics under normal conditions based on historical data by combining a label cleaning method—confident learning (CL)—with an unsupervised method—Gaussian mixture model (GMM)—for leak detection. The methodology is validated with synthetic and real measured data. Comparisons with four unsupervised methods demonstrate that the GMM method has superior identification of leak features from pressure data. For a real-world water distribution system in K city containing 91 pressure sensors, the average true positive rate is 0.78 and the average false positive rate is 0.11. This methodology is a promising tool to identify signs of leakage in large-scale water distribution networks (WDNs).

Confident Learning-Based Domain Adaptation for Hyperspectral Image Classification

Cross-domain hyperspectral image classification is one of the major challenges in remote sensing, especially for target domain data without labels. Recently, deep learning approaches have demonstrated effectiveness in domain adaptation. However, most of them leverage unlabeled target data only from a statistical perspective but neglect the analysis at the instance level. For better statistical alignment, existing approaches employ the entire unevaluated target data in an unsupervised manner, which may introduce noise and limit the discriminability of the neural networks. In this article, we propose confident learning-based domain adaptation (CLDA) to address the problem from a new perspective of data manipulation. To this end, a novel framework is presented to combine domain adaptation with confident learning (CL), where the former reduces the interdomain discrepancy and generates pseudo-labels for the target instances, from which the latter selects high-confidence target samples. Specifically, the confident learning part evaluates the confidence of each pseudo-labeled target sample based on the assigned labels and the predicted probabilities. Then, high-confidence target samples are selected as training data to increase the discriminative capacity of the neural networks. In addition, the domain adaptation part and the confident learning part are trained alternately to progressively increase the proportion of high-confidence labels in the target domain, thus further improving the accuracy of classification. Experimental results on four datasets demonstrate that the proposed CLDA method outperforms the state-of-the-art domain adaptation approaches. Our source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Li-ZK/CLDA-2022</uri> .

Detecting Uncertainty of Mortality Prediction Using Confident Learning.

Early mortality prediction is an actively researched problem that has led to the development of various severity scores and machine learning (ML) models for accurate and reliable detection of mortality in severely ill patients staying in intensive care units (ICUs). However, the uncertainty of such predictions due to irregular patient sampling, missing information, or high diversity of patient data has not yet been adequately addressed. In this paper, we used confident learning (CL) to incorporate sample-uncertainty information into our mortality prediction models and evaluated the performance of these models using a large dataset of 139,367 unique ICU admissions within the eICU Collaborative Research Database (eICU-CRD). The results of our study validate the importance of uncertainty quantification in patient outcome prediction and show that the state-of-the-art ML models augmented with CL are more robust against epistemic error and class imbalance.

Confident Learning: Estimating Uncertainty in Dataset Labels

Learning exists in the context of data, yet notions of confidence typically focus on model predictions, not label quality. Confident learning (CL) is an alternative approach which focuses instead on label quality by characterizing and identifying label errors in datasets, based on the principles of pruning noisy data, counting with probabilistic thresholds to estimate noise, and ranking examples to train with confidence. Whereas numerous studies have developed these principles independently, here, we combine them, building on the assumption of a class-conditional noise process to directly estimate the joint distribution between noisy (given) labels and uncorrupted (unknown) labels. This results in a generalized CL which is provably consistent and experimentally performant. We present sufficient conditions where CL exactly finds label errors, and show CL performance exceeding seven recent competitive approaches for learning with noisy labels on the CIFAR dataset. Uniquely, the CL framework is not coupled to a specific data modality or model (e.g., we use CL to find several label errors in the presumed error-free MNIST dataset and improve sentiment classification on text data in Amazon Reviews). We also employ CL on ImageNet to quantify ontological class overlap (e.g., estimating 645 missile images are mislabeled as their parent class projectile), and moderately increase model accuracy (e.g., for ResNet) by cleaning data prior to training. These results are replicable using the open-source cleanlab release.