Model-based Active Learning Research Articles

VisGIL: machine learning-based visual guidance for interactive labeling

Labeling of datasets is an essential task for supervised and semi-supervised machine learning. Model-based active learning and user-based interactive labeling are two complementary strategies for this task. We propose VisGIL which, using visual cues, guides the user in the selection of instances to label based on utility measures deduced from an active learning model. We have implemented the approach and conducted a qualitative and quantitative user study and a think-aloud test. The studies reveal that guidance by visual cues improves the trained model’s accuracy, reduces the time needed to label the dataset, and increases users’ confidence while selecting instances. Furthermore, we gained insights regarding how guidance impacts user behavior and how the individual visual cues contribute to user guidance. A video of the approach is available: https://ml-and-vis.org/visgil/.

MOSE: A Monotonic Selectivity Estimator Using Learned CDF

The accuracy of selectivity estimation is of vital importance to create good query plans. Traditional estimators such as histograms make several assumptions during estimation that can lead to huge errors. Recently the database community started exploring the usage of machine learning in selectivity estimation and won great achievements. However, due to the black box models they used, existing learning-based methods still face several new challenges, including high estimation latency, large training data demanding, and occurrence of illogical results. In this work, we propose a learning-based MOnotonic Selectivity Estimator (MOSE) to address these challenges. We first learn a multi-dimensional cumulative distribution function of the data in a supervised method and then compute selectivity for ad hoc query predicates at rum-time. We propose a novel regularizer and an effective attribute-aware calibration method to improve the estimation accuracy. To further improve the model efficiency, we design a mutual information based model ensemble method. With regard to the heavy cost of training data collection, we design a model-based active learning strategy to generate high-quality training data cost-effectively. We conduct extensive experiments on both real-world and synthetic datasets and the results show that MOSE outperforms the state-of-the-art methods in terms of accuracy and efficiency.

Active learning with simultaneous subject and variable selections

Training data are essential for learning classification models. Therefore, if only a limited number of labeled subjects are available for use as training samples, whereas a considerable amount of unlabeled data already exists, then it is always desirable enlarging the training set by labeling more subjects in order to ameliorate classification models. When it is costly in time and capital to label unlabeled subjects, it is crucial to know how many labeled subjects are necessary for training a satisfactory classification model. Although, active learning methods can gradually recruit new unlabeled subjects and disclose their label information to enlarge the size of the training set, there is a lack of discussion about the size of training samples in the literature. Hence, when/how to appropriately stop an active learning procedure is studied in this paper. Since the sequential subject recruiting strategy is used in active learning procedures, it is natural to adopt the idea of sequential analysis to dynamically and adaptively determine the training sample size for learning. In this study, we propose a stopping criterion for a linear model-based active learning procedure, such that this learning process will asymptotically achieve its best possible empirical performance, in terms of the area under receiver the operating characteristic curve (ROC), when the procedure is stopped. Other statistical properties of the proposed procedure, including estimation consistency and variable selection, are also studied. The numerical results using both synthesized and a real example are reported.

Greedy active learning algorithm for logistic regression models

We study a logistic model-based active learning procedure for binary classification problems, in which we adopt a batch subject selection strategy with a modified sequential experimental design method. Moreover, accompanying the proposed subject selection scheme, we simultaneously conduct a greedy variable selection procedure such that we can update the classification model with all labeled training subjects. The proposed algorithm repeatedly performs both subject and variable selection steps until a prefixed stopping criterion is reached. Our numerical results show that the proposed procedure has competitive performance, with smaller training size and a more compact model compared with that of the classifier trained with all variables and a full data set. We also apply the proposed procedure to a well-known wave data set (Breiman et al., 1984) and a MAGIC gamma telescope data set to confirm the performance of our method.

Model-based active learning to detect an isometric deformable object in the wild with a deep architecture

In the recent past, algorithms based on Convolutional Neural Networks (CNNs) have achieved significant milestones in object recognition. With large examples of each object class, standard datasets train well for inter-class variability. However, gathering sufficient data to train for a particular instance of an object within a class is impractical. Furthermore, quantitatively assessing the imaging conditions for each image in a given dataset is not feasible. By generating sufficient images with known imaging conditions, we study to what extent CNNs can cope with hard imaging conditions for instance-level recognition in an active learning regime.Leveraging powerful rendering techniques to achieve instance-level detection, we present results of training three state-of-the-art object detection algorithms namely, Fast R-CNN, Faster R-CNN and YOLO9000, for hard imaging conditions imposed into the scene by rendering. Our extensive experiments produce a mean Average Precision score of 0.92 on synthetic images and 0.83 on real images using the best performing Faster R-CNN. We show for the first time how well detection algorithms based on deep architectures fare for each hard imaging condition studied.

An efficient cardiac mapping strategy for radiofrequency catheter ablation with active learning

ObjectiveA major challenge in radiofrequency catheter ablation procedures is the voltage and activation mapping of the endocardium, given a limited mapping time. By learning from expert interventional electrophysiologists (operators), while also making use of an active-learning framework, guidance on performing cardiac voltage mapping can be provided to novice operators or even directly to catheter robots.MethodsA learning from demonstration (LfD) framework, based upon previous cardiac mapping procedures performed by an expert operator, in conjunction with Gaussian process (GP) model-based active learning, was developed to efficiently perform voltage mapping over right ventricles (RV). The GP model was used to output the next best mapping point, while getting updated towards the underlying voltage data pattern as more mapping points are taken. A regularized particle filter was used to keep track of the kernel hyperparameter used by GP. The travel cost of the catheter tip was incorporated to produce time-efficient mapping sequences.ResultsThe proposed strategy was validated on a simulated 2D grid mapping task, with leave-one-out experiments on 25 retrospective datasets, in an RV phantom using the Stereotaxis Niobe® remote magnetic navigation system, and on a tele-operated catheter robot. In comparison with an existing geometry-based method, regression error was reduced and was minimized at a faster rate over retrospective procedure data.ConclusionA new method of catheter mapping guidance has been proposed based on LfD and active learning. The proposed method provides real-time guidance for the procedure, as well as a live evaluation of mapping sufficiency.

An MRF Model-Based Active Learning Framework for the Spectral-Spatial Classification of Hyperspectral Imagery

Hyperspectral image classification has attracted extensive research efforts in the recent decades. The main difficulty lies in the few labeled samples versus high dimensional features. The spectral-spatial classification method using Markov random field (MRF) has been shown to perform well in improving the classification performance. Moreover, active learning (AL), which iteratively selects the most informative unlabeled samples and enlarges the training set, has been widely studied and proven useful in remotely sensed data. In this paper, we focus on the combination of MRF and AL in the classification of hyperspectral images, and a new MRF model-based AL (MRF-AL) framework is proposed. In the proposed framework, the unlabeled samples whose predicted results vary before and after the MRF processing step is considered as uncertain. In this way, subset is firstly extracted from the entire unlabeled set, and AL process is then performed on the samples in the subset. Moreover, hybrid AL methods which combine the MRF-AL framework with either the passive random selection method or the existing AL methods are investigated. To evaluate and compare the proposed AL approaches with other state-of-the-art techniques, experiments were conducted on two hyperspectral data sets. Results demonstrated the effectiveness of the hybrid AL methods, as well as the advantage of the proposed MRF-AL framework.