Active Data Selection Research Articles

Assisted Active Learning for Model-Based Method Development in Liquid Chromatography.

In recent decades, there has been a growing interest in fully automated methods for tackling complex optimization problems across various fields. Active learning (AL) and its variant, assisted active learning (AAL), incorporating guidance or assistance from external sources into the learning process, play key roles in this automation by enabling the autonomous selection of optimal experimental conditions to efficiently explore the problem space. These approaches are particularly valuable in situations wherein experimentation is costly or time-consuming. This study explores the application of AAL in model-based method development (MD) for liquid chromatography (LC) by using Bayesian statistics to incorporate historical data and analyte information for the generation of initial retention models. The process involves updating the model parameters based on new experiments, coupled with an active data selection method to choose the most informative experiment to run in a subsequent step. This iterative process balances model exploitation and experimental exploration until a satisfactory separation is achieved. The effectiveness of this approach is demonstrated via two practical examples, resulting in optimized separations in a limited number of experiments by optimizing the gradient slope. It is shown that the ability of AAL to leverage past knowledge and compound information to improve accuracy and reduce experimental runs offers a flexible alternative approach to fixed design methods.

Active Data Selection and Information Seeking

Bayesian inference typically focuses upon two issues. The first is estimating the parameters of some model from data, and the second is quantifying the evidence for alternative hypotheses—formulated as alternative models. This paper focuses upon a third issue. Our interest is in the selection of data—either through sampling subsets of data from a large dataset or through optimising experimental design—based upon the models we have of how those data are generated. Optimising data-selection ensures we can achieve good inference with fewer data, saving on computational and experimental costs. This paper aims to unpack the principles of active sampling of data by drawing from neurobiological research on animal exploration and from the theory of optimal experimental design. We offer an overview of the salient points from these fields and illustrate their application in simple toy examples, ranging from function approximation with basis sets to inference about processes that evolve over time. Finally, we consider how this approach to data selection could be applied to the design of (Bayes-adaptive) clinical trials.

Bayesian deep-learning for RUL prediction: An active learning perspective

Deep learning (DL) has been intensively exploited for remaining useful life (RUL) prediction in the recent decade. Although with high precision and flexibility, DL methods need sufficient run-to-failure data to guarantee their performance. However, run-to-failure data is fairly expensive to obtain in many industrial applications. How to economically achieve high accuracy with few run-to-failure data becomes a critical and emergent issue. In this study, a Bayesian deep-active-learning framework is proposed for RUL prediction, which goes beyond traditional passive learning and introduces a novel active learning perspective. We use Bayesian neural networks with Monte Carlo dropout inference to predict RUL with uncertainty quantification for samples without run-to-failure labels. The prediction uncertainty is further used to develop an acquisition function for actively selecting target samples to obtain their run-to-failure labels. A recursive model training and active data selection mechanism are then developed to maintain accuracy while reducing the size of the training data. Two practical examples, one from a public bearing dataset and the other from our lab testing on battery degradation, are presented to demonstrate the proposed method. Experimental results demonstrate that 20 and 40% of run-to-failure data can be saved for the bearing and the battery RUL prediction, respectively.

Representing globally accurate reactive potential energy surfaces with complex topography by combining Gaussian process regression and neural networks.

There has been increasing attention in using machine learning technologies, such as neural networks (NNs) and Gaussian process regression (GPR), to model multi-dimensional potential energy surfaces (PESs). A PES constructed using NNs features high accuracy and generalization capability, but a single NN cannot actively select training points as GPR does, resulting in expensive ab initio calculations as the molecular complexity increases. However, a PES constructed using GPR has a slow speed of evaluation and it is difficult to accurately describe a fast-changing potential. Herein, an efficient scheme for representing globally accurate reactive PESs with complex topography based on as few points as possible by incorporating active data selection of GPR into NN fitting is proposed. The validity of this strategy is tested using the BeH2+ system, and only 1270 points are automatically sampled. The generalization performance and speed of evaluation of the generated PES are much better than those of the GPR PES constructed using the same dataset. Moreover, an accurate NN PES is fitted by 12 122 points as a benchmark for comparison to further test the global accuracy of the PES obtained using this scheme, and the corresponding results present extremely consistent topography characteristics and calculated Be+(2S) + H2 reaction probabilities.

Data Efficient Lithography Modeling With Transfer Learning and Active Data Selection

Lithography simulation is one of the key steps in physical verification, enabled by the substantial optical and resist models. A resist model bridges the aerial image simulation to printed patterns. While the effectiveness of learning-based solutions for resist modeling has been demonstrated, they are considerably data-demanding. Meanwhile, a set of manufactured data for a specific lithography configuration is only valid for the training of one single model, indicating low data efficiency. Due to the complexity of the manufacturing process, obtaining enough data for acceptable accuracy becomes very expensive in terms of both time and cost, especially during the evolution of technology generations when the design space is intensively explored. In this paper, we propose a new resist modeling framework for contact layers, utilizing existing data from old technology nodes and active selection of data in a target technology node, to reduce the amount of data required from the target lithography configuration. Our framework based on transfer learning and active learning techniques is effective within a competitive range of accuracy, i.e., $3 \times -10 \times $ reduction on the amount of training data with comparable accuracy to the state-of-the-art learning approach.

Adaptive non-intrusive reduced order modeling for compressible flows

An adaptive non-intrusive reduced basis (RB) method based on Gaussian process regression (GPR) is proposed for parametrized compressible flows. Adaptivity is pursued in the offline stage. The reduced basis by proper orthogonal decomposition (POD) is constructed iteratively to achieve a specified tolerance. For GPR, active data selection is used at each iteration, with standard deviation as the error indicator. To improve accuracy for shock-dominated flows, a properly designed simplified problem (SP) is considered as input of the regression models in addition to using parameters directly. Furthermore, a surrogate error model is constructed to serve as an efficient error estimator for the GPR models. Several two- and three-dimensional cases are conducted, including the inviscid nozzle flow, the inviscid NACA0012 airfoil flow and the inviscid M6 wing flow. For all the cases, the trained models are able to make efficient predictions with reasonable accuracy in the online stage. The SP-based approach is observed to result in biased sampling towards transonic regions. The regression models are further applied in sensitivity analysis, from which the solution of the two-dimensional cases are shown to be significantly more sensitive to input parameters than the wing flow. This is consistent to the comparison of convergence histories between the parameter-based and the SP-based models. For cases of high sensitivity, the SP-based approach is superior and can help to significantly reduce the number of required snapshots to achieve a prescribed tolerance.

Reduced order modeling for nonlinear structural analysis using Gaussian process regression

A non-intrusive reduced basis (RB) method is proposed for parametrized nonlinear structural analysis undergoing large deformations and with elasto-plastic constitutive relations. In this method, a reduced basis is constructed from a set of full-order snapshots by the proper orthogonal decomposition (POD), and the Gaussian process regression (GPR) is used to approximate the projection coefficients. The GPR is carried out in the offline stage with active data selection, and the outputs for new parameter values can be obtained rapidly as probabilistic distributions during the online stage. Due to the complete decoupling of the offline and online stages, the proposed non-intrusive RB method provides a powerful tool to efficiently solve parametrized nonlinear problems with various engineering applications requiring multi-query or real-time evaluations. With both geometric and material nonlinearities taken into account, numerical results are presented for typical 1D and 3D examples, illustrating the accuracy and efficiency of the proposed method.

Construction of reactive potential energy surfaces with Gaussian process regression: active data selection

ABSTRACTGaussian process regression (GPR) is an efficient non-parametric method for constructing multi-dimensional potential energy surfaces (PESs) for polyatomic molecules. Since not only the posterior mean but also the posterior variance can be easily calculated, GPR provides a well-established model for active learning, through which PESs can be constructed more efficiently and accurately. We propose a strategy of active data selection for the construction of PESs with emphasis on low energy regions. Through three-dimensional (3D) example of H3, the validity of this strategy is verified. The PESs for two prototypically reactive systems, namely, H + H2O ↔ H2 + OH reaction and H + CH4 ↔ H2 + CH3 reaction are reconstructed. Only 920 and 4000 points are assembled to reconstruct these two PESs respectively. The accuracy of the GP PESs is not only tested by energy errors but also validated by quantum scattering calculations.

Active Semisupervised Clustering Algorithm with Label Propagation for Imbalanced and Multidensity Datasets

The accuracy of most of the existing semisupervised clustering algorithms based on small size of labeled dataset is low when dealing with multidensity and imbalanced datasets, and labeling data is quite expensive and time consuming in many real-world applications. This paper focuses on active data selection and semisupervised clustering algorithm in multidensity and imbalanced datasets and proposes an active semisupervised clustering algorithm. The proposed algorithm uses an active mechanism for data selection to minimize the amount of labeled data, and it utilizes multithreshold to expand labeled datasets on multidensity and imbalanced datasets. Three standard datasets and one synthetic dataset are used to demonstrate the proposed algorithm, and the experimental results show that the proposed semisupervised clustering algorithm has a higher accuracy and a more stable performance in comparison to other clustering and semisupervised clustering algorithms, especially when the datasets are multidensity and imbalanced.

Self-organizing nets for optimization.

Given some optimization problem and a series of typically expensive trials of solution candidates sampled from a search space, how can we efficiently select the next candidate? We address this fundamental problem by embedding simple optimization strategies in learning algorithms inspired by Kohonen's self-organizing maps and neural gas networks. Our adaptive nets or grids are used to identify and exploit search space regions that maximize the probability of generating points closer to the optima. Net nodes are attracted by candidates that lead to improved evaluations, thus, quickly biasing the active data selection process toward promising regions, without loss of ability to escape from local optima. On standard benchmark functions, our techniques perform more reliably than the widely used covariance matrix adaptation evolution strategy. The proposed algorithm is also applied to the problem of drag reduction in a flow past an actively controlled circular cylinder, leading to unprecedented drag reduction.

Discussion of the Papers by Atkinson, and Bates et al.

Discussion of the Papers by Atkinson, and Bates <i>et al.</i>

Information-Based Objective Functions for Active Data Selection

Learning can be made more efficient if we can actively select particularly salient data points. Within a Bayesian learning framework, objective functions are discussed that measure the expected informativeness of candidate measurements. Three alternative specifications of what we want to gain information about lead to three different criteria for data selection. All these criteria depend on the assumption that the hypothesis space is correct, which may prove to be their main weakness.