Training Set Generation Research Articles

A GPR-based emulator for semi-numerical reionization code SCRIPT: parameter inference from 21 cm data

Semi-numerical models of reionization typically involve a large number of unknown parameters whose values are constrained by comparing with observations. Increasingly often, exploring this parameter space using semi-numerical simulations can become computationally intensive, thus necessitating the use of emulators. In this work, we present a likelihood emulator based on Gaussian Process Regression (GPR) for our semi-numerical reionization code, SCRIPT, and use it for parameter inference using mock 21 cm power spectrum data and Bayesian MCMC analysis. A unique aspect of our methodology is the utilization of coarse resolution simulations to identify high-probability regions within the parameter space, employing only a moderate amount of computational time. Samples drawn from these high-probability regions are used to construct the training set for the emulator. The subsequent MCMC using this GPR-trained emulator is found to provide parameter posteriors that agree reasonably well with those obtained using conventional MCMC. The computing time for the analysis, which includes both generation of training sets and training the emulator, is reduced by approximately an order of magnitude. This methodology is particularly advantageous in scenarios where one wants to use different parametrizations of reionization models and/or needs to start with broad prior distributions on the parameters, offering an efficient and effective means of parameter inference.

Drag-based analytical optimal de-orbiting guidance from low earth orbit via Deep Neural Networks

Controlled de-orbiting is crucial for a low earth orbiting (LEO) satellite or capsule intending to land in a desired location and to prevent damage to people and property on the ground caused by debris. Drag modulation is one possible mechanism to control de-orbiting, exploiting atmospheric drag variation to reduce the necessary orbital energy from the initial conditions to the re-entry interface. In this context, this paper proposes a novel targeted de-orbiting Artificial Neural Network (ANN) and drag-based guidance algorithm for a LEO artificial satellite. It relies on the combination of empirical and analytical relations with Deep Neural Networks (DNNs) to estimate the main parameters of the optimal control law in real time. The training set is composed of several optimal control solutions generated with a previously developed optimal control algorithm, exploiting the formulation in equinoctial modified orbital parameters to manage the large time scale of the problem and the computational cost. An innovative procedure for the training set generation led to the achievement of general results with a limited number of samples. The successful outcome of the guidance algorithm on over 1000 cases demonstrates its robustness and generalization of results. To conclude, a Linear Quadratic Regulator (LQR) feedback control is applied to one case to deal with a more realistic and uncertain density model.

A deformability-based biochip for precise label-free stratification of metastatic subtypes using deep learning

Cellular deformability is a promising biomarker for evaluating the physiological state of cells in medical applications. Microfluidics has emerged as a powerful technique for measuring cellular deformability. However, existing microfluidic-based assays for measuring cellular deformability rely heavily on image analysis, which can limit their scalability for high-throughput applications. Here, we develop a parallel constriction-based microfluidic flow cytometry device and an integrated computational framework (ATMQcD). The ATMQcD framework includes automatic training set generation, multiple object tracking, segmentation, and cellular deformability quantification. The system was validated using cancer cell lines of varying metastatic potential, achieving a classification accuracy of 92.4% for invasiveness assessment and stratifying cancer cells before and after hypoxia treatment. The ATMQcD system also demonstrated excellent performance in distinguishing cancer cells from leukocytes (accuracy = 89.5%). We developed a mechanical model based on power-law rheology to quantify stiffness, which was fitted with measured data directly. The model evaluated metastatic potentials for multiple cancer types and mixed cell populations, even under real-world clinical conditions. Our study presents a highly robust and transferable computational framework for multiobject tracking and deformation measurement tasks in microfluidics. We believe that this platform has the potential to pave the way for high-throughput analysis in clinical applications, providing a powerful tool for evaluating cellular deformability and assessing the physiological state of cells.

Reducing the Cost of Neural Network Potential Generation for Reactive Molecular Systems.

Although machine learning potentials have recently had a substantial impact on molecular simulations, the construction of a robust training set can still become a limiting factor, especially due to the requirement of a reference ab initio simulation that covers all the relevant geometries of the system. Recognizing that this can be prohibitive for certain systems, we develop the method of transition tube sampling that mitigates the computational cost of training set and model generation. In this approach, we generate classical or quantum thermal geometries around a transition path describing a conformational change or a chemical reaction using only a sparse set of local normal mode expansions along this path and select from these geometries by an active learning protocol. This yields a training set with geometries that characterize the whole transition without the need for a costly reference trajectory. The performance of the method is evaluated on different molecular systems with the complexity of the potential energy landscape increasing from a single minimum to a double proton-transfer reaction with high barriers. Our results show that the method leads to training sets that give rise to models applicable in classical and path integral simulations alike that are on par with those based directly on ab initio calculations while providing the computational speedup we have come to expect from machine learning potentials.

On the Development of a Digital Twin for Underwater UXO Detection Using Magnetometer-Based Data in Application for the Training Set Generation for Machine Learning Models.

Scanning underwater areas using magnetometers in search of unexploded ordnance is a difficult challenge, where machine learning methods can find a significant application. However, this requires the creation of a dataset enabling the training of prediction models. Such a task is difficult and costly due to the limited availability of relevant data. To address this challenge in the article, we propose the use of numerical modeling to solve this task. The conducted experiments allow us to conclude that it is possible to obtain high compliance with the numerical model based on the finite element method with the results of physical tests. Additionally, the paper discusses the methodology of simplifying the computational model, allowing for an almost three times reduction in the calculation time without affecting model quality. The article also presents and discusses the methodology for generating a dataset for the discrimination of UXO/non-UXO objects. According to that methodology, a dataset is generated and described in detail including assumptions on objects considered as UXO and nonUXO.

Performance of two complementary machine-learned potentials in modelling chemically complex systems

Chemically complex multicomponent alloys possess exceptional properties derived from an inexhaustible compositional space. The complexity however makes interatomic potential development challenging. We explore two complementary machine-learned potentials—the moment tensor potential (MTP) and the Gaussian moment neural network (GM-NN)—in simultaneously describing configurational and vibrational degrees of freedom in the Ta-V-Cr-W alloy family. Both models are equally accurate with excellent performance evaluated against density-functional-theory. They achieve root-mean-square-errors (RMSEs) in energies of less than a few meV/atom across 0 K ordered and high-temperature disordered configurations included in the training. Even for compositions not in training, relative energy RMSEs at high temperatures are within a few meV/atom. High-temperature molecular dynamics forces have similarly small RMSEs of about 0.15 eV/Å for the disordered quaternary included in, and ternaries not part of training. MTPs achieve faster convergence with training size; GM-NNs are faster in execution. Active learning is partially beneficial and should be complemented with conventional human-based training set generation.

Deep learning-based attenuation for shear-wave leakage from ocean-bottom node data

The high-amplitude shear wave in the z-component of ocean-bottom node (OBN) data challenges conventional seismic data imaging and interpretation. The process of shear-wave attenuation from the z-component data can be regarded as a coherent noise reduction problem. However, conventional noise attenuation methods, such as filtering or matching and subtraction, are time-consuming or have limited accuracy for field data processing. Unlike conventional methods with indirect feature recognition, pretrained neural networks provide end-to-end denoising frameworks by directly predicting and subtracting the shear-wave-free data from the original z-component data. Here, we have developed a deep learning-based method to address the shear-wave leakage problem and adopted a training set generation strategy based on the field data. Denoised z-component data are not needed to train the network. The generation of the training set and the extraction of noise features are based on the data of the x- and p-components. A residual learning strategy is introduced to extract the noise from the field data other than those in the training set. Synthetic examples confirm the excellent performance of our training set generation strategy. Field 4C OBN examples show that our method can attenuate shear-wave noise without harming the signals and, therefore, can be integrated into multicomponent field data processing.

A methodological approach to enable natural language interaction in an Intelligent Tutoring System

In this paper, we present and evaluate the recent incorporation of a conversational agent into an Intelligent Tutoring System (ITS), using the open-source machine learning framework Rasa. Once it has been appropriately trained, this tool is capable of identifying the intention of a given text input and extracting the relevant entities related to the message content. We describe both the generation of a realistic training set in Spanish language that enables the creation of the required Natural Language Understanding (NLU) models and the evaluation of the resulting system. For the generation of the training set, we have followed a methodology that can be easily exported to other ITS. The model evaluation shows that the conversational agent can correctly identify the majority of the user intents, reporting an f1-score above 95%, and cooperate with the ITS to produce a consistent dialogue flow that makes interaction more natural.

Detecting non-natural language artifacts for de-noising bug reports

Textual documents produced in the software engineering process are a popular target for natural language processing (NLP) and information retrieval (IR) approaches. However, issue tickets often contain artifacts such as code snippets, log outputs and stack traces. These artifacts not only inflate the issue ticket sizes, but also can this noise constitute a real problem for some NLP approaches, and therefore has to be removed in the pre-processing of some approaches. In this paper, we present a machine learning based approach to classify textual content into natural language and non-natural language artifacts at line level. We show how data from GitHub issue trackers can be used for automated training set generation, and present a custom preprocessing approach for the task of artifact removal. The training sets are automatically created from Markdown annotated issue tickets and project documentation files. We use these generated training sets to train a Markdown agnostic model that is able to classify un-annotated content. We evaluate our approach on issue tickets from projects written in C++, Java, JavaScript, PHP, and Python. Our approach achieves ROC-AUC scores between 0.92 and 0.96 for language-specific models. A multi-language model trained on the issue tickets of all languages achieves ROC-AUC scores between 0.92 and 0.95. The provided models are intended to be used as noise reduction pre-processing steps for NLP and IR approaches working on issue tickets.

Determination of physically based pseudo-Voigt powder diffraction profile terms from the fundamental parameters approach

A methodology is developed where a fundamental parameters approach (FPA) description of a laboratory powder diffraction instrument (configured in divergent-beam Bragg–Brentano geometry) is used to determine GSAS-II profile parameters for peak asymmetry and instrumental peak widths. This allows the instrumental contribution to peak shapes to be robustly determined directly from a physical description of the instrument, even though GSAS-II does not directly implement FPA for peak shape computation. The FPA-derived parameters can be used as the starting point for instrument characterization, or to characterize sample broadening without the use of a standard to determine the instrument profile function. This new method can facilitate generation of training sets for machine learning. A plot is generated that shows the differences between the two approaches, demonstrating upper bounds for the accuracy of the GSAS-II profile model for a particular instrumental configuration.

Deep Learning-Assisted Demodulation for TeraHertz Communications Under Hybrid Distortions

TeraHertz wireless communication has been regarded as an effective technology to satisfy the ever-increasing requirements of high-rate services, where the imperfections of TeraHertz devices, including in-phase/quadrature imbalance, phase noise and nonlinearity, regarded as hybrid distortions, have to be investigated. Due to the hybrid distortions, the explicit system model could not be mathematically derived, which limits the performance of the state-of-the-art demodulation methods. Here, a deep learning-assisted demodulation methodology is proposed to improve the bit error rate performance. Specifically, a multiple-output deep feedforward neural network is designed to fit the mapping between the received signal and likelihood information, where the number of outputs equals to the size of the modulation set, thus enabling the demodulation of the overlapped received signals corresponding to different constellation points. Besides, a training set generation method is proposed to generate the training examples without the prior knowledge of likelihood information. Simulation results validate that the proposed learning-assisted methodology could improve the demodulation performance of the TeraHertz wireless systems under severe hybrid distortions.

Deep learning-based instance segmentation for the precise automated quantification of digital breast cancer immunohistochemistry images

The quantification of biomarkers on immunohistochemistry breast cancer images is essential for defining appropriate therapy for breast cancer patients, as well as for extracting relevant information on disease prognosis. This is an arduous and time-consuming task that may introduce a bias in the results due to intra- and inter-observer variability which could be alleviated by making use of automatic quantification tools. However, this is not a simple processing task given the heterogeneity of breast tumors that results in non-uniformly distributed tumor cells exhibiting different staining colors and intensity, size, shape, and texture, of the nucleus, cytoplasm and membrane.In this research work we demonstrate the feasibility of using a deep learning-based instance segmentation architecture for the automatic quantification of both nuclear and membrane biomarkers applied to IHC-stained slides. We have solved the cumbersome task of training set generation with the design and implementation of a web platform, which has served as a hub for communication and feedback between researchers and pathologists as well as a system for the validation of the automatic image processing models. Through this tool, we have collected annotations over samples of HE, ER and Ki-67 (nuclear biomarkers) and HER2 (membrane biomarker) IHC-stained images. Using the same deep learning network architecture, we have trained two models, so-called nuclei- and membrane-aware segmentation models, which, once successfully validated, have revealed to be a promising method to segment nuclei instances in IHC-stained images. The quantification method proposed in this work has been integrated into the developed web platform and is currently being used as a decision support tool by pathologists.

Exploring chemical and conformational spaces by batch mode deep active learning

Batch active learning allows the efficient generation of powerful training sets in chemistry and materials science.

Robust High-Throughput Phenotyping with Deep Segmentation Enabled by a Web-Based Annotator.

The abilities of plant biologists and breeders to characterize the genetic basis of physiological traits are limited by their abilities to obtain quantitative data representing precise details of trait variation, and particularly to collect this data at a high-throughput scale with low cost. Although deep learning methods have demonstrated unprecedented potential to automate plant phenotyping, these methods commonly rely on large training sets that can be time-consuming to generate. Intelligent algorithms have therefore been proposed to enhance the productivity of these annotations and reduce human efforts. We propose a high-throughput phenotyping system which features a Graphical User Interface (GUI) and a novel interactive segmentation algorithm: Semantic-Guided Interactive Object Segmentation (SGIOS). By providing a user-friendly interface and intelligent assistance with annotation, this system offers potential to streamline and accelerate the generation of training sets, reducing the effort required by the user. Our evaluation shows that our proposed SGIOS model requires fewer user inputs compared to the state-of-art models for interactive segmentation. As a case study of the use of the GUI applied for genetic discovery in plants, we present an example of results from a preliminary genome-wide association study (GWAS) of in planta regeneration in Populus trichocarpa (poplar). We further demonstrate that the inclusion of a semantic prior map with SGIOS can accelerate the training process for future GWAS, using a sample of a dataset extracted from a poplar GWAS of in vitro regeneration. The capabilities of our phenotyping system surpass those of unassisted humans to rapidly and precisely phenotype our traits of interest. The scalability of this system enables large-scale phenomic screens that would otherwise be time-prohibitive, thereby providing increased power for GWAS, mutant screens, and other studies relying on large sample sizes to characterize the genetic basis of trait variation. Our user-friendly system can be used by researchers lacking a computational background, thus helping to democratize the use of deep segmentation as a tool for plant phenotyping.

Cluster learning-assisted directed evolution.

Directed evolution, a strategy for protein engineering, optimizes protein properties (i.e., fitness) by expensive and time-consuming screening or selection of large mutational sequence space. Machine learning-assisted directed evolution (MLDE), which screens sequence properties in silico, can accelerate the optimization and reduce the experimental burden. This work introduces a MLDE framework, cluster learning-assisted directed evolution (CLADE), that combines hierarchical unsupervised clustering sampling and supervised learning to guide protein engineering. The clustering sampling selectively picks and screens variants in targeted subspaces, which guides the subsequent generation of diverse training sets. In the last stage, accurate predictions via supervised learning models improve final outcomes. By sequentially screening 480 sequences out of 160,000 in a four-site combinatorial library with five equal experimental batches, CLADE achieves the global maximal fitness hit rate up to 91.0% and 34.0% for GB1 and PhoQ datasets, respectively, improved from 18.6% and 7.2% obtained by random-sampling-based MLDE.

Automated Development of Molten Salt Machine Learning Potentials: Application to LiCl.

The in silico modeling of molten salts is critical for emerging "carbon-free" energy applications but is inhibited by the cost of quantum mechanically treating the high polarizabilities of molten salts. Here, we integrate configurational sampling using classical force fields with active learning to automate and accelerate the generation of Gaussian approximation potentials (GAP) for molten salts. This methodology reduces the number of expensive ab initio evaluations required for training set generation to O(100), enabling the facile parametrization of a molten LiCl GAP model that exhibits a 19 000-fold speedup relative to AIMD. The developed molten LiCl GAP model is applied to sample extended spatiotemporal scales, permitting new physical insights into molten LiCl's coordination structure as well as experimentally validated predictions of structures, densities, self-diffusion constants, and ionic conductivities. The developed methodology significantly lowers the barrier to the in silico understanding and design of molten salts across the periodic table.

Transient-optimized real-bogus classification with Bayesian convolutional neural networks – sifting the GOTO candidate stream

ABSTRACT Large-scale sky surveys have played a transformative role in our understanding of astrophysical transients, only made possible by increasingly powerful machine learning-based filtering to accurately sift through the vast quantities of incoming data generated. In this paper, we present a new real-bogus classifier based on a Bayesian convolutional neural network that provides nuanced, uncertainty-aware classification of transient candidates in difference imaging, and demonstrate its application to the datastream from the GOTO wide-field optical survey. Not only are candidates assigned a well-calibrated probability of being real, but also an associated confidence that can be used to prioritize human vetting efforts and inform future model optimization via active learning. To fully realize the potential of this architecture, we present a fully automated training set generation method which requires no human labelling, incorporating a novel data-driven augmentation method to significantly improve the recovery of faint and nuclear transient sources. We achieve competitive classification accuracy (FPR and FNR both below 1 per cent) compared against classifiers trained with fully human-labelled data sets, while being significantly quicker and less labour-intensive to build. This data-driven approach is uniquely scalable to the upcoming challenges and data needs of next-generation transient surveys. We make our data generation and model training codes available to the community.

CHELM: Convex Hull based Extreme Learning Machine for salient object detection

Machine learning based saliency detection methods have achieved better performance than traditional methods. Here, we propose a machine learning based method that utilizes Convex Hull and Extreme Learning Machine (ELM) for detecting salient object(s) in an image. The novelty of the proposed method lies in the generation of training set without using human annotations. Initially, an input image is segmented using SLIC algorithm at different scales to produce multiscale segmented images. This is followed by estimating two different saliency priors viz. (a) Convex Hull center prior and (b) contrast prior for each segmented image. These priors exploit foreground center and spatially weighted contrast respectively. Both of these estimated priors help in computing initial saliency of each segment across all scales. For each scale, the initial saliency map along with the Convex Hull based label map is employed on a segmented image to determine the positive (salient) and negative (background) training set. Distinctive features for each segment belonging to the training set are extracted and then passes to the Extreme Learning Machine for learning the ELM model. Afterwards, multiscale saliency maps of an image are found by applying the learned ELM model on distinctive features extracted from each segment across multiple scales. These multiscale saliency maps are linearly combined to obtain the final saliency map. The effectiveness of the proposed method is supported through extensive experimental results performed on six publicly available datasets viz. MSRA10K, DUT-OMRON, ECSSD, PASCAL-S, SED2, and THUR15K. The performance of the proposed method was compared with 11 state-of-the-art methods in terms of Precision, Recall, F-Measure, Receiver Operating Characteristics (ROC), and Area under the curve (AUC). The proposed method outperforms or comparable with compared methods in terms of all the performance measures.

도로포장에서 파손 및 크랙 탐지를 위한 이미지 레이블링 기술 분석 및 학습셋 생성 모델

도로포장에서 파손 및 크랙 탐지를 위한 이미지 레이블링 기술 분석 및 학습셋 생성 모델

Active learning for robust, high-complexity reactive atomistic simulations

Machine learned reactive force fields based on polynomial expansions have been shown to be highly effective for describing simulations involving reactive materials. Nevertheless, the highly flexible nature of these models can give rise to a large number of candidate parameters for complicated systems. In these cases, reliable parameterization requires a well-formed training set, which can be difficult to achieve through standard iterative fitting methods. Here, we present an active learning approach based on cluster analysis and inspired by Shannon information theory to enable semi-automated generation of informative training sets and robust machine learned force fields. The use of this tool is demonstrated for development of a model based on linear combinations of Chebyshev polynomials explicitly describing up to four-body interactions, for a chemically and structurally diverse system of C/O under extreme conditions. We show that this flexible training database management approach enables development of models exhibiting excellent agreement with Kohn-Sham density functional theory in terms of structure, dynamics, and speciation.