Initial Training Samples Research Articles

A robust ensemble surrogate model and its application in the in situ data modelling and analysis of a combine harvester

In this article, a robust ensemble model is proposed based on extended adaptive hybrid functions and fuzzy clustering. In the outlier detection stage, each sample is assigned memberships to judge whether it is an outlier or not, where the memberships are determined based on the responses of the ensemble surrogate model of each cluster. Then, the detected outliers are removed from the initial training samples, and the final prediction model is constructed based on the remaining normal samples. The results of numerical problems and the in-situ dataset from a combine harvester show that the proposed model can provide accurate detection results for outliers and accurate prediction results for new points. The sensitivity analysis based on the proposed robust ensemble model indicates that the angle of guide plate, the open rate of cleaning fan, and the height of header have a greater effect on the cleaning loss of combine harvester.

An active learning framework for personalized deep brain stimulation

Abstract Background: To personalize deep brain stimulation (DBS), we need to identify a link between DBS parameters and the neural response of an individual. The existing approach based on random or empirical sampling (RS) is time-consuming, costly, and practically impossible in a clinical setting. This limits the discovery of novel settings under challenging situations. To address this problem, we have developed a new algorithmic framework based on active learning (AL) that can obtain the best model between the DBS parameters and the brain response while minimizing the number of experiments. Methods: We used a computational Parkinson’s disease model to generate synthetic data. We swept the subthalamic nucleus DBS amplitude, frequency, and pulse width while estimating the globus pallidus internus (GPi) beta (13-30 Hz) power for each DBS parameter. This resulted in 200 different samples. We randomly selected 80% of this data for pool training and reserved the remaining 20% as unseen test data. Using three initial training samples, we trained two linear regression models (based on RS and AL) to obtain a link between DBS parameters and GPi beta power. We iteratively added one training sample at a time to both models based on AL and RS approaches until we had 20 training samples, then tested both models on unseen test data at each iteration and calculated the root mean squared error (RMSE). This process was repeated 1000 times. Results: The mean RMSE of the AL and RS approach was 0.043 and 0.039. Results showed the AL-based model outperformed the RS-based model by showing significantly less errors on the unseen dataset based on a two-sample t-test (p= 2.33e-07, N=1000). Conclusion: We validated that our AL approach outperforms the existing approach in identifying an individualized link between DBS parameters and brain response while minimizing the duration of the experimental procedure. Research Category and Technology and Methods Translational Research: 1. Deep Brain Stimulation (DBS) Keywords: Personalized Deep Brain Stimulation, Active Learning, Machine Learning

Stabbing of Intrusion with Learning Framework Using Auto Encoder Based Intellectual Enhanced Linear Support Vector Machine for Feature Dimensionality Reduction

Using an Intelligent Intrusion Detection System (IIDS) instead of less effective firewalls and other intrusion detection systems can increase network security. The system's overall effectiveness is determined by the accuracy and speed of IIDS' categorization and training algorithms. According to research, Stabbing Intrusion Using Learning Framework (SILF) is an innovative and intelligent method of learning attack features and lowering dimensionality. To improve Enhanced Long Short-Term Memory (ELSTM) prediction accuracy while minimising testing and training time, an auto-encoder approach, which is an efficient learning methodology for feature generation in an unsupervised way is applied. Initial training samples are fed into the classifier to increase the predictability of incursion and classification accuracy. Thus, model efficacy may be achieved linearly while alternative classifier approaches such as conventional SVM, Random Forest (RF), and Naive Bayesian (NB) are investigated and compared. In this research, an efficient Intelligent Intrusion Detection System using Auto Encoder with Enhanced LSTM (IIDS-AE-ELSTM) is proposed for feature dimensionality reduction. Testing and training have shown that the proposed model is more efficient than existing systems in terms of performance measures such as accuracy, precision, recall, and F-measure. A new method to intrusion detection is presented, which increases detection of network intrusions with dimensionality reduction. The Python environment is used in the proposed model to create an efficient dimensionality reduction model for intrusion detection.

Model-data-driven P-wave impedance inversion using ResNets and the normalized zero-lag cross-correlation objective function

Model-driven and data-driven inversions are two prominent methods for obtaining P-wave impedance, which is significant in reservoir description and identification. Based on proper initial models, most model-driven methods primarily use the limited frequency bandwidth information of seismic data and can invert P-wave impedance with high accuracy, but not high resolution. Conventional data-driven methods mainly employ the information from well-log data and can provide high-accuracy and high-resolution P-wave impedance owing to the superior nonlinear curve fitting capacity of neural networks. However, these methods require a significant number of training samples, which are frequently insufficient. To obtain P-wave impedance with both high accuracy and high resolution, we propose a model-data-driven inversion method using ResNets and the normalized zero-lag cross-correlation objective function which is effective for avoiding local minima and suppressing random noise. By using initial models and training samples, the proposed model-data-driven method can invert P-wave impedance with satisfactory accuracy and resolution. Tests on synthetic and field data demonstrate the proposed method's efficacy and practicability.

Prediction of Unstable Hydrodynamic Forces on Submerged Structures under the Water Surface Using a Data-Driven Modeling Approach

Catastrophic failures of partially or fully submerged structures, e.g., offshore platforms, hydrokinetic turbine blades, bridge decks, etc., due to the dynamic impact of free surface flows such as waves or floods have revealed the need to evaluate their reliability. In this respect, an accurate estimation of hydrodynamic forces and their relationship to instability in structures is required. The computational fluid dynamics (CFD) solver is known as a powerful tool to identify dynamic characteristics of flow; however, it commonly consumes a huge computational cost, especially in cases of re-simulations needed. In this paper, an efficient surrogate model based on the Gaussian process is developed to rapidly predict the nonlinear hydrodynamic pressure coefficients on submerged bodies near the water surface. For this purpose, a CFD model is first developed, which is based on a two-dimensional incompressible Navier–Stokes solver incorporating free surface treatment and turbulent flow models. Then, an experimental design is adopted to generate initial training samples considering the effect of the submerged body shape ratio and flow Re number. Surrogate models of hydrodynamic pressure coefficients and their instability based on Gaussian process modeling are established using the outcome from the CFD simulations, where optimal trend and correlation functions are also investigated. Once surrogate models are obtained, the mean and oscillation amplitudes of hydrodynamic pressure coefficients on a submerged rectangular body, which represents the shape of most civil structures, can be rapidly predicted without the attempt at re-simulation. The findings can be practically applied in rapidly assessing hydrodynamic forces and their instability of existing submerged civil structures or in designing new structures, where a suitable shape ratio should be adopted to avoid flow-induced instability of hydrodynamic forces.

A new active learning Kriging metamodel for structural system reliability analysis with multiple failure modes

Recently, the active learning Kriging (ALK) metamodel has proved to be an efficient method for structural system reliability analysis with multiple failure modes. A key step for enhancing the accuracy and efficiency of an ALK method is to select training samples around the limit state of the performance function as fast as possible. This study develops a new ALK metamodel based on the concept of significant domain where samples contribute the most to the failure of the system, which is termed as the ALK-SD, to achieve the above purpose. First, multiple Kriging models are established based on several initial training samples for each component using a modified DACE toolbox. Then, the significant domain of an engineering system is identified according to the type of the system. Thereafter, a new learning function H is defined to sequentially update the initial Kriging models by selecting the most probable points in the significant domain. Finally, four numerical examples and one engineering example are studied to illustrate the effectiveness of the proposed method, and the corresponding results are subsequently compared with those obtained via similar methods in literature in terms of the computational efficiency and the locations of newly added training samples. The results show that the proposed method can effectively and accurately evaluate the structural system reliability by outperforming its counterparts.

Mapping irrigated croplands in China using a synergetic training sample generating method, machine learning classifier, and Google Earth Engine

Agricultural irrigation is an important vehicle for increasing crop yield, but large-scale irrigation has posed great challenges to global and regional water availability and climate change via altering land–atmosphere interactions. The knowledge of irrigation distribution is essential to understand regional water cycles and guide agricultural management decision-making, but such information is scarce in China. We developed a remote sensing-dominated framework to map irrigated croplands in China at 500 m resolution using a synergetic training sample generating method, machine learning classifier, and a cloud computing platform (Google Earth Engine, GEE). To overcome the challenges of lacking nationwide training samples, we first produced two provisional irrigation maps by fusing statistics and MODIS-derived annual peak greenness indices. The two provisional irrigation maps were then spatially filtered with an existing irrigation product (GRIPC) to construct the initial training sample pool. Next, to enhance the robustness and cover more irrigated candidates, we screened and introduced the irrigated croplands in three land use/cover maps (CCI-LC, GLC_FCS, and NLCD) to supplement the training data pool. Afterward, we utilized locally adaptive random forest classifiers and data cubes (MODIS-derived spectral indices, climatic and topographic variables) to generate irrigation maps in each province of China on GEE. The resulting map outperformed other current irrigation maps with an overall accuracy of 79.2% . The map also showed a reasonable consistency with statistical data at the province and prefecture levels, with the determination coefficient (R2) of 0.89 and 0.77, respectively. In total, we identified 87.04 million hectares of irrigated croplands in mainland China in 2015. Using the resulting map and water use statistics, we found a high correlation between irrigation area and agricultural water use in Northwest, Northeast, and South China, and a low correlation in North China Plain. This map is expected to serve national water resource management and assist decision-making in improving agricultural adaption to climate change.

A high-dimension structural reliability method based on active learning Kriging and dimension reduction technique

For efficiently and accurately estimating the failure probability and the further sensitivity index of the high-dimension structures, a novel AS-AK-MCS method is proposed in this work. This method fully employs the merits of the active subspace-based dimension reduction technique, the active learning (AL) Kriging surrogate model, and the Monte Carlo simulation. In the proposed method, the intractable gradient information needed by the active subspace method is obtained by a crude Kriging model with initial training sample points. In the construction of the crude Kriging model, the proposed trend model selection criterion reduces the man-made error. Then the active subspace method converts the reliability analysis from the original high-dimension space into the low-dimension subspace, which facilities constructing an AL-Kriging model and also avoids the tricky “curse of dimension” problem. The state-of-the-art U learning function is applied as the points adding criterion in the active subspace. In order to demonstrate the effectivity and versatility of the proposed method, three representative examples including the linear/nonlinear and the explicit/implicit performance functions are studied for estimating the failure probability and the failure probability-based sensitivity index. Finally, the proposed method is applied to the reliability and sensitivity analysis of a composite radome structure.

Molecular Field Analysis Using Computational-Screening Data in Asymmetric N-Heterocyclic Carbene-Copper Catalysis toward Data-Driven In Silico Catalyst Optimization

Abstract A molecular-field-based regression analysis using computational screening data for N-heterocyclic carbene (NHC)-Cu-catalyzed asymmetric carbonyl additions of a silylboronate to aldehydes is reported. A computational screening was performed to collect enantioselectivity data (ΔΔG‡: energy differences between the transition states leading to each enantiomer) via transition-state (TS) calculations using density functional theory (DFT) methods. A molecular field analysis (MFA) was carried out using the obtained calculated ΔΔG‡ values and TS structures (30 samples in total). Important structural information for enantioselectivity extracted by the MFA was visualized on the TS structures, which provided insight into an asymmetric induction mechanism. Based on the obtained information, chiral NHC ligands were designed, which showed improved enantioselectivity in these carbonyl additions (designed ligands: up to 96% ee, initial training samples: up to 73% ee).

THE ALGORITHM TREE METHOD IN SOLVING THE TASK OF CLASSIFYING HYDROGRAPHIC DATA

Context. The work is dedicated to the identification of a simple and effective mechanism by which it is possible to build algorithmic classification trees (algorithmic tree models) on the basis of fixed initial information in the form of a discrete data training sample. The constructed algorithmic classification tree will unmistakably classify (recognize) the entire training sample on which the model is built, have a minimum structure (structural complexity) and consist of components – autonomous classification and recognition algorithms as the vertices of the structure (attributes of the tree).
 Objective. The aim of this work is to create a simple, effective and universal method of constructing classification (recognition) models based on the concept of algorithmic trees for arrays of real hydrographic data, where the obtained schemes of classification systems (classification tree structure) are characterized by a tree structure (construction) and autonomous classification algorithms (sets of generalized features) as their structural elements (construction blocks).
 Method. The general scheme of synthesizing classification trees in a form of algorithmic trees on the basis of a procedure of approximation of an array of discrete data by a set of elementary classifiers, which for the set initial training sample builds a tree-like structure, i.e. a model of the algorithmic tree, is suggested. Moreover, the constructed scheme consists of a set of autonomous classification and recognition algorithms evaluated at each step/stage of constructing the classification tree for this initial sample. A method for constructing an algorithmic classification tree has been developed, the main idea of which is to approximate step-by-step the initial sample of an arbitrary volume and structure by a set of elementary classification algorithms. The method of algorithmic tree in the formation of the current algorithmic tree vertex, node, generalized feature provides selection of the most effective, highquality elementary classifiers from the initial set and completion of only those paths in the tree structure where the largest number of errors (failures) occurs. The structural complexity of the algorithmic tree design is estimated based on the number of transitions, vertices and tiers of the model structure, which allows one to improve the quality of its subsequent analysis, provide an effective decomposition mechanism, and build algorithmic tree structures under fixed constraint sets. The method of the algorithmic tree synthesis allows one to build different types of tree-like recognition models with different initial sets of elementary classifiers with predetermined accuracy for a wide class of problems of the artificial intelligence theory.
 Results. The developed method of building algorithmic tree models allows one to work with training samples of a large amount of different types of information (discrete data) and provides high speed and economy of hardware resources in the process of generating the final classification scheme, as well as to build classification trees with predetermined accuracy.
 Conclusions. An approach to the synthesis of new recognition algorithms (schemes) based on a library (set) of already known algorithms (methods) and schemes has been developed. That is, an effective scheme for recognizing discrete objects based on stepby-step evaluation and selection of classification algorithms (generalized features) at each step of the scheme synthesis is presented. Based on the suggested concept of algorithmic classification trees, a model of the structure of the algorithm tree was built, which provided classification of flood situations for the Uzh river basin.

Incremental Learning Model for Load Forecasting without Training Sample

This article presents hourly load forecasting by using an incremental learning model called Online Sequential Extreme Learning Machine (OS-ELM), which can learn and adapt automatically according to new arrival input. However, the use of OS-ELM requires a sufficient amount of initial training sample data, which makes OS-ELM inoperable if sufficiently accurate sample data cannot be obtained. To solve this problem, a synthesis of the initial training sample is proposed. The synthesis of the initial sample is achieved by taking the first data received at the start of working and adding random noises to that data to create new and sufficient samples. Then the synthesis samples are used to initial train the OS-ELM. This proposed method is compared with Fully Online Extreme Learning Machine (FOS-ELM), which is an incremental learning model that also does not require the initial training samples. Both the proposed method and FOS-ELM are used for hourly load forecasting from the Hourly Energy Consumption dataset. Experiments have shown that the proposed method with a wide range of noise levels, can forecast hourly load more accurately than the FOS-ELM.

METHOD FOR SYNTHESIZING LOGICAL CLASSIFICATION TREES BASED ON THE SELECTION OF ELEMENTARY FEATURES

The general problem of constructing logical recognition and classification trees is considered. The object of this study is logical classification trees. The subject of the research is current methods and algorithms for constructing logical classification trees. The aim of the work is to create a simple and effective method for constructing recognition models based on classification trees for training samples of discrete information, which is characterized by elementary features in the structure of synthesized logical classification trees. A general method for constructing logical classification trees is proposed, which builds a tree structure for a given initial training sample, which consists of a set of elementary features evaluated at each step of building a model for this sample. A method for constructing a logical tree is proposed, the main idea of which is to approximate the initial sample of an arbitrary volume with a set of elementary features. When forming the current vertex of the logical tree, the node provides selection of the most informative, qualitative elementary features from the original set. This approach, when constructing the resulting classification tree, can significantly reduce the size and complexity of the tree, the total number of branches and tiers of the structure, and improve the quality of its subsequent analysis. The proposed method for constructing a logical classification tree makes it possible to build tree-like recognition models for a wide class of problems in the theory of artificial intelligence. The method developed and presented in this paper received a software implementation and was investigated when solving the problem of classifying geological data. The experiments carried out in this paper confirmed the operability of the proposed mathematical support and show the possibility of using it to solve a wide range of practical recognition and classification problems. Prospects for further research may consist in creating a limited method of the logical classification tree, which consists in maintaining a criterion for stopping the procedure for constructing a logical tree according to the depth of the structure, optimizing its software implementations, as well as experimental studies of this method for a wider range of practical tasks.

Method of algorithmic classification trees based on constraints

The general problem of constructing algorithmic recognition (classification) trees based on a limited method in the theory of artificial intelligence is considered. The object of this research is the concept of an algorithmic classification tree based on a bounded method. The subject of the research is actual methods, algorithms and schemes (limited method) for constructing algorithmic classification trees. A limited method for constructing algorithmic classification trees is proposed, which for a given initial training sample of any size builds a tree structure (algorithm tree model), which consists of a set of autonomous classification algorithms and recognition evaluated at each stage of constructing algorithm trees based on this initial sample. That is, a limited method for constructing an algorithmic classification tree is proposed, the main idea of which is a step-by-step approximation of the initial sample of an arbitrary volume and structure by a set of independent classification and recognition algorithms. This method, when forming the current vertex of the algorithmic tree, ensures that the most efficient (high-quality) autonomous classification algorithms are selected from the initial set and only those paths in the tree structure where the greatest number of classification errors occur are completed. The limited method of constructing an algorithmic classification tree makes it possible to build different types of tree-like recognition models with predefined accuracy for a wide class of problems in the theory of artificial intelligence. The limited method of the algorithmic classification tree developed and presented in this paper received a software implementation and was investigated and compared with the methods of logical classification trees, methods of the algorithmic classification tree (first and second types) when solving the problem of recognizing real geological data. Figs.: 2. Tabl.: 1. Refs.: 34 titles.

Time-dependent structural system reliability analysis model and its efficiency solution

For the widely existing time-dependent structural system (TDSS), the current model ignores the interaction among multiple modes in the minimum transformation of the performance function of each mode w.r.t. the time, it may misjudge the state of the system at some instants and result in the error of estimating the reliability. To avoid this possible mistake, this paper proposes an improved reliability analysis model for the TDSS. In the proposed model, the TDSS performance function is transformed into the single-mode time-dependent performance function firstly according to the relationship of multiple modes. Then the failure probability of the TDSS can be estimated by the single-mode time-dependent performance function. The relationship of multiple modes is strictly considered in the proposed model. On the basis of the improved model, this paper also presents the Kriging surrogate model combined with the Monte Carlo simulation method to estimate the failure probability of the TDSS. In this Kriging method, the initial training samples are uniformly selected in the sample space and a new extremum learning function is proposed. Theoretical analysis verifies the rationality of the proposed reliability analysis model, and the efficiency of the proposed method is illustrated by the results of numerical examples.

Construction and Monte Carlo Estimation of Wavelet Frames Generated by a Reproducing Kernel

We introduce a construction of multiscale tight frames on general domains. The frame elements are obtained by spectral filtering of the integral operator associated with a reproducing kernel. Our construction extends classical wavelets as well as generalized wavelets on both continuous and discrete non-Euclidean structures such as Riemannian manifolds and weighted graphs. Moreover, it allows to study the relation between continuous and discrete frames in a random sampling regime, where discrete frames can be seen as Monte Carlo estimates of the continuous ones. Pairing spectral regularization with learning theory, we show that a sample frame tends to its population counterpart, and derive explicit finite-sample rates on spaces of Sobolev and Besov regularity. Our results prove the stability of frames constructed on empirical data, in the sense that all stochastic discretizations have the same underlying limit regardless of the set of initial training samples.

Machine‐learning‐enhanced tail end prediction of structural response statistics in earthquake engineering

AbstractEvaluating the response statistics of nonlinear structures constitutes a key issue in engineering design. Hereby, the Monte Carlo method has proven useful, although the computational cost turns out to be considerably high. In particular, around the design point of the system near structural failure, a reliable estimation of the statistics is unfeasible for complex high‐dimensional systems. Thus, in this paper, we develop a machine‐learning‐enhanced Monte Carlo simulation strategy for nonlinear behaving engineering structures. A neural network learns the response behavior of the structure subjected to an initial nonstationary ground excitation subset, which is generated based on the spectral properties of a chosen ground acceleration record. Then using the superior computational efficiency of the neural network, it is possible to predict the response statistics of the full sample set, which is considerably larger than the initial training sample set. To ensure a reliable neural network response prediction in case of rare events near structural failure, we propose to extend the initial training sample set increasing the variance of the intensity. We show that using this extended initial sample set enables a reliable prediction of the response statistics, even in the tail end of the distribution.HIGHLIGHTSA new Monte Carlo method is developed that provides the response statistics of a nonlinear system in the tail end of the distribution.A nonstationary filter is applied to a Gaussian white noise to generate realistic artificial earthquake records.An extended training strategy using neural networks is proposed to improve the reliability of the method in the tail end of the distribution.The new strategy reveals a significant speedup as well as a prediction of the response statistics in the tail end with high confidence.

Prediction of the fact and degree of coronary artery disease using the processing of clinical and instrumental data by artificial intelligence

Aim of the study was to analyze the possibility of using neural network analysis to predict the severity of coronary bed lesion. The study was also designated to determine the performance and accuracy of the trained neural network model receiving input as the structured data and EGG images with the parameters and leads positioning differ from the training sample, and also to compare the efficiency of detecting transient myocardial ischemia with traditional diagnostic methods, such as 24-hour Holter monitoring, treadmill test. Neural network analysis of the available clinical, laboratory and instrumentation data allow to configure the network parameters for further prediction of coronary artery disease. The results obtained in the form of an AUC score allow to consider this method to be effective in the coronary artery disease diagnosis using recorded ECG tape with parameters and lead positioning differ from initial training sample. The efficiency of transient myocardial ischemia detection on the training sample of the trained neural network is higher in comparison with traditional diagnostic methods, such as 24-hour Holter monitoring, treadmill test.

New Adaptive Surrogate-Based Approach Combined Swarm Optimizer Assisted Less Tuning Cost of Dynamic Production-Inventory Control System

Today, most manufacturing control systems are complex and expensive, so they are limited to employ a small number of function evaluations for optimal design. Yet, looking for optimization methods with the less-computational cost is an open issue in engineering control systems. This paper aims to propose an effective adaptive optimization approach by integrating Kriging surrogate and Particle Swarm Optimization (PSO). In this method, a novel iterative adaptive approach is utilized using two sets of training samples including initial training and adaptive sample points. The initial training points are designed by space-filling design, while the adaptive points are generated using a new jackknife resampling approach. The proposed approach can effectively convergence towards the global optimal point using a small number of function evaluations. The efficiency and applicability of the proposed algorithm are evaluated using the optimal design of the fractional-order PID (FOPID) controller for some benchmark transfer functions. Then, the introduced approach is applied for tuning the parameters and the sensitivity analysis of the FOPID controller for a dynamic production-inventory control system. The results are in good agreement with the results reported in the literature, while the proposed approach is executed with a lower computational burden.

TRAINING DATA SAMPLING FOR CONVENTIONAL NEURAL NETWORKS CONFIGURING

The problem of generating training data for setting up the convolutional neural networks is considered, which is of great importance in the construction of intelligent medical diagnostic systems, where due to the lack of elements of the training sample, it is proposed to use the approaches of artificial data multiplication based on the initial training sample of a fixed size for the image processing (the results of the ultrasound, CT and MRI). It shows that the increase of the training sample resulted in less informative and poor quality elements, which can introduce extra errors in the goal achievement. To eliminate this situation the algorithm for assessing the quality of a sample element with the subsequent removal of uninformative elements is proposed.

FORMATION OF A LEARNING SET FOR THE TASK OF IMAGE PROCESSING

The problem of forming a training set for the task of image processing is considered. It is shown that this task is of great importance in the construction of intelligent medical diagnostic systems in which convolution neural networks are used for image processing (results of ultrasound, CT and MRI). Due to the lack of elements of the training sample, it is proposed, on the one hand, to use approaches of artificial data multiplication based on the initial training sample of a fixed volume, and on the other hand, to use methods that reduce the need for large training samples, both through the use of ensemble topology (hybrid neural networks), and by applying the transfer learning approach. An algorithm for the formation of a training set for image processing tasks is developed based on the modification of the initial input information with the calculation of the confidence measure of the obtained sample.