Virtual Sample Generation Method Research Articles

Improving prediction of groundwater quality in situations of limited monitoring data based on virtual sample generation and Gaussian process regression

The increasing pollution of aquifers by human activities over recent decades poses a threat to drinking water safety. While Gaussian Process Regression (GPR) is a robust tool for predicting and monitoring water quality, its effectiveness is hindered limitations of available data on model training and validation, known as the “small sample problem”. Various attempts to resolve this problem include virtual sample generation (VSG). This study aimed to increase the accuracy of GPR for predicting water quality in situations of limited datasets. Three VSG methods, namely Multi Distribution Mega-Trend Diffusion (MD-MTD), Generative Adversarial Network (GAN), and t-distributed stochastic nearest neighbor embedding (t-SNE) were compared for enhancing the accuracy of GPR model prediction of Strontium (Sr2+). The models were used to predict Sr2+ in the shallow aquifer system in Songyuan, Jilin Province. The results showed that t-SNE provided the most significant improvement to the accuracy of the GPR, with R2 increasing from 0.86 to 0.99 (12.98 %), followed by MD-MTD (R2 of 0.95, 9.39 %), with the least improvement obtained by GAN (R2 of 0.92, 5.98 %). Boxplots show that MD-MTD-GPR predictions do not fully capture observed data distributions. GANs accurately replicate the data distribution, while t-SNE-GPR achieves the highest prediction accuracy and handles data fluctuations. GPR accuracy improves with an increasing number of virtual samples but tends to decrease when the number exceeds 258 in this study. This study can guide the improvement of the accuracy of GPR for situations of limited datasets. The results of this study can help improve water quality management and drinking water safety in regions with sparse monitoring data.

Improving small sample prediction performance via novel nonlinear interpolation virtual sample generation with self-supervised learning

In the early stages of production processes, due to expensive experimental costs and demanding experimental conditions, the amounts of collected samples are relatively limited. Information and knowledge extracted from minimal experimental data may be distorted and unreliable. Tackling this task, some researchers have recommended virtual sample generation (VSG), aiming at improving prediction accuracy of forecasting models for small sample sets by creating informative artificial samples. To more effectively capture nonlinear feature representations in small data, in this paper, we propose a promising VSG method based on self-supervised learning architecture to generate optimal, feasible virtual samples. The suggested technology considers distilling nonlinear feature representations using a manifold algorithm. Based on these representations, we integrated Newton’s divided difference and Chebyshev interpolation to create nonlinear interpolation points. We feed those interpolation points into a U-net model and an encoder-decoder net model to produce virtual sample inputs and output, respectively. Additionally, in this paper, we developed a Siamese network model to select virtual samples resembling original data. In the experiments, two industrial datasets were utilized to demonstrate efficacy of the proposed method. These experimental results show that the proposed method can more successfully boost prediction accuracy compared to the other three cutting-edge VSG methods.

Study on rapid prediction of low concentration o-nitrotoluene in mononitrotoluene mixture by near infrared spectroscopy combined with novel calibration strategies

The determination of the o-nitrotoluene (o-MNT) content in separation process of mononitrotoluene (MNT) is of interest, since it affects the purity of m-nitrotoluene (m-MNT) and p-nitrotoluene (p-MNT). However, the analytical techniques traditionally used for its content determination are tedious and time consuming. Therefore, we explored the analysis of spectral data based on near-infrared spectroscopy (NIRS) and chemometrics, and extracted the spectral features of the o-MNT based on the interval selection algorithm. The calibration models for the o-MNT content based on samples with different concentration ranges were developed by PLS. Among them, the calibration model based on samples with 0.01–0.5 % concentration range has the best prediction performance. The calibration model was established with the determination coefficient of prediction (R2) of 0.959, root mean squared error of prediction (RMSEP) of 0.011 and ratio of standard deviation of the calibration set to standard error of prediction (RPD) of 4.899 for o-MNT. It is sufficient to meet the fast detection needs of the o-MNT content for process control in chemical industry. In addition, in order to reduce the demand of the model on samples and corresponding reference values, we explored the virtual sample generation method based on background difference compensation. The calibration model based on virtual samples was established with R2 of 0.74, RMSEP of 0.028 and RPD of 1.951. This study shows that the method based on NIRS and chemometrics has strong prediction performance for o-MNT in separation process of MNT, which is a guideline for controlling product purity of m-MNT and p-MNT. And the virtual sample generation method proposed in this study can significantly reduce the sample demand of calibration model.

A transferred hybrid surrogate model integrating Gaussian membership virtual sample generation for small sample prediction: Applications in metal tube bending

The high-performance virtual sample generation (VSG) method has been extensively introduced to solve the problem of small sample sizes. Data distribution information is a key element of current VSG methods at the data-driven level. Herein, we propose an improved VSG method with a Gaussian distribution and explore the relationship between the Gaussian function and data expansion. To obtain more feasible virtual samples, information expanded based on the Gaussian membership function (GMIE) was established. For further improvement, a hybrid surrogate model based on transfer (THSM) is proposed, which differs from the general hybrid surrogate model (HSM) methods that only mix single models. Using a prevailing evaluation method, our proposed method, which combines a Gaussian membership function with a hybrid surrogate model, outperforms other competing approaches in 12 numerical cases owing to its feasibility and efficacy. Additionally, the proposed approach is applied to a metal tube rotary draw bending (RDB) prediction problem to illustrate its ability to support complex engineering designs.

A virtual sample generation method based on manifold learning and a generative adversarial network for soft sensor models with limited data

BackgroundIn the modern industrial process, data-driven-based soft sensor technology plays a key role in intelligent measurement. However, due to the strong dependence of the prediction accuracy on the training data, it can be challenging to apply this technology to particular industrial processes for which obtaining enough training samples is difficult. MethodsTo solve the problem of insufficient training data for soft sensor models, this paper proposes the Potential of Heat-diffusion for Affinity-based Trajectory Embedding and generative adversarial network based virtual sample generation (PGAN-VSG) method, which combines Potential of Heat-diffusion for Affinity-based Trajectory Embedding (PHATE) based feature extraction, virtual feature generation (VFG) techniques, and PHATE based generative adversarial network (PGAN) to generate virtual samples. The methodology of the work consists of several phases. First, the PHATE is deployed to embed the original data into a low-dimensional space to obtain the original feature set, which preserves the local and global structure of the original data. Second, a number of candidate features are randomly generated in the neighborhood space of each original feature, and the features located in the sparse feature region are selected as virtual features. Furthermore, the PGAN with L1 distance loss is used to map the virtual features to the corresponding high-dimensional space to obtain virtual samples. To validate the performance of the proposed PGAN-VSG method, a multilayer perceptron (MLP) model was employed as a soft sensor model to showcase the effectiveness of the proposed approach. This was achieved through both a numerical example and an industrial case study involving the penicillin fermentation process. Significant findingsBy filling the sample scarcity region while ensuring the distribution of generated samples, the virtual sample generation (VSG) method proposed in this paper improves the prediction accuracy of the soft sensor model while also outperforming the classical VSG method.

Novel virtual sample generation method based on data augmentation and weighted interpolation for soft sensing with small data

Novel virtual sample generation method based on data augmentation and weighted interpolation for soft sensing with small data

Generating virtual samples to improve learning performance in small datasets with non-linear and asymmetric distributions

In today’s highly competitive environment, modeling the relationship between inputs and outputs using limited data for a management system at the early stages is important, but difficult. The virtual sample generation (VSG) method has been proposed in many studies to explore potential information to improve prediction performance of learning models for small datasets. However, those studies in general must assume an underlying distribution such as a linear triangular membership function or a Gaussian distribution to generate virtual samples. Thus, previous VSG methods may not effectively upgrade learning performance when the assumed distribution is not elastic enough for small datasets. To address this issue, in this paper, we proposed a novel VSG method called Newton-VSG method to generate virtual samples for small datasets with non-linear and asymmetric distributions. In the suggested method, we used Newton's method to estimate the minimum value of data range and the shape of data distribution. Further, we developed two deep learning models including Siamese network (SN) model for screening virtual sample input values and bagging auto-encoder (AE) model for predicting virtual sample output to ensure the quality of virtual samples. One real dataset for solidification cracking susceptibility test data and an other real dataset obtained from TFT-LCD process of a leading company in Taiwan were used to demonstrate the efficacy of the proposed method. On the partial least square regression (PLSR) and the back propagation neural network (BPNN) predictive models, we compared the proposed method with three state-of-the-art VSG methods in items of the mean absolute error (MAE) and the root mean squared error (RMSE). The experimental results demonstrated that the proposed method outperforms the other three VSG methods in prediction accuracy for small datasets.

A novel virtual sample generation method to improve the quality of data and the accuracy of data-driven models

Small data volume and data imbalance often lead to statistical failure and seriously restrict the accuracy of data-driven models, which has become a bottleneck problem, needing to be solved, in small sample modeling. The data expansion method has become the main way to solve small sample modeling. However, the randomness in the process of virtual sample generation and combination leads to many invalid data, resulting in poor consistency between the expanded data and the original data. For this reason, this paper proposes a virtual sample generation method based on acceptable area and joint probability distribution sampling (APS-VSG) to limit the randomness in the data expansion method, reduce the proportion of invalid data, improve data consistency after expansion, and improve the accuracy of the data-driven model under the condition of small samples. Firstly, the concept of “compact range of interaction (CRI)” was proposed, which further limits the domain estimation range of data to approximate the valid area of the data. Secondly, the prior knowledge was used to improve mega-trend-diffusion (MTD), and the CRI is delineated according to the trend dispersion to obtain the acceptable area of the virtual data. Finally, a joint probability distribution was constructed based on the true values of small samples in the acceptable area, and data sampling was conducted based on the probability distribution to generate virtual data. The experimental results of standard function datasets show that the virtual samples generated by the proposed method can ensure validity of more than 85%. The experimental results of the NASA li-ion battery dataset show that, compared with Interpolation, Noise, MD-MTD, GAN, and GMM-VSG methods, the error of the data-driven model trained with virtual data generated by the proposed method is significantly reduced. Compared with GAN and GMM-VSG, MSE, RMSE, MAE, and MAPE are reduced by at least 19.3%, 10.6%, 15.4%, and 16.7%, respectively.

Improved learning performance for small datasets in high dimensions by new dual-net model for non-linear interpolation virtual sample generation

The number of reliable samples obtained in early decision-making activity is usually relatively small. Due to variable distribution and incomplete structure of tiny datasets, it is challenging to create reliable and robust predictive modeling using classic statistical and machine learning models in small sample settings. The virtual sample generation (VSG) technique improves model learning accuracies for minimal datasets across diverse applications. Virtual samples on independent variables were generated using established VSG methods predicated on the assumption of a probability distribution or a membership function to fill data gaps. However, in the actual world, non-linear function interactions between variables are common. To address this issue, this paper developed a novel VSG method called Dual-VSG, which generates non-linear interpolation virtual samples using a self-supervised learning (SSL) framework to improve learning performance on small datasets. We generated non-linear interpolation virtual samples without labels by estimating non-linear functions and transforming them into a high-dimensional space using the proposed dual-net model. The weights of the dual-net model are transferred to a downstream task to generate virtual sample labels. To demonstrate the effectiveness of the suggested strategy, this research employed five datasets. On the Backpropagation Neural Networks (BPNN) predictive model, we compared the suggested method's prediction performance to two state-of-the-art VSG approaches. To assess prediction performance on a regression dataset, the Mean Absolute Percentage Error (MAPE) and the Root Mean Square Error (RMSE) are used. Furthermore, the classification accuracy (ACC) and the Fl measure are used to assess classification capability on classification datasets. In addition, the paired t-test was utilized to see if the suggested Dual-VSG approach differed significantly from the other VSG methods in terms of RMSE, MAPE, accuracy (ACC), or F1 score. For short datasets, the suggested Dual-VSG method outperforms those VSG methods, according to our experimental results.

A machine learning approach for corrosion small datasets

In this work, we developed a QSAR model using the K-Nearest Neighbor (KNN) algorithm to predict the corrosion inhibition performance of the inhibitor compound. To overcome the small dataset problems, virtual samples are generated and added to the training set using a Virtual Sample Generation (VSG) method. The generalizability of the proposed KNN + VSG model is verified by using six small datasets from references and comparing their prediction performances. The research shows that for the six datasets, the proposed model is able to make predictions with the best accuracy. Adding virtual samples to the training data helps the algorithm recognize feature-target relationship patterns, and therefore increases the number of chemical quantum parameters correlated with corrosion inhibition efficiency. This proposed method strengthens the prospect of ML for developing material designs, especially in the case of small datasets.

Application of Virtual Sample Generation and Screening in Process Parameter Optimization of Botanical Medicinal Materials.

The small sample problem widely exists in the fields of the chemical industry, chemistry, biology, medicine, and food industry. It has been a problem in process modeling and system optimization. The aim of this study is to focus on the problems of small sample size in modeling, the process parameters in the ultrasonic extraction of botanical medicinal materials can be obtained by optimizing the extraction rate model. However, difficulty in data acquisition results in problem of small sample size in modeling, which eventually reduces the accuracy of modeling prediction. A virtual sample generation method based on full factorial design (FFD) is proposed to solve the problem ofa small sample size. The experiments are first conducted according to the Box- Behnken Design (BBD) to obtain small-size samples, and the response surface function is established accordingly. Then, virtual sample inputs are obtained by the FFD, and the corresponding virtual sample outputs are calculated by the response surface function. Furthermore, a screening method of virtual samples is proposed based on an extreme learning machine (ELM). The connection weights of ELM are used for further optimization and screening of the generated virtual samples. The results show that virtual sample data can effectively expand the sample size. The precision of the model trained on semi-synthetic samples (small-size experimental simples and virtual samples) is higher than the model trained merely on small-size experimental samples. The virtual sample generation and screening methods proposed in this paper can effectively solve the modeling problem of small samples. The reliable process parameters can be obtained by optimizing the model trained by the semi-synthetic samples.

An approach based on multivariate distribution and Gaussian copulas to predict groundwater quality using DNN models in a data scarce environment

Machine Learning models have become a fruitful tool in water resources modelling. However, it requires a significant amount of datasets for training and validation, which poses challenges in the analysis of data scarce environments, particularly for poorly monitored basins. In such scenarios, using Virtual Sample Generation (VSG) method is valuable to overcome this challenge in developing ML models. The main aim of this manuscript is to introduce a novel VSG based on multivariate distribution and Gaussian Copula called MVD-VSG whereby appropriate virtual combinations of groundwater quality parameters can be generated to train Deep Neural Network (DNN) for predicting Entropy Weighted Water Quality Index (EWQI) of aquifers even with small datasets. The MVD-VSG is original and was validated for its initial application using sufficient observed datasets collected from two aquifers. The validation results showed that from only 20 original samples, the MVD-VSG provided enough accuracy to predict EWQI with an NSE of 0.87. However the companion publication of this Method paper is El Bilali et al. [1].•Development of MVD-VSG to generate virtual combinations of groundwater parameters in data scarce environment.•Training deep neural network to predict groundwater quality.•Validation of the method with sufficient observed datasets and sensitivity analysis.

Virtual sample generation method based on generative adversarial fuzzy neural network

Virtual sample generation method based on generative adversarial fuzzy neural network

SIFT-Flow-Based Virtual Sample Generation for Single-Sample Finger Vein Recognition

Finger vein recognition is considered to be a very promising biometric identification technology due to its excellent recognition performance. However, in the real world, the finger vein recognition system inevitably suffers from the single-sample problem: that is, only one sample is registered per class. In this case, the performance of many classical finger vein recognition algorithms will decline or fail because they cannot learn enough intra-class variations. To solve this problem, in this paper, we propose a SIFT-flow-based virtual sample generation (SVSG) method. Specifically, first, on the generic set with multiple registered samples per class, the displacement matrix of each class is obtained using the scale-invariant feature transform flow (SIFT-flow) algorithm. Then, the key displacements of each displacement matrix are extracted to form a variation matrix. After removing noise displacements and redundant displacements, the final global variation matrix is obtained. On the single sample set, multiple virtual samples are generated for the single sample according to the global variation matrix. Experimental results on the public database show that this method can effectively improve the performance of single-sample finger vein recognition.

High-accuracy estimation method of typhoon storm surge disaster loss under small sample conditions by information diffusion model coupled with machine learning models

Typhoon storm surge disaster is the most severe marine disaster in China. Accurate estimation of typhoon storm surge disaster loss (TSSDL) is significant for emergency decisions and economic sustainability development. However, the TSSDL estimation is limited by small sample conditions, resulting in the low accuracy of the TSSDL estimation model based on machine learning. To solve the problems of easy overfitting and poor generalization ability of machine learning models under small sample conditions, the high-accuracy TSSDL estimation method was proposed. Firstly, this estimation method combines Gaussian Noise with the Information Diffusion Model based on the Vibrating String equation (named GN-VSIDM) to generate virtual samples to augment the original training set. Then, the augmented training set was applied to train three machine learning models. The results are as follows: the virtual sample generation method, i.e., GN-VSIDM, solves the small sample problem in the TSSDL estimation process and improves the machine models’ accuracy and robustness. Based on the GN-VSIDM and the eXtreme Gradient Boosting (named XGBoost) methods, the joint model GN-VSIDM-XGBoost is the optimal TSSDL estimation model. Compared with the original XGBoost model, the RMSE and R2 of the GN-VSIDM-XGBoost model are 0.1089 and 0.8292, which reduces 25.67% and improves 19.88%, respectively. Besides, the GN-VSIDM-XGBoost model possesses excellent robustness. The GN-VSIDM overcomes the limitation on the performance of TSSDL estimation models based on machine learning under small sample conditions. This study provides an effective case and method for solving the small sample problem in disaster loss assessment.

Co-training based virtual sample generation for solving the small sample size problem in process industry

With the development of industrialization, the production scale and complexity of process industries are getting larger and larger. But, limited by the small amounts of samples and the uneven sample distribution in the process industry, it is difficult to establish accurate and efficient data-driven soft sensor models to predict some variables. To further develop the application of soft sensor models, generating new virtual samples based on the original sample distribution to extend the sample set is an ideal approach to solve this problem. In this paper, a novel virtual sample generation method based on the co-training of two K-Nearest Neighbor (KNN) models is proposed. First, according to the sparse parameter, sparse regions in each dimension of the feature space are identified. Second, the input features of virtual samples are generated in these sparse regions by performing interpolation operations. Third, the outputs of virtual samples are predicted by double KNN regressors based on co-training. The qualified virtual samples are screened and the model is updated using these virtual samples to improve the prediction accuracy of the double KNN models. To verify the effectiveness and superiority of the proposed virtual sample generation method based on the co-training (CTVSG), case studies are conducted using two standard functions and a Purified Terephthalic Acid (PTA) industrial dataset, where the effectiveness of CTVSG is confirmed.

A framework based on multivariate distribution-based virtual sample generation and DNN for predicting water quality with small data

Deep Neural Network (DNN) is a powerful tool for predicting and monitoring water quality. However, its application is only limited to well-monitored zones where the availability of data for training and validation phases. In this study, we attempt to develop a novel framework based on Multivariate distributions (MVD) (elliptical copulas)-based Virtual Sample Generation (VSG) method to broaden the application of DNN to predict water quality even with a small dataset. This framework is evaluated to predict the Entropy Weighted Water Quality Index (EWQI) using DNN and Electrical Conductivity, Temperature, and pH as input variables, in Berrechid and Chaouia aquifer systems, Morocco. Validation results showed that the virtual samples generated from 400, 50, 30, and 20 original samples improved the NSE from 0.88 to 0.92, from 0.53 to 0.91, from 0.42 to 0.91, and from 0.24 to 0.87, respectively. Besides, sensitivity analysis of the methodology to the virtual data sizes and the original samples showed that the RMSE and NSE of the DNN models have limits in function to virtual data sizes according to the first order Exponential Decay and logistic trends, respectively. These limits highly depend on original sample sizes. Such empirical trends are crucial for reproducing the proposed methodology in other sites to determine optimal virtual datasets. Overall, the proposed methodology provided new insights to improve the DNN model performances in predicting water quality with small datasets. Hence, it is useful to manage water quality in order to supply clean water for the population in poorly monitored zones.

Dual adversarial learning-based virtual sample generation method for data expansion of soft senors

Many key quality variables are difficult to measure in complex industrial processes for various reasons, such as working conditions or economic costs, leading to inefficient production monitoring. In recent years, soft sensors with outstanding performance in variable estimation have been widely used. However, quality samples collected from industrial sites are often limited, which results in incomplete datasets that cannot meet the training requirements of soft sensors and poor performance in model learning and prediction. In this paper, a new virtual sample generation method DA-GAN based on generative adversarial network (GAN) is proposed to provide extra training samples for soft sensors. Adversarial net-and adversarial sample-based dual adversarial learning is implemented to reduce the adversarial noise in the discriminator gradient, which can improve the convergence speed and learning stability of the generator and obtain virtual samples with higher similarity to the real data. Furthermore, a sample screening method based on asymmetric acceptable domain range expansion is introduced to choose high-quality virtual samples. Experimental results of two industrial case studies show that the virtual samples provided by DA-GAN are closer to real samples than several other widely used generation methods. The performance of the prediction model trained with the dataset added by the virtual samples yielded from DA-GAN can be better improved.

An extreme learning machine based virtual sample generation method with feature engineering for credit risk assessment with data scarcity

As a typical category of data scarcity, small sample often makes it difficult to build a reliable machine learning model in credit risk assessment, and thus many virtual sample generation (VSG) methods have been proposed for sample augmentation based on sample distribution. In particular, when small sample with low dimensionality exists in credit datasets simultaneously, it becomes more difficult to predict the customer credit status for credit institutions. In order to solve these issues, an extreme learning machine (ELM) based VSG methodology with feature engineering is proposed for credit risk assessment with data scarcity. In this methodology, ELM-based VSG methodology is first used to generate virtual samples and solve data instance scarcity (i.e., small sample) issue. Second, feature engineering is used to solve data attribute scarcity (i.e., low dimensionality) issue. Finally, various classifiers are used to predict the performance for generated virtual samples. For verification purpose, two public credit datasets are used to conduct credit classification with data scarcity. The experimental results show that the proposed methodology can effectively improve the classification performance for credit risk assessment with data scarcity.

Face Recognition in Single Sample Per Person Fusing Multi-Scale Features Extraction and Virtual Sample Generation Methods

Although face recognition has received a lot of attention and development in recent years, it is one of the research hotspots due to the low efficiency of Single Sample Per Person (SSPP) information in face recognition. In order to solve this problem, this article proposes a face recognition method based on virtual sample generation and multi-scale feature extraction. First, in order to increase the training sample information, a new NMF-MSB virtual sample generation method is proposed by combining the Non-negative Matrix Factorization (NMF) reconstruction strategy with Mirror transform(M), Sliding window(S), and Bit plane(B) sample extension methods. Second, a feature extraction method (named WPD-HOG-P) based on Wavelet Packet Decomposition, Histograms of Oriented Gradients, and image Pyramid is proposed. The proposed WPD-HOG-P method is beneficial to multi-scale facial image feature extraction. Finally, based on the extracted WPD-HOG-P features, the recognition model is established by using a grid search optimization support vector machine. Experimental results on ORL and FERET data sets show that the proposed method has higher recognition rates and lower computational complexity than the benchmark methods.