Virtual Sample Generation Method Research Articles

A novel virtual sample generation method based on a modified conditional Wasserstein GAN to address the small sample size problem in soft sensing

In the modern chemical industry process, soft sensing has been widely used. However, the lack of valid and sufficient data has made it difficult to apply advanced soft sensor modeling methods to realistic scenarios. In this paper, a novel soft sensing method based on deep learning is proposed to handle the problem of small data. Aiming at handling the issue of small sample size, a novel virtual sample generation method embedding a deep neural network as a regressor into conditional Wasserstein generative adversarial networks with gradient penalty (rCWGAN) is presented. In rCWGAN, conditional variables are introduced to make the training supervised and a dual training algorithm is specially designed. With the advanced structure and the designed training algorithm, rCWGAN has powerful sample generation capabilities and can well predict quality variables. Finally, an experiment on the purified terephthalic acid (PTA) solvent system is carried out for the validation of the presented rCWGAN. The simulation results indicate that the presented rCWGAN has good sample approximation ability and acceptable prediction accuracy with the small sample size task.

An approach for process optimisation of the Automated Fibre Placement (AFP) based thermoplastic composites manufacturing using Machine Learning, photonic sensing and thermo-mechanics modelling

The automated fibre placement (AFP) process is a complex manufacturing technique with many variables which affect the final part quality. Inverse Machine Learning (ML) models can be used as decision-aid tools for optimising thermoplastic composites manufacturing. However, a common challenge of ML application in manufacturing is the acquisition of relevant and sufficient data. To overcome this small-data learning problem, a hybrid approach has been proposed here which combines the benefits of ML algorithms such as the Artificial Neural Networks (ANN), virtual sample generation (VSG) methods, physics-based numerical simulations and data obtained from experiments and photonic sensors, to enhance the manufacturing process.

Novel SVD integrated with GBDT based Virtual Sample Generation and Its Application in Soft Sensor

With the coming of the big data era, data-driven based modeling approaches have become the hot research topic in recent years. Unfortunately, due to the limitation of the actual process, the data is basically in a steady state and it is difficult to obtain enough high-quality data, which is defined as the small sample (SS) problem. Recently, to deal with the SS problem, a virtual sample generation (VSG) approach based on the distribution of the original data has been taken into account. In this paper, a VSG method based on singular value decomposition (SVD) feature decomposition and gradient boosting decision tree (GBDT) prediction model (SVD-GBDT) is proposed. In the proposed SVD-GBDT method, firstly, the distribution characteristics of the original data are used to extract the main features and expand the number of samples by using the SVD algorithm. Then the GBDT algorithm is used to find the virtual output of the virtual samples by the SVD method. Finally, SVD and GBDT are combined to complete the sample expansion (SVD-GBDT-VSG). In this paper, we choose the purified terephthalic acid (PTA) industrial process to verify the effectiveness of the proposed methodology. Simulation results show that compared with related methods, the proposed SVD-GBDT-VSG algorithm in this paper can achieve sample expansion well and at the same time can effectively improve the accuracy performance of soft measurement.

Improved Virtual Sample Generation Method Using Enhanced Conditional Generative Adversarial Networks with Cycle Structures for Soft Sensors with Limited Data

Improved Virtual Sample Generation Method Using Enhanced Conditional Generative Adversarial Networks with Cycle Structures for Soft Sensors with Limited Data

A method for capacity prediction of lithium-ion batteries under small sample conditions

Accurate life prediction of lithium-ion battery is very important for the safe operation of battery system. At present, the data-driven life prediction method is an effective method. However, it is difficult to obtain full life cycle data of long-life lithium batteries, which leads to low accuracy of prediction results. In addition, the degradation of lithium-ion batteries has different trends in different stages, the commonly used methods are insufficient to describe global time variables which make it difficult to adapt to changes in different stages of lithium-ion battery capacity degradation. To solve the above problems, the paper proposes a deep adaptive continuous time-varying cascade network based on extreme learning machines (CTC-ELM) under the condition of small samples. First, a virtual sample generation method based on multi-population differential evolution is proposed, which uses multi-distribution overall trend diffusion technology to adaptively determine the virtual sample range, and combines with the improved differential evolution algorithm to achieve small sample data amplification. Then, a new prediction network with CTC-ELM is constructed. Finally, it is verified on different data sets. Experiments show that the method proposed can effectively expand the sample set of lithium-ion batteries and achieve high accuracy in the estimation of lithium-ion battery capacity.

A Gaussian mixture model based virtual sample generation approach for small datasets in industrial processes

Due to small-quantity and often imbalance of labeled samples, it is challenging to establish a robust and accurate prediction model through data-driven methods. To deal with the small dataset problem, new virtual samples may be generated via virtual sample generation (VSG) methods based on the trend of the original small raw dataset, thereby improving modeling performance. Effective VSG is desirable, but also challenging. Conventional VSG usually assumes that the raw sample set contains only a single operating mode. Taking multi-mode into account will improve the VSG based modeling performance since actual processes are often multi-mode. To this end, an information expansion function considering sample density and amount (IEDA) is first developed to expand the domain range of the attributes in this paper. Then, virtual samples under the multiple operating mode condition are generated by proposing a Gaussian mixture model based virtual sample generation (GMMVSG) method. Applications of GMMVSG on Tennessee Eastman benchmark process and an industrial hydrocracking process show significant improvement of modeling and predictions over other conventional VSG methods.

Enhanced virtual sample generation based on manifold features: Applications to developing soft sensor using small data

In the process industry, it is essential to establish a data-driven soft sensor to predict the key variable that is difficult to online measure directly. The accuracy performance of data-driven soft sensors relies heavily on data. Unfortunately, it is hard to acquire sufficient and informative data from the samples with limited number, which is called as the small sample problem. For handling the small sample problem, it is a good solution to generating virtual samples according to the distribution of original data. This paper proposes an enhanced method of virtual sample generation utilizing manifold features to develop soft sensors using small data. First, T-Distribution Stochastic Neighbor Embedding (t-SNE) is utilized to extract the features of input data. The main idea of generating virtual samples is to use the interpolation algorithm to obtain virtual t-SNE input features and then the random forest algorithm is utilized to get the virtual outputs using virtual t-SNE input features. Finally, virtual samples using the proposed t-SNE based virtual sample generation (t-SNE-VSG) can be achieved. For the sake of confirming the effectiveness and feasibility of the presented t-SNE-VSG, a standard data set is first used. What is more, a small data set from an actual industrial process of Purified Terephthalic Acid is used to establish a soft sensor model. The results from simulations show that the accuracy performance of the soft sensor established with small data can be effectively improved by adding the virtual samples generated by t-SNE-VSG. In addition, t-SNE-VSG achieves superior accuracy to state-of-the-art virtual sample generation methods.

Novel Virtual Sample Generation Using Target-Relevant Autoencoder for Small Data-Based Soft Sensor

In the complex industrial processes, the process data have the characteristics of imbalance and are incomplete due to the difficult-to-measure key variables, leading to the performance degradation of soft sensors. In order to deal with this problem, a novel noise injection virtual sample generation approach based on target-relevant autoencoder (NITAE-VSG) is proposed in this article. Different from the conventional VSG methods, the presented NITAE-VSG model can generate useful input and output virtual samples directly. To increase the diversity of virtual samples, the Gaussian noise (G_noise) is added into the features extracted from the target-relevant autoencoder (TAE) hidden layer. The generated virtual samples are combined with the original small samples to enhance the soft sensor model performance. To verify the reliable feasibility and superior effectiveness of the presented NITAE-VSG, two actual industrial cases: an industrial process of purified terephthalic acid shortened as PTA and a production system of ethylene are carried out. The results of simulations demonstrate that the presented NITAE-VSG can generate superior virtual samples to some other virtual sample generation (VSG) methods and can effectively improve the soft sensor modeling accuracy.

Unsupervised change detection method in SAR images based on deep belief network using an improved fuzzy C‐means clustering algorithm

Deep learning methods have recently displayed ground-breaking results for synthetic aperture radar image change detection problem. However, they still face the challenges of intrinsic noise and the difficulty of acquiring labeled data. To sort out these issues, we aim to develop a change detection approach specifically designed for analyzing synthetic aperture radar images based on Deep Belief Network as the deep architecture which includes unsupervised feature learning and supervised network fine-tuning. The deep neural networks can reach the final change maps directly from the two original images. A pre-classification based on Morphological Reconstruction and Membership Filtering is employed in order to minimize the effect of noise. Appropriate diversity samples are provided by a virtual sample generation method in order to mitigate overfitting raised by limited synthetic aperture radar data. Visual and quantitative analysis as well as comparisons with advanced algorithms show that our algorithm not only achieves better results but also requires less implementation time.

AEB effectiveness research methods based on reconstruction results of truth vehicle-to-TW accidents in China

Vehicle-to-TW accidents are one of the largest components of traffic accidents system in China, which always leads to severe accident consequences because the cyclists are vulnerable road users and drive at a high collision velocity. With the development of information and communication technology, automated Emergency Braking (AEB) system has been applied to modern vehicles, which can perform emergency braking automatically in dangerous situations and mitigate the consequence of accident. The purpose of this study is to propose a method to identify the mechanism that affects the effectiveness of AEB functions under real-life vehicle-to-TW traffic accident scenarios with Chinese county characteristics. Through the analysis and reconstruction for the sampling accident case-by-case, the whole process of the accident has been reproduced. The trajectory analysis program determines the accident triggering conditions and then generates the virtual physical parameters of the triggering conditions through the virtual sample generation method based on the initial sample. In parallel, the AEB effectiveness simulation program has been established. Plug in parameters of AEB system generated by the Latin hypercube sampling to the AEB effectiveness simulation program for getting the mechanism that affect the effectiveness of AEB functions. The AEB system parameters generated by Latin hypercube sampling are inserted into the AEB effectiveness simulation program to obtain the mechanism that affects the effectiveness of the AEB function. Under the multi-objective optimization problem of accident avoidance rate, technical cost and occupant comfort, the optimal parameters and multi-objective values of AEB are obtained by the NSGA-II algorithm. These results can adapt to Chinese actual traffic conditions to a certain extent, and provide a reliable basis for the research and development of AEB system in China.

A probabilistic sequence classification approach for early fault prediction in distribution grids using long short-term memory neural networks

As the global power grid must undergo a profound transformation in the coming decades to ensure reliable and cost-effective operation in a system with large shares of intermittent renewable energy generation, a critical element will be to leverage advanced data-driven predictive tools to optimise grid management activities. As it is expected that existing grids will be operated more to their limits, it is important to obtain better operational insights and estimations of the time to equipment failure to provide useful operational guidance and maintenance prioritisation support for grid operators. In this regard, this paper proposes a novel and real-time applicable method for fault prediction in 10 kV underground oil-insulated power cables using low-resolution data from a real case study from a Danish distribution system operator. The developed method is based on a sequence classification approach using long short-term memory neural networks where three different operational states are defined (Normal, Early warning, and Critical warning) to allow for prediction flexibility and better indication of the presence of systemic faults. Moreover, to enhance the data foundation, this paper investigates a Virtual Sample Generation method based on an adaptive Gaussian distribution. The capability of the proposed method yields satisfying results with prediction accuracy on the test set reaching as high as ~90%, hence proving the usefulness of the proposed approach and paving the way for smarter maintenance protocols.

Predictive maintenance within combined heat and power plants based on a novel virtual sample generation method

Combined heat and power plants are important assets in the current energy infrastructure, as they play an essential role in ensuring the security of energy supply. Therefore, they must have low downtime. Currently, preventive maintenance is the most popular maintenance strategy for combined heat and power plants. The preventive maintenance includes performing regular, scheduled maintenance checks and repairs, whether they are needed or not. The preventive maintenance is effective to a certain degree and reduces the downtime. However, there is a need for it to be upgraded to predictive maintenance. The current preventive maintenance strategy creates a barrier for combined heat and power plants to adopt a predictive maintenance framework as limited samples of faulty data will be available. To overcome this issue, this paper proposes a novel virtual sample generation method. The method creates a set of lifetime ratios based on the actual faults, Random Walks, and particle swarm optimisation faults to generate virtual samples. The method is implemented on two case studies. The first case study is used to evaluate the method on a benchmark example with publicly available data. The second case study is a set of feedwater pumps in the Danish Power Plant, Studstrupværket, run and operated by Ørsted. This case study is used for field testing and evaluation. The findings indicate that the method significantly improves the accuracy and robustness of the remaining-useful-lifetime model. The approach furthermore allows for using expert knowledge, which could be attractive for other energy applications.

Novel manifold learning based virtual sample generation for optimizing soft sensor with small data

Due to the extremely complex mechanism and strong non-linear characteristics of industrial processes, data-driven soft sensor technologies play a key role in the intelligent measurement of process industries. However, the information of the collected process data in the steady stage is quite limited and unreliable, causing the small sample problem. As a result, it becomes an intractable challenge to catch the nature of the process and build accurate soft sensor models. To solve this problem, this paper proposes a novel manifold learning based virtual sample generation method (Isomap-VSG) to generate feasible virtual samples in the information gaps for supplementing the original small sample space. To find data sparse regions reasonably, one kind of manifold learning methods called Isomap is used to visualize process data with high dimension. Then virtual samples can be generated by the interpolation method and extreme learning machine. The simulation results on a standard dataset and a real-world application demonstrate that, compared with other advanced methods, the proposed Isomap-VSG method can achieve better performance in terms of generating feasible virtual samples and improving the accuracy of soft sensor models using limited samples.

A Mega-Trend-Diffusion and Monte Carlo based virtual sample generation method for small sample size problem

Data-driven modeling has attracted wide attention in academia because of its effectiveness. However, Due to the lack of data, some traditional modeling methods, such as extreme learning machine (ELM), can’t achieve high learning accuracy. A novel approach based on Mega-Trend-Diffusion (MTD) and Monte Carlo is presented in this paper to deal with the problem, named Monte Carlo Mega-Trend-Diffusion (MCMTD). The proposed approach utilizes MTD to estimate the acceptable range of the attributions and Latin hypercube sampling method to sample. ELM is employed to establish the prediction model. In this paper, two real data sets, the multi-layer ceramic capacitors (MLCC) and the purified terephthalic acid (PTA), are used to verify the effectiveness and reasonability of MCMTD. The experimental results show that MCMTD can significantly enhance the accuracy and ability of the forecasting model.

A Novel Virtual Sample Generation Method to Overcome the Small Sample Size Problem in Computer Aided Medical Diagnosing

Deep neural networks are successful learning tools for building nonlinear models. However, a robust deep learning-based classification model needs a large dataset. Indeed, these models are often unstable when they use small datasets. To solve this issue, which is particularly critical in light of the possible clinical applications of these predictive models, researchers have developed approaches such as virtual sample generation. Virtual sample generation significantly improves learning and classification performance when working with small samples. The main objective of this study is to evaluate the ability of the proposed virtual sample generation to overcome the small sample size problem, which is a feature of the automated detection of a neurodevelopmental disorder, namely autism spectrum disorder. Results show that our method enhances diagnostic accuracy from 84%–95% using virtual samples generated on the basis of five actual clinical samples. The present findings show the feasibility of using the proposed technique to improve classification performance even in cases of clinical samples of limited size. Accounting for concerns in relation to small sample sizes, our technique represents a meaningful step forward in terms of pattern recognition methodology, particularly when it is applied to diagnostic classifications of neurodevelopmental disorders. Besides, the proposed technique has been tested with other available benchmark datasets. The experimental outcomes showed that the accuracy of the classification that used virtual samples was superior to the one that used original training data without virtual samples.

Building robust models for small data containing nominal inputs and continuous outputs based on possibility distributions

Learning with small data is challenging for most algorithms in regard to building statistically robust models. In previous studies, virtual sample generation (VSG) approaches have been verified as effective in terms of meeting this challenge. However, most VSG methods were developed for numerical inputs. Therefore, to address situations where data has nominal inputs and continuous outputs, a systemic VSG procedure is proposed to generate samples based on fuzzy techniques to further enhance modelling capability. Based on the concept of the data preprocess in the M5′ model tree, we reveal a useful procedure by which to extract the fuzzy relations between nominal inputs and continuous outputs. Further, with the idea of nonparametric operations, we employ trend similarity to present the fuzzy relations between inputs and outputs. Then, these relations are represented by possibility distributions, and sample candidates are created based on these distributions. Finally, the candidates filtered using \(\alpha\)-cut are regarded as qualified virtual samples. In the experiments, we demonstrate the effectiveness of our approach through a comparison with two other VSG approaches using five public datasets and two prediction models. Moreover, three parameters used in our approaches are discussed. However, determining how to find the most fit parameters requires further study in the future.

Using virtual samples to improve learning performance for small datasets with multimodal distributions

A small dataset that contains very few samples, a maximum of thirty as defined in traditional normal distribution statistics, often makes it difficult for learning algorithms to make precise predictions. In past studies, many virtual sample generation (VSG) approaches have been shown to be effective in overcoming this issue by adding virtual samples to training sets, with some methods creating samples based on their estimated sample distributions and directly treating the distributions as unimodal without considering that small data may actually present multimodal distributions. Accordingly, before estimating sample distributions, this paper employs density-based spatial clustering of applications with noise to cluster small data and applies the AICc (the corrected version of the Akaike information criterion for small datasets) to assess clustering results as an essential procedure in data pre-processing. Once the AICc shows that the clusters are appropriate to present the data dispersion of small datasets, each of their sample distributions is estimated by using the maximal p value (MPV) method to present multimodal distributions; otherwise, all of the data is inferred as having unimodal distributions. We call the proposed method multimodal MPV (MMPV). Based on the estimated distributions, virtual samples are created with a mechanism to evaluate suitable sample sizes. In the experiments, one real and two public datasets are examined, and the bagging (bootstrap aggregating) procedure is employed to build the models, where the models are support vector regressions with three kernel functions: linear, polynomial, and radial basis. The results show that the forecasting accuracies of the MMPV are significantly better than those of MPV, a VSG method developed based on fuzzy C-means, and REAL (using original training sets), based on most of the statistical results of the paired t test.

A Virtual Sample Generation Method Based on Differential Evolution Algorithm for Overall Trend of Small Sample Data: Used for Lithium-ion Battery Capacity Degradation Data

Considering the wide application of lithium-ion battery in life, the prediction of the remaining life of lithium-ion battery has become a research hotspot. Studies show, due to the improvement of the technology level of lithium-ion battery, its life is getting longer and longer. Even under the condition of accelerated life test, it is difficult to obtain enough available data for research in a short term. In order to solve the problem of how to accurately predict the residual life with the data-driven method under the condition of small sample size, an overall trend virtual sample generation method based on differential evolution (OT-DEVSG) is proposed. This method uses a differential evolution algorithm with better optimization performance, and improves the original mega-trend-diffusion (MTD) method, the range of virtual samples is effectively constrained and the trend of samples can be estimated more accurately. The method can effectively generate a virtual sample data sequence with time parameters, and adapt the virtual sample to the real-life sample trend, which solves the problem of insufficient degradation data of the lithium-ion batteries. Finally, we validate the effectiveness of the OT-DEVSG method with three existing data sets. The experimental results show that the proposed OT-DEVSG method is effective for solving the problem of long-term life prediction of lithium-ion batteries.

RETRACTED: A novel nonlinear VSG integrating ELM with noise injection for enhancing energy modelling and analysis on small data

Abstract Building a robust and accurate energy analysis model is considered as an important issue in the field of petrochemical industries. Under the circumstance of small samples, the accuracy of the energy analysis model is unacceptable. In order to solve this problem, a novel noise injection integrated with extreme learning machine based nonlinear virtual sample generation method is proposed. Through injecting noise in the output matrix of the hidden layer of Extreme learning machine (ELM), a virtual information matrix that is different from the original one generated using the original small dataset can be obtained. Then the newly generated information matrix is adopted to produce good-quality virtual samples for supplement knowledge for small samples. To authenticate the effectiveness of the proposed method, the proposed method is developed as an energy analysis model for an ethylene production process. Simulation results indicate that good virtual samples can be generated using the proposed method, and the accuracy of the energy analysis model is much improved with the aid of the newly generated virtual samples. The proposed method will effectively help production departments of petrochemical industries set more suitable targets of energy consumption and make better use of available resources.

Using an attribute conversion approach for sample generation to learn small data with highly uncertain features

Accelerating new product development has become an important marketing strategy for manufacturers who are competing globally. However, this may lead to the small data learning issue. Although machine learning algorithms are used to extract knowledge from training samples, algorithms may not output satisfactory predictions when training sizes are small. This paper provides a real case of a TFT-LCD (thin film transistor liquid crystal display) maker when a new strengthened cover glass is developed using chemical processes. With very little prior experience about the processes involved, engineers attempted to improve the yield rates by determining the parameters from a few pilot-run data. However, owing to the fact that the processes were different from those required to make TFT-LCD panels, the highly uncertain characteristics of the processes led to the use of two virtual sample generation (VSG) approaches, bootstrap aggregating (bagging) and the synthetic minority over-sampling technique, from which unsatisfactory results were obtained. Accordingly, this study was used to develop a systematic VSG method based on fuzzy theory to tackle the learning issue. The experimental results show that support vector regressions built with training sets containing the proposed samples present more precise predictions and thus can help engineers infer more correct manufacturing parameters.