Virtual Sample Generation Research Articles

Improving learning accuracy by using synthetic samples for small datasets with non-linear attribute dependency

Small-data problems are commonly encountered in the early stages of a new manufacturing procedure, presenting challenges to both academics and practitioners, as good performance is difficult to achieve with learning models when there is a lack of sufficient data. Virtual sample generation (VSG) has been shown to be an effective method to overcome this issue in a wide range of studies in various fields. Such works usually assume that the relations among attributes are independent of each other, and produce synthetic data by using sample distributions of these. However, the VSG technique may be ineffective if the real data has interrelated attributes. Therefore, this research provides a novel procedure to generate related virtual samples with non-linear attribute dependency. To construct a relational model between the independent and dependent attributes, we employ gene expression programming (GEP) to find the most suitable mathematical model. One practical dataset and three real UCI datasets are presented in this paper to verify the effectiveness of the proposed method, and the results show that the proposed approach has better learning accuracy with regard to a back-propagation neural (BPN) network than that of the well-known mega-trend-diffusion (MTD) and the multi regression analysis (MRA) approaches.

A latent information function to extend domain attributes to improve the accuracy of small-data-set forecasting

In the current highly competitive manufacturing environment, it is important to have effective and efficient control of manufacturing systems to obtain and maintain competitive advantages. However, developing appropriate forecasting models for such systems can be challenging in their early stages, as the sample sizes are usually very small, and thus there is limited data available for analysis. The technique of virtual sample generation is one way to address this issue, but this method is usually not directly applied to time series data. This research thus develops a Latent Information function to analyze data features and extract hidden information, in order to learn from small data sets considering timing factors. The experimental results obtained using the Synthetic Control Chart Time Series and aluminum price datasets show that the proposed method can significantly improve forecasting accuracy, and thus is considered an appropriate procedure to forecast manufacturing outputs based on small samples.

Development of Method for Three-Point Data Estimation and SVR-QSAR Model to Screen Anti Cancer Leads

Present work deals with generation of virtual samples as mathematical modeling of empirical data on the basis of empirical data. The generated samples were used for development of QSAR model. The method deals with extrapolation of sample vector in such a manner that there is conservation of the empirical data distribution. The data distribution has been judged with statistical parameters. The method was implemented with anticancer activity of Gossypol acetic acid against BCL2 target for colorectal cancer. Considering the virtual samples only for model development, model training showed a regression coefficient for leave one out cross validation as 0.996 with 66 virtual samples, and a regression coefficient with external test set data (51 samples) as 0.993. External test set data which were never used in the virtual sample generation showed predicted regression coefficient value of >0.61. On the basis of QSAR model, nine compounds were suggested as anti-BCL2 active compounds. The suggested compounds were further validated by docking study with Gossypol acetic acid and 'Tetrahydroisoquinoline amide substituted phenyl pyrazole' cocrystallized with chimeric BCL2-XL (PDBID: 2W3L) protein.

Mechanism of charge transport in castor oil-based polyurethane/carbon black composite (PU/CB)

In order to identify and quantify the conduction mechanisms involved in the transport process, castor oil based polyurethane/carbon black composite (PU/CB) was introduced in a statistical model that reproduced the ac conductivity of composite PU/CB measurements by the application of a transfer-matrix technique. The simulation is based on virtual composite sample generation represented by a two-dimensional model of a resistor–capacitor network. The network represents a microstructure that contains both insulating (polyurethane) and conductive regions (carbon black). Drude's formula was applied in the conductance calculation between two neighboring sites in the CB particles or aggregates, and Miller–Abrahams formula in conductance between two neighboring sites of the polymeric matrix. The theoretical–experimental adjustment indicates that, at low frequencies (below 103Hz), the transport mechanism by charges hopping in the polymeric matrix is dominant. On the other hand, at high frequencies (above 105Hz) electron conduction in carbon black, given by Drude's formula, is dominant. The ac conductivity in the polymeric matrix is determined by the competition of two processes, one connected to hopping between the nearest sites with large energy difference or to hopping between the sites beyond the nearest neighbors with small energy difference.

A novel virtual sample generation method based on Gaussian distribution

Traditional machine learning algorithms are not with satisfying generalization ability on noisy, imbalanced, and small sample training set. In this work, a novel virtual sample generation (VSG) method based on Gaussian distribution is proposed. Firstly, the method determines the mean and the standard error of Gaussian distribution. Then, virtual samples can be generated by such Gaussian distribution. Finally, a new training set is constructed by adding the virtual samples to the original training set. This work has shown that training on the new training set is equivalent to a form of regularization regarding small sample problems, or cost-sensitive learning regarding imbalanced sample problems. Experiments show that given a suitable number of virtual sample replicates, the generalization ability of the classifiers on the new training sets can be better than that on the original training sets.

Using past manufacturing experience to assist building the yield forecast model for new manufacturing processes

Polarizers are one of the key parts of Thin-Film Transistor Liquid-Crystal Displays (TFT-LCD), and their production requires high material costs. How to reduce manufacturing costs is thus a key task in this highly competitive global market. The precise yield forecast model considering learning effects that is proposed in this work is believed to be an effective approach to reduce both the raw material input-cost and inventory cost of overproduction. Support vector regression (SVR) model is one of the commonly used approaches to forecast the yield trend. However, in the early manufacturing stages for a new product, an SVR model is usually sensitive and unstable because of the use of insufficient data. Faced with this problem, this research aims at enhancing the SVR model by using past manufacturing experience and virtual samples to estimate the yield trend model for pilot products. This paper proposes a novel Quadratic-Curve Diffusion (QCD) method, wherein we derive a quadratic yield function (QYF) of the new manufacturing process for each key manufacturing variable by utilizing past manufacturing experience; and then use the QYF to generate virtual samples to assist building the overall yield forecast model of the new manufacturing process. The results show that the proposed method is superior to the performance of other forecast and virtual sample generation models.

Utilization of virtual samples to facilitate cancer identification for DNA microarray data in the early stages of an investigation

DNA microarray datasets are generally small in size, high dimensional with many non-discriminative genes, and non-linear with outliers. Their size/dimension ratio suggests that DNA microarray datasets are identified as small-sample problems. Recently, researchers have developed various gene selection algorithms to discover genes that are most relevant to a specific disease, and thus to reduce computation. Most gene selection algorithms improve learning performance and efficiency, but still suffer from the limitation of insufficient training samples in the datasets. Moreover, in the early stage of diagnosing a new disease, very limited data can be obtained. Therefore, the derived diagnostic model is usually unreliable to identify the new disease. Consequently, the diagnostic performance cannot always be robust, even with the gene selection algorithms. To solve the problem of very limited training dataset with non-linear data or outliers, we propose the method GVSG (Group Virtual Sample Generation), which is a non-linear Virtual Sample Generation algorithm. This non-linear method detects the characteristics in the very limited data, forms discrete groups of each discriminative gene, and systematically generates virtual samples for each of these to accelerate and stabilize the modeling process. The results show that this method significantly improves the learning accuracy in the early stage of DNA microarray data.

Learning management knowledge for manufacturing systems in the early stages using time series data

Acquiring knowledge in manufacturing systems in the early stages always has a challenging task due to the lack of sufficient data. This makes it hard for the derived management model to reach a reliable and stable level. Li and Lin (2006) developed a useful method that can deal with the problem of knowledge acquisition based on a small data set. However, this method assumes all data are collected at the same time, since they treat the data set as a source (from one population) of a priori knowledge for learning. In fact, instead of being a random data set, these collected data can be time dependent, that is, they tend to be a sequence of observations, occurring at different times. The consideration of this dependence property in the data will benefit the knowledge acquisition in the early stages by expanding the learning model from an independent model to a dependent model. This research expanded the intervalized kernel density estimator (IKDE) presented in Li and Lin (2006) to a more general form to improve the learning model in the early stages. The general model, called GIKDE, joints the concepts of time series and stochastic processes in order to deal with both independent and dependent data sets. The Virtual Sample Generation process based on GIKDE was also developed to produce extra information for expediting the learning. Results obtained from the application of the model to a control charts data, using a back-propagation neural network as the learning tool, show that this unique approach is an effective method of knowledge acquisition for a manufacturing system in the early stages.

A non-linearly virtual sample generation technique using group discovery and parametric equations of hypersphere

In manufacturing systems, only a small training dataset can be obtained in the early stages. A small training dataset usually leads to low learning accuracy with regard to classification of machine learning, and the knowledge derived is often fragile, and this is called the small sample problem. This research mainly aims at overcoming this problem using a special nonlinear classification technique to generate virtual samples to enlarge the training dataset for learning improvement. This research proposes a new sample generation method, named non-linear virtual sample generation (NVSG), which combines a unique group discovery technique and a virtual sample generation method using parametric equations of hypersphere. By applying a back-propagation neural network (BPN) as the learning tool, the computational experiments obtained from the simulated dataset and the real dataset quoted from the Iris Plant Database show that the learning accuracy can be significantly improved using NVSG method for very small training datasets.

Using virtual sample generation to build up management knowledge in the early manufacturing stages

Since only few examples can be obtained in the early stages in a manufacturing system and that fewer exemplars usually lead to a lower learning accuracy, this research uses intervalized kernel methods of Density Estimation to improve the small-data-set learning. Used techniques include the Intervalization Process to improve the kernel density estimation and virtual sample generation to produce extra information for expediting the learning. Results obtained from the provided example, using a back-propagation neural network as the learning tool, show that this unique approach is an effective method of scheduling knowledge creation for a system in the early stages.

Modeling of conductivity in composites with random resistor networks

Composite polymeric electrolytes (CPE) are well known systems in which an improvement in electrical conductivity of the polymer is achieved by the addition of a non-conductive additive (ceramic powders and immiscible polymers). A new class of conductivity models based on the application of the multi-value random resistor networks to the studies of CPE was developed by the authors. The first stage of the simulation is based on the virtual composite sample generation and its transformation into a form of the three-dimensional resistor network. The sample generation can take into account variable material parameters such as average grain diameter, shell thickness as well as their statistical distribution and different dependencies of local shell conductivity on the distance from the grain surface. The second stage of the algorithm is the use of the modified iteration procedure based on Kirkpatrick approach leading to the direct current resistivity of the virtual sample. The presented results were obtained for the matrix size 100 unit × 100 unit × 100 unit (with assumption 1 unit = 1 μm). The dependency of conductivity on the grain filler concentration was studied for varying grain sizes, changing shell thickness as well as for different statistical distributions of these parameters. The obtained results of simulation are in general agreement with the experimental data.

Random resistor network application to modelling of d.c. ionic conductivity in composite polymeric electrolytes

A new class of conductivity models based on the application of the random resistor network approach is developed to study composite polymeric electrolytes. The simulation is based on virtual composite sample generation and its transformation into the form of a three-dimensional resistor network. Average grain diameter and its statistical distribution, shell thickness and different dependencies of local shell conductivity on the distance from the grain surface can act as variables. The use of the modified iteration procedure based on the Kirkpatrick approach leads to direct current resistivity value. The results were obtained for the matrix size 100 × 100 × 100 units (with assumption 1 unit = 1 μm). The dependency of conductivity on the grain filler concentration was studied for varying grain sizes, changing shell thickness and for different statistical distributions of these parameters. The obtained results of simulation are in general agreement with the experimental data.

Virtual sample generation using a population of networks

Multilayer perceptrons have been shown to approximate any continuous functions with a desired precision. With insufficient training samples, however, the network can not learn the function properly and popular model selection methods such as cross validation can not be used. We propose a scheme to generate virtual samples using a population of networks. They are applied to regression problems and are shown to improve generalization and to solve the model selection problem at the same time.