Training Data Points Research Articles

ENHANCED TIME-DEPENDENT EVOLUTIONARY FUZZY SUPPORT VECTOR MACHINES INFERENCE MODEL FOR CASH FLOW PREDICTION AND ESTIMATE AT COMPLETION

This study conducts a mechanism enhancing the time series data treatment of the time-dependent evolutionary fuzzy support vector machine inference model (EFSIMT). The enhanced model, EFSIMET, was developed particularly to treat construction management problems that contain time series data. EFSIMET is an artificial intelligent hybrid system in which fuzzy logic (FL) deal with vagueness and approximate reasoning; support vector machine (SVM) acts as supervise learning tool; and fast messy genetic algorithm (fmGA) works to optimize FL and SVMs parameters simultaneously. Moreover, to capture the time series data characteristics, the inference model develops fmGA-based searching mechanism to seek suitable weight values to weight the training data points. This random-based searching mechanism has capacity to address the complex and dynamic nature of time series data; thus, it could improve the model's performance significantly. Nowadays, construction managementis facing complex and difficult problems due to the increasing uncertainties during project implementation. Therefore, the second objective of this study is proposed for the application of EFSIMET to treat two typical problems in construction: forecasting cash flow and estimate at completion. Through performance's comparison with previous works, the effectiveness and reliability of EFSIMET are proven. Hence, this model may be used as an intelligent decision support tool to assist the decision-making process to solve the construction management's difficulties.

Impact of training and validation sample selection on classification accuracy and accuracy assessment when using reference polygons in object-based classification

Reference polygons are homogenous areas that aim to provide the best available assessment of ground condition that the user can identify. Delineation of such polygons provides a convenient and efficient approach for researchers to identify training and validation data for supervised classification. However, the spatial dependence of training and validation data should be taken into account when the two data sets are obtained from a common set of reference polygons. We investigate the effect on classification accuracy and the accuracy estimates derived from the validation data when training and validation data are obtained from four selection schemes. The four schemes are composed of two sampling designs (simple random and systematic) and two methods for splitting sample points between training and validation (validation points in separate polygons from training points and validation points and training points split within the same polygons). A supervised object-based classification of the study region was repeated 30 times for each selection scheme. The selection scheme did not impact classification accuracy, but estimates of overall (OA), user's (UA), and producer's (PA) accuracies produced from the validation data overestimated accuracy for the study region by about 10%. The degree of overestimation was slightly greater when the validation sample points were allowed to be in the same polygons as the training data points. These results suggest that accuracy estimates derived from splitting training and validation within a limited set of reference polygons should be regarded with suspicion. To be fully confident in the validity of the accuracy estimates, additional validation sample points selected from the region outside the reference polygons may be needed to augment the validation sample selected from the reference polygons.

ANFIS AND NEURAL NETWORKS SYSTEMS FOR MULTIPLICITY DISTRIBUTIONS IN PROTON-PROTON INTERACTIONS

This article presents two systems that have the ability to simulate and predict the proton-proton (P-P) interaction. They are an adaptive neurofuzzy inference system (ANFIS) and a neural networks (NNs) system. The P-P-based ANFIS and NNs models calculate the multiplicity distribution of charged particles at different high energies. Simulation results of training charged particles using the ANFIS and NNs as tested with training data points showed perfect fitting to the experimental data. Prediction capabilities of the ANFIS and NNs checked with data points not used in training also proved to perform well. The results amply demonstrate the feasibility of these techniques in extracting the collision features and prove their effectiveness. It is found that the ANFIS shows better performance and trained more quickly than the NNs system.

A Generalized Directional Laplacian Distribution : Estimation, Mixture Models and Audio Source Separation

Directional or Circular statistics are pertaining to the analysis and interpretation of directions or rotations. In this work, a novel probability distribution is proposed to model multidimensional sparse directional data. The Generalized Directional Laplacian Distribution (DLD) is a hybrid between the Laplacian distribution and the von Mises-Fisher distribution. The distribution's parameters are estimated using Maximum-Likelihood Estimation over a set of training data points. Mixtures of Directional Laplacian Distributions (MDLD) are also introduced in order to model multiple concentrations of sparse directional data. The author explores the application of the derived DLD mixture model to cluster sound sources that exist in an underdetermined instantaneous sound mixture. The proposed model can solve the general <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">${K\times L~(K&lt; L)}$</tex></formula> underdetermined instantaneous source separation problem, offering a fast and stable solution.

Semi-supervised feature extraction for EEG classification

Two semi-supervised feature extraction methods are proposed for electroencephalogram (EEG) classification. They aim to alleviate two important limitations in brain---computer interfaces (BCIs). One is on the requirement of small training sets owing to the need of short calibration sessions. The second is the time-varying property of signals, e.g., EEG signals recorded in the training and test sessions often exhibit different discriminant features. These limitations are common in current practical applications of BCI systems and often degrade the performance of traditional feature extraction algorithms. In this paper, we propose two strategies to obtain semi-supervised feature extractors by improving a previous feature extraction method extreme energy ratio (EER). The two methods are termed semi-supervised temporally smooth EER and semi-supervised importance weighted EER, respectively. The former constructs a regularization term on the preservation of the temporal manifold of test samples and adds this as a constraint to the learning of spatial filters. The latter defines two kinds of weights by exploiting the distribution information of test samples and assigns the weights to training data points and trials to improve the estimation of covariance matrices. Both of these two methods regularize the spatial filters to make them more robust and adaptive to the test sessions. Experimental results on data sets from nine subjects with comparisons to the previous EER demonstrate their better capability for classification.

Improved adaptive pruning algorithm for least squares support vector regression

As the solutions of the least squares support vector regression machine (LS-SVRM) are not sparse, it leads to slow prediction speed and limits its applications. The defects of the existing adaptive pruning algorithm for LS-SVRM are that the training speed is slow, and the generalization performance is not satisfactory, especially for large scale problems. Hence an improved algorithm is proposed. In order to accelerate the training speed, the pruned data point and fast leave-one-out error are employed to validate the temporary model obtained after decremental learning. The novel objective function in the termination condition which involves the whole constraints generated by all training data points and three pruning strategies are employed to improve the generalization performance. The effectiveness of the proposed algorithm is tested on six benchmark datasets. The sparse LS-SVRM model has a faster training speed and better generalization performance.

Comparison of support vector machine, neural network, and CART algorithms for the land-cover classification using limited training data points

Support vector machine (SVM) was applied for land-cover characterization using MODIS time-series data. Classification performance was examined with respect to training sample size, sample variability, and landscape homogeneity (purity). The results were compared to two conventional nonparametric image classification algorithms: multilayer perceptron neural networks (NN) and classification and regression trees (CART). For 2001 MODIS time-series data, SVM generated overall accuracies ranging from 77% to 80% for training sample sizes from 20 to 800 pixels per class, compared to 67–76% and 62–73% for NN and CART, respectively. These results indicated that SVM’s had superior generalization capability, particularly with respect to small training sample sizes. There was also less variability of SVM performance when classification trials were repeated using different training sets. Additionally, classification accuracies were directly related to sample homogeneity/heterogeneity. The overall accuracies for the SVM algorithm were 91% (Kappa=0.77) and 64% (Kappa=0.34) for homogeneous and heterogeneous pixels, respectively. The inclusion of heterogeneous pixels in the training sample did not increase overall accuracies. Also, the SVM performance was examined for the classification of multiple year MODIS time-series data at annual intervals. Finally, using only the SVM output values, a method was developed to directly classify pixel purity. Approximately 65% of pixels within the Albemarle–Pamlico Basin study area were labeled as “functionally homogeneous” with an overall classification accuracy of 91% (Kappa=0.79). The results indicated a high potential for regional scale operational land-cover characterization applications.

Modified Feed-Forward Neural Network Structures and Combined-Function-Derivative Approximations Incorporating Exchange Symmetry for Potential Energy Surface Fitting

The classical interchange (permutation) of atoms of similar identity does not have an effect on the overall potential energy. In this study, we present feed-forward neural network structures that provide permutation symmetry to the potential energy surfaces of molecules. The new feed-forward neural network structures are employed to fit the potential energy surfaces for two illustrative molecules, which are H(2)O and ClOOCl. Modifications are made to describe the symmetric interchange (permutation) of atoms of similar identity (or mathematically, the permutation of symmetric input parameters). The combined-function-derivative approximation algorithm (J. Chem. Phys. 2009, 130, 134101) is also implemented to fit the neural-network potential energy surfaces accurately. The combination of our symmetric neural networks and the function-derivative fitting effectively produces PES fits using fewer numbers of training data points. For H(2)O, only 282 configurations are employed as the training set; the testing root-mean-squared and mean-absolute energy errors are respectively reported as 0.0103 eV (0.236 kcal/mol) and 0.0078 eV (0.179 kcal/mol). In the ClOOCl case, 1693 configurations are required to construct the training set; the root-mean-squared and mean-absolute energy errors for the ClOOCl testing set are 0.0409 eV (0.943 kcal/mol) and 0.0269 eV (0.620 kcal/mol), respectively. Overall, we find good agreements between ab initio and NN prediction in term of energy and gradient errors, and conclude that the new feed-forward neural-network models advantageously describe the molecules with excellent accuracy.

Crop classification modelling using remote sensing and environmental data in the Greater Platte River Basin, USA

With an ever expanding population, potential climate variability and an increasing demand for agriculture-based alternative fuels, accurate agricultural land-cover classification for specific crops and their spatial distributions are becoming critical to researchers, policymakers, land managers and farmers. It is important to ensure the sustainability of these and other land uses and to quantify the net impacts that certain management practices have on the environment. Although other quality crop classification products are often available, temporal and spatial coverage gaps can create complications for certain regional or time-specific applications. Our goal was to develop a model capable of classifying major crops in the Greater Platte River Basin (GPRB) for the post-2000 era to supplement existing crop classification products. This study identifies annual spatial distributions and area totals of corn, soybeans, wheat and other crops across the GPRB from 2000 to 2009. We developed a regression tree classification model based on 2.5 million training data points derived from the National Agricultural Statistics Service (NASS) Cropland Data Layer (CDL) in relation to a variety of other relevant input environmental variables. The primary input variables included the weekly 250 m US Geological Survey Earth Observing System Moderate Resolution Imaging Spectroradiometer normalized differential vegetation index, average long-term growing season temperature, average long-term growing season precipitation and yearly start of growing season. An overall model accuracy rating of 78% was achieved for a test sample of roughly 215 000 independent points that were withheld from model training. Ten 250 m resolution annual crop classification maps were produced and evaluated for the GPRB region, one for each year from 2000 to 2009. In addition to the model accuracy assessment, our validation focused on spatial distribution and county-level crop area totals in comparison with the NASS CDL and county statistics from the US Department of Agriculture (USDA) Census of Agriculture. The results showed that our model produced crop classification maps that closely resembled the spatial distribution trends observed in the NASS CDL and exhibited a close linear agreement with county-by-county crop area totals from USDA census data (R 2 = 0.90).

Recurrent sparse support vector regression machines trained by active learning in the time-domain

A method for the sparse solution of recurrent support vector regression machines is presented. The proposed method achieves a high accuracy versus complexity and allows the user to adjust the complexity of the resulting model. The sparse representation is guaranteed by limiting the number of training data points for the support vector regression method. Each training data point is selected based on the accuracy of the fully recurrent model using the active learning principle applied to the successive time-domain data. The user can adjust the training time by selecting how often the hyper-parameters of the algorithm should be optimised. The advantages of the proposed method are illustrated on several examples, and the experiments clearly show that it is possible to reduce the number of support vectors and to significantly improve the accuracy versus complexity of recurrent support vector regression machines.

Sparse multikernel support vector regression machines trained by active learning

A method for the sparse multikernel support vector regression machines is presented. The proposed method achieves a high accuracy versus complexity ratio and allows the user to adjust the complexity of the resulting models. The sparse representation is guaranteed by limiting the number of training data points for the support vector regression method. Each training data point is selected based on its influence on the accuracy of the model using the active learning principle. A different kernel function is attributed to each training data point, yielding multikernel regressor. The advantages of the proposed method are illustrated on several examples and the experiments show the advantages of the proposed method.

Prediction of diesel fuel cold properties using artificial neural networks

In this paper, two neural networks, multilayer perceptron and networks with radial-basis function, were used to predict important cold properties of commercial diesel fuels, namely cloud point and cold filter plugging point. The developed models predict the named properties using cetane number, density, viscosity, contents of total aromatics, and distillation temperatures at 10, 50, and 90 vol. % recovery as input data. The training algorithms, number of hidden layer neurons, and number of training data points were optimized in order to obtain a model with optimal predictive ability. The results indicated better prediction of cloud and cold filter plugging points in the case of multilayer perceptron networks. The obtained absolute error mean for the optimal neural network models (0.58°C for the cloud point and 1.46°C for the cold filter plugging point) are within the range of repeatability of standard cold properties determination methods.

Ultrasonic Determination of Water Concentration in Ethanol Fuel Using Artificial Neural Networks

Detection of water concentration in ethanol is important for several industries because of ethanol's widespread use in a number of products, including alcoholic beverages, solvents, and transportation fuels. The miscibility of ethanol with water and other polar liquids makes ethanol susceptible to adulteration beyond the specifications. In this study, adulteration of ethanol was simulated by mixing anhydrous ethanol with water at known concentrations. Ultrasound speed and temperature measurements were used to estimate the water concentration in ethanol using two mathematical models. The first model was based on statistical curve fitting to the experimental data, and the second model was based on a feed-forward, back-propagation neural network algorithm. A total of 651 data sets containing ultrasound speed and temperature as inputs and water concentration in ethanol as output were used to train the neural network algorithm. Both models were validated by preparing ethanol and water mixtures at known concentrations. Validation experiments showed that water concentration in ethanol-water mixtures can be determined by using ultrasound speed and mixture temperature with a standard error of prediction of 8.6% with neural network model and 12.4% with an empirical model. Improving the accuracy and increasing the number of data points for neural network training would reduce the prediction error. The measurements of ultrasound speed and temperature do not involve any moving parts, and the method is rapid and non-invasive, which makes the ultrasound measurement system suitable for online monitoring of water concentration in ethanol.

The Maximum Vector-Angular Margin Classifier and its fast training on large datasets using a core vector machine

Although pattern classification has been extensively studied in the past decades, how to effectively solve the corresponding training on large datasets is a problem that still requires particular attention. Many kernelized classification methods, such as SVM and SVDD, can be formulated as the corresponding quadratic programming (QP) problems, but computing the associated kernel matrices requires O(n2)(or even up to O(n3)) computational complexity, where n is the size of the training patterns, which heavily limits the applicability of these methods for large datasets. In this paper, a new classification method called the maximum vector-angular margin classifier (MAMC) is first proposed based on the vector-angular margin to find an optimal vector c in the pattern feature space, and all the testing patterns can be classified in terms of the maximum vector-angular margin ρ, between the vector c and all the training data points. Accordingly, it is proved that the kernelized MAMC can be equivalently formulated as the kernelized Minimum Enclosing Ball (MEB), which leads to a distinctive merit of MAMC, i.e., it has the flexibility of controlling the sum of support vectors like v-SVC and may be extended to a maximum vector-angular margin core vector machine (MAMCVM) by connecting the core vector machine (CVM) method with MAMC such that the corresponding fast training on large datasets can be effectively achieved. Experimental results on artificial and real datasets are provided to validate the power of the proposed methods.

Spatiotemporal pattern validation of chlorophyll-a concentrations in Lake Okeechobee, Florida, using a comparative MODIS image mining approach

A comparative analysis was conducted using three types of data-mining models produced from Moderate Resolution Imaging Spectroradiometer (MODIS) Terra Surface Reflectance 1-day or 8-day composite images to estimate chlorophyll-a (chl-a) concentrations in Lake Okeechobee, Florida. To understand the pros and cons of these three models, a genetic programming (GP) model was compared to an artificial neural network (ANN) model and multiple linear regression (MLR) model with respect to two different data sets related to model formulation. The first data set included the MODIS Terra bands from 1 to 7; the second data set extended the first data set by adding environmental parameters such as Secchi disc depth (SDD), total suspended solids (TSS), wind speed, water level, rainfall and air temperature collected around the lake in 2003 and 2004. The GP algorithm, which has an advantage in machine learning allowing us to select the appropriate input parameters that significantly impact the prediction accuracy, outperformed the other two models based on four statistical indices. Specifically, the GP modelling outputs revealed interesting determinations of chl-a concentrations for MODIS bands 3, 5, 6 and 7, corresponding to wavelengths 459–479, 1230–1250, 1628–1652 and 2105–2155 nm, respectively. The number of training data points is limited; therefore, the inclusion of additional environmental variables cannot improve the prediction accuracy of the GP-derived chl-a concentrations.

An enhanced Support Vector Machine classification framework by using Euclidean distance function for text document categorization

This paper presents the implementation of a new text document classification framework that uses the Support Vector Machine (SVM) approach in the training phase and the Euclidean distance function in the classification phase, coined as Euclidean-SVM. The SVM constructs a classifier by generating a decision surface, namely the optimal separating hyper-plane, to partition different categories of data points in the vector space. The concept of the optimal separating hyper-plane can be generalized for the non-linearly separable cases by introducing kernel functions to map the data points from the input space into a high dimensional feature space so that they could be separated by a linear hyper-plane. This characteristic causes the implementation of different kernel functions to have a high impact on the classification accuracy of the SVM. Other than the kernel functions, the value of soft margin parameter, C is another critical component in determining the performance of the SVM classifier. Hence, one of the critical problems of the conventional SVM classification framework is the necessity of determining the appropriate kernel function and the appropriate value of parameter C for different datasets of varying characteristics, in order to guarantee high accuracy of the classifier. In this paper, we introduce a distance measurement technique, using the Euclidean distance function to replace the optimal separating hyper-plane as the classification decision making function in the SVM. In our approach, the support vectors for each category are identified from the training data points during training phase using the SVM. In the classification phase, when a new data point is mapped into the original vector space, the average distances between the new data point and the support vectors from different categories are measured using the Euclidean distance function. The classification decision is made based on the category of support vectors which has the lowest average distance with the new data point, and this makes the classification decision irrespective of the efficacy of hyper-plane formed by applying the particular kernel function and soft margin parameter. We tested our proposed framework using several text datasets. The experimental results show that this approach makes the accuracy of the Euclidean-SVM text classifier to have a low impact on the implementation of kernel functions and soft margin parameter C.

Variable Sparsity Kernel Learning

This paper presents novel algorithms and applications for a particular class of mixed-norm regularization based Multiple Kernel Learning (MKL) formulations. The formulations assume that the given kernels are grouped and employ l1 norm regularization for promoting sparsity within RKHS norms of each group and ls, s≥2 norm regularization for promoting non-sparse combinations across groups. Various sparsity levels in combining the kernels can be achieved by varying the grouping of kernels---hence we name the formulations as Variable Sparsity Kernel Learning (VSKL) formulations. While previous attempts have a non-convex formulation, here we present a convex formulation which admits efficient Mirror-Descent (MD) based solving techniques. The proposed MD based algorithm optimizes over product of simplices and has a computational complexity of O(m2ntot log nmax/e2) where m is no. training data points, nmax,ntot are the maximum no. kernels in any group, total no. kernels respectively and e is the error in approximating the objective. A detailed proof of convergence of the algorithm is also presented. Experimental results show that the VSKL formulations are well-suited for multi-modal learning tasks like object categorization. Results also show that the MD based algorithm outperforms state-of-the-art MKL solvers in terms of computational efficiency.

Density-induced margin support vector machines

This paper proposes a new classifier called density-induced margin support vector machines (DMSVMs). DMSVMs belong to a family of SVM-like classifiers. Thus, DMSVMs inherit good properties from support vector machines (SVMs), e.g., unique and global solution, and sparse representation for the decision function. For a given data set, DMSVMs require to extract relative density degrees for all training data points. These density degrees can be taken as relative margins of corresponding training data points. Moreover, we propose a method for estimating relative density degrees by using the K nearest neighbor method. We also show the upper bound on the leave-out-one error of DMSVMs for a binary classification problem and prove it. Promising results are obtained on toy as well as real-world data sets.

Adaptive Learning in a World of Projections

This article presents a general tool for convexly constrained parameter/function estimation both for classification and regression tasks, in a timeadaptive setting and in (infinite dimensional) reproducing kernel Hilbert spaces (RKHS). The thematical framework is that of the set theoretic estimation formulation and the classical projections onto convex sets (POCS) theory. However, in contrast to the classical POCS methodology, which assumes a finite number of convex sets, our method builds upon our recent extension of the theory, which considers an infinite number of convex sets. Such a context is necessary to cope with the adaptive setting rationale, where data arrive sequentially. This article's goal is to review the advances that have taken place in this area over the years and present them, in simple geometric arguments, as an integral part and natural evolution of the classical POCS methodology. The structure of the resulting algorithms is such that it allows extension to general RKHS. In this perspective, two very powerful techniques, convex optimization and (implicit) mapping to RKHS, are combined, which provide a framework for a unifying treatment of linear and nonlinear modeling of both classification and regression tasks. Typical signal processing problems, such as filtering, smoothing, equalization, and beamforming, fall under this common umbrella. The methodology allows for the incorporation of a set of convex constraints, which encode a priori information. Convexity, rather than differentiability, is the only prerequisite for adopting error measures that quantify the model's fit against a set of training data points. Moreover, the complexity per iteration step remains linear with respect to the number of unknown parameters. The potential of the theory is demonstrated via numerical simulations for two typical problems; adaptive equalization and adaptive robust beamforming.

CORRELATION AND PREDICTION OF VLE OF WATER + CONGENER MIXTURES FOUND IN ALCOHOLIC BEVERAGES USING AN ARTIFICIAL NEURAL NETWORK

Artificial neural networks have been used for the correlation and prediction of vapor-liquid equilibrium data of binary water mixtures found in alcoholic beverage production. The main interest of the study is the accurate modeling of the bubble pressure and concentration of congeners in the vapor phase (substances different from ethanol and water), considered to be an important enological parameter in the alcoholic industry. Nine binary water + congener mixtures were considered for analysis. Vapor-liquid equilibrium data of these systems were taken from the literature (333 data points for training and 111 data points for testing), the artificial neural network results were compared with available literature data, and the accuracy of the modeling is discussed. The study shows that the neural network model is a good alternative method for the estimation of phase equilibrium properties for this type of mixture. Supplemental materials are available for this article. Go to the publisher's online edition of Chemical Engineering Communications to view the free supplemental file. http://www.informaworld.com/smpp/title~db=all~content=t713454788