Fast Training Algorithm Research Articles

AN EFFICIENT FUZZY NEURAL NETWORK TRAINING MODEL FOR SUPERVISED PATTERN CLASSIFICATION SYSTEM

Among the existing NN architectures, Multilayer Feedforward Neural Network (MFNN) with single hidden layer architecture has been scrutinized thoroughly as best for solving nonlinear classification problem. The training time is consumed more for very huge training datasets in the MFNN training phase. In order to reduce the training time, a simple and fast training algorithm called Exponential Adaptive Skipping Training (EAST) Algorithm was presented that improves the training speed by significantly reducing the total number of training input samples consumed by MFNN for training at every single epoch. Although the training performance of EAST achieves faster, it still lacks in the accuracy rate due to high skipping factor. In order to improve the accuracy rate of the training algorithm, Hybrid system has been suggested in which the neural network is trained with the fuzzified data. In this paper, a z-Score Fuzzy Exponential Adaptive Skipping Training (z-FEAST) algorithm is proposed which is based on the fuzzification of EAST. The evaluation of the proposed z-FEAST algorithm is demonstrated effectively using the benchmark datasets - Iris, Waveform, Heart Disease and Breast Cancer for different learning rate. Simulation study proved that z-FEAST training algorithm improves the accuracy rate.

Deep Learning in Agent-Based Models: A Prospectus

A very timely issue for economic agent-based models (ABMs) is their empirical estimation. This paper describes a line of research that could resolve the issue by using machine learning techniques, using multi-layer artificial neural networks (ANNs), or so called Deep Nets. The seminal contribution by Hinton et al. (2006) introduced a fast and efficient training algorithm called Deep Learning, and there have been major breakthroughs in machine learning ever since. Economics has not yet benefited from these developments, and therefore we believe that now is the right time to apply Deep Learning and multi-layered neural networks to agent-based models in economics.

A Z-Score Fuzzy Exponential Adaptive Skipping Training (Z-Feast) Algorithm for Efficient Pattern Classification

Among the existing NN architectures, Multilayer Feedforward Neural Network (MFNN) with single hidden layer architecture has been scrutinized thoroughly as best for solving nonlinear classification problem. The training time is consumed more for very huge training datasets in the MFNN training phase. In order to reduce the training time, a simple and fast training algorithm called Exponential Adaptive Skipping Training (EAST) Algorithm was presented that improves the training speed by significantly reducing the total number of training input samples consumed by MFNN for training at every single epoch. Although the training performance of EAST achieves faster, it still lacks in the accuracy rate due to high skipping factor. In order to improve the accuracy rate of the training algorithm, Hybrid system has been suggested in which the neural network is trained with the fuzzified data. In this paper, a z-Score Fuzzy Exponential Adaptive Skipping Training (z-FEAST) algorithm is proposed which is based on the fuzzification of EAST. The evaluation of the proposed z-FEAST algorithm is demonstrated effectively using the benchmark datasets - Iris, Waveform, Heart Disease and Breast Cancer for different learning rate. Simulation study proved that z-FEAST training algorithm improves the accuracy rate.

A fast training algorithm for extreme learning machine based on matrix decomposition

Extreme Learning Machine (ELM), a competitive machine learning technique for single-hidden-layer feedforward neural networks (SLFNNs), has proven to be efficient and effective algorithm for regression and classification problems. However, traditional ELM involves a large number of hidden nodes for complex real world regression and classification problems which increasing the computation burden. In this paper, a decomposition based fast ELM (DFELM) algorithm is proposed to effectively reduce the computational burden for large number of hidden nodes condition. Compared with ELM algorithm, DFELM algorithm has faster training time with a large number of hidden nodes maintaining the same accuracy performance. Experiment on three regression problems, six classification problems and a complex blast furnace modeling problem are carried out to verify the performance of DFELM algorithm. Moreover, the decomposition method can be extended to other modified ELM algorithms to further reduce the training time.

Product Identification Using Image Processing and Radial Basis Function Neural Networks

This paper presents a product identification using image processing and radial basis function neural networks. The system identified a specific product based on the shape of the product. An image processing had been applied to the acquired image and the product was recognized using the Radial Basis Function Neural Network (RBFNN). The RBF Neural Networks offer several advantages compared to other neural network architecture such as they can be trained using a fast two-stage training algorithm and the network possesses the property of best approximation. The output of the network can be optimized by setting suitable values of the center and the spread of RBF. In this paper, fixed spread value was used for every cluster. The system can detect all the four products with 100% successful rate using ±0.2 tolerance.

Soft-sensing in complex chemical process based on a sample clustering extreme learning machine model

In actual chemical processes, the fact that some essential variables cannot be directly measured makes the production quality out-of-control and even results in large economic losses. In this study, a novel sample clustering extreme learning machine (SC-ELM) model is developed to achieve timely and accurate measurement. SC-ELM is a fast training algorithm with an excellent generalization performance, and the combined sample clustering approach solves the non-optimal input weights of ELM. The network structure is designed by a fast leave-one-out cross-validation (FLOO-CV) method. Meanwhile, the validity of SC-ELM model is firstly tested by two classical regression datasets. With the comparison of other ELM models, SC-ELM is proved to be an effective model in both modeling accuracy and network structure. Then, SC-ELM is applied in measuring the quality index of a high-density polyethylene (HDPE) process running in a chemical plant, and the experiment results demonstrate that SC-ELM model can achieve quality estimation with higher measuring accuracy and less training time.

T2FELA: type-2 fuzzy extreme learning algorithm for fast training of interval type-2 TSK fuzzy logic system.

A challenge in modeling type-2 fuzzy logic systems is the development of efficient learning algorithms to cope with the ever increasing size of real-world data sets. In this paper, the extreme learning strategy is introduced to develop a fast training algorithm for interval type-2 Takagi-Sugeno-Kang fuzzy logic systems. The proposed algorithm, called type-2 fuzzy extreme learning algorithm (T2FELA), has two distinctive characteristics. First, the parameters of the antecedents are randomly generated and parameters of the consequents are obtained by a fast learning method according to the extreme learning mechanism. In addition, because the obtained parameters are optimal in the sense of minimizing the norm, the resulting fuzzy systems exhibit better generalization performance. The experimental results clearly demonstrate that the training speed of the proposed T2FELA algorithm is superior to that of the existing state-of-the-art algorithms. The proposed algorithm also shows competitive performance in generalization abilities.

Application of artificial neural network for adaptive speed control of PMSM drive with variable parameters

PurposeThe aim of the research was to find out a method of adaptive speed control robust against variation of selected parameters of system like moment of inertia, time constant of torque control loop or torque coefficient of the motor.Design/methodology/approachThe main goal of the research was achieved due to application of artificial neural network (ANN), which was trained on line on the base of speed control error. The good results were gained by elaboration of enough fast and precise training algorithm and proper ANN structure.FindingsThe work shows a structure of artificial neural network (ANN), applied as adaptive speed controller, and presents an algorithm of ANN training. Some versions of this algorithm were analysed and verified by simulation and experimental tests.Research limitations/implicationsThe research should be continued to determine a final version of training algorithm and its influence on controller properties.Practical implicationsThe elaborated adaptive controller can be easily used by applying microprocessor system available now on the market. The proposed control solution is robust against parameters variation as well as their imprecise identification. The controller has ability of self‐tuning which can have great practical advantage.Social implicationsSocial implications are difficult to determine.Originality/valueThe paper presents a new solution of adaptive speed controller, which means a new ANN structure and new training algorithm.

Fast Linear Adaptive Skipping Training Algorithm for Training Artificial Neural Network

Artificial neural network has been extensively consumed training model for solving pattern recognition tasks. However, training a very huge training data set using complex neural network necessitates excessively high training time. In this correspondence, a new fast Linear Adaptive Skipping Training (LAST) algorithm for training artificial neural network (ANN) is instituted. The core essence of this paper is to ameliorate the training speed of ANN by exhibiting only the input samples that do not categorize perfectly in the previous epoch which dynamically reducing the number of input samples exhibited to the network at every single epoch without affecting the network’s accuracy. Thus decreasing the size of the training set can reduce the training time, thereby ameliorating the training speed. This LAST algorithm also determines how many epochs the particular input sample has to skip depending upon the successful classification of that input sample. This LAST algorithm can be incorporated into any supervised training algorithms. Experimental result shows that the training speed attained by LAST algorithm is preferably higher than that of other conventional training algorithms.

On-line modeling via fuzzy support vector machines and neural networks

Unlike the other fuzzy neural networks, in this paper the fuzzy system and the neural network are separated, and they are corresponding to structure identification and parameter identification. The fuzzy model is generated automatically by on-line clustering method and fuzzy support vector machines. This fuzzy model is not updated, but its modeling error is compensated by a neural network. The neural network acts as the parameter identification model. The benefits of this new method are the structure is simple, and the physical meanings of each part of the model are clear. For the neural compensator, a new stable and fast training algorithm is proposed. Finally, this modeling method is successfully applied to model a magnetic tube recovery ratio in a metal company in China.

Learning Image Similarity from Flickr Groups Using Fast Kernel Machines

Measuring image similarity is a central topic in computer vision. In this paper, we propose to measure image similarity by learning from the online Flickr image groups. We do so by: Choosing 103 Flickr groups, building a one-versus-all multiclass classifier to classify test images into a group, taking the set of responses of the classifiers as features, calculating the distance between feature vectors to measure image similarity. Experimental results on the Corel dataset and the PASCAL VOC 2007 dataset show that our approach performs better on image matching, retrieval, and classification than using conventional visual features. To build our similarity measure, we need one-versus-all classifiers that are accurate and can be trained quickly on very large quantities of data. We adopt an SVM classifier with a histogram intersection kernel. We describe a novel fast training algorithm for this classifier: the Stochastic Intersection Kernel MAchine (SIKMA) training algorithm. This method can produce a kernel classifier that is more accurate than a linear classifier on tens of thousands of examples in minutes.

Paper currency number recognition based on binary tree and Adaboost algorithm

A fast weak classifier training algorithm and a fast caching strategy were used to accelerate Adaboost training.Integrated learning algorithm Adaboost can accurately construct two classifiers,so paper currency number recognition was formulated as a series of Adaboost two-class classification problems quickly and flexibly by using binary tree structure.The experimental results demonstrate that the fast Adaboost training algorithm can speed up the training and the paper currency number recognition system based on binary tree and Adaboost algorithm has good recognition rate and processing speed,and it has widely been used in currency counter,cash sorter and ATM.

Improved fast LBG training algorithm in Hadamard domain

To reduce computational complexity of the Linde-Buzo-Gray (LBG) algorithm, an improved fast training method in the Hadamard domain is proposed. This method uses an optimally-ordered Hadamard transform kernel based on the statistical energy distribution of transformed training vectors in k-dimensional space and a three-step elimination criterion to more efficiently reject impossible codevectors. Experimental results demonstrate the effectiveness of the proposed method in terms of arithmetical operations and search space.

Fuzzy Wavelet Neural Network Models for Prediction and Identification of Dynamical Systems

This paper presents fuzzy wavelet neural network (FWNN) models for prediction and identification of nonlinear dynamical systems. The proposed FWNN models are obtained from the traditional Takagi-Sugeno-Kang fuzzy system by replacing the THEN part of fuzzy rules with wavelet basis functions that have the ability to localize both in time and frequency domains. The first and last model use summation and multiplication of dilated and translated versions of single-dimensional wavelet basis functions, respectively, and in the second model, THEN parts of the rules consist of radial function of wavelets. Gaussian type of activation functions are used in IF part of the fuzzy rules. A fast gradient-based training algorithm, i.e., the Broyden-Fletcher-Goldfarb-Shanno method, is used to find the optimal values for unknown parameters of the FWNN models. Simulation examples are also given to compare the effectiveness of the models with the other known methods in the literature. According to simulation results, we see that the proposed FWNN models have impressive generalization ability.

Fast Adaboost Training Algorithm by Dynamic Weight Trimming

This paper presents a novel fast Adaboost training algorithm by dynamic weight trimming,which increases the training speed greatly when dealing with large datasets.At each iteration,the algorithm discards most of the samples with small weight and keeps only the samples with large weight to train the weak classifier.Then it checks the performance of the weak classifier on all the samples,if the weighted error is above 0.5,it will increase the number of training samples and retrain the weak classifier.During training,only a small portion of the samples are used to train the weak classifier,so the speed is increased greatly.

From Minimum Enclosing Ball to Fast Fuzzy Inference System Training on Large Datasets

While fuzzy inference systems (FISs) have been extensively studied in the past decades, the minimum enclosing ball (MEB) problem was recently introduced to develop fast and scalable methods in pattern classification and machine learning. In this paper, the relationship between these two apparently different data modeling techniques is explored. First, based on the reduced-set density estimator, a bridge between the MEB problem and the FIS is established. Then, an important finding that the Mamdani-Larsen FIS (ML-FIS) can be translated into a special kernelized MEB problem, i.e., a center-constrained MEB problem under some conditions, is revealed. Thus, fast kernelized MEB approximation algorithms can be adopted to construct ML-FIS in an efficient manner. Here, we propose the use of a core vector machine (CVM), which is a fast kernelized MEB approximation algorithm for support vector machine (SVM) training, to accomplish this task. The proposed fast ML-FIS training algorithm has the following merits: (1) the number of fuzzy rules can be automatically determined by the CVM training and (2) fast ML-FIS training on large datasets can be achieved as the upper bound on the time complexity of learning the parameters in ML-FIS is linear with the dataset size N and the upper bound on the corresponding space complexity is theoretically independent of N. Our experiments on simulated and real datasets confirm these advantages of the proposed training method, and demonstrate its superior robustness as well. This paper not only represents a very first study of the relationship between MEB and FIS, but it also points out the mutual transformation between kernel methods and FISs under the framework of the Gaussian mixture model and MEB.

An Improved Fast Training Algorithm for RBF Networks Using Symmetry-Based Fuzzy C-Means Clustering

In fuzzy C-means (FCM) clustering, each data point belongs to a cluster to a degree specified by a membership grade. FCM partitions a collection of vectors in c fuzzy groups and finds a cluster center in each group such that the dissimilarity measure is minimized. This paper presents a training algorithm for the radial basis function (RBF) network using symmetry-based Fuzzy C-means (SFCM) clustering method which is the modified version of FCM clustering method based on point symmetry distance measure. The training algorithm which uses SFCM clustering method to train the network has a number of advantages such as faster training time, more accurate predictions and reduced network architecture compared to the standard RBF networks. The proposed training algorithm has been implemented in the RBF networks created by the newrb function of MATLAB which uses gradient based iterative method as learning strategy, therefore the new network will undergo a hybrid learning process. The networks called Symmetry-based Fuzzy C-means Clustering–Radial Basis Function Network (SFCM/RBF) has been tested against the standard RBF network and the networks called standard Fuzzy C-means Clustering (FCM)-RBF network (FCM/RBF) in forecasting. The experimental models has been tested on three real world application problems, particularly in Air pollutant problem, Biochemical Oxygen Demand (BOD) problem, and Phytoplankton problem. Keywords: Fuzzy c-means clustering; SFCM; Radial basis function network; point symmetry distance; forecasting.

Provably Fast Training Algorithms for Support Vector Machines

Support Vector Machines are a family of algorithms for the analysis of data based on convex Quadratic Programming. We derive randomized algorithms for training SVMs, based on a variation of Random Sampling Techniques; these have been successfully used for similar problems. We formally prove an upper bound on the expected running time which is quasilinear with respect to the number of data points and polynomial with respect to the other parameters, i.e., the number of attributes and the inverse of a chosen soft margin parameter. [This is the combined journal version of the conference papers (Balcazar, J.L. et al. in Proceedings of 12th International Conference on Algorithmic Learning Theory (ALT’01), pp. 119–134, [2001]; Balcazar, J.L. et al. in Proceedings of First IEEE International Conference on Data Mining (ICDM’01), pp. 43–50, [2001]; and Balcazar, J.L. et al. in Proceedings of SIAM Workshop in Discrete Mathematics and Data Mining, pp. 19–29, [2002]).]

Parallel randomized sampling for support vector machine (SVM) and support vector regression (SVR)

A parallel randomized support vector machine (PRSVM) and a parallel randomized support vector regression (PRSVR) algorithm based on a randomized sampling technique are proposed in this paper. The proposed PRSVM and PRSVR have four major advantages over previous methods. (1) We prove that the proposed algorithms achieve an average convergence rate that is so far the fastest bounded convergence rate, among all SVM decomposition training algorithms to the best of our knowledge. The fast average convergence bound is achieved by a unique priority based sampling mechanism. (2) Unlike previous work (Provably fast training algorithm for support vector machines, 2001) the proposed algorithms work for general linear-nonseparable SVM and general non-linear SVR problems. This improvement is achieved by modeling new LP-type problems based on Karush–Kuhn–Tucker optimality conditions. (3) The proposed algorithms are the first parallel version of randomized sampling algorithms for SVM and SVR. Both the analytical convergence bound and the numerical results in a real application show that the proposed algorithm has good scalability. (4) We present demonstrations of the algorithms based on both synthetic data and data obtained from a real word application. Performance comparisons with SVMlight show that the proposed algorithms may be efficiently implemented.

Fast Constructive-Covering Algorithm for neural networks and its implement in classification

Almost all current training algorithms for neural networks are based on gradient descending technique, which causes long training time. In this paper, we propose a novel fast training algorithm called Fast Constructive-Covering Algorithm (FCCA) for neural network construction based on geometrical expansion. Parameters are updated according to the geometrical location of the training samples in the input space, and each sample in the training set is learned only once. By doing this, FCCA is able to avoid iterative computing and much faster than traditional training algorithms. Given an input sequence in an arbitrary order, FCCA learns “easy” samples first and “confusing” samples are easily learned after these “easy” samples. This sample reordering process is done on the fly based on geometrical concept. In addition, FCCA begins with an empty hidden layer, and adds new hidden neurons when necessary. This constructive learning avoids blind selection of neural network structure. The experimental work for classification problems illustrates the advantages of FCCA, especially in learning speed.