Single Layer Feedforward Network Research Articles

Face recognition based on extreme learning machine

Extreme learning machine (ELM) is an efficient learning algorithm for generalized single hidden layer feedforward networks (SLFNs), which performs well in both regression and classification applications. It has recently been shown that from the optimization point of view ELM and support vector machine (SVM) are equivalent but ELM has less stringent optimization constraints. Due to the mild optimization constraints ELM can be easy of implementation and usually obtains better generalization performance. In this paper we study the performance of the one-against-all (OAA) and one-against-one (OAO) ELM for classification in multi-label face recognition applications. The performance is verified through four benchmarking face image data sets.

Learning Associative Memories by Error Backpropagation

In this paper, a method for the design of Hopfield networks, bidirectional and multidirectional associative memories with asymmetric connections, is proposed. The given patterns can be assigned as locally asymptotically stable equilibria of the network by training a single-layer feedforward network. It is shown that the robustness in respect to acceptable noise in the input of the constructed networks is enhanced as the memory dimension increases and weakened as the number of the stored patterns grows. More important is that the remembered patterns are not necessarily of binary forms. Neural associative memories for storing gray-level images are constructed based on the proposed method. Numerical simulations show that the proposed method is efficient for the design of Hopfield-type recurrent neural networks.

Two-stage extreme learning machine for regression

Extreme learning machine (ELM) proposed by Huang et al. was developed for generalized single hidden layer feedforward networks (SLFNs) with a wide variety of hidden nodes. It proved to be very fast and effective especially for solving function approximation problems with a predetermined network structure. However, the method for determining the network structure of preliminary ELM may be tedious and may not lead to a parsimonious solution. In this paper, a systematic two-stage algorithm (named TS-ELM) is introduced to handle the problem. In the first stage, a forward recursive algorithm is applied to select the hidden nodes from the candidates randomly generated in each step and add them to the network until the stopping criterion achieves its minimum. The significance of each hidden node is then reviewed in the second stage and the insignificance ones are removed from the network, which drastically reduces the network complexity. The effectiveness of TS-ELM is verified by the empirical studies in this paper.

Constructive Approximation to Multivariate Function by Decay RBF Neural Network

It is well known that single hidden layer feedforward networks with radial basis function (RBF) kernels are universal approximators when all the parameters of the networks are obtained through all kinds of algorithms. However, as observed in most neural network implementations, tuning all the parameters of the network may cause learning complicated, poor generalization, overtraining and unstable. Unlike conventional neural network theories, this brief gives a constructive proof for the fact that a decay RBF neural network with n+1 hidden neurons can interpolate n+1 multivariate samples with zero error. Then we prove that the given decay RBFs can uniformly approximate any continuous multivariate functions with arbitrary precision without training. The faster convergence and better generalization performance than conventional RBF algorithm, BP algorithm, extreme learning machine and support vector machines are shown by means of two numerical experiments.

Combining Feature- and Correspondence-Based Methods for Visual Object Recognition

We present an object recognition system built on a combination of feature- and correspondence-based pattern recognizers. The feature-based part, called preselection network, is a single-layer feedforward network weighted with the amount of information contributed by each feature to the decision at hand. For processing arbitrary objects, we employ small, regular graphs whose nodes are attributed with Gabor amplitudes, termed parquet graphs. The preselection network can quickly rule out most irrelevant matches and leaves only the ambiguous cases, so-called model candidates, to be verified by a rudimentary version of elastic graph matching, a standard correspondence-based technique for face and object recognition. According to the model, graphs are constructed that describe the object in the input image well. We report the results of experiments on standard databases for object recognition. The method achieved high recognition rates on identity and pose. Unlike many other models, it can also cope with varying background, multiple objects, and partial occlusion.

Deterministic Neural Classification

This letter presents a minimum classification error learning formulation for a single-layer feedforward network (SLFN). By approximating the nonlinear counting step function using a quadratic function, the classification error rate is shown to be deterministically solvable. Essentially the derived solution is related to an existing weighted least-squares method with class-specific weights set according to the size of data set. By considering the class-specific weights as adjustable parameters, the learning formulation extends the classification robustness of the SLFN without sacrificing its intrinsic advantage of being a closed-form algorithm. While the method is applicable to other linear formulations, our empirical results indicate SLFN's effectiveness on classification generalization.

Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels

A framework to analyze inference performance in densely connected single-layer feed-forward networks is developed for situations where a given data set is composed of correlated patterns. The framework is based on the assumption that the left and right singular value bases of the given pattern matrix are generated independently and uniformly from Haar measures. This assumption makes it possible to characterize the objective system by a single function of two variables which is determined by the eigenvalue spectrum of the cross-correlation matrix of the pattern matrix. Links to existing methods for analysis of perceptron learning and Gaussian linear vector channels and an application to a simple but nontrivial problem are also shown.

Functional Link Artificial Neural Network for Classification Task in Data Mining

In solving classification task of data mining, the traditional algorithm such as multi-layer perceptron takes longer time to optimize the weight vectors. At the same time, the complexity of the network increases as the number of layers increases. In this study, we have used Functional Link Artificial Neural Networks (FLANN) for the task of classification. In contrast to multiple layer networks, FLANN architecture uses a single layer feed-forward network. Using the functionally expanded features FLANN overcomes the non-linearity nature of problems, which is commonly encountered in single layer networks. The features like simplicity of designing the architecture and low-computational complexity of the networks encourages us to use it in data mining task. An extensive simulation study is presented to demonstrate the effectiveness of the classifier.

Hermite interpolation by neural networks

In this paper we study Hermite interpolation by single-layer feedforward networks. We provide upper bound estimates for the minimum number of hidden units that can solve the Hermite interpolation problem exactly. We consider both the univariate and the multivariate cases.

Prediction of Cadmium Removal Using an Artificial Neural Network and a Neuro-Fuzzy Technique

The prediction of adsorption of cadmium by hematite using an adapted neural fuzzy model and a back propagation artificial neural network was compared. Adsorption was found to depend on the Cd concentration, agitation rate, temperature, pH, and the particle size of the hematite. The adaptive neuro-fuzzy inference system proved to be more efficient in predicting Cd adsorption than a single layered feed forward artificial neural network.

A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks

In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OS-ELM) and can learn data one-by-one or chunk-by-chunk (a block of data) with fixed or varying chunk size. The activation functions for additive nodes in OS-ELM can be any bounded nonconstant piecewise continuous functions and the activation functions for RBF nodes can be any integrable piecewise continuous functions. In OS-ELM, the parameters of hidden nodes (the input weights and biases of additive nodes or the centers and impact factors of RBF nodes) are randomly selected and the output weights are analytically determined based on the sequentially arriving data. The algorithm uses the ideas of ELM of Huang et al. developed for batch learning which has been shown to be extremely fast with generalization performance better than other batch training methods. Apart from selecting the number of hidden nodes, no other control parameters have to be manually chosen. Detailed performance comparison of OS-ELM is done with other popular sequential learning algorithms on benchmark problems drawn from the regression, classification and time series prediction areas. The results show that the OS-ELM is faster than the other sequential algorithms and produces better generalization performance.

Estimating the Number of Hidden Neurons in a Feedforward Network Using the Singular Value Decomposition

In this letter, we attempt to quantify the significance of increasing the number of neurons in the hidden layer of a feedforward neural network architecture using the singular value decomposition (SVD). Through this, we extend some well-known properties of the SVD in evaluating the generalizability of single hidden layer feedforward networks (SLFNs) with respect to the number of hidden layer neurons. The generalization capability of the SLFN is measured by the degree of linear independency of the patterns in hidden layer space, which can be indirectly quantified from the singular values obtained from the SVD, in a postlearning step. A pruning/growing technique based on these singular values is then used to estimate the necessary number of neurons in the hidden layer. More importantly, we describe in detail properties of the SVD in determining the structure of a neural network particularly with respect to the robustness of the selected model.

A Modified Backpropagation Training Algorithm for Feedforward Neural Networks*

In this paper, a new efficient learning procedure for training single hidden layer feedforward network is proposed. This procedure trains the output layer and the hidden layer separately. A new optimization criterion for the hidden layer is proposed. Existing methods to find fictitious teacher signal for the output of each hidden neuron, modified standard backpropagation algorithm and the new optimization criterion are combined to train the feedforward neural networks. The effectiveness of the proposed procedure is shown by the simulation results.

Fitting transducer characteristics to measured data using a virtual curve tracer

Abstract This paper presents an artificial neural network based approach of transducer modeling in a computer-based measurement system. Transducer’s static response characteristic is generally highly non-linear. However, if linear approximations were acceptable, for a given accuracy level, noise and measurement errors are always present. Therefore, curve-fitting procedures are required frequently to average these effects. With the facility of computation now available through digital computers and microprocessors, the problem of estimation of transducer’s input–output static response characteristics is being increasingly tackled using software techniques. However, for an inherent nonlinear transducer, a software solution depends upon the proper approach through mathematical modeling of the response curve. An artificial neural network based virtual curve tracer is proposed for modeling, testing and calibration of transducers. The implemented functions are described. Experimental results of the comparative study in modeling the static response characteristic of a strain-gauge pressure transducer using adaptive single-layer linear network and multilayer feed-forward back-propagation network are presented.

Training feedforward networks using simultaneous perturbation with dynamic tunneling

An efficient technique, namely simultaneous perturbation with dynamic tunneling for training single hidden layer feedforward network, is proposed. A sigmoidal hidden neuron is added to the single hidden layer neural network after training. Then the cascaded network is trained again using simultaneous perturbation. The dynamic tunneling technique is employed to detrap the local minima in training. The proposed technique is shown to give better convergence results for the selected problems, namely neuro-controller, encoder, adder, demultiplexer, XOR and L–T character recognition problem.

Estimation of conditional quantiles using artificial neural networks

In this paper, we give mean square convergence rates of conditional quantile estimators based on single hidden layer feed forward networks. Our results are formulated both for independent identicaly distributed (i.i.d.) random variables and for stationary mixing processes ( α-mixing and β-mixing). It turns out that the rates are the same as those for regression using neural networks. We use the same techniques as in Zhang (1998).

Neural Network Method for Solving Partial Differential Equations

A method is presented to solve partial differential equations (pde's) and its boundary and/or initial conditions by using neural networks. It uses the fact that multiple input, single output, single hidden layer feedforward networks with a linear output layer with no bias are capable of arbitrarily well approximating arbitrary functions and its derivatives, which is proven by a number of authors and well known in literature. Knowledge about the pde and its boundary and/or initial conditions is incorporated into the structures and the training sets of several neural networks. In this way we obtain networks of which some are specifically structured. To find the solution of the pde and its boundary and/or initial conditions we have to train all obtained networks simultaneously. Therefore we use an evolutionary algorithm to train the networks. We demonstrate the working of our method by applying it to two problems.

The predictability of security returns with simple technical trading rules

Technical traders base their analysis on the premise that the patterns in market prices are assumed to recur in the future, and thus, these patterns can be used for predictive purposes. This paper uses the daily Dow Jones Industrial Average Index from 1897 to 1988 to examine the linear and nonlinear predictability of stock market returns with simple technical trading rules. The nonlinear specification of returns are modelled by single layer feedforward networks. The results indicate strong evidence of nonlinear predictability in the stock market returns by using the past buy and sell signals of the moving average rules.

Convergence rates for single hidden layer feedforward networks

By allowing the training set to become arbitrarily large, appropriately trained and configured single hidden layer feedforward networks converge in probability to the smooth function that they were trained to estimate. A bound on the probabilistic rate of convergence of these network estimates is given. The convergence rate is calculated as a function of the sample size n. If the function being estimated has square integrable m th order partial derivatives then the L 2-norm estimation error approaches O p ( n − 1 2 ) for large m. Two steps are required for determining these bounds. A bound on the rate of convergence of approximations to an unknown smooth function by members of a special class of single hidden layer feedforward networks is determined. The class of networks considered can embed Fourier series. Using this fact and results on approximation properties of Fourier series yields a bound on L 2-norm approximation error. This bound is less than O( q − 1 2 ) for approximating a smooth function by networks with q hidden units. A modification of existing results for bounding estimation error provides a general theorem for calculating estimation error convergence rates. Combining this result with the bound on approximation rates yields the final convergence rates.

Control of dynamic systems using fuzzy logic and neural networks

The use of artificial neural network is proposed for high-speed processing of rules in fuzzy logic controller (FLC). the logic element of an FLC is replaced by a single hidden layer feedforward network. the input and output fuzzy subsets are expressed it of numerical patterns. the network is trained using the back-propagation algori to establish fuzzy associations between the input and output fuzzy subsets. the inference mechanism of the network is compared with that of compositional law of inference. In the proposed implementation of FLC, all the rules are processed in paralle. This implementation has potential for high-speed processing of rules if the network is realized in hardware. the use of neural networks in fuzzy logic self-organizing is also ivestigated. © 1993 John Wiley & Sons, Inc.