Single Layer Feedforward Network Research Articles

A New Approach to Cross-Modal Retrieval

The semantic correlation matching algorithm, which attempts to integrate semantic information based on the unified space obtained by CCA learning, achieves an improvement in the effect of cross-modal retrieval. However, there is still a lot of room for the algorithm to make further progress in the generation of shared subspace. This paper proposes a semantic correlation matching algorithm based on the single-layer feed forward network. It generates features through neural networks to represent weight and bias. Meanwhile, it maximizes the correlation between different modalities by means of cross-coupling. The experimental results show that the algorithm proposed in this paper is superior to the compared method in terms of cross-modal retrieval task of image-text and has better cross-modal retrieval ability.

Leakage Current Prediction of Composite Insulator using Artificial Neural Network

The uninterrupted power supply is one of the main concerns for the power utility which is often adversely affected by the flashover of the outdoor insulators. The main cause of the flashover of outdoor insulator is the contaminated pollution on its surface which is more severe for the insulators located nearer to the seashore. At such sites, insulators have to be washed regularly to avoid flashover which is an expensive and time-consuming process. So it is required to optimize the washing schedule. When the contamination is severe on the surface of the insulator, it allows the flow of the leakage current (LC) which turns into the flashover. The LC is a good indication for predicting the flashover. LC measurement and instrumentation system in real tower insulator is complex and expensive. In this paper, a prediction method is developed which predicts the LC at the level of starting of the arc which may turns into a flashover. An artificial neural network based model is developed which predicts the leakage current for the different polluted condition and humidity level. An experimental setup is prepared and subsequently data is taken to acquire LC on different relative humidity and equivalent salt deposit density (ESDD). Subsequently, a neural network model is constructed with the experimental data to predict LC. A single layer feed forward network based a predictive model performs LC prediction with the average error of 5.46% from the real values which is acceptable in case of alarming situations.

Online Extreme Learning Machine-Based Channel Estimation and Equalization for OFDM Systems

Machine learning-based channel estimation and equalization methods may improve the robustness and bit error rate (BER) performance of communication systems. However, the implementation of these methods has been blocked by some limitations, mainly including channel model-based offline training and high-computational complexity for training deep neural network (DNN). To overcome those limitations, we propose an online fully complex extreme learning machine (C-ELM)-based channel estimation and equalization scheme with a single hidden layer feedforward network (SLFN) for orthogonal frequency-division multiplexing (OFDM) systems against fading channels and the nonlinear distortion resulting from an high-power amplifier (HPA). Computer simulations show that the proposed scheme can acquire the information of channels accurately and has the ability to resist nonlinear distortion and fading without pre-training and feedback link between receiver and transmitter. Furthermore, the robustness of the proposed scheme is well investigated by extensive simulations in various fading channels, and its excellent generalization ability is also discussed and compared with the DNN.

Cross-Domain Extreme Learning Machines for Domain Adaptation

Extreme learning machines (ELMs), as “generalized” single hidden layer feedforward networks, have been proved to be effective and efficient for classification and regression problems. Traditional ELMs assume that the training and testing data are drawn from the same distribution, which however is often violated in real-world applications. In this paper, we propose a unified cross-domain ELM (CDELM) framework to address domain adaptation problems, in which the distributions of training data ( source domain ) and testing data ( target domain ) are different but related. CDELM not only fully leverages labeled source data and unlabeled target data simultaneously to construct an adaptive target classifier but also maintains the computational efficiency of ELMs. Specifically, CDELM adapts the source classifier to target domain by matching the projected means of both domains, and explores the structure property of target domain by using manifold regularization to make the final classifier more adaptable to target data. Based on the framework, two algorithms CDELM-M and CDELM-C are proposed, which aim at minimizing the marginal and conditional distribution distance between source and target domains, respectively. Moreover, CDELM-C can further enhance the classification accuracy by multiple iterations. Comprehensive experimental studies on artificial datasets and public text and image datasets demonstrate that both CDELM-M and CDELM-C are competitive with several state-of-the-art domain adaptation learning methods in terms of the classification accuracy and efficiency.

Investigation of Thermal Performance of Cylindrical Heat Pipe Using Silver Nanofluid: Part-II: Prediction Using Artificial Neural Network

Heat pipes are one of the heat transfer devices adopted in various applications due to its high heat transfer capability. Recently, heat pipes are charged with different nanofluids to increases its performance. Thermal performance of heat pipe charged with nanofluids depends of various factors related to operating conditions, working fluids, heat pipe, etc. Thus, extensive experimentation work is required to recommend appropriate parameters for different industrial applications. In such cases, in order to reduce experiments and to propose the appropriate combination of parameters, experimentation results need to be modeled using fundamentals of simulation. In the Part I of the paper, extensive experiments have been carried out on cylindrical heat pipe with silver nanofluid as working medium. In the present paper, total of 96 artificial neural network models have been created and trained on the basis of the total 700 data sets presented in Part I of the paper. Out of 700 data sets, eight data sets have been randomly separated for post-training simulation of the model, while 692 data sets have been employed for formulating the ANN model using the ANN tool of MATLAB student version available at the college. The models have been prepared to predict the thermal performance of cylindrical heat pipe under study when one, two and three parameters have been considered as output neurons one by one and six parameters as input neurons. Out of six input parameters, heat load, average evaporator and average condenser temperature are related to operating condition, size and concentration of silver nanoparticles are related to working fluid, and inclination angle is related to position of heat pipe. Trained 96 ANN models have been simulated using eight data sets, and the best ANN model has been proposed on the basis of minimum value of error in prediction in terms of MSE, NMSE, MAE, MARD and MRE. The best ANN model identified for one, two and three output neurons is single-layer feed-forward backpropagation network with six neurons of hidden layer, cascade feed-forward backpropagation network having three neurons at hidden layer and single-layer feed-forward backpropagation network with 12 neurons of hidden layer, respectively. It is also concluded that for single output neuron, any one type of error in prediction can be considered to select the best ANN model. But for more number of output neurons, comparison of various types of error in prediction is essential to judge the best ANN model.

Stacked ensemble extreme learning machine coupled with Partial Least Squares-based weighting strategy for nonlinear multivariate calibration.

With its simple theory and strong implementation, extreme learning machine (ELM) becomes a competitive single hidden layer feed forward networks for nonlinear multivariate calibration in chemometrics. To improve the generalization and robustness of ELM further, stacked generalization is introduced into ELM to construct a modified ELM model called stacked ensemble ELM (SE-ELM). The SE-ELM is to create a set of sub-models by applying ELM repeatedly to different sub-regions of the spectra and then combine the predictions of those sub-models according to a weighting strategy. Three different weighting strategies are explored to implement the proposed SE-ELM, such as the Winner-takes-all (WTA) weighting strategy, the constraint non-negative least squares (CNNLS) weighing strategy and the partial least squares (PLS) weighting strategy. Furthermore, PLS is suggested to be selected as the optimal weighting method that can handle the multi-colinearity among the predictions yielded by all the sub-models. The experimental assessment of the three SE-ELM models with different weighting strategies is carried out on six real spectroscopic datasets and compared with ELM, back-propagation neural network (BPNN) and Radial basis function neural network (RBFNN), statistically tested by the Wilcoxon signed rank test. The obtained experimental results suggest that, in general, all the SE-ELM models are more robust and more accurate than traditional ELM. In particular, the proposed PLS-based weighting strategy is at least statistically not worse than, and frequently better than the other two weighting strategies, BPNN, and RBFNN.

An empirical evaluation of extreme learning machine: application to handwritten character recognition

Extreme learning machine (ELM), a randomized learning paradigm for single hidden layer feed-forward network, has gained significant attention for solving problems in diverse domains due to its faster learning ability. The output weights in ELM are determined by an analytic procedure, while the input weights and biases are randomly generated and fixed during the training phase. The learning performance of ELM is highly sensitive to many factors such as the number of nodes in the hidden layer, the initialization of input weight and the type of activation functions in the hidden layer. Although various works on ELM have been proposed in the last decade, the effect of the all these influencing factors on classification performance has not been fully investigated yet. In this paper, we test the performance of ELM with different configurations through an empirical evaluation on three standard handwritten character datasets, namely, MNIST, ISI-Kolkata Bangla numeral, ISI-Kolkata Odia numeral and a newly developed NIT-RKL Bangla numeral dataset. Finally, we derive some best ELM figurations which can serve as general guidelines to design ELM based classifiers.

Segment-level probabilistic sequence kernel and segment-level pyramid match kernel based extreme learning machine for classification of varying length patterns of speech

In this work, we address some issues in the classification of varying length patterns of speech represented as sets of continuous-valued feature vectors using kernel methods. Kernels designed for varying length patterns are called as dynamic kernels. We propose two dynamic kernels namely segment-level pyramid match kernel (SLPMK) and segment-level probabilistic sequence kernel (SLPSK) for classification of long duration speech, represented as varying length sets of feature vectors using extreme learning machine (ELM). SLPMK and SLPSK are designed by partitioning the speech signal into increasingly finer segments and matching the corresponding segments. SLPSK is built upon a set of Gaussian basis functions, where half of the basis functions contain class-specific information while the other half implicates the common characteristics of all the speech utterances of all classes. The computational complexity of SVM training algorithms is usually intensive, which is at least quadratic with respect to the number of training examples. It is difficult to deal with the immense amount of data using traditional SVMs. For reducing the training time of classifier we propose to use a simple algorithm namely ELM. ELM refers to a wider type of generalized single hidden layer feedforward networks (SLFNs) whose hidden layer need not be tuned. In our work, we proposed to explore kernel based ELM to exploit dynamic kernels. We study the performance of the ELM-based classifiers using the proposed SLPSK and SLPMK for speech emotion recognition and speaker identification tasks and compare with other kernels for varying length patterns. Experimental studies showed that proposed ELM-based approach offer a 10–12% of relative improvement over baseline approach, and a 3–9% relative improvement over ELMs/SVMs using other state-of-the-art dynamic kernels.

A sequential learning method with Kalman filter and extreme learning machine for regression and time series forecasting

In many regression and time series forecasting problems, the input data is not fully available at the beginning of the training phase. Conventional machine learning methods for batch data are not able to handle this problem. The sequential version of ELM, called Online Sequential Extreme Learning Machine (OS-ELM), addresses this problem through the least squares recursive solution for updating the network output weights. However, the implementation of OS-ELM and its extensions suffer from the problem of multicollinearity and its side effect on the variance of the weight estimates. This paper introduces a new method of sequential learning for handling the effects of multicollinearity. The proposed method, called Kalman Learning Machine (KLM), uses the Kalman filter to sequentially update the output weights of a Single Layer Feedforward Network (SLFN) based on OS-ELM. An extension of the proposed method, called Extended Kalman Learning Machine (EKLM), is presented in order to address the problem of nonlinear data. The proposed method was compared with some of the most recent and effective methods for handling the effects of multicollinearity in sequential learning problems. The experiments performed showed that the proposed method performs better than most state-of-the-art methods considering both the prediction error and training time.

Guaranteed Recovery of One-Hidden-Layer Neural Networks via Cross Entropy

We study model recovery for data classification, where the training labels are generated from a one-hidden-layer neural network with sigmoid activations, also known as a single-layer feedforward network, and the goal is to recover the weights of the neural network. We consider two network models, the fully-connected network (FCN) and the non-overlapping convolutional neural network (CNN). We prove that with Gaussian inputs, the empirical risk based on cross entropy exhibits strong convexity and smoothness uniformly in a local neighborhood of the ground truth, as soon as the sample complexity is sufficiently large. This implies that if initialized in this neighborhood, gradient descent converges linearly to a critical point that is provably close to the ground truth. Furthermore, we show such an initialization can be obtained via the tensor method. This establishes the global convergence guarantee for empirical risk minimization using cross entropy via gradient descent for learning one-hidden-layer neural networks, at the near-optimal sample and computational complexity with respect to the network input dimension without unrealistic assumptions such as requiring a fresh set of samples at each iteration.

EEG classification using sparse Bayesian extreme learning machine for brain–computer interface

Mu rhythm is a spontaneous neural response occurring during a motor imagery (MI) task and has been increasingly applied to the design of brain–computer interface (BCI). Accurate classification of MI is usually rather difficult to be achieved since mu rhythm is very weak and likely to be contaminated by other background noises. As an extension of the single layer feedforward network, extreme learning machine (ELM) has recently proven to be more efficient than support vector machine that is a benchmark for MI-related EEG classification. With probabilistic inference, this study introduces a sparse Bayesian ELM (SBELM)-based algorithm to improve the classification performance of MI. SBELM is able to automatically control the model complexity and exclude redundant hidden neurons by combining advantageous of both ELM and sparse Bayesian learning. The effectiveness of SBELM for MI-related EEG classification is validated on a public dataset from BCI Competition IV IIb in comparison with several other competing algorithms. Superior classification accuracy confirms that the proposed SBELM-based algorithm is a promising candidate for performance improvement of an MI BCI.

Point cloud occlusion recovery with shallow feedforward neural networks

3D scenes reconstructed from point clouds, acquired by either laser scanning or photogrammetry, are subject to data voids generated by occluding objects. Modeling from incomplete data is usually a manual process in which human interpretation plays an essential role. This paper presents a machine learning algorithm based on neural networks capable of recovering point cloud occlusions for surfaces that can be approximated with injective functions. Starting from the point clouds acquired around the occlusion, a set of single-layer feedforward networks with a variable number of neurons is trained and validated with a subset of the original cloud, which is preliminarily decimated using local curvature to reduce CPU cost. The averaged result of the best neural networks is evaluated on a spatial domain that contains the 2D projection of the void, obtaining a complete 3D point cloud for the occluded volume. Criteria for choosing the number of neurons and the activation function for hidden and output layers are illustrated and discussed. Results are presented for both simulated and real occlusions, describing the pros and cons of the proposed method.

An Empirical Study of the Extreme Learning Machine for Twitter Sentiment Analysis

Extreme Learning Machine (ELM) method is proposed for single hidden layer feed-forward networks (SLFNs). The ELM employs feed-forward neural network architecture and works with randomly determined input weights. In this aspect, ELM depends on principle that enables to determine weights and biases in the network. In the first phase of ELM that can be named as feature mapping, the usage of random values differs the ELM from other methods that employ a kernel function for feature mapping such as Support Vector Machines (SVM) and Deep Neural Networks. After the feature mapping, the main goal of the ELM is to learn weights between hidden and output layers by minimizing the error. The ELM has gained much more popularity recently; and can be utilized for classification, regression, and dimension reduction. In literature, Twitter sentiment analysis is generally considered as a classification task. Therefore, in this study, the basic ELM is utilized for Twitter sentiment analysis and compared with the SVM which is one of the most successful machine learning algorithms used for sentiment analysis. Experiments are conducted on two different Turkish datasets. Experimental results show that the performance of the two methods are slightly different, but SVM outperforms basic ELM.

Nonlinear Characterization of the MRE Isolator Using Binary-Coded Discrete CSO and ELM

Magnetorheological elastomer (MRE) isolator has been proved as a promising semi-active control device for structural vibration control. For its engineering application, developing an accurate and robust model is definitely necessary and also a challenging task. Most of the present models, belonging to parametric models, need to identify various model parameters and sometimes are not capable of perfectly capturing the unique characteristics of the device. In this work, a novel nonparametric model is proposed to characterize the inherent dynamics of the MRE isolator with the features of hysteresis and nonlinearity. Initially, dynamic tests are conducted to evaluate the performance of the isolator under various loading conditions, including harmonic, random, and seismic excitations. Then, on the basis of the captured experimental results, a hybrid learning method is designed to forecast the nonlinear responses of the device with known external inputs. In this method, a type of single hidden layer feed-forward network, called extreme learning machine (ELM), is developed to forecast the nonlinear responses (shear force) of the device with captured velocity, displacement, and current level. To obtain optimal performance of the developed model, an improved binary-coded discrete cat swarm optimization (BCDCSO) method is adopted to select optimal inputs and neuron number in the hidden layer for the network development. The performance of the proposed method is verified through the comparison between experimental results and model predictions. Due to the noise influence in the practical condition, the robustness of the proposed method is also validated via adding noise disturbance into the supplying currents. The results show that the proposed method outperforms the standard ELM in terms of characterization of the MRE isolator, even though the captured responses are polluted with external measurement noises.

Tree Crown Detection, Delineation and Counting in UAV Remote Sensed Images: A Neural Network Based Spectral–Spatial Method

UAVs are fast emerging as a remote sensing platform to complement satellite based remote sensing. Agriculture and ecology is one of the important applications of UAV remote sensing, also known as low altitude remote sensing (LARS). This work demonstrates the use and potential of LARS in agriculture, particularly small holder open field agriculture. Two UAVs are used for remote sensing. The first UAV is a fixed wing aircraft with a high spatial resolution visible spectrum also known as RGB camera as a payload. The second UAV is a quadrotor UAV with an RGB camera interfaced to an on-board single board computer as the payload. LARS was carried out to acquire aerial high spatial resolution RGB images of different farms. Spectral–spatial classification of high spatial resolution RGB images for detection, delineation and counting of tree crowns in the image is presented. Supervised classification is carried out using extreme learning machine (ELM), a single hidden layer feed forward network neural network classifier. ELM was modelled for RGB values as input feature vectors and binary (tree and non-tree pixels) output class. Due to similarities in spectral intensities, some of the non-tree pixels were classified as tree pixels and in order to remove them, spatial classification was performed on the image. Spatial classification was carried out using thresholded geometrical property filtering techniques. Threshold values chosen for carrying out spatial classification were analysed to obtain optimal values. Finally in the delineation and counting, the connected tree crowns were segmented using Watershed algorithm performed on the image after marking individual tree crowns using Distance Transform method. Five representative UAV images captured at different altitudes with different crowns of banana plant, mango trees and coconut trees were used to demonstrate the performance of the proposed method. The performance was compared with the traditional KMeans spectral–spatial method of clustering. Results and comparison of performance parameters of KMeans spectral–spatial and ELM spectral–spatial classification methods are presented. Results indicate that ELM performed better than KMeans.

Solar photovoltaic power forecasting using optimized modified extreme learning machine technique

Prediction of photovoltaic power is a significant research area using different forecasting techniques mitigating the effects of the uncertainty of the photovoltaic generation. Increasingly high penetration level of photovoltaic (PV) generation arises in smart grid and microgrid concept. Solar source is irregular in nature as a result PV power is intermittent and is highly dependent on irradiance, temperature level and other atmospheric parameters. Large scale photovoltaic generation and penetration to the conventional power system introduces the significant challenges to microgrid a smart grid energy management. It is very critical to do exact forecasting of solar power/irradiance in order to secure the economic operation of the microgrid and smart grid. In this paper an extreme learning machine (ELM) technique is used for PV power forecasting of a real time model whose location is given in the Table 1. Here the model is associated with the incremental conductance (IC) maximum power point tracking (MPPT) technique that is based on proportional integral (PI) controller which is simulated in MATLAB/SIMULINK software. To train single layer feed-forward network (SLFN), ELM algorithm is implemented whose weights are updated by different particle swarm optimization (PSO) techniques and their performance are compared with existing models like back propagation (BP) forecasting model.

User behavior prediction in social networks using weighted extreme learning machine with distribution optimization

With the increasing presence of online social networks (OSN), there is a growing interest in accurately predicting user behaviors based on the data collected from OSN. Unlike traditional data, data collected from OSN are often imbalanced, which could make the user behavior prediction based on such data challenging. In this article, a novel data-driven method is proposed using extreme learning machine (ELM). ELM uses a single layer feedforward network (SLFN), and it has been attracting strong attentions in the research community because of its fast training speed and good generalization performance. However, the traditional ELM is mainly designed for balanced data learning in which the overall distribution of data is not considered. Hence, the prediction accuracy of ELM may be worse than some traditional neural network (NN) learning algorithms for some practical tasks. To address this issue, we enhance ELM by considering the distribution of data through the use of L2 norm. Specifically, the weighted strategy is integrated into our proposed method by setting weight for each class based on its size. For social network imbalanced data, some experiments are conducted using our weighted ELM based on the overall distribution, named ODW-ELM, to deal with binary classification and multi-class classification tasks. The experimental results show that ODW-ELM can achieve better performance in balanced datasets and imbalanced datasets. And ODW-ELM can also achieve better performance for classification application in social network while classifying blogs according to their popularity.

Improving Extreme Learning Machine by Competitive Swarm Optimization and its application for medical diagnosis problems

Extreme Learning Machine (ELM) is swiftly gaining popularity as a way to train Single hidden Layer Feedforward Networks (SLFN) for its attractive properties. ELM is a fast learning network with remarkable generalization performance. Although ELM generally can outperform traditional gradient descent-based algorithms such as Backpropagation, its performance can be highly affected by the random selection of the input weights and hidden biases of SLFN. Moreover, ELM networks tend to have more hidden neurons due to this random selection. In this paper, we propose a new model that uses Competitive Swarm Optimizer (CSO) to optimize the values of the input weights and hidden neurons of ELM. Two versions of ELM are considered: the classical ELM and the regularized version. The goal of the model is to increase the generalization performance, stabilize the classifier, and to produce more compact networks by reducing the number of neurons in the hidden layer. The proposed model is experimented based on 15 medical classification problems. Experimental results demonstrate that the proposed model can achieve better generalization performance with smaller number of hidden neurons and with higher stability. In addition, it requires much less training time compared to other metaheuristic based ELMs.

Long-Term Battery Voltage, Power, and Surface Temperature Prediction Using a Model-Based Extreme Learning Machine

A battery’s state-of-power (SOP) refers to the maximum power that can be extracted from the battery within a short period of time (e.g., 10 s or 30 s). However, as its use in applications is growing, such as in automatic cars, the ability to predict a longer usage time is required. To be able to do this, two issues should be considered: (1) the influence of both the ambient temperature and the rise in temperature caused by Joule heat, and (2) the influence of changes in the state of charge (SOC). In response, we propose the use of a model-based extreme learning machine (Model-ELM, MELM) to predict the battery future voltage, power, and surface temperature for any given load current. The standard ELM is a kind of single-layer feedforward network (SLFN). We propose using a set of rough models to replace the active functions (such as logsig()) in the ELM for better generalization performance. The model parameters and initial SOC in these “rough models” are randomly selected within a given range, so little prior knowledge about the battery is required. Moreover, the identification of the complex nonlinear system can be transferred into a standard least squares problem, which is suitable for online applications. The proposed method was tested and compared with RLS (Recursive Least Square)-based methods at different ambient temperatures to verify its superiority. The temperature prediction accuracy is higher than ±1.5 °C, and the RMSE (Root Mean Square Error) of the power prediction is less than 0.25 W. It should be noted that the accuracy of the proposed method does not rely on the accuracy of the state estimation such as SOC, thereby improving its robustness.

An Incremental Optimal Weight Learning Machine of Single-Layer Neural Networks

An optimal weight learning machine with growth of hidden nodes and incremental learning (OWLM-GHNIL) is given by adding random hidden nodes to single hidden layer feedforward networks (SLFNs) one by one or group by group. During the growth of the networks, input weights and output weights are updated incrementally, which can implement conventional optimal weight learning machine (OWLM) efficiently. The simulation results and statistical tests also demonstrate that the OWLM-GHNIL has better generalization performance than other incremental type algorithms.