Hidden Layer Output Matrix Research Articles

Research on Prediction of News Public Opinion Guiding Power Based on Neural Network

The development of big data technology and the popularity of we-media platforms make the prediction of news public opinion more complicated, which means that the complexity and dynamic nature of news public opinion require higher accuracy of prediction. With the rapid popularization of we media technology, traditional single-model algorithm is difficult to effectively predict network public opinion under the current background. Therefore, this paper proposes an algorithm based IGA-RBF neural network to deal with the complicated news public opinion prediction. Firstly, the ARMA (autoregressive moving average model) prediction model is constructed and the BRF neural network is combined. Then IGA is introduced to optimize BRF neural network, and the column vector of output matrix of hidden layer is optimized globally. The algorithm uses k-means clustering to select parameters in RBF network. The experimental results demonstrate that the model algorithm makes up for the shortcomings of the single prediction algorithm, improves the accuracy of prediction, and has better prediction results of public opinion trends.

Reduce Training Error of Extreme Learning Machine by Selecting Appropriate Hidden Layer Output Matrix

Reduce Training Error of Extreme Learning Machine by Selecting Appropriate Hidden Layer Output Matrix

Extreme Learning Machine Based on Calculating the Output Weight of Partial Robust M-regression

In order to improve the nonlinear mapping capability and learning performance of extreme learning machine (ELM), a new learning algorithm called extreme learning machine based on calculating the output weight of partial robust M-regression is proposed. This algorithm introduces the partial robust M-regression into the extreme learning machine algorithm. Firstly, the hidden layer output matrix H is calculated by extreme learning machine. Secondly, the matrix H and vector Y are weighted by weighted strategy. Then, PRM algorithm is used to establish the regression model between the weighted matrix Hw and vector Yw, and calculate its regression coefficient, namely output weight of the ELM Algorithm. The proposed method predicts the Mackay’s robot arm regression and sediment concentration of the Yellow River Basin to verify the effectiveness of the method. The simulation results show that the proposed PRMELM algorithm is superior to the original extreme learning machine algorithm in prediction accuracy and generalization performance.

Stochastic configuration network ensembles with selective base models

Studies have demonstrated that stochastic configuration networks (SCNs) have good potential for rapid data modeling because of their sufficient adequate learning power, which is theoretically guaranteed. Empirical studies have verified that the learner models produced by SCNs can usually achieve favorable test performance in practice but more in-depth theoretical analysis of their generalization power would be useful for constructing SCN-based ensemble models with enhanced generalization capacities. In particular, given a collection of independently developed SCN-based learner models, it is useful to select certain base learners that can potentially obtain preferable test results rather than considering all of the base models together, before simply taking their average in order to build an effective ensemble model. In this study, we propose a novel framework for building SCN ensembles by exploring key factors that might potentially affect the generalization performance of the base model. Under a mild assumption, we provide a comprehensive theoretical framework for examining a learner model’s generalization error, as well as formulating a novel indicator that contains measurement information for the training errors, output weights, and a hidden layer output matrix, which can be used by our proposed algorithm to find a subset of appropriate base models from a pool of randomized learner models. A toy example of one-dimensional function approximation, a case study for developing a predictive model for forecasting student learning performance, and two large-scale data sets were used in our experiments. The experimental results indicate that our proposed method has some remarkable advantages for building ensemble models.

Two-Stage Game Strategy for Multiclass Imbalanced Data Online Prediction

For multiclass imbalanced data online prediction, how to design a self-adapted model is a challenging problem. To address this issue, a novel dynamic multi-classification algorithm which uses two-stage game strategy has been put forward. Different from typical imbalanced classification methods, the proposed approach provided a self-updating model quantificationally, which can match the changes of arriving sample chunk automatically. In data generation phase, two dynamic ELMs with game theory are utilized for generating the lifelike minority class to equilibrate the distribution of different samples. In model update phase, both the current prediction performance and the cost sensitivity are taken into consideration simultaneously. According to the suffer loss and the shifty imbalance ratio, the proposed method develops the relationship between new weight and individual model, and an aggregate model of game theory is adopted to calculate the combination weight. These strategies help the algorithm reduce fitting error of sequence fragments. Also, alterative hidden-layer output matrix can be calculated according to the current fragment, thus building the steady network architecture in the next chunk. Numerical experiments are conducted on eight multiclass UCI datasets. The results demonstrate that the proposed algorithm not only has better generalization performance, but also improves the predictive ability of ELM method for minority samples.

On robust randomized neural networks for regression: a comprehensive review and evaluation

Data from real-world regression problems are quite often contaminated with outliers. In order to efficiently handle such undesirable samples, robust parameter estimation methods have been incorporated into randomized neural network (RNN) models, usually replacing the ordinary least squares (OLS) method. Despite recent successful applications to outlier-contaminated scenarios, significant issues remain unaddressed in the design of reliable outlier-robust RNN models for regression tasks. For example, the number of hidden neurons impacts directly on the norm of the estimated output weights, since the OLS method will rely on an ill-conditioned hidden-layer output matrix. Another design concern involves the high sensitivity of RNNs to the randomization of the hidden layer weights, an issue that can be suitably handled, e.g., by intrinsic plasticity techniques. Bearing these concerns in mind, we describe several ideas introduced in previous works concerning the design of RNN models that are both robust to outliers and numerically stable. A comprehensive evaluation of their performances is carried out across several benchmarking regression datasets taking into account accuracy, weight norms, and training time as figures of merit.

Tool wear condition monitoring based on wavelet transform and improved extreme learning machine

In machining process, tool wear is an inevitable consequence which progresses rapidly leading to a catastrophic failure of the system and accidents. Moreover, machinery failure has become more costly and has undesirable consequences on the availability and the productivity. Consequently, developing a robust approach for monitoring tool wear condition is needed to get accurate product dimensions with high quality surface and reduced stopping time of machines. Prognostics and health management has become one of the most challenging aspects for monitoring the wear condition of cutting tools. This study focuses on the evaluation of the current health condition of cutting tools and the prediction of its remaining useful life. Indeed, the proposed method consists of the integration of complex continuous wavelet transform (CCWT) and improved extreme learning machine (IELM). In the proposed IELM, the hidden layer output matrix is given by inverting the Moore–Penrose generalized inverse. After the decomposition of the acoustic emission signals using CCWT, the nodes energy of coefficients have been taken as relevant features which are then used as inputs in IELM. The principal idea is that a non-linear regression in a feature space of high dimension is involved by the extreme learning machine to map the input data via a non-linear function for generating the degradation model. Then, the health indicator is obtained through the exploitation of the derived model which is in turn used to estimate the remaining useful life. The method was carried out on data of the real world collected during various cuts of a computer numerical controlled tool.

PREDIKSI HARGA EMAS MENGGUNAKAN FEED FORWARD NEURAL NETWORK DENGAN METODE EXTREME LEARNING MACHINE

The prediction of gold price aims to find out the gold price in the future on the basis of historical data on gold prices in the past, so it can be used as a consideration by gold investors to investing in gold. Prediction methods that do not require assumptions, one of which is Artificial Neural Networks. In this study, using Artificial Neural Networks, Feed Forward Neural Network with Extreme Learning Machine (ELM). ELM is a non-iterative algorithm so ELM has advantages in process speed. The input weight and bias for this method are determined randomly. After that, to find the final weight using the Moore-Penrose Generalized Inverse calculation on the hidden layer output matrix. The best model selection criteria uses the Mean Absolute Percentage Error (MAPE). This study shows that the results of the training and testing process from the model 1 input neuron and 7 hidden neurons are very good, because it produces MAPE training = 0.6752% and MAPE testing = 0.8065%. Also gives a very good prediction result because it has MAPE = 0.5499% Keywords: gold price, Extreme Learning Machine, MAPE

A Fast Algorithm for Multi-Class Learning from Label Proportions

Learning from label proportions (LLP) is a new kind of learning problem which has attracted wide interest in machine learning. Different from the well-known supervised learning, the training data of LLP is in the form of bags and only the proportion of each class in each bag is available. Actually, many modern applications can be successfully abstracted to this problem such as modeling voting behaviors and spam filtering. However, time-consuming training is still a challenge for LLP, which becomes a bottleneck especially when addressing large bags and bag sizes. In this paper, we propose a fast algorithm called multi-class learning from label proportions by extreme learning machine (LLP-ELM), which takes advantage of an extreme learning machine with fast learning speed to solve multi-class learning from label proportions. Firstly, we reshape the hidden layer output matrix and the training data target matrix of an extreme learning machine to adapt to the proportion information instead of the real labels. Secondly, a robust loss function with a regularization term is formulated and two efficient solutions are provided to different cases. Finally, various experiments demonstrate the significant speed-up of the proposed model with better accuracies on different datasets compared with several state-of-the-art methods.

Ensemble OS-ELM based on combination weight for data stream classification

For online classification, how to design a self-adapted model is a challenging task. To make the model easily adaptable for the fast-changing data stream, a novel ensemble OS-ELM has been put forward. Different from traditional ensemble methods, the proposed approach provided a new self-adapted weight update algorithm. In online learning stage, both the current prediction accuracy and history record are considered. Based on suffer loss and the norm of output layer vector, an aggregate model of game theory is adopted to calculate the combination weight. This strategy fully considers the differences of individual learners. It helps the ensemble method reduce the fitting error of sequence fragment. Also, alterative hidden-layer output matrix can be calculated according to the current fragment, thus building the steady network architecture in the next chunk. So interactive parameter optimization is avoided and the automatic model is suitable for online learning. Numerical experiments are conducted on eight different kinds of UCI datasets. The results demonstrate that the proposed algorithm not only has better generalisation performance but also provides faster learning procedure.

Prediction model of permeability index for blast furnace based on the improved multi-layer extreme learning machine and wavelet transform

The permeability index of the blast furnace is a significant symbol to measure the smooth operation of the blast furnace. This paper proposes a novel prediction model for permeability index of the blast furnace based on the multi-layer extreme learning machine (ML-ELM), the principal component analysis (PCA) method and wavelet transform (called as W-PCA-ML-ELM prediction model). This modified ML-ELM algorithm is based on the ML-ELM algorithm and the PCA method (named as PCA-ML-ELM). The PCA method is applied on the ML-ELM algorithm to improve the algebraic property of the last hidden layer output matrix which deteriorates its generalization performance due to the high multicollinearity. Because the production data of the blast furnace field contain noises, this paper applies the wavelet transform to remove the noise. Comparing with other prediction models which are based on the ML-ELM, the ELM, the BP and the SVM, simulation results illustrate that the better generalization performance and stability of the proposed W-PCA-ML-ELM prediction model.

Multivariate Chaotic Time Series Prediction Based on ELM–PLSR and Hybrid Variable Selection Algorithm

In this paper, a novel method (Hybrid–ELM–PLSR) is proposed based on hybrid variable selection algorithm and improved extreme learning machine (ELM) for multivariate chaotic time series prediction. The hybrid variable selection algorithm combines the advantages of filter and wrapper, effectively balancing the calculation speed and prediction accuracy. Moreover, for ELM, multicollinearity, which can result in ill-condition, is always existent among the hidden layer output matrix. And the optimal number of hidden nodes is also difficult to be determined. Therefore,in order to overcome these problems, an improved ELM (ELM–PLSR) is proposed based on partial least square regression (PLSR). It can effectively enhance the stability performance and prediction performance of ELM. Hybrid–ELM–PLSR can be divided into three stages. At first, filter is used to rearrange the input variables through the correlations with desired variables. Then wrapper is used to select the optimal variable subset through evaluating the prediction performance of different subsets. Finally, ELM–PLSR is used to build the prediction model. The simulation experiment results based on San Francisco river runoff dataset demonstrate that the proposed method is effective for multivariate chaotic time series. And the prediction accuracy and reliability are higher than other methods.

Analysis on fast training speed of extreme learning machine and replacement policy

Extreme learning machine is known for its fast learning speed while maintaining acceptable generalisation. Its learning process can be divided into two parts: (1) randomly assigns input weights and biases in hidden layer, and (2) analytically determines output weights by the use of Moore-Penrose generalised inverse. Through the analysis from theory and experiment aspects we point out that it is the random weights assignment rather than the analytical determination with generalised inverse that leads to its fast training speed. In fact, the calculation of generalised inverse of hidden layer output matrix based on singular value decomposition (SVD) has very low efficiency especially on large scale data, and even directly cannot work. Considering this high calculation complexity reduces the learning speed of ELM conjugate gradient is introduced as a replacement of Moore-Penrose generalised inverse and conjugate gradient based ELM (CG-ELM) is proposed. Numerical simulations show that, in most cases, CG-ELM achieved faster speed than ELM in the condition of maintaining similar generalisation. Even in the case that ELM cannot work because of the huge amount of data CG-ELM attains good performance, which illustrates that Moore-Penrose generalised inverse is not the contribution of fast learning speed of ELM from experiment view.

On-line sequential extreme learning machine based on recursive partial least squares

This paper proposes the online sequential extreme learning machine algorithm based on the recursive partial least-squares method (OS-ELM-RPLS). It is an improvement to the online sequential extreme learning machine based on recursive least-squares (OS-ELM-RLS) introduced in (Huang et al., 2005 [1]). Like in the batch extreme learning machine (ELM), in OS-ELM-RLS the input weights of a single-hidden layer feedforward neural network (SLFN) are randomly generated, however the output weights are obtained by a recursive least-squares (RLS) solution. However, due to multicollinearities in the columns of the hidden-layer output matrix caused by presence of redundant input variables or by the large number of hidden-layer neurons, the problem of estimation the output weights can become ill-conditioned. In order to circumvent or mitigate such ill-conditioning problem, it is proposed to replace the RLS method by the recursive partial least-squares (RPLS) method. OS-ELM-RPLS was applied and compared with three other methods over three real-world data sets. In all the experiments, the proposed method always exhibits the best prediction performance.

An Improved Model Based on BPNN for Class-D Power Amplifier

Abstract An improved behavioral model based on BP neural network and approximation theory is proposed to describe the nonlinearity of Class-D power amplifier in this paper. Firstly, the hidden layer neurons are activated by a group of Chebyshev orthogonal polynomial basis functions (So the BP neural network is also called as Chebyshev orthogonal basis neural network in this paper), and the algorithm based on pseudo-inverse is used, which could immediately determine the neural network weights. Then, the input vector and the hidden-layer output matrix of neural network have to be normalized to enhance the neural network convergence precision, and the optimized algorithm, based on the relationship between the average error function and the number of hidden layer neurons, is used to determine the required optimal number of neurons in hidden layer. Finally, simulation results show that the improved behavioral model based on Chebyshev orthogonal basis neural network has faster computing speed and higher accuracy than traditional model.

Robust extreme learning machine

The output weights computing of extreme learning machine (ELM) encounters two problems, the computational and outlier robustness problems. The computational problem occurs when the hidden layer output matrix is a not full column rank matrix or an ill-conditioned matrix because of randomly generated input weights and biases. An existing solution to this problem is Singular Value Decomposition (SVD) method. However, the training speed is still affected by the large complexity of SVD when computing the Moore–Penrose (MP) pseudo inverse. The outlier robustness problem may occur when the training data set contaminated with outliers then the accuracy rate of ELM is extremely affected. This paper proposes the Extended Complete Orthogonal Decomposition (ECOD) method to solve the computational problem in ELM weights computing via ECODLS algorithm. And the paper also proposes the other three algorithms, i.e. the iteratively reweighted least squares (IRWLS-ELM), ELM based on the multivariate least-trimmed squares (MLTS-ELM), and ELM based on the one-step reweighted MLTS (RMLTS-ELM) to solve the outlier robustness problem. However, they also encounter the computational problem. Therefore, the ECOD via ECODLS algorithm is also used successfully in the three proposed algorithms. The experiments of regression problems were conducted on both toy and real-world data sets. The outlier types are one-sided and two-sided outliers. Each experiment was randomly contaminated with outliers, of one type only, with 10%, 20%, 30%, 40%, and 50% of the total training data size. Meta-metrics evaluation was used to measure the outlier robustness of the proposed algorithms compared to the existing algorithms, i.e. the minimax probability machine regression (MPMR) and the ordinary ELM. The experimental results showed that ECOD can effectively replace SVD. The ECOD is robust to the not full column rank or the ill-conditional problem. The speed of the ELM training using ECOD is also faster than the ordinary training algorithm. Moreover, the meta-metrics measure showed that the proposed algorithms are less affected by the increasing number of outliers than the existing algorithms.

A New Learning Algorithm for Single hidden Layer Feedforward Neural Networks

high dimensional pattern recognition problems, the learning speed of gradient based training algorithms (back-propagation) is generally very slow. Local minimum, improper learning rate and over-fitting are some of the other issues. Extreme learning machine was proposed as a non-iterative learning algorithm for single-hidden layer feed forward neural network (SLFN) to overcome these issues. The input weight and biases are chosen randomly in ELM which makes the classification system of non- deterministic behavior. In this paper, a new learning algorithm is presented in which the input weights and the hidden layer biases of SLFN are assigned from basis vectors generated by training space. The output weights and biases are decided through simple generalized inverse operation on output matrix of hidden layer. This makes very fast learning speed and better generalization performance in comparison to conventional learning algorithm as well as ELM.

Multivariate nonlinear modelling of fluorescence data by neural network with hidden node pruning algorithm

A hidden node pruning algorithm (HNPA) has been proposed as a method of configuration optimization and training in multilayer feedforward network. By this approach, a network initially bearing excessive hidden nodes is trained to sufficient precision and is pruned to the optimal size. Upon pruning, significant hidden nodes are determined by singular value decomposition (SVD) of output matrix of hidden layer and are retained. Weights and biases are preset intentionally in the pruned system, then training continues. The method has been tested with simulated nonlinear data and then applied to the modelling of a nonlinear fluorescence data of a real multicomponent analytical system, and satisfactory quantitative results have been achieved.