Single-hidden Layer Feedforward Neural Networks Research Articles

Efficient Ensemble Broad Learning System Based on Dropout and DropConnect

Broad Learning System is an emerged efficient algorithm for training single hidden layer feedforward neural network (SLFN) with fast speed and good generalization ability. However, it is very difficult to determine the appropriate broad learning system structure and broad learning system may perform overfitting due to the dependence between nodes in processing fully connected network. In order to deal with these problems, efficient ensemble broad learning system based on Dropout and DropConnect are proposed in this paper. The proposed Dropout Ensemble Broad Learning System randomly discards hidden nodes to improve diversity between individuals and reduce the synergy between nodes to improve prediction stability. The DropConnect Ensemble Broad Learning System randomly drops connect weights to generate more complementary models by adding input attribute disturbance. The experimental results and statistical analysis on UCI data sets confirm that the proposed method can solve the problem of model overfitting caused by the strong dependence between the nodes of ensemble broad learning system and also show that the proposed approaches outperform the original BLS on prediction stability and classification accuracy.

Prediksi Harga Kelapa Sawit Menggunakan Metode Extreme Learning Machine

Palm oil is one of the keys to the Indonesian economy and the main commodity for attracting foreign investment. The palm oil and palm kernel industry generates most of the foreign currency from palm oil. The price of palm oil often goes up and down every month resulting in instability in the income received by people who own oil palm plantations. The aim of predicting palm oil prices is to carry out appropriate planning or steps for palm oil business actors. One way to overcome this problem is to make predictions. One method that can make predictions is the Extreme Learning Machine (ELM). ELM is an artificial neural network method used to predict palm oil prices. The ELM method is a feedforward method with a single hidden layer which is better known as a single hidden layer feedforward neural network (SLFNs). In this research, the best implementation was 5 inputs with 20 neurons in the hidden layer with output in the form of palm oil price predictions. Based on the tests carried out, the research produced the smallest error rate of 0.0027111424247658633 using 20 neurons in the hidden layer so that the latest data prediction test results for 5 price rotations in September rotation 1 were 1400.314191, September rotation 2 were 1846.798921, September rotation 3 amounted to 1505.430419, September rotation 4 amounted to 2301.853412, September rotation 5 amounted to 2645.082489 in palm oil price predictions.

Extreme Learning Machine Classification Method based on Twin Strategy Salp Swarm Algorithm

Extreme Learning Machines (ELM) are a type of Single Hidden Layer Feedforward Neural Network (SLFN). In recent years, they have attracted attention for their powerful approximation ability and fast learning speed. Compared with traditional neural network algorithms, ELM have the advantages of simple structure, fast learning speed, and good generalization performance. However, since the input weights and biases of ELM are randomly generated, there may be some suboptimal or unnecessary input weights and biases. In addition, ELM may require more hidden nodes, which may slow down its response to unknown test data. To address these issues, a twin strategy Salp Swarm Algorithm (TSSA) is proposed to increase the population diversity to improve the convergence speed of the algorithm through chaotic initialization, while shock inertia weights and learning paradigms are introduced into the follower position updating to enhance the stochasticity of the particles. Classification tests are conducted on six datasets using the proposed TSSA, and the experimental results show that the proposed TSSA-ELM has higher accuracy and better generalization performance than some existing ELM variants.

CTFNet: Long-Sequence Time-Series Forecasting Based on Convolution and Time-Frequency Analysis.

Although current time-series forecasting methods have significantly improved the state-of-the-art (SOTA) results for long-sequence time-series forecasting (LSTF), they still have difficulty in capturing and extracting the features and dependencies of long-term sequences and suffer from information utilization bottlenecks and high-computational complexity. To address these issues, a lightweight single-hidden layer feedforward neural network (SLFN) combining convolution mapping and time-frequency decomposition called CTFNet is proposed with three distinctive characteristics. First, time-domain (TD) feature mining-in this article, a method for extracting the long-term correlation of horizontal TD features based on matrix factorization is proposed, which can effectively capture the interdependence among different sample points of a long time series. Second, multitask frequency-domain (FD) feature mining-this can effectively extract different frequency feature information of time-series data from the FD and minimize the loss of data features. Integrating multiscale dilated convolutions, simultaneously focusing on both global and local context feature dependencies at the sequence level, and mining the long-term dependencies of the multiscale frequency information and the spatial dependencies among the different scale frequency information, break the bottleneck of data utilization, and ensure the integrity of feature extraction. Third, highly efficient-the CTFNet model has a short training time and fast inference speed. Our empirical studies with nine benchmark datasets show that compared with state-of-the-art methods, CTFNet can reduce prediction error by 64.7% and 53.7% for multivariate and univariate time series, respectively.

Mixture extreme learning machine algorithm for robust regression

The extreme learning machine (ELM) is a well-known approach for training single hidden layer feedforward neural networks (SLFNs) in machine learning. However, ELM is most effective when used for regression on datasets with simple Gaussian distributed error because it often employs a squared loss in its objective function. In contrast, real-world data is often collected from unpredictable and diverse contexts, which may contain complex noise that cannot be characterized by a single distribution. To address this challenge, we propose a robust mixture ELM algorithm, called Mixture-ELM, that enhances modeling capability and resilience to both Gaussian and non-Gaussian noise. The Mixture-ELM algorithm uses an adjusted objective function that blends Gaussian and Laplacian distributions to approximate any continuous distribution and match the noise. The Gaussian mixture accurately models the residual distribution, while the inclusion of the Laplacian distribution addresses the limitations of the Gaussian distribution in identifying outliers. We derive a solution to the novel objective function using the expectation maximization (EM) and iteratively reweighted least squares (IRLS) algorithms. We evaluate the effectiveness of the algorithm through numerical simulation and experiments on benchmark datasets, thereby demonstrating its superiority over other state-of-the-art machine learning methods in terms of robustness and generalization.

Identification Of Hijaiyah Letters Image Using Extreme Learning Machine Method

The development of digital image processing technology has many benefits, one of which is the identification of an object, such as the identification of the image of the hijaiyah letter. Hijaiyah letters are the letters of the arabic alphabet as the original language of the Qur'an. In essence, humans have the ability to recognize and distinguish hijaiyah letter patterns from one another, but this is not the case with computers, using digital images and machine learning, in this study an identification concept was built by recognizing the image of hijaiyah letters using one of the machine learning methods. namely the extreme learning machine (ELM) method. Extreme learning machine (ELM) is a feedforward neural network with one hidden layer or better known as single hidden layer feedforward neural networks (SLFNs). Therefore, the purpose of this study is that the computer can identify objects as well as human capabilities and see how accurate the results obtained in the ELM method are. The digital image identification process using the extreme learning machine (ELM) method is carried out in two stages, namely training and testing, where previously the preprocessing process was carried out first by changing the color of the RGB image to HSV and processing the color v, then segmentation was carried out with the aim of separating the objects (foreground) with the background, then to make it easier to recognize the pattern, a morphological process is carried out. From the simulation carried out on the test data, the results obtained an average accuracy of 90% with an error of 10%.

Integrating multivariate fuzzy neural networks into fuzzy inference system for enhanced decision making

In this paper, we introduce a novel family of operators for single-hidden layer feed-forward neural networks that operate on multivariate fuzzy-valued functions. These operators are based on the fractional mean value of the approximating function. The construction of these operators involves the use of sigmoidal functions, providing multiple choices for their implementation. We demonstrate the modeling capabilities of our operators employing a novel combined neural network system that incorporates the basic fuzzy inference mechanism and fuzzy neural networks. This system represents the connection strength among the output neurons using multivariate fuzzy-valued functions, enhancing decision-making processes. Additionally, we prove the convergence properties of these operators with respect to the Pompeiu-Hausdorff metric. By controlling the asymptotic decay of the sigmoidal function, we achieve accurate Jackson-type estimations using the modulus of continuity of fuzzy-valued functions.

Functional extreme learning machine.

Extreme learning machine (ELM) is a training algorithm for single hidden layer feedforward neural network (SLFN), which converges much faster than traditional methods and yields promising performance. However, the ELM also has some shortcomings, such as structure selection, overfitting and low generalization performance. This article a new functional neuron (FN) model is proposed, we takes functional neurons as the basic unit, and uses functional equation solving theory to guide the modeling process of FELM, a new functional extreme learning machine (FELM) model theory is proposed. The FELM implements learning by adjusting the coefficients of the basis function in neurons. At the same time, a simple, iterative-free and high-precision fast parameter learning algorithm is proposed. The standard data sets UCI and StatLib are selected for regression problems, and compared with the ELM, support vector machine (SVM) and other algorithms, the experimental results show that the FELM achieves better performance.

Using a Hybrid Neural Network and a Regularized Extreme Learning Machine for Human Activity Recognition with Smartphone and Smartwatch.

Mobile health (mHealth) utilizes mobile devices, mobile communication techniques, and the Internet of Things (IoT) to improve not only traditional telemedicine and monitoring and alerting systems, but also fitness and medical information awareness in daily life. In the last decade, human activity recognition (HAR) has been extensively studied because of the strong correlation between people's activities and their physical and mental health. HAR can also be used to care for elderly people in their daily lives. This study proposes an HAR system for classifying 18 types of physical activity using data from sensors embedded in smartphones and smartwatches. The recognition process consists of two parts: feature extraction and HAR. To extract features, a hybrid structure consisting of a convolutional neural network (CNN) and a bidirectional gated recurrent unit GRU (BiGRU) was used. For activity recognition, a single-hidden-layer feedforward neural network (SLFN) with a regularized extreme machine learning (RELM) algorithm was used. The experimental results show an average precision of 98.3%, recall of 98.4%, an F1-score of 98.4%, and accuracy of 98.3%, which results are superior to those of existing schemes.

A novel hybrid Extreme Learning Machine and Teaching–Learning-Based​ Optimization algorithm for skin cancer detection

Skin cancers, such as melanoma, can be difficult to spot in their early stages because they often resemble benign moles. Early detection of melanoma is crucial as it increases the chances of successful treatment and prevents cancer from spreading to other areas of the body. Machine learning algorithms and computer vision techniques are versatile techniques for detecting melanoma. However, current research has limitations, such as inaccurate detection and longer computation times. This paper proposes a novel hybrid Extreme Learning Machine (ELM) and Teaching–Learning-Based Optimization (TLBO) algorithm as a versatile technique for detecting melanoma. ELM is a single-hidden layer feed-forward neural network that can be trained quickly and accurately, while TLBO is an optimization algorithm used to fine-tune the network’s parameters for improved performance. Together, these techniques can classify skin lesions as benign or malignant images, potentially improving melanoma detection accuracy.

STWD-SFNN: Sequential three-way decisions with a single hidden layer feedforward neural network

The three-way decisions strategy has been employed to construct network topology in a single hidden layer feedforward neural network (SFNN). However, this model has a general performance, and does not consider the process costs, since it has fixed threshold parameters. Inspired by the sequential three-way decisions (STWD), this paper proposes STWD with an SFNN (STWD-SFNN) to enhance the performance of networks on structured datasets. STWD-SFNN adopts multi-granularity levels to dynamically learn the number of hidden layer nodes from coarse to fine, and set the sequential threshold parameters. Specifically, at the coarse granular level, STWD-SFNN handles easy-to-classify instances by applying strict threshold conditions, and with the increasing number of hidden layer nodes at the fine granular level, STWD-SFNN focuses more on disposing of the difficult-to-classify instances by applying loose threshold conditions, thereby realizing the classification of instances. Moreover, STWD-SFNN considers and reports the process cost produced from each granular level. The experimental results verify that STWD-SFNN has a more compact network on structured datasets than other SFNN models, and has better generalization performance than the competitive models. All models and datasets can be downloaded from https://github.com/wuc567/Machine-learning/tree/main/STWD-SFNN.

Small-signal Modeling of InP HBT based on PSO-ELM Neural Network

Extreme Learning Machine (ELM) is a new learning algorithm for single-hidden layer feedforward neural network, which has been widely used in lots of fields. However, it still has the insufficiency of randomly determining the hidden layer threshold and output weight, which leads to ill-conditioned output. In order to avoid the risk of decreasing prediction accuracy caused by this possibility, the ELM is optimized using particle swarm algorithm. A Particle Swarm Optimization (PSO) enhanced ELM algorithm is proposed to accurately model the small-signal properties of InP Heterojunction Bipolar Transistors (HBTs). PSOELM algorithm solves the problem of unstable prediction data caused by random determination of input weights in ELM. Comparing the modeling effects of the PSO-ELM model and the ELM model under different bias conditions for a 1 μm×15 μm InP HBT, it is proved that the PSO-ELM algorithm has better consistency with the measured data.

Empirical validation of ELM trained neural networks for financial modelling

The purpose of this work is to compare predictive performance of neural networks trained using the relatively novel technique of training single hidden layer feedforward neural networks (SFNN), called Extreme Learning Machine (ELM), with commonly used backpropagation-trained recurrent neural networks (RNN) as applied to the task of financial market prediction. Evaluated on a set of large capitalisation stocks on the Australian market, specifically the components of the ASX20, ELM-trained SFNNs showed superior performance over RNNs for individual stock price prediction. While this conclusion of efficacy holds generally, long short-term memory (LSTM) RNNs were found to outperform for a small subset of stocks. Subsequent analysis identified several areas of performance deviations which we highlight as potentially fruitful areas for further research and performance improvement.

Pfizer Stock Price Prediction Based on Extreme Learning Machine

Nowadays, an Extreme learning machine (ELM) is known to be a fast learning algorithm of single-hidden layer feedforward neural network (SLFNs), and overcomes the disadvantages of the classical learning algorithm in neural network methods multiple iterations, huge search space and a large number of calculations, only needs to set the appropriate numbers of hidden layer nodes, assigns the weight of input and deviation of hidden layers without iteration. Research shows that the stock market is a very complex nonlinear system, which requires artificial intelligence theory, statistics theory and economic theory to study the stock price forecast. In this paper, ELM is introduced in predicting the stock price of Pfizer company, and by comparing it with SVM and BP, we analyze its feasibility and advantage in stock price prediction. The experiment results show that ELM is of high accuracy of prediction and apparent advantages in parameter selection and learning speed．

A Novel Key Features Screening Method Based on Extreme Learning Machine for Alzheimer's Disease Study.

The Extreme Learning Machine (ELM) is a simple and efficient Single Hidden Layer Feedforward Neural Network(SLFN) algorithm. In recent years, it has been gradually used in the study of Alzheimer’s disease (AD). When using ELM to diagnose AD based on high-dimensional features, there are often some features that have no positive impact on the diagnosis, while others have a significant impact on the diagnosis. In this paper, a novel Key Features Screening Method based on Extreme Learning Machine (KFS-ELM) is proposed. It can screen for key features that are relevant to the classification (diagnosis). It can also assign weights to key features based on their importance. We designed an experiment to screen for key features of AD. A total of 920 key functional connections screened from 4005 functional connections. Their weights were also obtained. The results of the experiment showed that: (1) Using all (4,005) features to diagnose AD, the accuracy is 95.33%. Using 920 key features to diagnose AD, the accuracy is 99.20%. The 3,085 (4,005 - 920) features that were screened out had a negative effect on the diagnosis of AD. This indicates the KFS-ELM is effective in screening key features. (2) The higher the weight of the key features and the smaller their number, the greater their impact on AD diagnosis. This indicates that the KFS-ELM is rational in assigning weights to the key features for their importance. Therefore, KFS-ELM can be used as a tool for studying features and also for improving classification accuracy.

Improved swarm-wavelet based extreme learning machine for myoelectric pattern recognition

Myoelectric signal generated by muscles is one of the bio-signals which is used by humans to control equipments. To achieve this purpose, a good myoelectric pattern recognition (M-PR) is required. The applied classifier and extracted feature sets greatly affect the success of M-PR. This paper proposes a hybrid and fast classifier, extreme learning machine (ELM) which is enhanced by improved hybrid particle swarm optimization with wavelet mutation (improved swarm-wavelet). ELM is, in essence, a single-hidden layer feed-forward neural network that keeps off iterative learning to save the training time. In addition to improving the actual performance of M-PR, we also evaluate the optimization of ELM using improved swarm-wavelet in this paper. The swarm-wavelet is improved by using the particle refresh and applying velocity improvement to avoid trapped in local minima. In ELM, the improved swarm-wavelet is used to find the most suitable parameters to increase the classification accuracy. Furthermore, this paper provides comparisons of improved swarm-wavelet-ELM, swarm-wavelet-ELM and standard swarm-ELM. The experimental results show that the improved swarm-wavelet-ELM, our proposed method, is the most accurate classifier with mean accuracy of 99.6%.

A COMPARATIVE STUDY OF SEVERAL SLFN-BASED CLASSIFICATION ALGORITHMS FOR URBAN AND RURAL LAND USE

Abstract. In the study of urban sustainable development, accurate classification of land use has become an important basis for monitoring urban dynamic changes. Hence it is necessary to develop the appropriate recognition model for urban-rural land use. Although deep learning algorithms have become a research hotspot in image classification tasks in recent years, and many good results have been achieved. But other machine learning algorithms are not going away. Compared deep learning with machine learning, there are some advantages and disadvantages in data dependence, hardware dependence, feature processing, problem solving methods, execution time, and interpretability, etc. Especially in the classification for remote sensing images, the continuous research and development of traditional machine learning algorithms is still of great significance. In this paper, the performances of several SLFN-based classification algorithms were studied and compared, including ELM, RBF K-ELM, mixed K-ELM, A-ELM and SVM. Extreme Learning Machine (ELM) is a new algorithm for single-hidden-layer feedforward neural network (SLFN). It has simple structure, fast speed and is easy to train. In some applications, however, standard ELM is prone to be overfitting and its performance will be affected seriously when outliers exist. In order to explore the performance of ELM and its improved algorithm for urban-rural land use classification, comparative experiments between three improved ELM algorithms (RBF K-ELM, mixed K-ELM and A-ELM), ELM and SVM with image data from several study areas were performed, and the classification accuracy and efficiency were analysed. The results show that the three improved ELM algorithms perform better than the standard ELM and SVM both in overall accuracy and Kappa coefficient. However, it is worth noting that the computation efficiency of RBF K-ELM and mixed K-ELM decreases greatly with larger image, the time cost is much more than other algorithms. Compared with other algorithms, A-ELM has the advantages of higher Overall Accuracy and less classification time.

An Input Weights Dependent Complex-Valued Learning Algorithm Based on Wirtinger Calculus

Complex-valued neural network is a kind of learning model which can deal with problems in complex domain. Fully complex extreme learning machine (CELM) is a much faster training algorithm than the complex backpropagation (CBP) scheme. However, it is at the cost of using more hidden nodes to obtain the comparable performance. An upper-layer-solution-aware algorithm has been proposed for training single-hidden layer feedforward neural networks, which performs much better than its counterparts, pseudo-inverse learning (PIL)/extreme learning machine and gradient decent-based backpropagation neural networks. Consequently, there exist two challenges that need to be dealt with: 1) How to combine the advantages of CBP and CELM to develop a novel complex learning algorithm? and 2) What is the convergent behavior of the presented algorithm? In this article, an input weights dependent complex-valued (IWDCV) learning algorithm based on Wirtinger calculus has been proposed, which effectively solves the nonanalytic problem of the common activation functions during training neural networks. In addition, the monotonicity of the error function and the deterministic convergence of the proposed model have been strictly proved, which theoretically guarantee the efficiency and effectiveness of the given model, IWDCV. Finally, for real and complex-valued problems, a variety of simulations have been done to demonstrate the comparable performance of the proposed algorithm which support the theoretical observations as well.

Optimized Extreme Learning Machine

Abstract: Extreme Learning Machine (ELM) is a learning method for single-hidden layer feedforward neural network (SLFN) training. The ELM strategy speeds up learning by generating input weights and biases for hidden nodes at random rather than modifying network parameters, making it much faster than the standard gradient-based approach. In this project, an ELM optimized by Hybrid Particle Swarm Optimization approach is presented to optimize the input weights and hidden biases for ELM. We will analyze and obtain results for benchmark datasets. The Optimized Extreme Learning Machine algorithm's output is compared to publicly available data. Later we will compare different algorithms and check which one gives better output metrics. Keywords: ELM, SLFN, PSO, Gradient-based approach, Optimization

A novel multivariate signal processing-based fault diagnosis approach of rotating machinery under various operating conditions

Compared with signals collected by the single sensor, the collected multivariate signals contain more information to reflect the state of mechanical equipment, which has a positive effect on fault diagnosis. However, different acquisition channels and various operating conditions interfere with the extraction of fault features of rotating machinery. To solve this problem, taking rolling bearings as an example in this paper, a novel method is adopted to alleviate these interferences and combined with an improved extreme learning machine (ELM) to achieve intelligent fault diagnosis of rolling bearings under various operating conditions. First, adaptive projection intrinsically transformed multivariate empirical mode decomposition is used to decompose multivariate signals and obtain intrinsic mode functions of each channel to construct feature matrices. Then, nuisance attribute projection (NAP) is employed to alleviate the interference components in the feature matrix, which are originated from different channels and operating conditions. Finally, vectors belonging to the processed feature matrix as samples are input into the proposed weighted extreme learning machine (WELM) for intelligent fault classification. The weighted matrix of the WELM can compress the dimension of the sample and extract sensitive features, and the effectiveness of the proposed fault diagnosis model via the above methods is verified by experiments. Furthermore, comparative experiments show that the proposed fault diagnosis model has higher accuracy than the model combined with NAP and traditional single-hidden layer feedforward neural network or ELM. Therefore, the proposed fault diagnosis model may potentially aid experts on fault diagnosis of rotating machinery under various operating conditions.