Hidden Layer Feedforward Neural Network Research Articles

Efficient Ensemble Broad Learning System Based on Dropout and DropConnect

Broad Learning System is an emerged efficient algorithm for training single hidden layer feedforward neural network (SLFN) with fast speed and good generalization ability. However, it is very difficult to determine the appropriate broad learning system structure and broad learning system may perform overfitting due to the dependence between nodes in processing fully connected network. In order to deal with these problems, efficient ensemble broad learning system based on Dropout and DropConnect are proposed in this paper. The proposed Dropout Ensemble Broad Learning System randomly discards hidden nodes to improve diversity between individuals and reduce the synergy between nodes to improve prediction stability. The DropConnect Ensemble Broad Learning System randomly drops connect weights to generate more complementary models by adding input attribute disturbance. The experimental results and statistical analysis on UCI data sets confirm that the proposed method can solve the problem of model overfitting caused by the strong dependence between the nodes of ensemble broad learning system and also show that the proposed approaches outperform the original BLS on prediction stability and classification accuracy.

Prediksi Harga Kelapa Sawit Menggunakan Metode Extreme Learning Machine

Palm oil is one of the keys to the Indonesian economy and the main commodity for attracting foreign investment. The palm oil and palm kernel industry generates most of the foreign currency from palm oil. The price of palm oil often goes up and down every month resulting in instability in the income received by people who own oil palm plantations. The aim of predicting palm oil prices is to carry out appropriate planning or steps for palm oil business actors. One way to overcome this problem is to make predictions. One method that can make predictions is the Extreme Learning Machine (ELM). ELM is an artificial neural network method used to predict palm oil prices. The ELM method is a feedforward method with a single hidden layer which is better known as a single hidden layer feedforward neural network (SLFNs). In this research, the best implementation was 5 inputs with 20 neurons in the hidden layer with output in the form of palm oil price predictions. Based on the tests carried out, the research produced the smallest error rate of 0.0027111424247658633 using 20 neurons in the hidden layer so that the latest data prediction test results for 5 price rotations in September rotation 1 were 1400.314191, September rotation 2 were 1846.798921, September rotation 3 amounted to 1505.430419, September rotation 4 amounted to 2301.853412, September rotation 5 amounted to 2645.082489 in palm oil price predictions.

Extreme Learning Machine Classification Method based on Twin Strategy Salp Swarm Algorithm

Extreme Learning Machines (ELM) are a type of Single Hidden Layer Feedforward Neural Network (SLFN). In recent years, they have attracted attention for their powerful approximation ability and fast learning speed. Compared with traditional neural network algorithms, ELM have the advantages of simple structure, fast learning speed, and good generalization performance. However, since the input weights and biases of ELM are randomly generated, there may be some suboptimal or unnecessary input weights and biases. In addition, ELM may require more hidden nodes, which may slow down its response to unknown test data. To address these issues, a twin strategy Salp Swarm Algorithm (TSSA) is proposed to increase the population diversity to improve the convergence speed of the algorithm through chaotic initialization, while shock inertia weights and learning paradigms are introduced into the follower position updating to enhance the stochasticity of the particles. Classification tests are conducted on six datasets using the proposed TSSA, and the experimental results show that the proposed TSSA-ELM has higher accuracy and better generalization performance than some existing ELM variants.

Mixture extreme learning machine algorithm for robust regression

The extreme learning machine (ELM) is a well-known approach for training single hidden layer feedforward neural networks (SLFNs) in machine learning. However, ELM is most effective when used for regression on datasets with simple Gaussian distributed error because it often employs a squared loss in its objective function. In contrast, real-world data is often collected from unpredictable and diverse contexts, which may contain complex noise that cannot be characterized by a single distribution. To address this challenge, we propose a robust mixture ELM algorithm, called Mixture-ELM, that enhances modeling capability and resilience to both Gaussian and non-Gaussian noise. The Mixture-ELM algorithm uses an adjusted objective function that blends Gaussian and Laplacian distributions to approximate any continuous distribution and match the noise. The Gaussian mixture accurately models the residual distribution, while the inclusion of the Laplacian distribution addresses the limitations of the Gaussian distribution in identifying outliers. We derive a solution to the novel objective function using the expectation maximization (EM) and iteratively reweighted least squares (IRLS) algorithms. We evaluate the effectiveness of the algorithm through numerical simulation and experiments on benchmark datasets, thereby demonstrating its superiority over other state-of-the-art machine learning methods in terms of robustness and generalization.

Identification Of Hijaiyah Letters Image Using Extreme Learning Machine Method

The development of digital image processing technology has many benefits, one of which is the identification of an object, such as the identification of the image of the hijaiyah letter. Hijaiyah letters are the letters of the arabic alphabet as the original language of the Qur'an. In essence, humans have the ability to recognize and distinguish hijaiyah letter patterns from one another, but this is not the case with computers, using digital images and machine learning, in this study an identification concept was built by recognizing the image of hijaiyah letters using one of the machine learning methods. namely the extreme learning machine (ELM) method. Extreme learning machine (ELM) is a feedforward neural network with one hidden layer or better known as single hidden layer feedforward neural networks (SLFNs). Therefore, the purpose of this study is that the computer can identify objects as well as human capabilities and see how accurate the results obtained in the ELM method are. The digital image identification process using the extreme learning machine (ELM) method is carried out in two stages, namely training and testing, where previously the preprocessing process was carried out first by changing the color of the RGB image to HSV and processing the color v, then segmentation was carried out with the aim of separating the objects (foreground) with the background, then to make it easier to recognize the pattern, a morphological process is carried out. From the simulation carried out on the test data, the results obtained an average accuracy of 90% with an error of 10%.

Lithium-Ion Battery State of Health Prediction With a Robust Collaborative Augmented Hidden Layer Feedforward Neural Network Approach

Lithium-ion (Li-ion) batteries play an important role in providing necessary energy when acting as a main or backup source of electricity. Indeed, the unavailability of battery aging discharge data in most real-world applications makes the State of Health (SoH) assessment very challenging. Alternatively, accelerated aging is therefore adopted to emulate the degradation process and to achieve an SoH estimate. However, accelerated aging generates limited deterioration patterns suffering from a higher level of complexity due to the non-linearity and non-stationarity imposed by harsh conditions. In this context, this paper aims to provide a predictive model capable of solving incomplete data problems by providing two main solutions for each of the problems of complexity and missing patterns, respectively. First, to overcome the problem of lack of patterns, a robust collaborative feature extractor (RCFE) is designed by collaborating between a set of improved restricted Boltzmann machines (I-RBMs) to be able to share learning knowledge among different locally trained I-RBMs to create a more generalized global extraction model. Second, a set of RCFEs is then evolved through a neural network with an augmented hidden layer (NAHL) to enhance the predictive ability by further exploring representation learning to overcome pattern complexity issues. The designed RCFE-NAHL is trained to predict SoH using constant current (CC) discharge characteristics by implying multiple characteristics recorded through the constant voltage (CV) charging process as indicators of health. The proposed SoH prediction approach performances are evaluated on a set of battery life cycles from the well-known NASA database. In this context, the achieved results clearly highlight the higher accuracy and robustness of the proposed learning model.

Functional extreme learning machine.

Extreme learning machine (ELM) is a training algorithm for single hidden layer feedforward neural network (SLFN), which converges much faster than traditional methods and yields promising performance. However, the ELM also has some shortcomings, such as structure selection, overfitting and low generalization performance. This article a new functional neuron (FN) model is proposed, we takes functional neurons as the basic unit, and uses functional equation solving theory to guide the modeling process of FELM, a new functional extreme learning machine (FELM) model theory is proposed. The FELM implements learning by adjusting the coefficients of the basis function in neurons. At the same time, a simple, iterative-free and high-precision fast parameter learning algorithm is proposed. The standard data sets UCI and StatLib are selected for regression problems, and compared with the ELM, support vector machine (SVM) and other algorithms, the experimental results show that the FELM achieves better performance.

STWD-SFNN: Sequential three-way decisions with a single hidden layer feedforward neural network

The three-way decisions strategy has been employed to construct network topology in a single hidden layer feedforward neural network (SFNN). However, this model has a general performance, and does not consider the process costs, since it has fixed threshold parameters. Inspired by the sequential three-way decisions (STWD), this paper proposes STWD with an SFNN (STWD-SFNN) to enhance the performance of networks on structured datasets. STWD-SFNN adopts multi-granularity levels to dynamically learn the number of hidden layer nodes from coarse to fine, and set the sequential threshold parameters. Specifically, at the coarse granular level, STWD-SFNN handles easy-to-classify instances by applying strict threshold conditions, and with the increasing number of hidden layer nodes at the fine granular level, STWD-SFNN focuses more on disposing of the difficult-to-classify instances by applying loose threshold conditions, thereby realizing the classification of instances. Moreover, STWD-SFNN considers and reports the process cost produced from each granular level. The experimental results verify that STWD-SFNN has a more compact network on structured datasets than other SFNN models, and has better generalization performance than the competitive models. All models and datasets can be downloaded from https://github.com/wuc567/Machine-learning/tree/main/STWD-SFNN.

Retracted: Potential Fault Diagnosis Method and Classification Accuracy Detection of IGBT Device Based on Improved Single Hidden Layer Feedforward Neural Network.

[This retracts the article DOI: 10.1155/2021/6036118.].

Big Data Classification in IOT Healthcare Application Using Optimal Deep Learning

The Internet of Things (IoT) system is composed of several numbers of sensor nodes and systems, which are wirelessly interlinked to the internet. Generally, big data is the storage of a huge amount of information, which causes the classification process to be very challenging. Numerous big data classification approaches are implemented, but the computational time and secure handling of information are the major problems. The aim of the study is the development of big data approach in Internet of Things (IoT) healthcare application. Hence, this paper presents the proposed Dragonfly Rider Competitive Swarm Optimization-based Deep Residual Network (DRCSO-based DRN) for big data classification in IoT. First, the IoT nodes are simulated, and the heart disease patient data are collected through sensors. The routing is done using the Multi-objective Fractional Gravitational Search Algorithm (Multi-objective FGSA). In the Base Station (BS), the big data classification is done. Here, the classification is done using MapReduce (MR) framework, which includes two phases, like mapper and the reducer phase. The input data is initially fed to the mapper phase in the map-reduce (MR) framework. In the mapper phase, feature selection is carried out based on Dragonfly Rider Optimization Algorithm (DROA) in order to select the appropriate features for further processing. The DROA is modeled through merging Dragonfly Algorithm (DA) and Rider Optimization Algorithm (ROA). In the reducer phase, the classification is performed using DRN, which is trained by the developed DRCSO algorithm. The DRCSO is modeled by incorporating DA, ROA and Competitive Swarm Optimization (CSO). In addition, the performance of the developed method is outperformed than the existing approaches such as Linguistic Fuzzy Rules with Canopy Mapreduce (LFR-CM) + Fuzzy classifier, Machine learning-dependent k-nearest neighbors (FML-KNN), MapReduce-Fuzzy Integral-dependent Ensemble Learning Model+Single hidden layer feedforward neural network (MR-FI-ELM + SLFN) and DROA-recurrent neural network (RNN) based on the accuracy, average residual energy and throughput with the value of 0.929, 0.086[Formula: see text]J and 86.585. The proposed method is used to manage and derive meaningful information from the patient’s medical records, medical examinations results and hospital records.

Enhanced Salp Search Algorithm for Optimization Extreme Learning Machine and Application to Dew Point Temperature Prediction

Extreme learning machine (ELM) is popular as a method of training single hidden layer feedforward neural networks. However, the ELMs optimized by the traditional gradient descent algorithms cannot fundamentally solve the influence of the random selection of the input weights and biases. Therefore, this paper proposes a method of extreme learning machine optimized by an enhanced salp search algorithm (NSSA-ELM). Salp search algorithm (SSA) is a metaheuristic algorithm, to improve the performance of SSA exploration and avoid getting stuck in local optima, the neighborhood centroid opposite‑based learning is used to optimize SSA. This method maintains the diversity of the population, which is conducive to avoid local optimization and accelerate convergence. This paper performs classification tests on NSSA and other metaheuristic-optimized ELMs on ten datasets, and regression tests on 5 datasets. Finally, the prediction ability of dew point temperature is evaluated. The meteorological data of five climatically representative cities in China from 2016 to 2022 were collected to predict the dew point temperature. The experimental results show that the NSSA-ELM is the best model, and its generalization performance and accuracy are better than other models.

Empirical validation of ELM trained neural networks for financial modelling

The purpose of this work is to compare predictive performance of neural networks trained using the relatively novel technique of training single hidden layer feedforward neural networks (SFNN), called Extreme Learning Machine (ELM), with commonly used backpropagation-trained recurrent neural networks (RNN) as applied to the task of financial market prediction. Evaluated on a set of large capitalisation stocks on the Australian market, specifically the components of the ASX20, ELM-trained SFNNs showed superior performance over RNNs for individual stock price prediction. While this conclusion of efficacy holds generally, long short-term memory (LSTM) RNNs were found to outperform for a small subset of stocks. Subsequent analysis identified several areas of performance deviations which we highlight as potentially fruitful areas for further research and performance improvement.

Automated layer-wise solution for ensemble deep randomized feed-forward neural network

The randomized feed-forward neural network is a single hidden layer feed-forward neural network that enables efficient learning by optimizing only the output weights. The ensemble deep learning framework significantly improves the performance of randomized neural networks. However, the framework’s capabilities are limited by traditional hyper-parameter selection approaches. Meanwhile, different random network architectures, such as the existence or lack of a direct link and the mapping of direct links, can also strongly affect the results. We present an automated learning pipeline for the ensemble deep randomized feed-forward neural network in this paper, which integrates hyper-parameter selection and randomized network architectural search via Bayesian optimization to ensure robust performance. Experiments on 46 UCI tabular datasets show that our strategy produces state-of-the-art performance on various tabular datasets among a range of randomized networks and feed-forward neural networks. We also conduct ablation studies to investigate the impact of various hyper-parameters and network architectures.

Fractal adaptive weight synthesized-local directional pattern-based image classification using enhanced tree seed algorithm.

Coral reefs are one of the most prominent marine ecosystems on the Earth; they are threatened due to various factors, including growing anthropogenic impacts and the effects of global change. Hence, the automatic classification of coral species is significant for tracking and detecting threatened and susceptible coral species. This paper proposes a new feature descriptor known as fractal adaptive weight synthesized-local directional pattern (FAWS-LDP) method to classify coral images using enhanced tree seed algorithm with extreme learning machine (ETSA-ELM) technique. The proposed feature descriptor (FAWS-LDP) inherits the advantage of both fractal pixel intensity information and local directional characteristics by indexing both feature vector values. Finally, the extracted features are imported to the extreme learning machine (ELM) network for classification. The ELM classifier is a single hidden layer feed-forward neural network with a faster learning speed and produces good generalization performance. The random selection of input weight and biases of the ELM classifier produces non-optimal or unnecessary input biases and weights to the network. Hence, to fine-tune the parameters of the ELM classifier, an enhanced tree seed algorithm (ETSA) is proposed. The proposed ETSA is a new learning technique to overcome the drawbacks like local optima and a lower coverage rate. The classification performance of ELM employing the ETSA optimizer is compared to the original TSA and other well-known metaheuristic algorithm (MHA) trainers, such as genetic algorithm (GA), particle swarm optimization (PSO), and artificial bee colony (ABC), utilizing model performance measures like specificity, sensitivity, and classification accuracy. The evaluation of coral classification datasets reveals that the proposed ETSA-ELM produces consistently superior performance to existing methods. Finally, the proposed feature descriptor technique is statistically analyzed using a non-parametric Friedman test to demonstrate the efficiency.

A Novel Key Features Screening Method Based on Extreme Learning Machine for Alzheimer's Disease Study.

The Extreme Learning Machine (ELM) is a simple and efficient Single Hidden Layer Feedforward Neural Network(SLFN) algorithm. In recent years, it has been gradually used in the study of Alzheimer’s disease (AD). When using ELM to diagnose AD based on high-dimensional features, there are often some features that have no positive impact on the diagnosis, while others have a significant impact on the diagnosis. In this paper, a novel Key Features Screening Method based on Extreme Learning Machine (KFS-ELM) is proposed. It can screen for key features that are relevant to the classification (diagnosis). It can also assign weights to key features based on their importance. We designed an experiment to screen for key features of AD. A total of 920 key functional connections screened from 4005 functional connections. Their weights were also obtained. The results of the experiment showed that: (1) Using all (4,005) features to diagnose AD, the accuracy is 95.33%. Using 920 key features to diagnose AD, the accuracy is 99.20%. The 3,085 (4,005 - 920) features that were screened out had a negative effect on the diagnosis of AD. This indicates the KFS-ELM is effective in screening key features. (2) The higher the weight of the key features and the smaller their number, the greater their impact on AD diagnosis. This indicates that the KFS-ELM is rational in assigning weights to the key features for their importance. Therefore, KFS-ELM can be used as a tool for studying features and also for improving classification accuracy.

Soil-cutting simulation and dual-objective optimization on tillage process parameters of micro-tiller by smoothed particle Galerkin modeling and genetic algorithm

Precision tillage is a technology-based approach enabled by understanding the soil-tool interaction, which is of central importance to improving agricultural operations. Numerical simulation offers a valuable way to investigate soil behavior in tilling process which is typically difficult or impossible to obtain via physical experiments. Inspired by the recent development in the numerical analysis of processes involving large deformation (e.g., cutting of blood clots), smoothed particle Galerkin (SPG) method was used to simulate the interaction between rotary blade and soil in the tilling process. Motion of soil particles, torque on the rotary blade, and power consumption were obtained for performance analysis by simulation and soil bin experiments. It was showed that SPG method was an effective tool to simulate soil-cutting process which provided quite acceptable predictions compared with soil bin tests. Results also supported the presence of a direct relationship between process parameters (tillage depth, forward velocity, and rotation velocity) and micro-tiller’s working performance. Instead of conventional polynomial regression, extreme learning machine (ELM), a single hidden layer feedforward neural network, was used to model this relationship, which had better flexibility and scalability in this multi-input and multi-output problem. Using training data from SPG modeling, this physics-free fast prediction method provided satisfactory consistency with numerical simulation results but with considerably less time, which served as the basis for efficient parameter optimization in tillage. Reduction of torque on the rotary blade and power consumption of micro-tiller in mechanical tilling operation benefits the sustainability of the environment and farming systems. Taking these two as dual-minimization objectives, NSGA-II, a genetic algorithm-based iterative procedure together with ELM modeling was implemented to optimize operation parameters.

A Disturbance Observer-Based Intelligent Finite-Time Sliding Mode Flight Controller Design for an Autonomous Quadrotor

This article presents an intelligent control strategy for the position and attitude tracking of a quadrotor using a nonsingular fast-terminal sliding mode and disturbance observer. The quadrotor system is divided into an underactuated position and the fully actuated attitude subsystem. A single hidden layer feed-forward neural network is used for the estimation of unknown dynamics associated with the attitude subsystem. Moreover, a nonlinear disturbance observer is designed to tackle the unknown bounded external disturbances acting on the attitude subsystem by supplying its estimation in the control laws that help in mitigating the chattering phenomena. The proposed methodology ensures the finite-time convergence of the tracking error and does not prone to the singularity problem in control. The closed-loop finite-time stability is investigated using the Lyapunov theory. Besides, an expression for the convergence time has been derived. The effectiveness of the proposed scheme is initially assessed using numerical simulations for the way-point tracking as well as circle tracking tasks and then validated in real-time using DJI Matrice 100 by performing a lemniscate tracking task. To show the superiority of the proposed methodology over existing methods, a comparative study has been done and found the improved tracking performance in the proposed approach.

Implementation of CBIR Method for Identification of Corn Disease Using Extreme Learning Machine

Various artificial neural network methods and digital image processing techniques have been applied in agriculture. Previous researchers have proposed methods for identification or detection of plant diseases, such as using K-NN, support vector machine (SVM), backpropagation and so on. The disease attack can reduce the amount of agricultural productivity and result in other material losses for farmers. Plant diseases can be observed from the physical and color changes of the affected parts such as roots, stems and leaves. However, with the development of computer vision technology, observations can be assisted by image processing methods and artificial neural networks, for automation in agriculture. This study proposes a method of applying digital image processing with artificial neural networks to identify corn plant diseases. The purpose of this research is as a comparative study of the proposed method with other conventional methods. The methods used are content based image retrieval (CBIR) and extreme learning machine (ELM). Content based image retrieval (CBIR) is the process of searching for an image in a database by comparing the features in the query image with the features that have been stored in the image database. Meanwhile, extreme learning machine (ELM) is an artificial neural network with one hidden layer or single hidden layer feedforward neural network (SLFNs). The feature extraction used consists of 3 features, namely color, shape, and texture feature extraction. The dataset consists of 3 types of images, namely images of healthy leaves, leaf spot disease, and rust disease. The ELM artificial neural network serves as a classifier that classifies the types of diseases based on the given feature extraction. The proposed system is able to identify the type of disease based on the pattern that appears on the leaf surface with the highest precision of 0.9922.

Extreme Learning Machine Using Bat Optimization Algorithm for Estimating State of Health of Lithium-Ion Batteries

An accurate estimation of the state of health (SOH) of lithium-ion batteries is essential for the safe and reliable operation of electric vehicles. As a single hidden-layer feedforward neural network, extreme learning machine (ELM) has the advantages of a fast learning speed and good generalization performance. The bat algorithm (BA) is a swarm intelligence optimization algorithm based on bat echolocation for foraging. In this study, BA was creatively applied to improve the ELM neural network, forming a BA-ELM model, and it was applied to SOH estimation for the first time. First, through Pearson and Spearman correlation analysis, six variables were determined as the input variables of the model. The actual remaining capacity of the battery was determined as the output variable. Second, BA was used to optimize the connection weights and bias in ELM to construct the BA-ELM model. Third, the battery data set was trained and tested with BA-ELM, ELM, Elman, back propagation (BP), radial basis function (RBF), and general regression neural network (GRNN) models. Five statistical error indicators, and the radar chart, scatter plot, and violin diagram were used to compare the estimation effects. The results show that the evaluation function of BA-ELM can converge quickly and effectively optimize the network model of ELM. The RMSE of the BA-ELM model was 0.5354%, and the MAE was 0.4326%, which is the smallest among the 6 models. The RMSE values of the other model were 2.27%, 3.53%, 3.07%, 3.86%, 3.24%, respectively, indicating the BA-ELM has good potential for future applications.

A study of extreme learning machine on small sample-sized classification problems

Extreme learning machine (ELM) is a special type of single hidden layer feedforward neural network that emphasizes training speed and optimal generalization. The ELM model proposes that the weights of hidden neurons need not be tuned, and the weights of output neurons can be calculated by finding the Moore-Penrose generalized inverse method. Thus, the ELM classifier is suitable to use in a homogeneous ensemble model due to the untuned random hidden weights which promote diversity even with the same training data. This paper studies the effectiveness of the ELM ensemble models in solving small sample-sized classification problems. The research involves two variants of the ensemble model: the normal ELM ensemble with majority voting (ELE), and the random subspace method (RS-ELM). To simulate the small sample cases, only 30% of the total data will be used as the training data. Experiment results show that the RS-ELM model can outperform a multi-layer perceptron (MLP) model under the assumptions of a Friedman test. Furthermore, the ELE model has similar performance as an MLP model under the same assumptions.