Layer Feedforward Neural Network Research Articles

Efficient Ensemble Broad Learning System Based on Dropout and DropConnect

Broad Learning System is an emerged efficient algorithm for training single hidden layer feedforward neural network (SLFN) with fast speed and good generalization ability. However, it is very difficult to determine the appropriate broad learning system structure and broad learning system may perform overfitting due to the dependence between nodes in processing fully connected network. In order to deal with these problems, efficient ensemble broad learning system based on Dropout and DropConnect are proposed in this paper. The proposed Dropout Ensemble Broad Learning System randomly discards hidden nodes to improve diversity between individuals and reduce the synergy between nodes to improve prediction stability. The DropConnect Ensemble Broad Learning System randomly drops connect weights to generate more complementary models by adding input attribute disturbance. The experimental results and statistical analysis on UCI data sets confirm that the proposed method can solve the problem of model overfitting caused by the strong dependence between the nodes of ensemble broad learning system and also show that the proposed approaches outperform the original BLS on prediction stability and classification accuracy.

Prediksi Harga Kelapa Sawit Menggunakan Metode Extreme Learning Machine

Palm oil is one of the keys to the Indonesian economy and the main commodity for attracting foreign investment. The palm oil and palm kernel industry generates most of the foreign currency from palm oil. The price of palm oil often goes up and down every month resulting in instability in the income received by people who own oil palm plantations. The aim of predicting palm oil prices is to carry out appropriate planning or steps for palm oil business actors. One way to overcome this problem is to make predictions. One method that can make predictions is the Extreme Learning Machine (ELM). ELM is an artificial neural network method used to predict palm oil prices. The ELM method is a feedforward method with a single hidden layer which is better known as a single hidden layer feedforward neural network (SLFNs). In this research, the best implementation was 5 inputs with 20 neurons in the hidden layer with output in the form of palm oil price predictions. Based on the tests carried out, the research produced the smallest error rate of 0.0027111424247658633 using 20 neurons in the hidden layer so that the latest data prediction test results for 5 price rotations in September rotation 1 were 1400.314191, September rotation 2 were 1846.798921, September rotation 3 amounted to 1505.430419, September rotation 4 amounted to 2301.853412, September rotation 5 amounted to 2645.082489 in palm oil price predictions.

Extreme Learning Machine Classification Method based on Twin Strategy Salp Swarm Algorithm

Extreme Learning Machines (ELM) are a type of Single Hidden Layer Feedforward Neural Network (SLFN). In recent years, they have attracted attention for their powerful approximation ability and fast learning speed. Compared with traditional neural network algorithms, ELM have the advantages of simple structure, fast learning speed, and good generalization performance. However, since the input weights and biases of ELM are randomly generated, there may be some suboptimal or unnecessary input weights and biases. In addition, ELM may require more hidden nodes, which may slow down its response to unknown test data. To address these issues, a twin strategy Salp Swarm Algorithm (TSSA) is proposed to increase the population diversity to improve the convergence speed of the algorithm through chaotic initialization, while shock inertia weights and learning paradigms are introduced into the follower position updating to enhance the stochasticity of the particles. Classification tests are conducted on six datasets using the proposed TSSA, and the experimental results show that the proposed TSSA-ELM has higher accuracy and better generalization performance than some existing ELM variants.

CTFNet: Long-Sequence Time-Series Forecasting Based on Convolution and Time-Frequency Analysis.

Although current time-series forecasting methods have significantly improved the state-of-the-art (SOTA) results for long-sequence time-series forecasting (LSTF), they still have difficulty in capturing and extracting the features and dependencies of long-term sequences and suffer from information utilization bottlenecks and high-computational complexity. To address these issues, a lightweight single-hidden layer feedforward neural network (SLFN) combining convolution mapping and time-frequency decomposition called CTFNet is proposed with three distinctive characteristics. First, time-domain (TD) feature mining-in this article, a method for extracting the long-term correlation of horizontal TD features based on matrix factorization is proposed, which can effectively capture the interdependence among different sample points of a long time series. Second, multitask frequency-domain (FD) feature mining-this can effectively extract different frequency feature information of time-series data from the FD and minimize the loss of data features. Integrating multiscale dilated convolutions, simultaneously focusing on both global and local context feature dependencies at the sequence level, and mining the long-term dependencies of the multiscale frequency information and the spatial dependencies among the different scale frequency information, break the bottleneck of data utilization, and ensure the integrity of feature extraction. Third, highly efficient-the CTFNet model has a short training time and fast inference speed. Our empirical studies with nine benchmark datasets show that compared with state-of-the-art methods, CTFNet can reduce prediction error by 64.7% and 53.7% for multivariate and univariate time series, respectively.

A fast online stacked regressor to handle concept drifts

The number of algorithms based on Extreme Learning Machine (ELM), which train Single Layer Feedforward Neural Networks (SLFN), has increased in the past years due to their fast training stages and good generalization performances. Many variants have been proposed, which deal with outliers, with online learning and other characteristics. Recently, Fast Deep Stacked Network (FDSN) was proposed, which is an alternative to large ELM networks with similar performances using less memory than ELM-based algorithms. Regarding online methods, which deal with data that arrives over time considering concept drift, ensemble approaches are one of the most promising categories of these methods. FDSN can be viewed as an ensemble of SLFN, where the network output is the output of the most recent SLFN. In this paper, we propose Online Sequential FDSN (OSFDSN), which is similar to FDSN, but each of its SLFN modules has a weighted contribution to the network output. These weights are dynamically calculated, according to the most recent data. We compare our method with some other similar techniques with implementations publicly available. Our conducted experiments show that the proposed technique is statistically equivalent to the others when considering RMSE, while having a faster updating scheme.

Mixture extreme learning machine algorithm for robust regression

The extreme learning machine (ELM) is a well-known approach for training single hidden layer feedforward neural networks (SLFNs) in machine learning. However, ELM is most effective when used for regression on datasets with simple Gaussian distributed error because it often employs a squared loss in its objective function. In contrast, real-world data is often collected from unpredictable and diverse contexts, which may contain complex noise that cannot be characterized by a single distribution. To address this challenge, we propose a robust mixture ELM algorithm, called Mixture-ELM, that enhances modeling capability and resilience to both Gaussian and non-Gaussian noise. The Mixture-ELM algorithm uses an adjusted objective function that blends Gaussian and Laplacian distributions to approximate any continuous distribution and match the noise. The Gaussian mixture accurately models the residual distribution, while the inclusion of the Laplacian distribution addresses the limitations of the Gaussian distribution in identifying outliers. We derive a solution to the novel objective function using the expectation maximization (EM) and iteratively reweighted least squares (IRLS) algorithms. We evaluate the effectiveness of the algorithm through numerical simulation and experiments on benchmark datasets, thereby demonstrating its superiority over other state-of-the-art machine learning methods in terms of robustness and generalization.

Identification Of Hijaiyah Letters Image Using Extreme Learning Machine Method

The development of digital image processing technology has many benefits, one of which is the identification of an object, such as the identification of the image of the hijaiyah letter. Hijaiyah letters are the letters of the arabic alphabet as the original language of the Qur'an. In essence, humans have the ability to recognize and distinguish hijaiyah letter patterns from one another, but this is not the case with computers, using digital images and machine learning, in this study an identification concept was built by recognizing the image of hijaiyah letters using one of the machine learning methods. namely the extreme learning machine (ELM) method. Extreme learning machine (ELM) is a feedforward neural network with one hidden layer or better known as single hidden layer feedforward neural networks (SLFNs). Therefore, the purpose of this study is that the computer can identify objects as well as human capabilities and see how accurate the results obtained in the ELM method are. The digital image identification process using the extreme learning machine (ELM) method is carried out in two stages, namely training and testing, where previously the preprocessing process was carried out first by changing the color of the RGB image to HSV and processing the color v, then segmentation was carried out with the aim of separating the objects (foreground) with the background, then to make it easier to recognize the pattern, a morphological process is carried out. From the simulation carried out on the test data, the results obtained an average accuracy of 90% with an error of 10%.

An informative teaching model of English-Chinese translation in universities based on the perceptual machine model

Abstract With the development of globalization, there is an increasing demand for English-Chinese translation talents in the country and society. In this paper, a text classifier based on multilayer perceptron is proposed, an MLP structure containing two layers of feedforward neural network is designed for the characteristics of English-Chinese translation corpus, the training samples are selected with text height mainly below 60 pixels, and the best sliding window of 13×13 is selected by comparison, supplemented by the smoothing method of mathematical morphology. The constructed MLP-based information-based teaching model is then applied to teaching English-Chinese translation in colleges and universities and is mainly evaluated in terms of student evaluation and teaching effectiveness. There were 28.48 percentage points more English-Chinese translation students who thought the class was more interesting than the original, while 20.56% and 20.50% more students evaluated the class as more helpful and less helpful, respectively. The average score of English-Chinese translation students increased by 15.88% compared to the original one through the comparison test between traditional and informational teaching modes. The MLP-based informatization teaching can mobilize students’ enthusiasm to participate in English-Chinese translation learning, strengthen the guidance of translation methods, and improve students’ translation levels.

Lithium-Ion Battery State of Health Prediction With a Robust Collaborative Augmented Hidden Layer Feedforward Neural Network Approach

Lithium-ion (Li-ion) batteries play an important role in providing necessary energy when acting as a main or backup source of electricity. Indeed, the unavailability of battery aging discharge data in most real-world applications makes the State of Health (SoH) assessment very challenging. Alternatively, accelerated aging is therefore adopted to emulate the degradation process and to achieve an SoH estimate. However, accelerated aging generates limited deterioration patterns suffering from a higher level of complexity due to the non-linearity and non-stationarity imposed by harsh conditions. In this context, this paper aims to provide a predictive model capable of solving incomplete data problems by providing two main solutions for each of the problems of complexity and missing patterns, respectively. First, to overcome the problem of lack of patterns, a robust collaborative feature extractor (RCFE) is designed by collaborating between a set of improved restricted Boltzmann machines (I-RBMs) to be able to share learning knowledge among different locally trained I-RBMs to create a more generalized global extraction model. Second, a set of RCFEs is then evolved through a neural network with an augmented hidden layer (NAHL) to enhance the predictive ability by further exploring representation learning to overcome pattern complexity issues. The designed RCFE-NAHL is trained to predict SoH using constant current (CC) discharge characteristics by implying multiple characteristics recorded through the constant voltage (CV) charging process as indicators of health. The proposed SoH prediction approach performances are evaluated on a set of battery life cycles from the well-known NASA database. In this context, the achieved results clearly highlight the higher accuracy and robustness of the proposed learning model.

Integrating multivariate fuzzy neural networks into fuzzy inference system for enhanced decision making

In this paper, we introduce a novel family of operators for single-hidden layer feed-forward neural networks that operate on multivariate fuzzy-valued functions. These operators are based on the fractional mean value of the approximating function. The construction of these operators involves the use of sigmoidal functions, providing multiple choices for their implementation. We demonstrate the modeling capabilities of our operators employing a novel combined neural network system that incorporates the basic fuzzy inference mechanism and fuzzy neural networks. This system represents the connection strength among the output neurons using multivariate fuzzy-valued functions, enhancing decision-making processes. Additionally, we prove the convergence properties of these operators with respect to the Pompeiu-Hausdorff metric. By controlling the asymptotic decay of the sigmoidal function, we achieve accurate Jackson-type estimations using the modulus of continuity of fuzzy-valued functions.

Prediction of Women’s Fashion Product Expense in Hanoi City Using Artificial Neural Network

A person's fashion product expense plays an important role in building a garment company's product and pricing strategy. This article presents the results of predicting the fashion expenses of women aged 25 to 55 in Hanoi city. A linear multivariable model determined by the Bayesion Model Average (BMA) was applied to predict expenditures based on a set of 150 women survey data. The 3 layers feed-forward neural network model with 8 input neurons, 4 (+1) hidden layer neurons, and 1 output layer neuron, The Sigmoid activation function, trained by the error backpropagation algorithm has been established on software R specifically predicts the value of women's fashion expense with the above data set. The results show that the artificial neural network (ANN) has been set up allowing to prediction of fashion expenses accurately. The expenditure values predicted by the ANN and the actual value are linearly correlated with R2 = 0.987; The prediction results by ANN are more accurate than the defined multivariable linear model.

Functional extreme learning machine.

Extreme learning machine (ELM) is a training algorithm for single hidden layer feedforward neural network (SLFN), which converges much faster than traditional methods and yields promising performance. However, the ELM also has some shortcomings, such as structure selection, overfitting and low generalization performance. This article a new functional neuron (FN) model is proposed, we takes functional neurons as the basic unit, and uses functional equation solving theory to guide the modeling process of FELM, a new functional extreme learning machine (FELM) model theory is proposed. The FELM implements learning by adjusting the coefficients of the basis function in neurons. At the same time, a simple, iterative-free and high-precision fast parameter learning algorithm is proposed. The standard data sets UCI and StatLib are selected for regression problems, and compared with the ELM, support vector machine (SVM) and other algorithms, the experimental results show that the FELM achieves better performance.

Retracted: Potential Fault Diagnosis Method and Classification Accuracy Detection of IGBT Device Based on Improved Single Hidden Layer Feedforward Neural Network.

[This retracts the article DOI: 10.1155/2021/6036118.].

A self-organizing modular neural network based on empirical mode decomposition with sliding window for time series prediction

Time series is mostly with a chaotic nature and non-stationary characteristic in real-word, which makes it difficult to be modeled and predicted accurately. To solve this problem, we introduce a novel self-organizing modular neural network based on the empirical mode decomposition with the sliding window mechanism (SWEMD-MNN) for time series prediction. In SWEMD-MNN, the improved empirical mode decomposition with sliding window (SWEMD) is developed to decompose time series online, which can effectively alleviate the limitation that the traditional EMD-based models cannot handle the long term or online problem and end effect. Thus, SWEMD-MNN can decompose time series based on time characteristic effectively and dynamically, and improve the prediction accuracy of the classical modular neural networks dividing time series based on sample space. Then time subseries are dynamically assigned to the subnetworks with a single layer feedforward neural network using the sample entropy and Euclidean distance for learning. Experimental investigations using benchmark chaotic and real-world time series show that SWEMD-MNN can decompose time series effectively and dynamically, and provides a better prediction accuracy than the fully coupled networks and other MNN models for time series prediction.

A novel hybrid Extreme Learning Machine and Teaching–Learning-Based​ Optimization algorithm for skin cancer detection

Skin cancers, such as melanoma, can be difficult to spot in their early stages because they often resemble benign moles. Early detection of melanoma is crucial as it increases the chances of successful treatment and prevents cancer from spreading to other areas of the body. Machine learning algorithms and computer vision techniques are versatile techniques for detecting melanoma. However, current research has limitations, such as inaccurate detection and longer computation times. This paper proposes a novel hybrid Extreme Learning Machine (ELM) and Teaching–Learning-Based Optimization (TLBO) algorithm as a versatile technique for detecting melanoma. ELM is a single-hidden layer feed-forward neural network that can be trained quickly and accurately, while TLBO is an optimization algorithm used to fine-tune the network’s parameters for improved performance. Together, these techniques can classify skin lesions as benign or malignant images, potentially improving melanoma detection accuracy.

STWD-SFNN: Sequential three-way decisions with a single hidden layer feedforward neural network

The three-way decisions strategy has been employed to construct network topology in a single hidden layer feedforward neural network (SFNN). However, this model has a general performance, and does not consider the process costs, since it has fixed threshold parameters. Inspired by the sequential three-way decisions (STWD), this paper proposes STWD with an SFNN (STWD-SFNN) to enhance the performance of networks on structured datasets. STWD-SFNN adopts multi-granularity levels to dynamically learn the number of hidden layer nodes from coarse to fine, and set the sequential threshold parameters. Specifically, at the coarse granular level, STWD-SFNN handles easy-to-classify instances by applying strict threshold conditions, and with the increasing number of hidden layer nodes at the fine granular level, STWD-SFNN focuses more on disposing of the difficult-to-classify instances by applying loose threshold conditions, thereby realizing the classification of instances. Moreover, STWD-SFNN considers and reports the process cost produced from each granular level. The experimental results verify that STWD-SFNN has a more compact network on structured datasets than other SFNN models, and has better generalization performance than the competitive models. All models and datasets can be downloaded from https://github.com/wuc567/Machine-learning/tree/main/STWD-SFNN.

Character level vehicle license detection using multi layered feed forward back propagation neural network

Real-world traffic situations, including smart traffic monitoring, automated parking systems, and car services are increasingly using vehicle license detection systems (VLDS). Vehicle license plate identification is still a challenge with current approaches, particularly in more complicated settings. The use of machine learning and deep learning algorithms, which display improved classification accuracy and resilience, has been a significant recent breakthrough. Deep learning-based license plate identification using neural networks is proposed in this article. The number plate is detected using a multi layered feed forward back propagation neural network (MLFFBPNN). In this method, there are 3 layers namely input, hidden, and output layers has been utilized. Each layer has been related with interconnection weights. In feed forward of information, initially a set of randomly chosen weights are feed to the input data and an output has been determined. Back propagation training algorithm is utilized to train the network. Then character level identification is performed. The suggested proposed method is compared to the region-based convolutional neural network (RCNN) method in terms of accuracy and computational efficiency. The proposed method produced the character level recognition accuracy of 89%. It is improved by 4% when compared with the RCNN recognition method.

A novel DCNN-ELM hybrid framework for face mask detection

The Coronavirus disease (2019) has caused massive destruction of human lives and capital around the world. The latest variant Omicron is proved to be the most infectious of all its previous counterparts – Alpha, Beta and Delta. Various measures are identified, tested and implemented to minimize the attack on humans. Face masks are one of those measures that are shown to be very effective in containing the infection. However, it requires continuous monitoring for law enforcement. In the present manuscript, a detailed research investigation using different ablation studies is carried out to develop the framework for face mask recognition using pre-trained deep convolution neural networks (DCNN) models used in conjunction with a fast single layer feed-forward neural network (SLFNN) commonly known as Extreme Learning Machine (ELM) as classification technique. The ELM is well known for its real time data processing capabilities and has been successfully applied both for regression and classification problems of image processing and biomedical domain. It is for the first time that in this paper we have proposed the use of ELM as classifier for face mask detection. As a precursor to this, for feature selection, six pre-trained DCNNs such as Xception, Vgg16, Vgg19, ResNet50, ResNet 101 and ResNet152 are tested for this purpose. The best testing accuracy is obtained in case of ResNet152 transfer learning model used with ELM as the classifier. The performance evaluation through different ablation studies on testing accuracy explicitly proves that ResNet152 - ELM hybrid architecture is not only the best among the selected transfer learning models but also proves so when it is compared with several other classifiers used for the face mask detection operation. Through this investigation, novelty of the use of ResNet152 + ELM for face mask detection framework in real time domain is established.

Assessment of back-side activation of titania thin film using a fixed-bed photocatalytic-reactor: Kinetic study, operating cost and ANN modeling

This work is concerned with construction, application and modeling of a thin-film fixed photocatalytic bed multi lamp photo-reactor as an organic pollutant treatment system. Five photocatalytic units were located in a cylindrical vessel with a recycling current. In each unit, a thin layer of titania nanoparticles was coated on the outer surface of a quartz tube which a UV lamp was installed inside it. By this configuration, the UV lamps irradiation can approach directly to the photocatalyst particles before penetration into the reaction media. The effect of initial pH, initial DR16 (as model pollutant) dosage, H2O2 (as auxiliary oxidant) concentration and number of applied activated photocatalytic units were studied. The process was monitored via the kinetic rate constant and operating cost factors. The photocatalytic degradation of DR16 molecules in the constructed multi lamp photo-reactor followed a pseudo-first-order kinetic, and well described by the Langmuir-Hinshelwood equation. Under the initial pH of 3, 150 mg/L of H2O2 and illumination of the five lamps, complete degradation of the target pollutant was achieved after 60 min. Under these conditions, the rate constant and operating cost were 0.078 min−1 and 1.13 US$/m3 respectively. The real photon flux inside the reactor and quantum efficiency of the photocatalytic degradation of DR16, using five turned on lamps, were evaluated as 10.84 × 10−6 (einstein.s−1) and 0.009 respectively Also, the photocatalytic process was well modeled (with mean square error of 16.81 ×10−5) by a three layered feed forward back propagation artificial neural network consists of trainlm as training function, 7 hidden neurons and tansig-tansig as transfer functions calibrated at 156th epoch.

Small-signal Modeling of InP HBT based on PSO-ELM Neural Network

Extreme Learning Machine (ELM) is a new learning algorithm for single-hidden layer feedforward neural network, which has been widely used in lots of fields. However, it still has the insufficiency of randomly determining the hidden layer threshold and output weight, which leads to ill-conditioned output. In order to avoid the risk of decreasing prediction accuracy caused by this possibility, the ELM is optimized using particle swarm algorithm. A Particle Swarm Optimization (PSO) enhanced ELM algorithm is proposed to accurately model the small-signal properties of InP Heterojunction Bipolar Transistors (HBTs). PSOELM algorithm solves the problem of unstable prediction data caused by random determination of input weights in ELM. Comparing the modeling effects of the PSO-ELM model and the ELM model under different bias conditions for a 1 μm×15 μm InP HBT, it is proved that the PSO-ELM algorithm has better consistency with the measured data.