Functional Link Neural Network Research Articles

Müntz–Legendre neural network construction for solving delay optimal control problems of fractional order with equality and inequality constraints

In this paper, an artificial intelligence approach using neural networks is described to solve a class of delay optimal control problems of fractional order with equality and inequality constraints. In the proposed method, a functional link neural network based on the Muntz–Legendre polynomial is developed. The problem is first transformed into an equivalent problem with a fractional dynamical system without delay, using a Pade approximation. According to the Pontryagin’s minimum principle for optimal control problems of fractional order and by constructing an error function, the authors then define an unconstrained minimization problem. The authors use trial solutions for the states, Lagrange multipliers and control functions where these trial solutions are constructed by a single-layer Muntz–Legendre neural network model. The authors then exploit an unconstrained optimization scheme for adjusting the network parameters (weights and bias) and to minimize the computed error function. Some numerical examples are given to illustrate the effectiveness of the proposed method.

Stacked autoencoder based deep random vector functional link neural network for classification

Extreme learning machine (ELM), which can be viewed as a variant of Random Vector Functional Link (RVFL) network without the input–output direct connections, has been extensively used to create multi-layer (deep) neural networks. Such networks employ randomization based autoencoders (AE) for unsupervised feature extraction followed by an ELM classifier for final decision making. Each randomization based AE acts as an independent feature extractor and a deep network is obtained by stacking several such AEs. Inspired by the better performance of RVFL over ELM, in this paper, we propose several deep RVFL variants by utilizing the framework of stacked autoencoders. Specifically, we introduce direct connections (feature reuse) from preceding layers to the fore layers of the network as in the original RVFL network. Such connections help to regularize the randomization and also reduce the model complexity. Furthermore, we also introduce denoising criterion, recovering clean inputs from their corrupted versions, in the autoencoders to achieve better higher level representations than the ordinary autoencoders. Extensive experiments on several classification datasets show that our proposed deep networks achieve overall better and faster generalization than the other relevant state-of-the-art deep neural networks.

Price Prediction for Agricultural Commodities in Bandung Regency Based on Functional Link Neural Network and Artifical Bee Colony Algorithms

In Indonesia, fluctuating agricultural commodity prices often impacts society negatively. In this study, farmers in Bandung Regency, West Java, Indonesia, were chosen as a case study. Fluctuating agricultural commodity prices can lead to farmers suffering losses due to the sale price being smaller or equal to the cost of planting. Price is influenced by crop productivity, while planting productivity is strongly influenced by weather. A system is developed in this study to predict the price of agricultural commodities based on price, productivity and weather history using a Functional Link Neural Network (FLNN) algorithm optimized with the Artificial Bee Colony (ABC) algorithm. The price prediction results can be used as recommendations for farmers as to whether they should plant or not. In addition, the prediction results are compared to the Artificial Neural Network (ANN) algorithm with Backpropagation algorithm as the learning algorithm. From the experimental result, the best Mean Absolute Percentage Error (MAPE) value was obtained with FLNN-ABC: 7.68% for the predicted price of chili and 10.59% for the predicted price of onion.

Fuzzy Clustering in Cascaded Feature Space

The success of fuzzy clustering heavily relies on the proper feature space constructed by the input data. For nonspherical and overlapped clusters, kernel fuzzy clustering is more effective because it finds more proper feature space compared to conventional fuzzy clustering. Unfortunately, poor scalability of kernel fuzzy clustering is induced by the construction of a kernel matrix. To solve the problem, random feature-based method was presented to approximate the kernel function. More interestingly, these features exposed in the approximate feature space are directly manipulable. Inspired by the architecture of functional-link neural network, to exploit more information from both the original data space and the approximate kernel space, a new feature space called cascaded feature space (CF) is constructed in this paper. By performing classical fuzzy c-means (FCM) in CF space, a new fuzzy clustering framework called FCM-CF is developed. To reduce computational complexity and perform FCM in CF space well, dimension reduction methods are adopted to generate two variants of FCM-CF. The experiment results of our proposed algorithms verify their superiority in comparison of other classical fuzzy clustering algorithms.

Utilizing hybrid functional fuzzy wavelet neural networks with a teaching learning-based optimization algorithm for medical disease diagnosis

Accurate medical disease diagnosis is considered to be an important classification problem. The main goal of the classification process is to determine the class to which a certain pattern belongs. In this article, a new classification technique based on a combination of The Teaching Learning-Based Optimization (TLBO) algorithm and Fuzzy Wavelet Neural Network (FWNN) with Functional Link Neural Network (FLNN) is proposed. In addition, the TLBO algorithm is utilized for training the new hybrid Functional Fuzzy Wavelet Neural Network (FFWNN) and optimizing the learning parameters, which are weights, dilation and translation. To evaluate the performance of the proposed method, five standard medical datasets were used: Breast Cancer, Heart Disease, Hepatitis, Pima-Indian diabetes and Appendicitis. The efficiency of the proposed method is evaluated using 5-fold cross-validation and 10-fold cross-validation in terms of mean square error (MSE), classification accuracy, running time, sensitivity, specificity and kappa. The experimental results show that the efficiency of the proposed method for the medical classification problems is 98.309%, 91.1%, 91.39%, 88.67% and 93.51% for the Breast Cancer, Heart Disease, Hepatitis, Pima-Indian diabetes and Appendicitis datasets, respectively, in terms of accuracy after 30 runs for each dataset with low computational complexity. In addition, it has been observed that the proposed method has efficient performance compared with the performance of other methods found in the related previous studies.

A new genetically optimized tensor product functional link neural network: an application to the daily exchange rate forecasting

The training speed for multilayer neural networks is slow due to the multilayering. Therefore, removing the hidden layers, provided that the input layer is endowed with additional higher order units is suggested to avoid such problem. Tensor product functional link neural network (TPFLNN) is a single layer with higher order terms that extend the network’s structure by introducing supplementary inputs to the network (i.e., joint activations). Although the structure of the TPFLNN is simple, it suffers from weight combinatorial explosion problem when its order becomes excessively high. Furthermore, similarly to many neural network methods, selection of proper weights is one of the most challenging issues in the TPFLNN. Finding suitable weights could help to reduce the number of needed weights. Therefore, in this study, the genetic algorithm (GA) was used to find near-optimum weights for the TPFLNN. The proposed method is abbreviated as GA–TPFLNN. The GA–TPFLNN was used to forecast the daily exchange rate for the Euro/US Dollar, and Japanese Yen/US Dollar. Simulation results showed that the GA–TPFLNN produced more accurate forecasts as compared to the standard TPFLNN, GA, GA–TPFLNN with backpropagation, GA-functional expansion FLNN, multilayer perceptron, support vector regression, random forests for regression, and naive methods. The GA helps the TPFLNN to find low complexity network structure and/or near-optimum parameters which leads to this better result.

Short-term solar power prediction using multi-kernel-based random vector functional link with water cycle algorithm-based parameter optimization

A new hybrid model combining the kernel functions along with the random vector functional link neural network (RVFLN) is proposed in this paper for an effective solar power prediction. The conventional RVFLN is known for its fast learning speed, simple architecture and good generalization capabilities and allows direct connection between input and output nodes along with nonlinear enhancement nodes with random weights. However, the bottleneck of selecting the number of hidden enhancement nodes and mapping functions is still a challenging problem. To overcome these deficiencies of the conventional RVFLN, kernel functions are used for both the direct links and the hidden nodes to provide better stability, generalization and regression accuracy. Instead of using a single kernel for the enhancement nodes, this paper proposes an optimal kernel function that comprises a linear combination of weighted local kernel and a global kernel to improve the prediction accuracy of the solar power generation. This optimal kernel will be known as multi-kernel RVFLN (MK-RVFLN), and its parameters are optimized using an efficient metaheuristic evaporation-based water cycle (EVWCA-MKRVFLN) to provide accurate prediction of solar power. To validate its superior prediction accuracy, two solar power plants of 25 and 100 MW capacity in the states of New York and California are considered for 5-min- and 60-min-ahead prediction in the months of January, April, July and October. The result analysis shows that the MK-RVFLN algorithm attains better performance than many other techniques.

Intelligent Secure Ecosystem Based on Metaheuristic and Functional Link Neural Network for Edge of Things

Internet of Things (IoT) has evolved for building smart environments in a distributed system, where the data produced by IoT devices are transmitted through Edge computing devices to streamline the flow of traffic from IoT devices to a distributed network. In such a scenario, the attacker introduces many attacks to the edge before forwarding them to distributed servers. This necessitates intrusion detection systems for such environments to mitigate security attacks. This paper has projected a basis for characterization of intrusive behaviors in a distributed system based on the functional link neural nets response weighted-average and teaching–learning metaheuristic with elitism on weight-space. The proposed technique makes use of teaching–learning metaheuristic optimization to obtain suitable parameters for the functional link neural net. Furthermore, the processing of duplicate parameters is successfully avoided by using mutation operation. In addition to this, in this paper the proposed method is found to be more efficient in terms of computational burden.

Optimization of sampling intervals for tracking control of nonlinear systems: A game theoretic approach

This paper presents a near optimal adaptive event-based sampling scheme for tracking control of an affine nonlinear continuous-time system. A zero-sum game approach is proposed by introducing a novel performance index. The optimal value function, i.e., the solution to the associatedHamilton–Jacobi–Issac (HJI) equation is approximated using a functional link neural network (FLNN) with event-based aperiodic state feedback information as inputs. The saddle point approximated optimal solution is employed to design the near optimal event-based control policy and the sampling condition. An impulsive weight update scheme is designed to guarantee local ultimate boundedness of the closed-loop parameters, which is analyzed via extension of Lyapunov stability theory for the impulsive hybrid dynamical systems. Zeno-freeness of the event-sampling scheme is enforced and its effect on stability is analyzed. Finally, numerical simulation results are included to corroborate the analytical design, which shows a 48.82% reduction of feedback communication and computational load.

A SARIMA-RVFL hybrid model assisted by wavelet decomposition for very short-term solar PV power generation forecast

A very short-term solar PV power generation forecast can be extremely helpful for real-time balancing operation in an electricity market which in turn will profit both energy suppliers as well as customers. However, the intermittency of solar PV power introduces inaccuracies in its forecast. To address this challenge, the research paper has studied the effect of wavelet decomposition of solar PV power time series on its forecast. A novel and time adaptive, Seasonal Autoregressive Integrated Moving Average (SARIMA)-Random Vector Functional Link (RVFL) neural network hybrid model assisted by Maximum Overlap Discrete Wavelet Transform (MODWT) has been proposed. The solar PV power generation data obtained from roof-top solar PV plants installed at IIT Gandhinagar is used to develop and validate the forecast models. Various numerical forecast accuracy measures have been calculated which show an improvement in accuracy and adaptability of proposed forecast model over constituent models.

Short-term mixed electricity demand and price forecasting using adaptive autoregressive moving average and functional link neural network

A new hybrid adaptive autoregressive moving average (ARMA) and functional link neural network (FLNN) trained by adaptive cubature Kalman filter (ACKF) is presented in this paper for forecasting day-ahead mixed short-term demand and electricity prices in smart grids. The hybrid forecasting framework is intended to capture the dynamic interaction between the electricity consumers and the forecasted prices resulting in the shift of demand curve in electricity market. The proposed model comprises a linear ARMA-FLNN obtained by using a nonlinear expansion of the weighted inputs. The nonlinear functional block helps introduce nonlinearity by expanding the input space to higher dimensional space through basis functions. To train the ARMA-FLNN, an ACKF is used to obtain faster convergence and higher forecasting accuracy. The proposed method is tested on several electricity markets, and the performance metrics such as the mean average percentage error (MAPE), and error variance are compared with other forecasting methods, indicating the improved accuracy of the approach and its suitability for a real-time forecasting.

Functional-Link Neural Network for Nonlinear Equalizer in Coherent Optical Fiber Communications

We propose and experimentally demonstrate a simple nonlinear equalizer based on functional-link neural network (FLNN). The nonlinear stochastic mapping enables FLNN to serve as a nonlinear network, so we construct an FLNN with the signals from the two polarizations and the mapped features as input to combat the fiber nonlinearity in coherent optical transmission systems. The FLNN can use the Moore-Penrose generalized inverse or the ridge regression to solve the weights, which can speed up the training process, and avoid the iterative and time-consuming training process that exist universally in most of the deep neural networks. We also extend the FLNN to the multi-channel transmissions. All of the received signals from different channels are stretched as the input and then we use a joint FLNN to extract features and equalize the nonlinear distortions. We conduct simulations and experiments to verify the proposed scheme. In the simulation and experiment, we transmit a 128 Gb/s polarization division multiplexed 16-QAM (PDM-16-QAM) signal over 1000-km and 600-km standard single mode fiber (SSMF), respectively. Both the simulation and experimental results show that the FLNN has similar performance as deep neural network (DNN), which can improve the transmission performance in the nonlinear region. Moreover, the FLNN can avoid the gradient dissipation and local minimum problems in DNN, which simplify the training process. We also extend the proposed scheme in a five-channel ( $5\times160$ Gb/s) multiplexed transmission system. In simulation, we use joint FLNN and joint DNN to compensate the nonlinear distortions, respectively. We find that the BERs of the five channels can be below 7% HD-FEC with nonlinear equalizer.

Adaptive RCMAC Neural Network Dynamic Surface Control for Permanent-Magnet Synchronous Motors Driven Two-Axis X-Y Table

In this paper, a robust adaptive dynamic surface control (RADSC) is proposed with a recurrent cerebellar model articulation controller (RCMAC) based on the function link neural network (FLNN) for two-axis X-Y table derived by two permanent-magnet synchronous motors (PMSMs) servo drives. Initially, an optimal computed torque controller (OCTC) is used to ensure the stability of the two-axis X-Y table system. But there may be destroyed in the control performance due to the presence of parameter uncertainties. This might be due to the fact that linear optimal control possesses inherent robustness within a specified spectrum of model uncertainties. Due to the above reasons, the RADSC is developed according to the system requirements in order to increase the robustness of the control system. In the proposed control scheme, a dynamic surface controller (DSC) and an RCMAC uncertainty observer with a robust controller are combined to constitute the RADSC scheme. In the proposed RADSC, the DSC is utilized to get rid of the complexity of explosion present in the backstepping design. Furthermore, the RCMAC uncertainty observer is developed such that it can approximate the nonlinear parameter uncertainty terms online, whereas the robust controller is designed to recover the residual of the approximation error of the RCMAC. Based on the Lyapunov stability analysis, the online adaptive control laws are derived. The experimental results confirm that the x-axis and y-axis motions are controlled separately, whereas it can be concluded that the proposed RADSC's dynamic behaviors accomplish robust tracking performance though there were parameter uncertainties.

A Novel Hybrid FLANN-PsO Technique for real Time Fingerprint Classification

In this paper we are presenting a Particle swarm optimized functional link neural network for classifying a collection of real time fingerprints in the field of biometric recognition. From the collected fingerprints the feature vectors are extracted as a collection of different angle oriented features using the Gabor filter bank. The classes of the fingerprints are assigned as per the Henry System. For classification a novel FLANN-PSO algorithm is used and tested for accuracy through different parameters and different angular features of the fingerprints. In this work we have obtained an accuracy of 98% for real time collected fingerprint images. It has been compared with other classifiers and the results obtained of this work in terms of accuracy and MSE value has shown appreciable improvement over the other algorithms.

FPGA Implementation of a Functional Neuro-Fuzzy Network for Nonlinear System Control

This study used Xilinx Field Programmable Gate Arrays (FPGAs) to implement a functional neuro-fuzzy network (FNFN) for solving nonlinear control problems. A functional link neural network (FLNN) was used as the consequent part of the proposed FNFN model. This study adopted the linear independent functions and the orthogonal polynomials in a functional expansion of the FLNN. Thus, the design of the FNFN model could improve the control accuracy. The learning algorithm of the FNFN model was divided into structure learning and parameter learning. The entropy measurement was adopted in the structure learning to determine the generated new fuzzy rule, whereas the gradient descent method in the parameter learning was used to adjust the parameters of the membership functions and the weights of the FLNN. In order to obtain high speed operation and real-time application, a very high speed integrated circuit hardware description language (VHDL) was used to design the FNFN controller and was implemented on FPGA. Finally, the experimental results demonstrated that the proposed hardware implementation of the FNFN model confirmed the viability in the temperature control of a water bath and the backing control of a car.

A fresh look at functional link neural network for motor imagery-based brain–computer interface

BackgroundArtificial neural networks (ANNs) are one of the widely used classifiers in the brain–computer interface (BCI) systems-based on noninvasive electroencephalography (EEG) signals. Among the different ANN architectures, the most commonly applied for BCI classifiers is the multilayer perceptron (MLP). When appropriately designed with optimal number of neuron layers and number of neurons per layer, the ANN can act as a universal approximator. However, due to the low signal-to-noise ratio of EEG signal data, overtraining problem may become an inherent issue, causing these universal approximators to fail in real-time applications. New methodIn this study we introduce a higher order neural network, namely the functional link neural network (FLNN) as a classifier for motor imagery (MI)-based BCI systems, to remedy the drawbacks in MLP. ResultsWe compare the proposed method with competing classifiers such as linear decomposition analysis, naïve Bayes, k-nearest neighbours, support vector machine and three MLP architectures. Two multi-class benchmark datasets from the BCI competitions are used. Common spatial pattern algorithm is utilized for feature extraction to build classification models. Comparison with existing method(s)FLNN reports the highest average Kappa value over multiple subjects for both the BCI competition datasets, under similarly preprocessed data and extracted features. Further, statistical comparison results over multiple subjects show that the proposed FLNN classification method yields the best performance among the competing classifiers. ConclusionsFindings from this study imply that the proposed method, which has less computational complexity compared to the MLP, can be implemented effectively in practical MI-based BCI systems.

A novel bilinear functional link neural network filter for nonlinear active noise control

The actual active noise control (ANC) systems often exist some nonlinearity in such cases: (1) the reference signals are chaotic noise; (2) the primary path is nonlinear; (3) the secondary path is nonlinear or non-minimum phase. To solve these problems, the well-known functional link neural network (FLNN) filter using trigonometric expansions is applied successfully in nonlinear ANC (NANC) systems. However, the FLNN filter still has some limitations, for example it does not have the ability to solve the problems of cross terms and acoustical feedback. For modifying the performance of FLNN filter, we propose a new bilinear FLNN (BFLNN) filter combining the trigonometric function expansion, feedback output signals and the products of input signals and feedback output signals. The proposed BFLNN filter is based on a recursive structure which can represent the real nonlinear systems with fewer coefficients than non-recursive models do. Moreover, to some extent the recursive structure can compensate the acoustical feedback between reference microphone and secondary speaker. However, considering the stability of the recursive structure, this paper also offers a sufficient stability condition for the proposed BFLNN filter. Through analysis, the products of input signals and output signals can provide more comprehensive expression for the cross-term problem. To demonstrate the effectiveness of the proposed BFLNN filter, numerous simulation studies considering the problems of cross terms and acoustical feedback in the presence of different reference signals and acoustic paths are conducted in simulation section, and the results prove that the proposed BFLNN filter is more suitable for NANC systems.

Thermal Environment Prediction for Metro Stations Based on an RVFL Neural Network

With the improvement of China’s metro carrying capacity, people in big cities are inclined to travel by metro. The carrying load of these metros is huge during the morning and evening rush hours. Coupled with the increase in numbers of summer tourists, the thermal environmental quality in early metro stations will decline badly. Therefore, it is necessary to analyze the factors that affect the thermal environment in metro stations and establish a thermal environment change model. This will help to support the prediction and analysis of the thermal environment in such limited underground spaces. In order to achieve relatively accurate and rapid on-line modeling, this paper proposes a thermal environment modeling method based on a Random Vector Functional Link Neural Network (RVFLNN). This modeling method has the advantages of fast modeling speed and relatively accurate prediction results. Once the preprocessed data is input into this RVFLNN for training, the metro station thermal environment model will be quickly established. The study results show that the thermal model based on the RVFLNN method can effectively predict the temperature inside the metro station.

Enhanced channel estimation in OFDM systems with neural network technologies

Orthogonal frequency division multiplexing (OFDM) provides an effective and low complexity means of eliminating inter-symbol interference for transmission over frequency selective fading channels. In OFDM systems, channel state information (CSI) is required for the OFDM receiver to perform coherent detection or diversity combining, if multiple transmit and receive antennas are deployed. In practice, CSI can be reliably estimated at the receiver by transmitting pilots along with data symbols. In this paper, we investigate and compare various efficient pilot-based channel estimation schemes by neural network technologies for OFDM systems. We present further the application of functional link neural fuzzy network (FLNFN) for channel estimation in the investigated OFDM systems. We compared bit error rates of the proposed neural network with that of the other neural network technologies, the least square (LS) algorithm, and the minimum mean square error (MMSE) algorithm. Our results demonstrate that the proposed FLNFN algorithm can enhance the performance of channel estimation in existing OFDM channel environments.

Rapid temperature prediction method for electronic equipment cabin

An accurate model is very important for thermal prediction and thermal management of airborne electronic module. Thermal Network Model (TNM) is the commonly used to analyze transient thermal response. It cannot describe the nonlinear or time-varying temperature process very well. In order to realize a fast thermal modeling with a relatively high accuracy, this paper proposes a modeling method based on sliding time window Random Vector Functional Link Neural Network (RVFLNN). The input and output variables of RVFLNN are determined by analyzing the heat transfer relationship of studied system. This method can rapidly realize thermal modeling without a time-consuming iterative training process. In order to overcome the variability of studied system, the sliding time window technology is specially introduced to improve the model prediction accuracy. This method is applied to analyze thermal experimental data of electronic equipment cabin. The temperature prediction performance of presented method is compared with the traditional Artificial Neural Network Model (ANNM). Comparison results show that the proposed modeling method has the advantages of fast modeling and good prediction accuracy. This study can provide an effective way to describe a complex dynamic heat transfer process adaptively and accurately, which may help the thermal control scheme design.