Random Vector Functional Link Research Articles

An Elitist Artificial Electric Field Algorithm Based Random Vector Functional Link Network for Cryptocurrency Prices Forecasting

An Elitist Artificial Electric Field Algorithm Based Random Vector Functional Link Network for Cryptocurrency Prices Forecasting

Evolution-Driven Randomized Graph Convolutional Networks

Randomized neural networks (NNs), such as random vector functional link (RVFL) and extreme learning machine (ELM), have been widely applied in various classification problems owing to their computational efficiency and universal approximation capability. However, such approaches are designed for regular Euclidean data and lack the ability to generalize to complex structured data. Moreover, their randomly generated parameters often lead to a suboptimal decision boundary with a growing requirement of hidden neurons. In this article, we first propose a plain framework, termed randomized graph convolutional networks (RGCNs), to generalize the classic randomized NNs to the non-Euclidean domain. Then, a hybrid framework called evolution-driven RGCN (EvoRGCN) is presented by using adaptive differential evolution with novelty search strategy to seek the globally optimal graph embedding for the plain RGCN. Finally, we recast the classic ELM and RVFL under the proposed frameworks, resulting in four novel semi-supervised models, including the plain models [i.e., graph convolutional extreme learning machines (GCELMs) and graph convolutional RVFL (GCRVFL)] and the optimized models (i.e., O-GCELM and O-GCRVFL). We show that our approaches are the natural generalization of the traditional randomized NNs in the non-Euclidean domain. Furthermore, our approaches not only retain the advantages of the classic approaches but also enable them to handle graph data. We compare our approaches against many existing methods across regular datasets and graph benchmarks, demonstrating that the proposed approaches dramatically outperform the compared methods with better generalization ability and robustness. Particularly, we quantitatively show the performance ranking of different randomized NNs, i.e., O-GCRVFL <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$&gt; $ </tex-math></inline-formula> O-GCELM <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\approx $ </tex-math></inline-formula> GCRVFL <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$&gt; $ </tex-math></inline-formula> GCELM <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\approx $ </tex-math></inline-formula> RVFL <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$&gt; $ </tex-math></inline-formula> ELM.

A Bayesian Approach To Distributed Anomaly Detection In Edge AI Networks

The challenge of anomaly detection is to obtain an accurate understanding of expected behaviour which is intensified when the data are distributed heterogeneously. Transmitting raw data to a central site incurs high communication overhead and raises privacy issues. The concept of Edge AI allows computation to be performed at the edge site allowing for quick decision making in mission critical scenarios such as self-driving cars. A model is learnt locally and its parameters are transmitted and aggregated. However, existing methods of aggregation do not account for variance and heterogeneous distribution of data. They also do not consider edge constraints such as limited computational, memory and communication capabilities of edge devices. In this work, a fully Bayesian approach is employed by means of a Bayesian Random Vector Functional Link AutoEncoder being incorporated with Expectation Propagation for distributed training. Our anomaly detection system operates without any transmission of raw data, is robust under inhomogeneous network densities and under uneven and biased data distributions. It allows for asynchronous updates to converge in a few iterations and is a relatively simple neural network addressing edge constraints without compromising on performance as compared to existing more complex models.

An Effective Online Sequential Stochastic Configuration Algorithm for Neural Networks

Random Vector Functional-link (RVFL) networks, as a class of random learner models, have received careful attention from the neural network research community due to their advantages in obtaining fast learning algorithms and models, in which the hidden layer parameters are randomly generated and remain fixed during the training phase. However, its universal approximation ability may not be guaranteed if the random parameters are not properly selected in an appropriate range. Moreover, the resulting random learner’s generalization performance may seriously deteriorate once the RVFL network’s structure is not well-designed. Stochastic configuration (SC) algorithm, which incrementally constructs a universal approximator by obtaining random hidden parameters under a specified supervisory mechanism, instead of fixing the selection scope in advance and without any reference to training information, can effectively circumvent these awkward issues caused by randomness. This paper extends the SC algorithm to an online sequential version, termed as an OSSC algorithm, by means of recursive least square (RLS) technique, aiming to copy with modeling tasks where training observations are sequentially provided. Compared to the online sequential learning of RVFL networks (OS-RVFL in short), our proposed OSSC algorithm can avoid the awkward setting of certain unreasonable range for the random parameters, and can also successfully build a random learner with preferable learning and generalization capabilities. The experimental study has shown the effectiveness and advantages of our OSSC algorithm.

Error-output recurrent multi-layer Kernel Reservoir Network for electricity load time series forecasting

Electricity is one of the most consumed commodities in the modern world. Electricity load prediction models are used to plan distribution operations to balance the equilibrium of demand and supply. This necessity has increased the number of recent research works. They employed several learning algorithms, such as support vector regression, to predict demands. However, these algorithms have high computational cost and too many user-defined parameters that directly impact their performance. Recently, randomization-based learning algorithms have been widely tested because they performed well at a lower cost. However, still, there was a main drawback: uncertainty in approximation and learning. This work employed a kernel trick to solve the uncertainty problem. A kernel with reservoir-state layers was used to solve the problem. The kernel reservoir-state layers from the echo state network not only transformed features into high-dimensional space, but also enhanced the forecasting ability by learning temporal information. Additionally, the proposed model also had a multi-step prediction ability that used previous forecasting errors to update the output weights in the current step to prevent an accumulated error problem. We compared our proposed model with single-layer and multi-layer variants of Extreme Learning Machine, Echo State Network, and Random Vector Functional Link on ten electrical load data sets. The proposed model showed the best performance on 9/10 data sets in terms of Mean Square Error or Symmetric Mean Absolute Percentage Error. These findings implied that the proposed algorithm was superior in forecasting long-term electricity load.

Spatio Prediction of Soil Capability Modeled with Modified RVFL Using Aptenodytes Forsteri Optimization and Digital Soil Assessment Technique

To meet the needs of Egypt’s rising population, more land must be cultivated. Land evaluation is vital to achieving sustainable agricultural production. To determine the soil capability in the northeast Nile Delta region of Egypt, the present study introduces a new form of integration between the Agriculture Land Evaluation System (ALES Arid) model and the machine learning (ML) approach. The soil capability indicators required for the ALES Arid model were determined for the 47 collected soil profiles covering the study area. These indicators include soil pH, soil salinity, the sodium adsorption ratio (SAR), the exchangeable sodium percentage (ESP), the organic matter (OM) content, the calcium carbonate (CaCO3) content, the gypsum content, the clay percentage, and the slope. The ALES Arid model was run using these indicators, and soil capability indexes were obtained. Using GIS, these indexes helped to classify the study area into four capability classes, ranging from good to very poor soils. To predict the soil capability, three machine learning algorithms named traditional RVFL, sine cosine algorithm (SCA), and AFO were also applied to the same soil criteria. The developed ML method aims to enhance the prediction of soil capability. This method depends on improving the performance of Random Vector Functional Link (RVFL) using an optimization technique named Aptenodytes Forsteri Optimization (AFO). The operators of AFO were used to determine the best parameters of RVFL since traditional RVFL is sensitive to parameters. To assess the performance of the developed AFO-RVFL method, a set of real collected data was used. The experimental results illustrate the high efficacy of AFO-RVFL in the spatial prediction of soil capability. The correlations found in this study are critical for understanding the overall techniques for predicting soil capability.

Graph ensemble deep random vector functional link network for traffic forecasting

Traffic forecasting is crucial to achieving a smart city as it facilitates public transportation management, autonomous driving, and the resource relocation of the sharing economy. Traffic forecasting belongs to the challenging spatiotemporal forecasting task, which is highly demanding because of the complicated geospatial correlation between traffic nodes, inconsistent and highly non-linear temporal patterns due to various events, and sporadic traffic accidents. Previous graph neural network (GNN) models built for transportation forecasting feature the sophisticated structure and heavy computation cost as they combine the deep neural network and graph machine learning to capture the spatiotemporal dynamics for the whole transportation network. However, it may be more practical for practitioners to perform node-wise forecasting for specific nodes of interest rather than network-wise forecasting. To mitigate the gaps mentioned above, we propose a novel graph ensemble deep random vector functional link network (GEdRVFL) to forecast the future traffic volume by combining the well-performing ensemble deep random vector functional link (EdRVFL) with the graph convolution layer for a specific node and realize the node-wise traffic forecasting. After a comprehensive comparison with the state-of-the-art models, our model beats the others in four out of five cases measured by mean absolute scaled error.

Random vector functional link forests and extreme learning forests applied to UAV automatic target recognition

This paper proposes two novel machine learning algorithms, namely Random Vector Functional Link Forests and Extreme Learning Forests, to develop an improved unmanned aerial vehicles automatic target recognition system. Such models take advantage of the stochastic procedure followed by Random Forests, where random subsets of instances and features are selected to build diverse Decision Trees. However, different from the usual uni-variate split criterion from Decision Tree algorithms, we propose and employ the novel Random Vector Functional Link Tree or Extreme Learning Tree, where each decision split is performed using the fast non-linear mapping of multiple features provided by either Random Vector Functional Link or Extreme Learning Machines. To prove the efficacy of the novel algorithm, experiments are performed using 90 binary classification problems to compare the performance of the proposed algorithm against other state-of-the-art ensemble learning techniques. Statistical analysis indicates the success of the proposed algorithms in terms of both predictive performance and computational complexity. While the model with deeper trees outperforms classical ensembles in terms of predictive performance (1.41% error reduction) and has similar results to state-of-the-art ensemble models, the model with shallow trees outperforms all ensembles in terms of computational burden (at least 36% faster). Finally, the novel methods are applied to develop an automatic target recognition system for unmanned aerial vehicles, achieving a valuable trade-off in terms of area under receiver operating characteristic curve (0.9309), F1 score (0.8190), accuracy (0.8646), and computational time (4.14 s).

Significant wave height forecasting using hybrid ensemble deep randomized networks with neurons pruning

The reliable control of wave energy devices highly relies on the forecasts of wave heights. However, the dynamic characteristics and significant fluctuation of waves’ historical data pose challenges to precise predictions. Neural networks offer a promising solution to forecast the wave heights by extracting meaningful features from historical observations. This paper proposes a novel hybrid random vector functional link network with the ensemble and deep learning benefits. Hierarchical stacks of hidden layers are constructed to enforce the deep representations of the time series. Individual output layers follow all enhancement layers to adopt ensemble learning. A neuron pruning strategy is proposed to remove the noisy information from the random features and boost the network’s performance. Besides, the proposed network is further utilized to forecast the additive and multiplicative residuals from the ARIMA method. Finally, the ensemble of additive-ARIMA-edRVFL, multiplicative-ARIMA-edRVFL, and edRVFL achieves the best average rankings around two for three forecasting horizons. The proposed ensemble achieves an average ranking of 1.33 on four-hours ahead of forecasting in terms of root mean square error and mean absolute scaled error. Extensive experiments are conducted on twelve time series of the significant wave height. The comparative results demonstrate the superiority of the proposed model over other state-of-the-art methods. The source codes are available on https://github.com/P-N-Suganthan/CODES.

Distributed random vector functional link network with subspace-based local connections

In order to solve the problem of poor generalization ability and high computational complexity of random vector functional link (RVFL) network when dealing with large-scale data classification, we design and implement a distributed RVFL network with subspace-based local connections in Spark framework (DRVFL-SLC). Firstly, in order to take advantage of the partition parallelism of resilient distributed dataset (RDD), the large-scale dataset stored in the Hadoop distributed file system HDFS is randomly divided into random sample partition (RSP) data blocks and each RSP data block corresponds to a partition of the RDD, where the RSP data block is a subset of data that maintains probability distribution consistency with the big data at a given significance level. After that, the mapPartitions transformation is invoked on the RDD containing multiple partitions in a distributed environment and this operation trains the corresponding optimal RVFL-SLC efficiently in parallel. Then, the collect execution operator is used to efficiently fuse the optimal RVFL-SLC corresponding to each partition of the RDD to obtain DRVFL-SLC for realizing the classification of big data. Finally, the feasibility and effectiveness of DRVFL-SLC are verified based on several large-scale data set with at least million records on a Spark cluster deployed with 6 computing nodes. The results show that DRVFL-SLC has a good speedup ratio, scalability and scale growth, and can achieve better generalization performance than RVFL-SLC trained on a single machine with full data.

Evolving convolution neural network by optimal regularization random vector functional link for computational color constancy

To automatically eliminate the influence of external illumination on the acquired image, restore the true color information of the object, and provide true and stable color features for computer vision tasks, our work is based on the convolutional neural network (CNN) VGG19 for feature extraction of image information. Moreover, it proposes a model based on atom search optimization improved by chaotic-logistic maps to optimize the regularization random vector functional link (RRVFL). The optimized RRVFL replaces CNN’s regression layer to estimate the image illumination and then restore the image. First, we used chaotic-logistic maps to optimize the initial value of search agent of the atom search optimization (CASO) algorithm and then used the search agent segment mapping method proposed in our study to simultaneously optimize the number of nodes in hidden layer, regularization factors, input weight, and the bias of RRVFL hidden layer. This avoids the problem of serious fluctuation of prediction results and low accuracy caused by the randomness of the RRVFL parameters. After the predicted illumination information was obtained with the optimized RRVFL, the image was restored using the diagonal transformation method. Comparative experiments showed that the average angular errors of the illumination estimation of the V19-CASO-RRVFL algorithm proposed in our study were 1.73, 0.30, 0.3406, and 0.0976 lower than those of the gray-edge-2, GE-CASO-RRVFL, DS-Net, and color constancy on deep residual learning algorithms, respectively.

Reservoir Prediction Model via the Fusion of Optimized Long Short-Term Memory Network (LSTM) and Bidirectional Random Vector Functional Link (RVFL)

An accurate and stable reservoir prediction model is essential for oil location and production. We propose an predictive hybrid model ILSTM-BRVFL based on an improved long short-term memory network (IAOS-LSTM) and a bidirectional random vector functional link (Bidirectional-RVFL) for this problem. Firstly, the Atomic Orbit Search algorithm (AOS) is used to perform collective optimization of the parameters to improve the stability and accuracy of the LSTM model for high-dimensional feature extraction. At the same time, there is still room to improve the optimization capability of the AOS. Therefore, an improvement scheme to further enhance the optimization capability is proposed. Then, the LSTM-extracted high-dimensional features are fed into the random vector functional link (RVFL) to improve the prediction of high-dimensional features by the RVFL, which is modified as the bidirectional RVFL. The proposed ILSTM-BRVFL (IAOS) model achieves an average prediction accuracy of 95.28%, compared to the experimental results. The model’s accuracy, recall values, and F1 values also showed good performance, and the prediction ability achieved the expected results. The comparative analysis and the degree of improvement in the model results show that the high-dimensional extraction of the input data by LSTM is the most significant improvement in prediction accuracy. Secondly, it introduces a double-ended mechanism for IAOS to LSTM and RVFL for parameter search.

Detection of mild cognitive impairment in type 2 diabetes mellitus based on machine learning using privileged information

Type 2 diabetes mellitus (T2DM) patients may develop into mild cognitive impairment (MCI) or even dementia. However, there is lack of reliable machine learning model for detection MCI in T2DM patients based on machine learning method. In addition, the brain network changes associated with MCI have not been studied. The aim of this study is to develop a machine learning based algorithm to help detect MCI in T2DM. There are 164 participants were included in this study. They were divided into T2DM-MCI (n = 56), T2DM-nonMCI (n = 49), and normal controls (n = 59) according to the neuropsychological evaluation. Functional connectivity of each participant was constructed based on resting-state magnetic resonance imaging (rs-fMRI). Feature selection was used to reduce the feature dimension. Then the selected features were set into the cascaded multi-column random vector functional link network (RVFL) classifier model using privileged information. Finally, the optimal model was trained and the classification performance was obtained using the testing data. The results show that the proposed algorithm has outstanding performance compared with classic methods. The classification accuracy of 73.18 % (T2DM-MCI vs NC) and 79.42 % (T2DM-MCI vs T2DM-nonMCI) were achieved. The functional connectivity related to T2DM-MCI mainly distribute in the frontal lobe, temporal lobe, and central region (motor cortex), which could be used as neuroimaging biomarkers to recognize MCI in T2DM patients. This study provides a machine learning model for diagnosis of MCI in T2DM patients and has potential clinical significance for timely intervention and treatment to delay the development of MCI.

Rock physics analysis from predicted Poisson's ratio using RVFL based on Wild Geese Algorithm in scarab gas field in WDDM concession, Egypt

Some of the important rock physics parameters, such as the shear-wave velocity and Poisson's ratio, are conventionally calculated from compressional and shear sonic well logs. Although these parameters are vital for geomechanical purposes, these types of shear sonic logs are rarely recorded for most wells. Therefore, this study aims to use ordinary well log and seismic data to predict the Poisson's ratio using some of the machine learning algorithms that are based on a proposed model calculated from a modified version of the Random Vector Functional Link (RVFL) using the Wild Geese Algorithm (WGA). This is applied as a case study in the Scarab gas field in the West Delta Deep Marine (WDDM) concession, Egypt. The main aim of using WGA is to determine the best configuration from the parameters of RVFL to enhance the process of prediction. The rock physics templates are used for interpreting the lithology and pore-fluid from well log data and RVFL-WGA. This is achieved using the cross-plot of P-impedance versus Poisson's ratio, Lambda-Rho versus Mu-Rho, Poisson's ratio versus bulk modulus and P-impedance versus Vp/Vs ratio from both methods. All cross plots are color-coded by the shale volume and hydrocarbon saturation.

Performance Enhancement of the Unbalanced Text Classification Problem Through a Modified Chi Square-Based Feature Selection Technique

This paper proposes a modified chi square-based feature selection algorithm in conjunction with a random vector functional link network-based text classifier for improving the classification performance of multi-labeled text documents with unbalanced class distributions. In the proposed feature selection method, maximum features are selected from classes that have a great deal of training and testing documents as an improvement towards original chi-square method. On two benchmark datasets that are multi-labeled, multi-class, and unbalanced, a comparison of the model with three conventional selection techniques such as chi-square, term frequency-inverse document frequency, and mutual information is accumulated for assessing its effectiveness. Additionally, the proposed model is compared with four different classifiers. In the study, it was found that the proposed model performs better in terms of precision, recall, f-measure, and hamming losses and is able to select the majority of true positive documents despite an unbalanced class distribution for both the datasets.

EEG-based emotion recognition using random Convolutional Neural Networks

Emotion recognition based on electroencephalogram (EEG) signals is helpful in various fields, including medical healthcare. One possible medical application is to diagnose emotional disorders in patients. Humans tend to work and communicate efficiently when in a good mood. On the other hand, negative emotions can harm physical and mental health. Traditional EEG-based methods usually extract time-domain and frequency-domain features before classifying them. Convolutional Neural Networks (CNN) enables us to extract features and classify them end-to-end. However, most CNN methods use backpropagation to train their models, which can be computationally expensive, primarily when a complex model is used. Inspired by the successes of Random Vector Functional Link and Convolutional Random Vector Functional Link, we propose using a randomized CNN model for emotion recognition that removes the need for a backpropagation method. Also, we expand our randomized CNN method to a deep and ensemble version to improve emotion recognition performance. We do experiments on the commonly used publicly available Database for Emotion Analysis using the Physiological Signals (DEAP) dataset to evaluate our randomized CNN models. Results on the DEAP dataset show our models outperform all other models, with at least 95% accuracy for all subjects. Our ensemble version outperforms our shallow version, winning the shallow version in most subjects.

Random vector functional link network with L21 norm regularization for robot visual servo control with feature constraint

Random vector functional link network with L21 norm regularization for robot visual servo control with feature constraint

Do EEMD based decomposition-ensemble models indeed improve prediction for crude oil futures prices?

The ensemble empirical mode decomposition (EEMD) based decomposition-ensemble models are widely applied to crude oil futures prices prediction. However, we argue whether EEMD really improves the prediction as the one-time decomposition would cause the exposure of future features, which may result in a misleadingly high accuracy. Therefore, a sliding decomposition-ensemble paradigm SW-EEMD-RVFL is proposed, which conducts the decomposition only of historical series in the sliding window for each forecasting iteration. The random vector functional link (RVFL) neural network is employed as the forecasting approach. The Brent and West Texas Intermediate (WTI) crude oil futures are taken to verify the model by comprehensively comparing individual econometric, artificial intelligence models and their hybrid forms. The findings show that, although hybrid models perform well in the in-sample data, SW-EEMD based models cannot outperform neither the EEMD based models or individual models in the out-of-sample data. EEMD cannot improve the prediction when only based on the decomposition of historical series, which is not consistent with most researches. Except only focusing on the improving accuracy, this work also gives possible explanations of the forecasting results from both methodologies and theories, as well as demonstrates the Efficient Markets Hypothesis (EMH) in terms of artificial intelligence methods.

Tensor based stacked fuzzy neural network for efficient data regression.

Random vector functional link and extreme learning machine have been extended by the type-2 fuzzy sets with vector stacked methods, this extension leads to a new way to use tensor to construct learning structure for the type-2 fuzzy sets-based learning framework. In this paper, type-2 fuzzy sets-based random vector functional link, type-2 fuzzy sets-based extreme learning machine and Tikhonov-regularized extreme learning machine are fused into one network, a tensor way of stacking data is used to incorporate the nonlinear mappings when using type-2 fuzzy sets. In this way, the network could learn the sub-structure by three sub-structures’ algorithms, which are merged into one tensor structure via the type-2 fuzzy mapping results. To the stacked single fuzzy neural network, the consequent part parameters learning is implemented by unfolding tensor-based matrix regression. The newly proposed stacked single fuzzy neural network shows a new way to design the hybrid fuzzy neural network with the higher order fuzzy sets and higher order data structure. The effective of the proposed stacked single fuzzy neural network are verified by the classical testing benchmarks and several statistical testing methods.

An accuracy-maximization learning framework for supervised and semi-supervised imbalanced data

While we attempt to develop the balanced error rate (BER) minimization learning framework for randomized learning of feedforward neural networks to deal with imbalanced datasets, it remains unclear whether the BER minimization learning framework can be effectively extended into its semi-supervised version. This paper proposes a new concept of accuracy maximization for randomized learning methods on imbalanced datasets for the first time, and theoretically proves that it is equivalent to the minimization of the generalized BER for the use of the selected neural networks. In particular, accuracy maximization can be easily extended to semi-supervised scenarios as its semi-supervised version is proved to be linearly dependent on its original. In this paper, based on the proposed accuracy maximization concept, we propose an accuracy-maximization learning framework, and further develop a new accuracy-maximization extreme learning machine AMELM by taking Extreme Learning Machine (ELM) as a typical randomized learning method for feedforward neural networks so as to handle challenging data issues such as the class imbalance and label scarcity. It is worth noting that the proposed accuracy maximization based framework is not only suitable for ELM, but can be extended to different randomized learning methods, such as Random Vector Functional Link Network (RVFL), and Schmidt Neural Network (SNN) for supervised and semi-supervised imbalanced data. The efficacy of AMELM is tested on extensive benchmark datasets. Experimental results show that AMELM can achieve satisfactory performances on labeled or partially labeled imbalanced data. Also, AMELM obtains at least comparable classification performance to other baseline methods yet has fewer hyperparameters to tune, showing its potential for practical applications.