Random Vector Functional Link Network Research Articles

Soil data augmentation and model construction based on spectral difference and content difference

Soil analysis makes for developing precision agriculture and monitoring land quality, while the models available for spectroscopy-based chemometrics are constrained by limited samples from small areas. The paper proposed sample expansion and model construction based on spectral difference and content difference, realizing data augmentation and deep learning applied to original samples with limited numbers. The spectral subtraction based on maximum or minimum values exploited the maximum or minimum values to acquire the spectral difference and content difference, which provided a new data form for model construction. Keeping enhanced samples whose spectral difference and content difference were all zero was useful for improving model performance. Augmentation of all data or training data based on maximum or minimum values-based spectral subtraction, which sorted the contents and made them the maximum or minimum values in sequence, achieved sample expansion by the spectral difference and content difference. The model utilized the random vector functional link (RVFL) network, extreme learning machine (ELM), and one-dimensional convolutional neural network (1D CNN), which could predict the content of new samples through ensemble averaging when predicting content difference. The experimental result showed the model of the spectral subtraction based on maximum or minimum values had a similar performance to that of the original samples. Augmentation of all data improved model performance by only RVFL and ELM. Augmentation of training data verified 1D CNN was better than RVFL and ELM. The paper implements a new data augmentation method and applies CNN to original samples with inadequate numbers, which lays the foundation for an improved model and applying spectral preprocessing.

Study on applicable coverage extension of theory-based generalization errors bounds to the variants of RVFL network and ELM

This study investigates the coverage extendibility of generalized prediction error estimation for the random vector functional link (RVFL) network and generalized extreme learning machine (GELM). GELM is an ELM applying the limiting behavior of Moore–Penrose generalized inverse (M–P GI) to the output matrix of the last hidden layer of ELM. The RVFL network, the ELM, and their variants are currently employed in various areas due to their fast learning and are expected to be adopted in more applications. Additionally, obtaining prediction error bounds quickly for these networks could greatly increase their utilities. Therefore, exploring the error bounds of the variants is crucial. Recently, the upper and lower bounds of generalized prediction error and their associated tail probabilities for the RVFL network and GELM were analyzed. This study aims to extend the coverage of the existing error bounds to variants of RVFL and ELM. To this end, we investigate each solution of the objective functions for the existing RVFL and ELM variants, compare it with the solution obtained by using the limiting behavior of M–P GI, and then discuss the applicability of the probability inequalities for the prediction error to the variant. Specifically, we conceptualize a restricted solution vector and prove that the restricted solution of a variant of RVFL is identical to that of GELM, with an appropriately selected regularization parameter. The mechanism used in the proof can be applied to other variants, which have different objective functions but have similar forms of solution. Experiments are performed with the CAIDA (Center for Applied Internet Data Analysis) dataset. Numerical results verify the analysis, showing that the generalization prediction error bounds and their associated tail probabilities for ELM (RVFL) variants in applicability coverage are almost identical to those of the GELM (RVFL). In addition, performance-based network similarity is introduced and computed with the obtained results. Based on this analysis, we can conclude that the probability inequalities for prediction error derived can be directly applied to some variants of the RVFL network and ELM, such as robust regularized RVFL and ridge regression ELM. However, it cannot apply directly to the networks that use information obtained during the learning process in its learning, such as RVFL+ and its variants. The significance of this study is that it enables us to figure out the relationship between the size-restricted final weight vector of RVFL and the final weight vectors of ELM, the effect of the relationship on generalized prediction error estimation, and the performance-based similarity between networks. This study provides a theoretical basis for the generalized prediction error's upper and lower bounds and their associated tail probabilities for ELM (RVFL) variants.

RDCSAE-RKRVFLN: A unified deep learning framework for robust and accurate DOA estimation

This paper introduces an innovative unified deep learning (DL) model, “reduced deep convolutional stack autoencoder (RDCSAE)-robust kernel-based random vector functional link network (RKRVFLN)” (RDCSAE-RKRVFLN), for addressing practical challenges like phase and gain uncertainty, mutual coupling effects, and element position errors in direction of arrival (DOA) estimation. The RDCSAE utilizes deep convolutional neural network (DCNN) and stack autoencoder (SAE) for robust feature extraction. The RKRVFLN integrates a concise SAE output with kernel-based learning. Thus, the integration of RDCSAE (suitable for unsupervised feature extraction) and RKRVFLN (for supervised learning) in the developed model results in efficient use of limited training data. Performance metrics of this method include among others the accuracy (separation between sources), root mean square error (RMSE), signal-to-noise ratio (SNR), etc. The method performs better than existing data driven methods in terms of computational efficiency, faster learning speed, enhanced model generalization, higher accuracy, and shorter event regression time and empirical methods. In the presence of above mentioned four perturbations, this method successfully resolves multiple sources with a resolution of 0.3° with a RMSE of 0.00376. For real time application, the paper implements RDCSAE-RKRVFLN on a high-speed reconfigurable Xilinx Virtex-5 field-programmable gate array (FPGA) realizing a digital DOA estimator for one source. The hardware estimates the DOA with a maximum error of 0.01953125 which follows closely the simulation results indicating the efficiency and feasibility of the method.

Recurrent ensemble random vector functional link neural network for financial time series forecasting

Financial time series forecasting is crucial in empowering investors to make well-informed decisions, manage risks effectively, and strategically plan their investment activities. However, the non-stationary and non-linear characteristics inherent in time series data pose significant challenges when accurately predicting future forecasts. This paper proposes a novel Recurrent ensemble deep Random Vector Functional Link (RedRVFL) network for financial time series forecasting. The proposed model leverages randomly initialized and fixed weights for the recurrent hidden layers, ensuring stability during training. Furthermore, incorporating stacked hidden layers enables deep representation learning, facilitating the extraction of complex patterns from the data. The proposed model generates the forecast by combining the outputs of each layer through an ensemble approach. A comparative analysis was conducted against several state-of-the-art models over financial time-series datasets, and the results demonstrated the superior performance of our proposed model in terms of forecasting accuracy and predictive capability.

Random vector functional link networks for function approximation on manifolds

The learning speed of feed-forward neural networks is notoriously slow and has presented a bottleneck in deep learning applications for several decades. For instance, gradient-based learning algorithms, which are used extensively to train neural networks, tend to work slowly when all of the network parameters must be iteratively tuned. To counter this, both researchers and practitioners have tried introducing randomness to reduce the learning requirement. Based on the original construction of Igelnik and Pao, single layer neural-networks with random input-to-hidden layer weights and biases have seen success in practice, but the necessary theoretical justification is lacking. In this study, we begin to fill this theoretical gap. We then extend this result to the non-asymptotic setting using a concentration inequality for Monte-Carlo integral approximations. We provide a (corrected) rigorous proof that the Igelnik and Pao construction is a universal approximator for continuous functions on compact domains, with approximation error squared decaying asymptotically like O(1/n) for the number n of network nodes. We then extend this result to the non-asymptotic setting, proving that one can achieve any desired approximation error with high probability provided n is sufficiently large. We further adapt this randomized neural network architecture to approximate functions on smooth, compact submanifolds of Euclidean space, providing theoretical guarantees in both the asymptotic and non-asymptotic forms. Finally, we illustrate our results on manifolds with numerical experiments.

Enhancing short-term wind speed prediction based on an outlier-robust ensemble deep random vector functional link network with AOA-optimized VMD

Wind speed prediction is a crucial aspect in the utilization of wind energy. In this paper, a wind speed prediction model based on an outlier-robust ensemble deep random vector functional link network (ORedRVFL) and arithmetic optimization algorithm-optimized variational mode decomposition (AOA-VMD) is designed. First, the penalty factor and the number of mode decompositions of VMD are optimized using the AOA algorithm and the original data are decomposed using the optimized VMD. Then the decomposed data is predicted using the ensemble deep random vector functional link network (edRVFL) model. The edRVFL uses rich intermediate features for the final decision, which can make the final result closer to the real data. In order to strengthen the anti-interference ability to the outliers, this paper robustly improves the edRVFL model, and the improved model is called ORedRVFL. ORedRVFL reduces the impact of outliers by introducing regularization and norm to balance the relationship between training error and weights. The experiments have proved that the model proposed in this paper outperforms other models in terms of anti-interference ability and prediction accuracy.

Domain-incremental learning without forgetting based on random vector functional link networks

Incremental learning is a paradigm that extends knowledge by learning from new data, often used to add new classes to an existing model or to learn a new domain. It imposes strict limitations on the model’s access to data from previous tasks, making it similar to the human learning process. The main challenge of incremental learning is catastrophic forgetting, where previous knowledge is severely forgotten while learning new tasks. In this work, we propose a novel approach for domain-incremental learning. Inspired by the Normal Equation, we accumulate the Gram Matrix from each task’s hidden layer output to update a simplified RVFL model. This algorithm achieves performance comparable to joint training while strictly adhering to privacy restrictions. With issues such as forgetting, storage requirements and privacy protection be addressed, this algorithm has the potential to play a crucial role in the field of edge computing and other related fields.

Fast and Accurate Short-Term Load Forecasting with a Hybrid Model

Short-term electric load forecasting (STLF) plays a pivotal role in modern power system management, bolstering forecasting accuracy and efficiency. This enhancement assists power utilities in formulating robust operational strategies, consequently fostering economic and social advantages within the systems. Existing methods employed for STLF either exhibit poor forecasting performance or require longer computational time. To address these challenges, this paper introduces a hybrid learning approach comprising variational mode decomposition (VMD) and random vector functional link network (RVFL). The RVFL network, serving as a universal approximator, showcases remarkable accuracy and fast computation, owing to the randomly generated weights connecting input and hidden layers. Additionally, the direct links between hidden and output layers, combined with the availability of a closed-form solution for parameter computation, further contribute to its efficiency. The effectiveness of the proposed VMD-RVFL was assessed using electric load datasets obtained from the Australian Energy Market Operator (AEMO). Moreover, the effectiveness of the proposed method is demonstrated by comparing it with existing benchmark forecasting methods using two performance indices such as root mean square error (RMSE) and mean absolute percentage error (MAPE). As a result, our proposed method requires less computational time and yielded accurate and robust prediction performance when compared with existing methods.

An effective diagnosis of schizophrenia using kernel ridge regression-based optimized RVFL classifier

Schizophrenia (SCZ) is a severe mental and debilitating neuropsychiatric disorder that disrupts a person’s thought processes, emotions, and behavior. Due to misdiagnosis, self-denial, and social stigma, many SCZ cases go untreated. Magnetic resonance imaging (MRI) is an excellent noninvasive tool for soft tissue contrast imaging because it provides crucial data on tissue structure size, position, and shape. The Resnet50 network is a deep residual learning framework used for feature extraction. Random-vector functional link network (RVFL) is an example of a single-hidden-layer feedforward network in which input features and hidden layer features are fed to the output layer. In this paper, we introduced a kernel ridge regression-based random vector functional link (KRR-RVFL) classifier which focuses on addressing the linearity issues in RVFL by designating the kernel function in the input layer for the precise diagnosis of SCZ. The genetic algorithm (GA) seeks to minimize the loss function by optimizing the weights and biases of the KRR-RVFL network. The classification performance is investigated on the SCZ and cognitive normal (CN) subjects, collected from the available open neuro platform, including 99 participants. The results of the suggested network show superior performance to the recent state-of-the-art networks in terms of accuracy 93.66%, sensitivity 92.22%, specificity 95.17%, precision 95.33%, F-measure 93.74%, and G-mean 93.68%. The performance metrics demonstrated the applicability of this framework for assisting clinicians in the automatic, precise evaluation of SCZ.

Event-Triggered Relearning Modeling Method for Stochastic System with Non-Stationary Variable Operating Conditions

This study presents a novel event-triggered relearning framework for neural network modeling, designed to improve prediction precision in dynamic stochastic complex industrial systems under non-stationary and variable conditions. Firstly, a sliding window algorithm combined with entropy is applied to divide the input and output datasets across different operational conditions, establishing clear data boundaries. Following this, the prediction errors derived from the neural network under different operational states are harnessed to define a set of event-triggered relearning criteria. Once these conditions are triggered, the relevant dataset is used to recalibrate the model to the specific operational condition and predict the data under this operating condition. When the predicted data fall within the training input range of a pre-trained model, we switch to that model for immediate prediction. Compared with the conventional BP neural network model and random vector functional-link network, the proposed model can produce a better estimation accuracy and reduce computation costs. Finally, the effectiveness of our proposed method is validated through numerical simulation tests using nonlinear Hammerstein models with Gaussian noise, reflecting complex stochastic industrial processes.

Ship order book forecasting by an ensemble deep parsimonious random vector functional link network

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model’s complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry’s decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-.

Robust noise-aware algorithm for randomized neural network and its convergence properties

The concept of randomized neural networks (RNNs), such as the random vector functional link network (RVFL) and extreme learning machine (ELM), is a widely accepted and efficient network method for constructing single-hidden layer feedforward networks (SLFNs). Due to its exceptional approximation capabilities, RNN is being extensively used in various fields. While the RNN concept has shown great promise, its performance can be unpredictable in imperfect conditions, such as weight noises and outliers. Thus, there is a need to develop more reliable and robust RNN algorithms. To address this issue, this paper proposes a new objective function that addresses the combined effect of weight noise and training data outliers for RVFL networks. Based on the half-quadratic optimization method, we then propose a novel algorithm, named noise-aware RNN (NARNN), to optimize the proposed objective function. The convergence of the NARNN is also theoretically validated. We also discuss the way to use the NARNN for ensemble deep RVFL (edRVFL) networks. Finally, we present an extension of the NARNN to concurrently address weight noise, stuck-at-fault, and outliers. The experimental results demonstrate that the proposed algorithm outperforms a number of state-of-the-art robust RNN algorithms.

Respiratory Motion Prediction with Empirical Mode Decomposition-Based Random Vector Functional Link

The precise prediction of tumor motion for radiotherapy has proven challenging due to the non-stationary nature of respiration-induced motion, frequently accompanied by unpredictable irregularities. Despite the availability of numerous prediction methods for respiratory motion prediction, the prediction errors they generate often suffer from large prediction horizons, intra-trace variabilities, and irregularities. To overcome these challenges, we have employed a hybrid method, which combines empirical mode decomposition (EMD) and random vector functional link (RVFL), referred to as EMD-RVFL. In the initial stage, EMD is used to decompose respiratory motion into interpretable intrinsic mode functions (IMFs) and residue. Subsequently, the RVFL network is trained for each obtained IMF and residue. Finally, the prediction results of all the IMFs and residue are summed up to obtain the final predicted output. We validated this proposed method on the benchmark datasets of 304 respiratory motion traces obtained from 31 patients for various prediction lengths, which are equivalent to the latencies of radiotherapy systems. In direct comparison with existing prediction techniques, our hybrid architecture consistently delivers a robust and highly accurate prediction performance. This proof-of-concept study indicates that the proposed approach is feasible and has the potential to improve the accuracy and effectiveness of radiotherapy treatment.

Alzheimer's Disease Diagnosis via Intuitionistic Fuzzy Random Vector Functional Link Network

Alzheimer's disease (AD) is a prominent neurodegenerative disorder, which leads to memory loss and cognitive impairment. The progression is irreversible and shows atrophies in cerebral cortex. Multiple studies revealed that the early diagnosis and early treatment can slow the progress of dementia, and hence, further atrophies can be controlled. Brain imaging data, such as magnetic resonance imaging (MRI), have been prominently used for the diagnosis of AD. Multiple approaches have been proposed for the diagnosis of AD. We propose a novel intuitionistic fuzzy random vector functional link network (IFRVFL) for the diagnosis of AD. Unlike standard random vector functional link (RVFL) network, extreme learning machine (ELM), and kernel ridge regression (KRR), which uses a uniform weighting approach for generating the optimal classifier, the proposed IFRVFL uses a fuzzy weighted approach for generating the optimal classifier. A uniform weighting scheme assumes that all the data samples are equally important; however, in real-world scenarios, this assumption may not hold true due to the presence of outliers and noise. Hence, it results in lower generalization. The proposed IFRVFL assigns each sample an intuitionistic fuzzy number (IFN), which is a function of membership and nonmembership score of a sample. The membership score is a function of the sample distance from the centroid of its corresponding class and the nonmembership score is a function of sample distance from the centroid as well as the neighborhood of the given sample. The proposed IFRVFL effectively reduces the influence of outliers. To evaluate the efficiency of the proposed IFRVFL model, we employed it for the diagnosis of AD. Experimental results demonstrate that the proposed IFRVFL model is superior in mild cognitive impairment (MCI) versus AD case. Thus, IFRVFL can be used in the clinical setting for the early diagnosis of AD. Furthermore, to check the robustness of the proposed IFRVFL model, we also evaluated it on benchmark datasets. Experimental results and the statistical tests reveal that the proposed IFRVFL is better in comparison to baseline models. The source code of the proposed IFRVFL formulation is available at https://github.com/mtanveer1/.

Extended Features Based Random Vector Functional Link Network for Classification Problem

Random vector functional link (RVFL) network has been successfully employed in diverse domains such as computer vision and machine learning, due to its universal approximation capability. Recently, the shallow RVFL architecture has been extended to deep architectures. In deep architectures, multiple hidden layers are stacked for extracting informative features from the original feature space. Therefore, having rich features, deep models are very successful compared to shallow models. In this article, we propose an extended feature RVFL (efRVFL) model that is trained over extended feature space generated analytically from the original feature space. The proposed efRVFL model has three types of features, i.e., original features, supervised randomized (newly generated) features, and unsupervised randomized features, in its feature matrix. The proposed efRVFL model with additional features has capability to capture nonlinear hidden relationships within the dataset. The proposed efRVFL model is an unstable classifier, and thus, its performance can be improved further via ensemble learning. Ensemble models are stable and accurate and have better generalization performance than single models. Therefore, we also propose an ensemble of extended feature RVFL (en-efRVFL) model. Each base model of en-efRVFL is trained over different feature spaces so that more accurate and diverse base models can be generated. The outcome of the base models is integrated via average voting scheme. Empirical evaluation over <inline-formula> <tex-math notation="LaTeX">$46$</tex-math> </inline-formula> UCI classification datasets demonstrates that the proposed efRVFL and en-efRVFL models have better performance than RVFL and other given deep models. Furthermore, the experimental results over <inline-formula> <tex-math notation="LaTeX">$12$</tex-math> </inline-formula> sparse datasets show that the proposed en-efRVFL model has a winning performance among several deep feedforward neural networks (FNNs).

Fusing Higher and Lower-Order Biological Information for Drug Repositioning via Graph Representation Learning

Drug repositioning is a promising drug development technique to identify new indications for existing drugs. However, existing computational models only make use of lower-order biological information at the level of individual drugs, diseases and their associations, but few of them can take into account higher-order connectivity patterns presented in biological heterogeneous information networks (HINs). In this work, we propose a novel graph representation learning model, namely FuHLDR, for drug repositioning by fusing higher and lower-order biological information. Specifically, given a HIN, FuHLDR first learns the representations of drugs and diseases at a lower-order level by considering their biological attributes and drug-disease associations (DDAs) through a graph convolutional network model. Then, a meta-path-based strategy is designed to obtain their higher-order representations involving the associations among drugs, proteins and diseases. Their integrated representations are thus determined by fusing higher and lower-order representations, and finally a Random Vector Functional Link Network is employed by FuHLDR to identify novel DDAs. Experimental results on two benchmark datasets demonstrate that FuHLDR performs better than several state-of-the-art drug repositioning models. Furthermore, our case studies on Alzheimer's disease and Breast neoplasms indicate that the rich higher-order biological information gains new insight into drug repositioning with improved accuracy.

Ensemble deep random vector functional link network using privileged information for Alzheimer's disease diagnosis.

In this paper, deep RVFL and its ensembles are enabled to incorporate privileged information, however, the standard RVFL model and its deep models are unable to use privileged information. Privileged information-based approach commonly seen in human learning. To fill this gap, we incorporate learning using privileged information (LUPI) in deep RVFL model and propose deep RVFL with LUPI framework (dRVFL+). Privileged information is available while training the models. To make the model more robust, we propose ensemble deep RVFL+ with LUPI framework (edRVFL+). Unlike traditional ensemble approach wherein multiple base learners are trained, the proposed edRVFL+ optimises a single network and generates an ensemble via optimization at different levels of random projections of the data. Both dRVFL+ and edRVFL+ efficiently utilise the privileged information which results in better generalization performance. In LUPI framework, half of the available features are used as normal features and rest as the privileged features. However, we propose a novel approach for generating the privileged information. To the best of our knowledge, this is first time that a separate privileged information is generated. The proposed models are employed for the diagnosis of Alzheimer's disease. Experimental results show the promising performance of both the proposed models.

Diagnosis for autism spectrum disorder children using T1-based gray matter and arterial spin labeling-based cerebral blood flow network metrics.

Autism Spectrum Disorder (ASD) is a complex neurodevelopmental condition characterized by impairments in motor skills, communication, emotional expression, and social interaction. Accurate diagnosis of ASD remains challenging due to the reliance on subjective behavioral observations and assessment scales, lacking objective diagnostic indicators. In this study, we introduced a novel approach for diagnosing ASD, leveraging T1-based gray matter and ASL-based cerebral blood flow network metrics. Thirty preschool-aged patients with ASD and twenty-two typically developing (TD) individuals were enrolled. Brain network features, including gray matter and cerebral blood flow metrics, were extracted from both T1-weighted magnetic resonance imaging (MRI) and ASL images. Feature selection was performed using statistical t-tests and Minimum Redundancy Maximum Relevance (mRMR). A machine learning model based on random vector functional link network was constructed for diagnosis. The proposed approach demonstrated a classification accuracy of 84.91% in distinguishing ASD from TD. Key discriminating network features were identified in the inferior frontal gyrus and superior occipital gyrus, regions critical for social and executive functions in ASD patients. Our study presents an objective and effective approach to the clinical diagnosis of ASD, overcoming the limitations of subjective behavioral observations. The identified brain network features provide insights into the neurobiological mechanisms underlying ASD, potentially leading to more targeted interventions.

Short-term household load forecasting based on Stacking-SCN

The household load has high volatility and uncertainty, which plays an increasingly important role in short-term power load forecasting for individual residential customers in the operation and planning of future power grids. There is still room for improvement in the prediction accuracy of traditional machine learning and deep learning models. This paper firstly analyzes the complexity of household load data and formulates forecasting problems. Then, the principle of stochastic configuration network (SCN), stacking algorithm and a stacking integrated stochastic configuration network (Stacking-SCN) is introduced, which constructs a short-term household load forecasting model. By adaptively constraining the output weight distribution of the network model, the prediction accuracy can be improved. To show the prediction performance of the algorithm, the experiment comparing the Stacking-SCN model with incremental random vector functional link network (IRVFLN) models, the traditional SCN, ensemble-SCN (E-SCN), deep neural networks (DNN) and long short-term memory (LSTM), and verify it with RMSE, MAE and APE performance metrics. Experimental results show that Stacking-SCN can more effectively express the time series relationship of household short-term load, has higher prediction accuracy, effectively prevents network over-fitting problems, and improves the accuracy of prediction capabilities.

Imbalanced Real-time Fault Diagnosis Based on Minority-Prioritized Online Semi-supervised Random Vector Functional Link Network

Imbalanced Real-time Fault Diagnosis Based on Minority-Prioritized Online Semi-supervised Random Vector Functional Link Network