Stochastic Configuration Networks Research Articles

Evolutionary stochastic configuration networks for industrial data analytics

Stochastic configuration network (SCN) with compact architecture is expected for data modeling. However, the hidden-node parameters (HNPs) randomly configured may result in a slow learning process due to the redundant nodes embedded in the model. To resolve this problem, an evolutionary SCN based on an improved differential evolution (DE) algorithm is proposed in this paper. Specifically, the improved DE reuses the assignment information of last hidden node to find an appropriate search scope for the current one; employs a space reduction method to seed a promising population in the scope; and develops a performance-aware scheme to adjust the scale factor of mutation operators. The proposed evolutionary SCNs are compared with other methods on six datasets and then applied for two real-world applications. Experimental results demonstrate that the proposed method obtains superior performance in terms of compactness and accuracy, with great potential for real-world data analysis.

Spectral data analysis based on bagging stochastic configuration networks

Although stochastic configuration networds(SCN) has the universal approximation property and faster learning speed,during the process of model construction,the randomness of weight and biases assignment as well as the uncertainty of model structure lead to instability.Inspired by bagging,an ensemble model named bagging SCN is proposed to adress the limitation of the single model.Firstly,multiple different training subsets are extracted by bootstrap sampling.Then the SCN submodels are trained on each subset.Finally,the median output of these submodels is taken as the final prediction.Predictions made by bagging SCN are tested on two public datasets.The performance of bagging SCN is then compared with other techniques,including SCN,bagging SCN with other aggregating rules.Experimental results demonstrate that bagging SCN proposed in this study exhibits good stability and high prediction accuracy,making it suitable for quantitative analysis of spectral data.

Enhancing deep stochastic configuration networks: Efficient training via low-rank matrix approximation

Deep Stochastic Configuration Networks (DeepSCN) represent a class of randomized learner models distinguished by their universal approximation capabilities, rapid learning processes, and ease of implementation. These models surpass other randomized learners due to an embedded supervisory mechanism. Although the feature matrix is meticulously constructed, the predictive accuracy of DeepSCN also critically depends on the quality of the model parameters obtained from solving the corresponding system of linear equations. As the number of hidden neurons increases, the resulting expansion of the feature matrix renders matrix inversion computationally expensive, potentially unstable, and memory-intensive. This study addresses these challenges by integrating randomized algorithm-based low-rank matrix approximation to compute model parameters efficiently. A theoretical error bound for the obtained model parameters is derived, ensuring precision and accuracy. The effectiveness of the algorithm is validated on benchmark datasets for both classification and regression tasks. The results demonstrate that this integration significantly enhances the reliability and stability of DeepSCN, improving performance and scalability while greatly reducing computational demands. The method maintains stability despite substantial variations in the number of hidden neurons and mitigates overfitting caused by matrix inversion, ensuring stable and reliable training outcomes. This study advances DeepSCN training methodologies and large-scale data processing.

Objective Sliding Mode Control for the Flotation Motor Replacement Process Based on a Stepwise Stochastic Configuration Network

Flotation motors face constant vibration and friction in the working process, resulting in wear and often need to be replaced, reducing production efficiency. In order to ensure the efficient operation of flotation equipment, we designed a manipulator to assist the work. The manipulator model is difficult to be accurate, and the diversity of external information brings great challenges to the precise control of the manipulator. Therefore, an Objective Sliding Mode Control (OSMC) strategy combined with Stepwise Stochastic Configuration Networks (SSCNs) is developed to solve the above problems while reducing the oscillation of the system. The SSCN is distinguished by its innovative node selection mechanism for the hidden layer, which operates in two distinct phases: initially, ‘single time’, where weights and biases are randomly assigned through a uniform distribution, and subsequently, ‘double time’, where these parameters are adjusted based on a normal distribution centered around the values selected in the first phase. This approach enhances the system’s robustness by incorporating a disturbance observer based on SSCN, designed to accurately estimate and compensate for system uncertainties. The simulation model realistically integrates these uncertainties and disturbances, demonstrating the effectiveness of the proposed OSMC in satisfying the stringent control requirements of mechanical arms under uncertain conditions. This advancement offers a promising new method for the precise control of robotic systems in challenging environments.

Long-term Traffic Flow Prediction using Stochastic Configuration Networks for Smart Cities

Accurate predictions of traffic flow are very meaningful to city managers. With such information, traffic systems can better coordinate traffic signals and reduce congestion. By understanding traffic patterns, navigation systems can provide real-time routing suggestions that avoid traffic jams, save time, and reduce fuel consumption. However, traffic flow will be interfered with by multiple factors such as collection time and place. In this paper, we propose to use stochastic configuration networks (SCNs) to predict the traffic flow. The network is trained through stepwise construction, and the network parameters are effectively optimized based on the approximation theorem and convergence analysis optimization mechanism. The proposed network automatically adjusts its structure according to the complexity of traffic flow to better adapt to the complex non-linearity of traffic flow. We observed that the proposed model achieves better prediction performance overall and greater flexibility in the length of the prediction period compared to the benchmarks using the Guangzhou urban traffic flow dataset. It's worth noting that SCNs consistently outperform other models across different prediction intervals. They yield RMSE improvements of up to 10.73% for 10-minute predictions, 5.02% for 30-minute predictions, and 11.21% for 60-minute predictions compared to the least effective models. The R-value also exhibits steady enhancement, increasing up to 0.78%, 0.65%, and 2.33% for 10-minute, 30-minute, and 60-minute predictions, respectively. These notable advancements, combined with the model's computational efficiency, especially in short-term predictions, underscore the effectiveness and practicality of SCNs in traffic flow prediction tasks.

Multi-level optimizing of parameters in stochastic configuration networks based on cloud model and nutcracker optimization algorithm

As a state-of-the-art neural network model, stochastic configuration networks (SCNs) are widely employed in diverse fields due to their exceptional approximation capabilities. Similar to other neural network models, an excessive number of parameters can potentially compromise the generalization ability of SCNs, including hyper-parameters such as the stochastic scale factors (λ) and the maximum number of nodes (Lmax), as well as model parameters like input weight (w) and input bias (b). To tackle this issue, this study proposes a multi-level parameter optimization approach, termed stochastic configuration network with cloud models (CMSCNs). Firstly, the optimal parameter range is determined based on the concept of “cloud droplet” from the cloud model. Herein, mathematical expectation (Ex) is substituted by a polynomial function constructed with λ as the dependent variable and other parameters as independent variables. Secondly, we employ the nutcracker optimization algorithm (NOA) to optimize w and b, using residuals as evaluation indices to identify their optimal combination. Thirdly, singular value decomposition (SVD) is integrated to compress the network structure of CMSCNs for enhanced computational efficiency. Finally, 18 public real datasets and submergence depth data from an oil well are utilized to assess the performance of CMSCNs. The experimental results demonstrate that our proposed method offers improved generalizability and stability while also exhibiting significant potential in practical applications.

Stochastic configuration networks for short-term power load forecasting

The accuracy of power load forecasting can enhance the operational efficacy and economic efficiency of the power system. This paper utilizes the stochastic configuration network (SCN) and improved marine predator algorithm (IMPA) for power systems short-term load forecasting. First, the daily load data is clustered using the kernel fuzzy C-means (KFCM), and the clustering results are utilized as training input data for the load forecasting model. Then, load forecasting is performed using SCN. Next, IMPA is developed by integrating the opposite-based differential evolution (ODE) and normal mutation perturbation strategy for the purpose of preventing the algorithm from converging to local minima. Finally, IMPA is employed to optimize the critical parameters of the load forecasting model. The experimental results clearly demonstrate that the proposed algorithm performs favorably in terms of the prediction accuracy compared to other modeling algorithms.

Robust multi-target regression with improved stochastic configuration networks and its applications

To improve the accuracy and robustness of stochastic configuration networks (SCNs) for resolving multi-target regression tasks, this paper proposes a robust modeling approach based on improved stochastic configuration networks. A parallel implementation of SCN models is designed to incrementally generate the hidden nodes, which enhances the diversity of hidden layer mapping through information superposition and spanning connection. We employ an elastic net regularization model to sparsely constrain the model parameters to characterize the correlation among multiple targets. Then, the mixture Laplace distributions are used as the prior distribution of each target modeling error, and the output weights of the SCN model are re-evaluated by maximizing a posteriori estimation to enhance model’s robustness with respect to some uncertainties presented in training samples. The modelling performance of the proposed solution is tested on six standard datasets and the historical data of a municipal solid waste incineration process. The experimental results show that the proposed modeling technique has advantages in terms of both the prediction accuracy and the robustness.

Modular stochastic configuration network with attention mechanism for soft measurement of water quality parameters in wastewater treatment processes

Due to the complex and multi-distributed nature of wastewater treatment process data, stochastic configuration networks (SCNs) face difficulties such as large network structure and complex construction process when dealing with data modeling tasks. To address these challenges, this paper proposes a modular stochastic configuration network with attention mechanism, which integrates the advantages of modular partitioning, attention mechanism, and SCN. The model employs a fuzzy mean clustering strategy to simplify and decompose the complex multi-case and multi-distribution nonlinear modeling task, thereby reducing the modeling complexity of a single model. Furthermore, the model utilizes a stochastic configuration algorithm to learn the subtasks and construct corresponding sub-networks. The output weights are then assigned to each sub-network based on the attention mechanism to obtain the model’s final output. The validity of the proposed method is validated through two benchmark experiments. Then the model is applied to measure effluent ammonia nitrogen. The results indicate that compared with the classical randomized and modularized network models, the proposed model exhibits certain advantages in terms of learning accuracy and generalization performance.

Temporal online self-learning stochastic configuration networks: A study on strip deviation prediction

Stochastic configuration networks (SCNs), with the rapid learning abilities and universal approximation properties, have garnered growing interest in the industrial domain in recent years. In the hot rolling process, strip deviation affects product quality and production safety. In this paper, a novel model named temporal online self-learning stochastic configuration networks (TOSL-SCNs) designed for early prediction of strip deviation in the hot rolling process is introduced. To address the challenges of time series prediction, conventional SCNs are extended to a two-dimensional format, enabling effective processing of time series data. Given the dynamic nature of industrial environments, the model incorporates an online self-learning capability to adapt to nonstationary data. Additionally, to ensure the stability of the model’s predictive performance, a data buffer strategy is introduced to dynamically refresh the training dataset. This paper demonstrates the capabilities of TOSL-SCNs in time series modeling, online self-learning, and data buffer strategy from multiple perspectives. The case study on strip deviation prediction highlights the necessity of these capabilities in industrial applications.

Multimodal graph learning with framelet-based stochastic configuration networks for emotion recognition in conversation

The multimodal emotion recognition in conversation (ERC) task presents significant challenges due to the complexity of relationships and the difficulty in achieving semantic fusion across various modalities. Graph learning, recognized for its capability to capture intricate data relations, has been suggested as a solution for ERC. However, existing graph-based ERC models often fail to address the fundamental limitations of graph learning, such as assuming pairwise interactions and neglecting high-frequency signals in semantically-poor modalities, which leads to an over-reliance on text. While these issues might be negligible in other applications, they are crucial for the success of ERC. In this paper, we propose a novel framework for ERC, namely multimodal graph learning with framelet-based stochastic configuration networks (i.e., Frame-SCN). Specifically, framelet-based stochastic configuration networks, which employ 2D directional Haar framelets to extract both low- and high-pass components, are introduced to learn the unified semantic embeddings from multimodal data, mitigating prediction biases caused by an excessive reliance on text without introducing an unnecessarily large number of parameters. Also, we develop a modality-aware information extraction module that is able to extract both general and sensitive information in a multimodal semantic space, alleviating potential noise issues. Extensive experiment results demonstrate that our proposed Frame-SCN outperforms many state-of-the-art approaches on two widely used multimodal ERC datasets.

Stochastic configuration network modeling method based on information superposition and mixture correntropy

Stochastic configuration network modeling method based on information superposition and mixture correntropy

Multivariable adaptive decoupling control based on stochastic configuration networks with serial-parallel switching distribution

A multivariable adaptive decoupling control scheme is proposed based on stochastic configuration networks with serial-parallel switching distribution (SPSCN). Firstly, a linear controller is designed by combining a PID controller, feedback decoupling, and one-step optimal control. Secondly, a nonlinear controller is presented to deal with higher-order nonlinear terms and unknown external perturbations. SPSCN is used to improve the prediction accuracy of unmodeled dynamics. It combines uniform and normal search strategies in a serial-parallel fashion, aiming at improving the node quality and reducing the model complexity. The approximation performance of the SPSCN algorithm is demonstrated by performing approximation experiments with two functions and four benchmark datasets. Compared with the generalized minimum variance control (GMVC) algorithm in controlling the process of cement raw material decomposition, our proposed control scheme outperforms.

An empirical study on prediction of seismic activity using stochastic configuration networks

An empirical study on prediction of seismic activity using stochastic configuration networks

Intelligent fault diagnosis for unbalanced battery data using adversarial domain expansion and enhanced stochastic configuration networks

An accurate and efficient fault diagnosis method for battery systems is crucial to ensuring the safety of battery packs. Addressing the issue of insufficient actual fault data in battery operations, this paper proposes an intelligent fault diagnosis method based on feature-enhanced stochastic configuration networks and adversarial domain expansion of imbalanced battery fault data (AFDEM-FESCN). Firstly, we designed an adversarial fault domain data expansion method (AFDEM). By learning the distribution of fault data through adversarial training, the distribution of sample domains is balanced, thereby reducing model bias. Subsequently, we adjusted the distribution of SCN iterative parameters and added a linear feature layer. This enhances the feature extraction capability of the network through distribution overlay, enabling fault diagnosis. Finally, the effectiveness and feasibility of the proposed method were validated through a practical battery system fault diagnosis case, achieving a diagnostic accuracy of 92.1%. Experimental results demonstrate that the AFDEM-FESCN method exhibits good accuracy in battery system fault diagnosis, providing an effective solution to the challenge of imbalanced data in intelligent fault diagnosis.

Intuitionistic Fuzzy Stochastic Configuration Networks for Solving Binary Classification Problems

Intuitionistic Fuzzy Stochastic Configuration Networks for Solving Binary Classification Problems

Knowledge-informed randomized machine learning and data fusion for anomaly areas detection in multimodal 3D images

We consider a long-standing yet hard and largely open machine learning problem of anomaly areas detection in multimodal 3D images. Purely data-driven methods often fail in such tasks because rarely incorporating domain-specific knowledge into the algorithm and do not fully utilize information from multiple modalities. We address these issues by proposing a novel framework with data fusion technology to leverage domain-specific knowledge and multimodal labeled data, as well as employ the power of randomized learning techniques. To demonstrate the proposed framework efficiency, we apply it to the challenging task of detecting subtle pathologies in MRI scans. A distinct feature of the resulting solution is that it explicitly incorporates evidence-based medical knowledge about pathologies into the feature maps. Our experiments show that the method is capable of achieving lesion detection in 71% of subjects by using just one such feature. Integrating information from all feature maps and data modalities enhances detection rate to 78%. Using stochastic configuration networks to initialize the weights of the classification model enables to increase precision metric by 18% as compared to deterministic approaches. This demonstrates the possibility and practical viability of building efficient and interpretable randomised algorithms for automated anomaly detection in complex multimodal data.

Self-organizing hierarchical incremental learning framework and universal approximation analysis based on stochastic configuration mechanism

Conventional machine learning algorithms face significant limitations when dealing with high-dimensional data. Besides, deep learning models often require substantial computational resources and have a high processing time despite their excellent performance. Hence, this paper proposes an expanded stochastic configuration network and a self-organizing hierarchical incremental learning (SHIL) framework to overcome these challenges. Specifically, this study introduces a novel supervised hierarchical clustering tree based on the minimum redundancy maximum correlation algorithm, which mines internal data structures to construct diverse hierarchies. Subsequently, by exploiting the parent-child node relationships in the tree structure, SHIL defines the maximum number of nodes as the switching condition between levels uses the supervisory mechanism as the parameter selection criterion, and adopts the tolerance error as the termination criterion for the training. Furthermore, the universal approximation property of the SHIL framework is provided. The proposed SHIL framework is validated on several benchmark datasets, image datasets, and industrial robot cases, with the corresponding experimental results demonstrating that SHIL significantly improves computational efficiency and ensures high accuracy.

Industry Image Classification Based on Stochastic Configuration Networks and Multi-Scale Feature Analysis.

For industry image data, this paper proposes an image classification method based on stochastic configuration networks and multi-scale feature extraction. The multi-scale features are extracted from images of different scales using deep 2DSCN, and the hidden features of multiple layers are also connected together to obtain more informational features. The integrated features are fed into SCNs to learn a classifier which improves the recognition rate for different categories. In the experiments, a handwritten digit database and an industry hot-rolled steel strip database are used, and the comparison results demonstrate the proposed method can effectively improve the classification accuracy.

A Recurrent Stochastic Configuration Network for Dynamic System Modeling and Its Application

In order to model nonlinear dynamic systems quickly and accurately, a recurrent stochastic configuration network (RSCN) with universal approximation ability is proposed in this paper. The method consists of network parameter learning and network structure determination. The network parameter learning strategy consists of input weights and the biases of hidden nodes generated randomly through a supervision mechanism, while the output weights of hidden nodes are determined by solving the least-squares problem. The network structure is determined incrementally by the training error of a single-layer recurrent neural network. The effectiveness of RSCN is tested and evaluated by using three nonlinear dynamic functions and historical data from a municipal solid waste incineration (MSWI) process. The experimental results show that the proposed method can improve modeling accuracy and reduce training time, which is valuable for applications in the field of industrial process modeling.