Stochastic Configuration Networks Research Articles

Industry Image Classification Based on Stochastic Configuration Networks and Multi-Scale Feature Analysis

For industry image data, this paper proposes an image classification method based on stochastic configuration networks and multi-scale feature extraction. The multi-scale features are extracted from images of different scales using deep 2DSCN, and the hidden features of multiple layers are also connected together to obtain more informational features. The integrated features are fed into SCNs to learn a classifier which improves the recognition rate for different categories. In the experiments, a handwritten digit database and an industry hot-rolled steel strip database are used, and the comparison results demonstrate the proposed method can effectively improve the classification accuracy.

A Recurrent Stochastic Configuration Network for Dynamic System Modeling and Its Application

In order to model nonlinear dynamic systems quickly and accurately, a recurrent stochastic configuration network (RSCN) with universal approximation ability is proposed in this paper. The method consists of network parameter learning and network structure determination. The network parameter learning strategy consists of input weights and the biases of hidden nodes generated randomly through a supervision mechanism, while the output weights of hidden nodes are determined by solving the least-squares problem. The network structure is determined incrementally by the training error of a single-layer recurrent neural network. The effectiveness of RSCN is tested and evaluated by using three nonlinear dynamic functions and historical data from a municipal solid waste incineration (MSWI) process. The experimental results show that the proposed method can improve modeling accuracy and reduce training time, which is valuable for applications in the field of industrial process modeling.

OPT-CO: Optimizing pre-trained transformer models for efficient COVID-19 classification with stochastic configuration networks

Building upon pre-trained ViT models, many advanced methods have achieved significant success in COVID-19 classification. Many scholars pursue better performance by increasing model complexity and parameters. While these methods can enhance performance, they also require extensive computational resources and extended training times. Additionally, the persistent challenge of overfitting, due to limited COVID-19 dataset sizes, remains a hurdle. To address these challenges, we proposed a novel method to optimize pre-trained transformer models for efficient COVID-19 classification with stochastic configuration networks (SCNs), referred to as OPT-CO. We proposed two optimization methods: sequential optimization (SeOp) and parallel optimization (PaOp), by incorporating optimizers in a sequential and parallel manner, respectively. Our method can enhance model performance without necessitating a significant parameter expansion. Additionally, we introduced OPT-CO-SCN to avoid overfitting problems through the adoption of random projection for head augmentation. The experiments were carried out to evaluate the performance of our proposed model based on two publicly available datasets. Based on the evaluation results, our method achieved superior, performance surpassing other state-of-the-art methods.

Greedy deep stochastic configuration networks ensemble with boosting negative correlation learning

Deep stochastic configuration networks (DSCNs) employ data-dependent supervision mechanism to randomly assign node parameters and incrementally construct the deep neural network structure, thereby ensuring the model's universal approximation property. To build a random neural networks ensemble model with better generalization performance, we propose a novel greedy deep stochastic configuration networks ensemble model based on boosting negative correlation learning, termed as GDSCNE. Firstly, greedy optimization strategy based on inequality constraints is utilized to generate random parameters of base components with multi-layer architecture, which can accelerate the decline of network residuals when configuring a new node. Additionally, boosting negative correlation learning framework is presented for the base components ensemble process, which uses least square approach with negative correlation learning penalty term to update the ensemble output weights for each base component, subsequently, boosting method is applied to construct a stronger ensemble model by adaptive weighting through the results of base components. Finally, we evaluated GDSCNE on the popular regression benchmark datasets from the KEEL, experimental results demonstrate that GDSCNE outperforms state-of-the-art random learning algorithms in terms of regression accuracy and generalization performance across several regression datasets with varying sizes.

Finite-time compensation control with dead-zone estimation for a rehabilitative walker considering internal disturbance forces

This study discusses a finite-time compensation tracking control method for a rehabilitative training walker. The dynamic model with input dead zone was constructed to describe the walker, and a finite-time disturbance forces observation method was proposed based on the impact mechanism on tracking performance. This approach is novel in that the disturbance forces were observed in reverse through their effects on tracking performance, thus successfully obtaining the disturbance forces of the walker. To ensure the practical finite-time stability of the system, the nonlinear finite-time compensation tracking controller with stochastic configuration networks (SCN) dead-zone estimation was built for the rehabilitative walker. Simulation results and comparative analyses confirmed that the proposed compensation control method effectively restrains dead zone and internal disturbance forces.

Accelerated MPC: A real-time model predictive control acceleration method based on TSMixer and 2D block stochastic configuration network imitative controller

Modern industrial processes demand real-time model predictive control (MPC) method within the constraints of limited computing resources. This paper introduces an accelerated MPC method to address the issue. Specifically, we employ the time series mixing model (TSMixer) to construct a predictive model capable of accurately forecasting the behavior of the system under control. Furthermore, we extend the capabilities of TSMixer by integrating a feature classification model (TSMixer with FCM). This enhanced version takes into account both future information and static variables, resulting in superior accuracy compared to the transformer model while maintaining a significantly smaller parameter count. Finally, to further enhance MPC’s responsiveness, we develop a two-dimensional block stochastic configuration network (2D-BSCN) imitative controller. This innovative network clones the behavior of rolling optimization, effectively replacing online optimization to reduce computational time. The experimental results show that the accelerated MPC has an excellent performance in numerical simulation and actual industrial processes. Notably, the algorithm achieves a smaller FLOPs and tracking time than other state-of-the-art models, well within the requirement for industrial process control.

Modeling and reliability analysis of the micro-milling process considering stochastic tool wear with surrogate models

Micro-milling technology is widely employed in manufacturing micro-precision parts due to its flexible processing and high accuracy. Compared to the macro-milling process, micro-milling has a smaller machining size, which leads to tool wear, tool runout, and other factors being more sensitive to the impact of machining quality. This work proposes a reliability evaluation framework for machining systems based on the micro-milling mechanical model and its surrogate method. Establish a cutting force model under shear and plow conditions with periodic variations in instantaneous uncut thickness. Subsequently, the probability distribution of the cutting parameters is obtained based on Bayesian updates. An improved buffer failure probability method is proposed to quickly get the micro-milling system failure probability. Developing new ensemble and adaptive sampling methods and introducing stochastic configuration network (SCN), an adaptive stochastic configuration network ensemble (ASCNE) model is established to alleviate the time-consuming problem of repeated calculations of mechanical models in reliability analysis. Experimental results indicate that the mechanical model can achieve good predictive performance, and the constructed ASCNE model can achieve high-precision surrogate effects. Additionally, the reliability analysis results of the system can guide tool replacement during the machining process, ensuring safe and efficient execution of the micro-milling.

A novel stochastic configuration network with enhanced feature extraction for industrial process modeling

Stochastic configuration networks (SCNs), possessing sound generalization performance and low computational burden, have been extensively investigated in data analysis field. But in practice, it performs poor. This is mainly caused by two aspects: (1) the increment of random hidden nodes with incompact constraints may bring the emergence of redundant nodes, thereby resulting in poor modeling performance, and (2) SCNs have limited feature extraction capabilities. To address these issues, a novel SCN inserting feature layer, termed as FSCN, is presented in this paper. First, the embedded feature layer establishes a connection between the input and hidden layers, thus mapping the input data into a suitable feature space. Second, an inequality constraint based on Greville iterative method is established. It can not only facilitate the construction of high-quality hidden nodes but also guarantee convergence of the built model. Then, the Greville iterative method is also employed for output weight updating. Finally, comparative experiments on a real-valued nonlinear function, four real-world data sets and the modeling of two real industrial processes are conducted and the comparison results with other modeling methods show that FSCN has superior performance in terms of both model accuracy and compactness.

An information entropy-based fuzzy stochastic configuration network for robust data modeling

In data-driven modeling, the generalization performance of a neural network based on the minimum mean square error criterion may be affected when the samples contain noise. To handle uncertain data modeling problems, this study contributes a novel robust data modeling method called information entropy-based fuzzy stochastic configuration networks (IEFSCNs), which incorporates the maximum correntropy criterion (MCC) to reflect the squared error of the training sample set and uses the quantized minimum error entropy (QMEE) criterion to mine the distribution structure of the modeling data and suppress outlier data. Fuzzy systems are designed to generate nodes in the fuzzy transformation layer. A nonlinear enhancement layer is established as a hidden layer to enhance the representational capacity of the target system. The fuzzy transformation and nonlinear enhancement layers are fully connected to determine the outputs of the IEFSCNs. In addition, an incremental parameter learning algorithm is developed based on a supervision mechanism for optimizing the cost function, and its convergence property is demonstrated. Owing to the superiorities of the cost function by synthesizing MCC and QMEE, IEFSCNs can improve the modeling accuracy and reduce noise suppression. The performances are evaluated using various regression and classification tasks. Compared with existing methods, including SCNs, RVFL, F-SCNs, RSC-KDE, RSC-MCC, and SM-RSC, the proposed method demonstrates an effective robust modeling performance by reducing network structure redundancy while enhancing its ability to suppress noisy data.

An adaptive strategy for time‐varying batch process fault prediction based on stochastic configuration network

AbstractFault prediction ensures safe and stable production, and cuts maintenance costs. Due to the changing operating conditions that lead to the changes in the characteristics of industrial processes, there is a need to monitor the fault state of batch processes in real‐time and to accurately predict fault trends. An adaptive slow feature analysis‐neighborhood preserving embedding‐improved stochastic configuration network (SFA‐NPE‐ISCN) algorithm for batch process fault prediction is proposed. Firstly, SFA is used to extract the time‐varying features of process data and establish the update index of the NPE model. Then, to extract local nearest‐neighbor features and reconstruct them by the NPE model with adaptive update capability, square prediction error (SPE) statistics are constructed as fault state features based on the reconstructed error. Further, the hunter‐prey optimization (HPO) algorithm optimizes the weights and biases in the stochastic configuration network, and the singular value decomposition (SVD) and QR decomposition of column rotation are introduced to solve the ill‐posed problem of SCN and obtain the prediction model of ISCN. Finally, the obtained statistics SPE is formed into a time series, and the ISCN model is used to predict the process state trend. The effectiveness of the proposed algorithm is verified by case studies of industrial‐scale penicillin fermentation processes and the Hot strip mill process.

Improved stochastic configuration networks with vision patch fusion method for industrial image classification

This paper contributes to the advancement of stochastic configuration neural networks (SCN) in the field of visual applications. The proposed image classification randomized algorithm is an extension of deep stochastic configuration networks (DeepSCN), known as an improved SCN with vision patch fusion (Vi-SCN). Compared to existing two dimensional stochastic configuration networks (2DSCN) and DeepSCN, we have developed an incremental modeling method that employs a stochastic configuration patch fusion method. This method extracts randomly fused image features from three-channel high-resolution image data, improving the network's ability to extract features. Moreover, we have introduced a strategy for dynamically determining the depth structure of the network, enabling flexible adjustments to the network structure and the number of nodes in each layer based on the complexity of image recognition tasks. Through a series of comparisons on four image classification benchmark datasets, we assess the superior learning performance of our design compared to 2DSCN and DeepSCN. Furthermore, to evaluate the performance of Vi-SCN in practical visual application scenarios, we collect three industrial datasets with distinct characteristics. Vi-SCN demonstrate outstanding performance in all three tasks, showing its significant potential in image recognition.

A new two-stage decomposition and integrated hybrid model for short-term wind speed prediction

Accurate wind speed prediction is of essential importance for the stability and safe operation of power systems. Given the complexity of wind speed sequence, this paper proposed a new two-stage decomposition and integrated hybrid model to improve the accuracy of wind speed prediction. A two-stage decomposition method combining robust local mean decomposition (RLMD), sample entropy (SE) and variational modal decomposition (VMD) was used to decompose the wind speed signal in the data preprocessing stage. Firstly, the wind speed signal was decomposed into various components by RLMD, and the complexity of each component was calculated using the SE to classify them into random, detail component and trend component. Then, a secondary decomposition of the random component with the highest SE was performed using the VMD. In the prediction stage, two different prediction models were used for prediction depending on the smoothness of each component. Stochastic configuration networks (SCN) was used to predict the detail and trend components with relatively smoothness. Echo state network (ESN) was used to predict the components of the secondary decomposition. Finally, the actual wind speed data were compared by different prediction models, which illustrated that the prediction method proposed in this paper had good prediction accuracy and generalizability.

Universal approximation property of stochastic configuration networks for time series

For the purpose of processing sequential data, such as time series, and addressing the challenge of manually tuning the architecture of traditional recurrent neural networks (RNNs), this paper introduces a novel approach-the Recurrent Stochastic Configuration Network (RSCN). This network is constructed based on the random incremental algorithm of stochastic configuration networks. Leveraging the foundational structure of recurrent neural networks, our learning model commences with a modest-scale recurrent neural network featuring a single hidden layer and a solitary hidden node. Subsequently, the node parameters of the hidden layer undergo incremental augmentation through a random configuration process, with corresponding weights assigned structurally. This iterative expansion continues until the network satisfies predefined termination criteria. Noteworthy is the adaptability of this algorithm to handle time series data, exhibiting superior performance compared to traditional recurrent neural networks with similar architectures. The experimental results presented in this paper underscore the efficacy of the proposed RSCN for sequence data processing, showcasing its advantages over conventional recurrent neural networks in the context of the performed experiments.

Stochastic configuration networks with CPU-GPU implementation for large-scale data analytics

Stochastic configuration networks (SCNs) are a class of randomized learner models that have garnered increasing attention in data analytics. In the original implementation of SCN, the pseudo-inverse of the hidden layer output matrix is used to evaluate output weights. However, it is quite challenging to compute the pseudo-inverse of the hidden layer output matrices for large-scale datasets with high complexity, which occurs in many real-world applications. This paper aims to accelerate the learning process of SCNs by employing the well-known alternating direction method of multipliers (ADMM) solver and CPU-GPU heterogeneous computing technique. Empirical results on two benchmark datasets demonstrate the efficiency and effectiveness of the proposed method for large-scale data analytics.

Working condition recognition of fused magnesium furnace based on stochastic configuration networks and reinforcement learning

Automatic and accurate recognition of abnormal working conditions of fused magnesia furnace is of great significance to the safe and reliable production of fused magnesia. Aiming at the defects of manual judgment of abnormal working conditions in the production process of fused magnesium furnace and the existing recognition method of abnormal working conditions based on machine learning, this paper proposes a working condition recognition model for fused magnesium furnace based on stochastic configuration networks and reinforcement learning. Firstly, a hybrid data augmentation method of generative and non-generative is used to obtain high-quality sample data with salient features. Secondly, based on ResNeXt, multi-scale local features are extracted under the condition of limited parameter quantity by controlling the cardinality. Combining a mixed model that enjoys the benefit of both self-attention and convolution (ACmix) and bidirectional feature pyramid network (BiFPN), the extracted local feature maps of different scales are cross-scale fused and focused, and more differentiated detailed feature information of the region of interest is retained. Thirdly, based on Transformer, the working condition recognition network of fused magnesium furnace is constructed to improve the global correlation between adjacent local features in the spatial dimension. The fused features are sent to stochastic configuration networks to establish a classification criterion for working condition recognition of fused magnesium furnace with generalization ability. Finally, reinforcement learning is used to evaluate the credibility of uncertain recognition results of samples in real time, and a self-optimizing adjustment action strategy at the Transformer encoding layer is defined. Build a library of Transformer models with different encoding layers, which is adapt to the different feature extraction requirements of multi-modal working samples. The experimental results show that the method in this paper has better recognition performance and generalization ability than other algorithms.

Online Self-Learning Stochastic Configuration Networks for Nonstationary Data Stream Analysis

Online Self-Learning Stochastic Configuration Networks for Nonstationary Data Stream Analysis

A cooperative stochastic configuration network based on differential evolutionary sparrow search algorithm for prediction

Stochastic configuration network (SCN) is a powerful prediction model whose performance is significantly influenced by the configuration of the network parameters. To improve the prediction accuracy of the network, a cooperative stochastic configuration network (CSCN) based on a novel differential evolutionary sparrow search algorithm (DESSA), termed as DESSA-CSCN, is proposed. In the CSCN, the number of hidden layer nodes is adaptively adjusted according to the number of iterations, and the parameters of hidden nodes are cooperatively optimized by using a population-based metaheuristic algorithm. A sparse matrix is introduced to mitigate parameter overfitting caused by the increased number of hidden layer nodes. During parameter optimization, the fitness function is constructed by using the supervision mechanism of the SCN, and the DESSA is utilized as the metaheuristic algorithm to update the weights and biases. In order to verify the effectiveness of the DESSA-CSCN, several simulation experiments have been conducted. The performance of the DESSA is evaluated by the CEC2017 test suit, and the simulation results show that the DESSA exhibits better convergence accuracy and can jump out of local optima more effectively than other algorithms. The performance of the DESSA-CSCN is evaluated by 4 datasets from KEEL, and the simulation results indicate that the DESSA-CSCN achieves better prediction accuracy and faster prediction speed than other models.

Multi-target regression via stochastic configuration networks with modular stacked structure

Multi-target regression via stochastic configuration networks with modular stacked structure

Multi-task stochastic configuration network with autonomous linking and its application in wastewater treatment processes

Stochastic configuration networks (SCNs) have been widely used for modeling complex industrial process due to their rapid learning speed, ease of implementation, and universal approximation capability. For modeling water quality parameters in wastewater treatment processes (WWTP), however, multiple complex tasks are often required to be modelled simultaneously. In this paper, a multi-task stochastic configuration network with autonomous linking characteristic is proposed to further develop the modeling capability of SCNs to deal with multi-tasks and achieve simultaneous measurement of multiple critical water quality parameters in the WWTP. The method can autonomously construct corresponding common nodes and proprietary nodes according to the distribution characteristics of different tasks to model the shared and private information among these tasks. Specifically, the relevant information between these tasks is explored by constructing common nodes; then personalized approximation of each task is achieved by constructing proprietary nodes for different tasks, thus improving the overall modeling performance of the model. A series of benchmark experiments and an industrial case from WWTP are carried out to verify the superiority of the proposed method. Experimental results demonstrate that our proposed method has a promising potential for multi-task data modeling.

A novel method of rolling bearings fault diagnosis based on singular spectrum decomposition and optimized stochastic configuration network

Fault diagnosis of rolling bearings is essential for the safe operation of rotating machinery. However, in the production process, the rolling bearings have a complex working environment embedded with weak fault signals and a large number of interfering signals, which is a considerable challenge to automatically and accurately detect bearing fault type from the actual vibration signal. Therefore, a novel fault diagnosis scheme is proposed based on singular spectrum decomposition (SSD) and an optimized stochastic configuration network (SCN). Firstly, SSD is used to pre-process the original rolling bearings vibration signal to obtain several singular spectral components (SSCs) and the practical component is selected according to the maximum correlation coefficient for signal reconstruction. Furthermore, time domain and power spectrum entropy (PSE) features of the reconstructed signal are extracted to obtain a fault information-rich feature sets. In addition, the parameters of the SCN are optimized by marine predators algorithm (MPA) to enhance the learning ability and generalization performance of the SCN. Finally, the feature sets are input into MPA-SCN to achieve fault classification. Experimental results exhibit that the proposed method has higher accuracy in rolling bearings fault diagnosis compared with other methods, which provides a high-efficiency solution for rolling bearings fault diagnosis.