Stochastic Configuration Networks Research Articles

Industrial data classification using stochastic configuration networks with self-attention learning features

Industrial data classification using stochastic configuration networks with self-attention learning features

Stochastic configuration network for structural reliability analysis

Structural reliability analysis is of critical importance for ensuring the safety of many engineering systems. This paper proposes an efficient structural reliability analysis method based on stochastic configuration networks (SCNs), which is constructed by learning the real performance function of the structure. In this article, two kinds of innovative surrogate models of performance function are constructed based on the trained SCNs. To improve the approximation accuracy and efficiency of SCNs for complex high-dimensional performance functions, the improved high-dimensional model representation is applied to decompose the multivariate function into a combination of several low-dimensional functions. An innovative SCN-based learning strategy is proposed to simultaneously approximate all the low-dimensional component functions. By performing Monte Carlo Simulation on the established SCN-based surrogate models, we can successfully evaluate the failure probability of many complex multi-dimensional engineering structures. The proposed SCN-based reliability analysis method provides a powerful tool for estimating the failure probability of complex multi-dimensional engineering structures. Effectiveness of the proposed method in terms of accuracy and efficiency as well as stability is demonstrated through several numerical experiments.

Predicting hot-strip finish rolling thickness using stochastic configuration networks

In the hot-rolling metal forming process, the consistency and accuracy of the thickness of the metal strip are the most important factors for the product quality control. The current method of utilizing a mechanism prediction model with pre-defined parameters does not perform well due to some limits on the model assumptions and environmental interference. Manually tuning these parameters of the mechanism model may even result in worse performance. To resolve this problem, an advanced randomized learner model, termed stochastic configuration network (SCN), is employed to build a data-driven prediction model which can be trained by using a dataset collected from a real-world hot-rolling production site. Based on the rolling theory and gray relational analysis (GRA), 36 features are selected as the inputs of the prediction model. Experimental results with comparisons show that our proposed method is feasible and outperforms other machine learning methods, such as deep learning models and the random vector functional link (RVFL) model.

Experimental evaluation of stochastic configuration networks: Is SC algorithm inferior to hyper-parameter optimization method?

To overcome the pitfalls of Random Vector Functional Link (RVFL), a network called Stochastic Configuration Networks (SCN) has been proposed. By constraining and adaptively selecting the range of randomized parameters using the Stochastic Configuration (SC) algorithm, SCN claims to be potent in building an incremental randomized learning system according to residual error minimization. The SC has three variants depending on how the range of output weights are updated. In this work, we first relate the SCN to appropriate literature. Subsequently, we show that the major parts of the SC algorithm can be replaced by a generic hyper-parameter optimization method to obtain overall better results.

Deep stochastic configuration networks with different random sampling strategies

Stochastic configuration networks (SCNs) are a class of randomized learner models that ensure the universal approximation property, whereby random weights and biases are drawn from the uniform distribution and selected by a supervisory mechanism. This paper looks into the impact of the distribution of random weights on the performance of SCNs. In the light of a fundamental principle in machine learning, that is, a model with smaller parameters holds improved generalization, we recommend using symmetric zero-centered distributions in constructing SCNs to improve the generalization performance. Further, we introduce a scalar in the distributions to make the SCN model adaptively feasible to different datasets. Simulation results are reported for both regression and classification tasks over twenty-one benchmark datasets using SCN. Results are also presented on ten regression datasets using a deep implementation of SCN, known as deep stochastic configuration networks (DeepSCN).

Stochastic configuration network based on improved whale optimization algorithm for nonstationary time series prediction

AbstractThe stochastic configuration network (SCN), a type of randomized learning algorithm, can solve the infeasible problem in random vector functional link (RVFL) by establishing a supervisory mechanism. The advantages of fast learning, convergence and not easily falling into local optima make SCN popular. However, the prediction effect of SCN is affected by the parameter settings and the nonstationarity of input data. In this paper, a hybrid model based on variational mode decomposition (VMD), improved whale optimization algorithm (IWOA), and SCN is proposed. The SCN will predict relatively stable data after decomposition by VMD, and parameters of SCN are optimized by IWOA. The IWOA diversifies the initial population by employing logistic chaotic map based on bit reversal and improves the search ability by using Lévy flight. The exploration and exploitation of IWOA are superior to those of other optimization algorithms in multiple benchmark functions and CEC2020. Moreover, the proposed model is applied to the prediction of the nonstationary wind speeds in four seasons. We evaluate the performance of the proposed model using four evaluation indicators. The results show that the R2 of the proposed model under four seasons are more than 0.999, and the root mean square error, mean absolute error, and symmetric mean absolute percentage error are less than 0.3, 0.17, and 13%, respectively, which are almost 1/10, 1/10, and 1/4 those of SCN, respectively.

Robust stochastic configuration networks for industrial data modelling with Student’s-t mixture distribution

Data collected from industrial sites commonly contains outliers or noise that obey unknown distributions, making it challenging to establish an accurate data-driven model. Therefore, this paper proposes a novel robust stochastic configuration network based on a Student’s-t mixture distribution (termed as SM-RSC). Firstly, a stochastic configuration algorithm is employed to determine the number of hidden nodes, the input weights and biases. Secondly, the maximum a posteriori (MAP) estimate is used to evaluate the output weights of the SCN learner model under the assumption that outliers or noises obey the Student’s-t mixture distribution. Because the output weights cannot be solved directly due to the unknown hyper-parameters of the mixture distribution, we apply the well-known expectation–maximization (EM) algorithm for optimizing the hyper-parameters of the mixture distribution and update the output weights iteratively. The proposed algorithm is evaluated by a function approximation, four benchmark datasets, and a case study on industrial data modelling for a waste incineration process. The results show that SM-RSC performs favorably compared with other methods.

A Wavelet Packet Transform and Convolutional Neural Network Method Based Ultrasonic Detection Signals Recognition of Concrete.

This paper proposes a new intelligent recognition method for concrete ultrasonic detection based on wavelet packet transform and a convolutional neural network (CNN). To validate the proposed data-based method, a case study is presented where the K-fold cross-validation was adopted to produce the performance analysis and classification experiments. Moreover, three evaluation indicators, precision, recall, and F-score, are calculated for analyzing the classification performance of the trained models. As a result, the obtained four-classifying CNN reaches more than 99% detection accuracy while the lowest recognition accuracy is not less than 92.5% on the testing dataset for the six-classifying CNN model. Compared with the existing stochastic configuration network (SCN) models, the presented method achieves the design objective with better recognition performance. The calculation results of the six-classifying and five-classifying models and related research clearly indicate the remaining challenging tasks for intelligent recognition algorithms in extracting features and classifying mass data from various concrete defects precisely and efficiently.

Stochastic configuration networks for self-blast state recognition of glass insulators with adaptive depth and multi-scale representation

The operating state of insulators is directly related to the stability of power transmission line. The existing methods for insulator state recognition cannot achieve satisfactory performance. In this paper, the self-blast state recognition of glass insulators is investigated by using an adaptive learning representation. To increase the adaptability of the network to different scales, we propose a solution based on multi-scale information throughout the entire process, beginning from a low-scale to high-scale subnetworks. The multi-scale information is aggregated in parallel way to take advantage of rich information representation. Then, an imitation of the human thinking pattern is employed. Utilizing entropy-based cost function, we update the parameters of the learner model in real-time. Based on the constraint of the evaluation index, adaptive depth representation for training glass insulators that are unsatisfied with the reliability evaluation is constructed to realize the self-optimizing regulation of feature space. Correspondingly, a stochastic configuration networks (SCNs) classifier is re-constructed to fit for the update multi-hierarchies knowledge space to carry out the re-recognition process. Finally, fuzzy integration is employed to ensemble multi-hierarchies network to improve the model’s generalization. The recognition results on aerial dataset of insulators images demonstrate the effectiveness of our proposed approach.

Performance of soft sensors based on stochastic configuration networks with nonnegative garrote

Performance of soft sensors based on stochastic configuration networks with nonnegative garrote

Stochastic configuration networks for imbalanced data classification

Stochastic configuration networks for imbalanced data classification

Heterogeneous feature ensemble modeling with stochastic configuration networks for predicting furnace temperature of a municipal solid waste incineration process

Heterogeneous feature ensemble modeling with stochastic configuration networks for predicting furnace temperature of a municipal solid waste incineration process

Deep stochastic configuration networks with optimised model and hyper-parameters

The selection of hyper-parameters in building neural networks is often left to end-users. However, this may lead to poor performance regardless of the amount of experience of the user. With deep implementation, this becomes even more challenging due to a large number of hyper-parameters. Deep stochastic configuration networks (DeepSCNs) have become increasingly popular over the past years because of their universal approximation property, fast learning and easy implementation. So far, understanding and setting the hyper-parameters for this type of network is still unexplored. This paper defines a suitable search space for DeepSCN, a performance estimation strategy for finding suitable hyper-parameters with search strategies Monte-Carlo tree search (MCTS) and random search. Simulations are performed using both searches over four benchmark datasets, and results indicate some significant improvements in modelling performance. Furthermore, a case study is presented to demonstrate that an optimised model and hyper-parameters can be found using MCTS.

Stochastic configuration networks for multi-dimensional integral evaluation

Complex multi-dimensional integrals are widely used in various engineering problems. This paper proposes a novel numerical integration method based on stochastic configuration networks (SCNs), which is constructed by learning the integrand function. A corresponding primitive function based on a simple functional expression of the trained SCN can be analytically derived, and a general functional relation between the integrand and the primitive function is established based on SCN parameters. By repeatedly applying the derived functional relations, we can successfully evaluate many complex multi-dimensional integrals. The SCN-based numerical integral method provides a powerful tool for solving complex multi-dimensional integrals. Effectiveness of the proposed method in terms of both computational accuracy and stability is demonstrated through numerical experiments.

Hybrid Parallel Stochastic Configuration Networks for Industrial Data Analytics

As a class of randomized learner model, stochastic configuration networks (SCNs) have been successfully applied in a few data analytics tasks. Given the industrial big data modeling tasks, however, the original SCNs potentially lead to excessive training time. To this end, this article extends SCNs to a hybrid parallel version, termed hybrid parallel SCNs (HPSCNs). In the hybrid parallel learning algorithm, two SCNs are synchronously constructed. The difference between the two of them is that one employs a point-incremental algorithm, and another one adopts a block-incremental algorithm. Moreover, a data parallel method is established based on a dynamic block strategy to accelerate the establishment of candidate node pool for each SCN. Additionally, this article proposes an adaptive hyperparameter adjustment method, which allows the hyperparameters in the supervisory mechanism to be automatically adjusted along with the learning process. Comparative experiments are first conducted through four large-scale benchmark datasets, followed by the fully discussion on the algorithm parameters. Finally, a practical industrial application case is made, where a grinding particle size soft sensor is developed based on HPSCN, showing the effectiveness of the proposed algorithm.

Estimating Evapotranspiration of Pomegranate Trees Using Stochastic Configuration Networks (SCN) and UAV Multispectral Imagery

Evapotranspiration (ET) estimation is important in precision agriculture water management, such as evaluating soil moisture, drought monitoring, and assessing crop water stress. As a traditional method, evapotranspiration estimation using crop coefficient (Kc) has been commonly used. Since there are strong similarities between the Kc curve and the vegetation index curve, the crop coefficient Kc is usually estimated as a function of the vegetation index. Researchers have developed linear regression models for the Kc and the normalized difference vegetation index (NDVI), usually derived from satellite imagery. However, the spatial resolution of the satellite image is often insufficient for crops with clumped canopy structures, such as vines and trees. Therefore, in this article, the authors used Unmanned Aerial Vehicles (UAVs) to collect high-resolution multispectral imagery in a pomegranate orchard located at the USDA-ARS, San Joaquin Valley Agricultural Sciences Center, Parlier, CA. The Kc values were measured from a weighing lysimeter and the NDVI values were derived from UAV imagery. Then, the authors established a relationship between the NDVI and Kc by using a linear regression model and a stochastic configuration networks (SCN) model, respectively. Based on the research results, the linear regression model has an R2 of 0.975 and RMSE of 0.05. The SCN regression model has an R2 and RMSE value of 0.995 and 0.046, respectively. Compared with the linear regression model, the SCN model improved performance in predicting Kc from NDVI. Then, actual evapotranspiration was estimated and compared with lysimeter data in an experimental pomegranate orchard. The UAV imagery provided a spatial and tree-by-tree view of ET distribution.

Fast stochastic configuration network based on an improved sparrow search algorithm for fire flame recognition

Flame image recognition is of great significance in the fire detection and prevention. In this paper, in order to improve the accuracy of fire recognition, a fast stochastic configuration network (FSCN) method based on an improved sparrow search algorithm (ISSA) is proposed. In the design of fast stochastic configuration network (FSCN), the gradual increase of hidden layer nodes in the original stochastic configuration network (SCN) is canceled, and the number of them is set directly. An improved sparrow search algorithm (ISSA) is used to generate the input weights and biases of hidden layer nodes. At the same time, the supervisory mechanism is retained to judge the weights and biases of all hidden layer nodes, and ISSA is used to regenerate corresponding weights and biases for the nodes that do not meet the constraints in the supervisory mechanism. In the ISSA, sine map, adaptive adjustment of hyper-parameters and mutation strategy are used to improve the optimization ability of the original sparrow search algorithm (SSA). Some parameters in FSCN are optimized by ISSA to make it have better classification performance. Finally, the image processing technology is used to extract features from the flame images and the interference images, and then the feature vectors are obtained to train the ISSA-FSCN. Several simulation experiments have been carried out to verify effectiveness of the proposed ISSA-FSCN method. In the performance verification of ISSA on CEC test suit, ISSA averagely outperforms other algorithms by 33.6% in the average results of 20 functions. In the performance verification of FSCN, the average results of accuracy, precision, recall, F1 and auc are compared. In the experiment 1, ISSA-FSCN averagely outperforms other algorithms by 19.7%, 14.7%, 12.8%, 14.5% and 23.0%. In the experiment 2, ISSA-FSCN averagely outperforms other algorithms by 2.3%, 2.1%, 2.5%, 6.0%, 3.2%. In the experiment 3, ISSA-FSCN averagely outperforms other algorithms by 5.9%, 4.0%, 4.1%, 4.1%, 6.8%.

Broad stochastic configuration network for regression

Stochastic configuration network (SCN) is an incremental learning approach with universal approximation property for analyzing high-dimensional and large-scale data. It can start with a small structure and gradually increase hidden layer nodes through the supervision mechanism. Broad learning system (BLS), as a novel structure, has been widely used in various fields because of its high efficiency, generalization performance and scalability. Unlike deep learning models, BLS is proposed based on random vector functional link network (RVFLN) which can map input features to a more appropriate feature space and the mapped features are enhanced as enhancement nodes. Therefore, to enhance the feature learning ability and improve the effectiveness of SCN, inspired by BLS architecture, a novel broad stochastic configuration network (BSCN) is proposed, then we give the structure and theory of BSCN. In this model, the original features are transformed into mapping features in the feature layer and the mapping features are enhanced in the enhancement layer, then the input weights and biases of enhancement nodes are determined according to the supervision mechanism and the output weight matrix can be calculated through the standard least squares. The experimental results on function approximation problems and real-world datasets indicate that BSCN improves the performance of SCN, and it has higher regression accuracy and stability.

A Lightweight Learning Method for Stochastic Configuration Networks Using Non-Inverse Solution

Stochastic configuration networks (SCNs) face time-consuming issues when dealing with complex modeling tasks that usually require a mass of hidden nodes to build an enormous network. An important reason behind this issue is that SCNs always employ the Moore–Penrose generalized inverse method with high complexity to update the output weights in each increment. To tackle this problem, this paper proposes a lightweight SCNs, called L-SCNs. First, to avoid using the Moore–Penrose generalized inverse method, a positive definite equation is proposed to replace the over-determined equation, and the consistency of their solution is proved. Then, to reduce the complexity of calculating the output weight, a low complexity method based on Cholesky decomposition is proposed. The experimental results based on both the benchmark function approximation and real-world problems including regression and classification applications show that L-SCNs are sufficiently lightweight.

Hybrid modeling for submergence depth of the pumping well using stochastic configuration networks with random sampling

In oilfield production, the submergence depth of a pumping well is an important indicator for measuring the fullness of the pumping pump, determining reasonable oil well production parameters, and formulating efficient production measures. In order to solve the problems of high security risks, high production costs and large errors, a method of submergence depth prediction based on a hybrid model is proposed in this paper. Firstly, according to the production process of oil recovery, the analytical model is built to ensure that the submergence depth prediction meets the actual production requirements; then, the data-driven model is used to compensate the prediction results of the analytical model, which in order to reduce the influence of various production parameters on the prediction accuracy. When using stochastic configuration networks (SCNs) to build the data model, it may encounter the problem that the generalization ability is easily affected by the noise of training set, outliers and multi-states. So, a SCNs modeling method based on random sampling strategy is proposed to solve this problem. Random sampling was conducted on the training set, and different training subsets are evaluated by defining V indexes, which not only considers the prediction accuracy, but also considers the stability of the model. Through comparisons of SCNs and SCNs with random sampling, the latter approach can reduce the prediction error at least by 11.65% in the training phase, and at least 62.19% in the testing phase. The simulation experiment results of predicted submergence depth show that, compared with other methods, the proposed SCNs with random sampling method can reduce the average prediction error at least by 1.95%, 30.52%, 0.28% and 34.36% at RMSE, MAE, MSE and MAPE for the testing set. These prove that the proposed model has advantages in comprehensive prediction performance and can meet actual production needs.