Multi-level optimizing of parameters in stochastic configuration networks based on cloud model and nutcracker optimization algorithm

Abstract

Listen

Translate article

As a state-of-the-art neural network model, stochastic configuration networks (SCNs) are widely employed in diverse fields due to their exceptional approximation capabilities. Similar to other neural network models, an excessive number of parameters can potentially compromise the generalization ability of SCNs, including hyper-parameters such as the stochastic scale factors (λ) and the maximum number of nodes (Lmax), as well as model parameters like input weight (w) and input bias (b). To tackle this issue, this study proposes a multi-level parameter optimization approach, termed stochastic configuration network with cloud models (CMSCNs). Firstly, the optimal parameter range is determined based on the concept of “cloud droplet” from the cloud model. Herein, mathematical expectation (Ex) is substituted by a polynomial function constructed with λ as the dependent variable and other parameters as independent variables. Secondly, we employ the nutcracker optimization algorithm (NOA) to optimize w and b, using residuals as evaluation indices to identify their optimal combination. Thirdly, singular value decomposition (SVD) is integrated to compress the network structure of CMSCNs for enhanced computational efficiency. Finally, 18 public real datasets and submergence depth data from an oil well are utilized to assess the performance of CMSCNs. The experimental results demonstrate that our proposed method offers improved generalizability and stability while also exhibiting significant potential in practical applications.