Greedy stochastic configuration networks for ill-posed problems

Abstract

The inequality constraint in stochastic configuration networks (SCNs) is key to their universal approximation capability. SCNs incrementally add new units to neural networks under a supervisory mechanism. However, with an increasing number of hidden nodes, the output weights which are evaluated by the least squares method, are frequently unstable. This instability phenomenon (arbitrarily small perturbations leading to large changes in the solutions) can be classified as an ill-posed problem. Instability is one of the most important manifestations of ill-posed problems, and the Tikhonov regularization method, which is widely used to address ill-posed problems, is undesirable. This study examined the supervisory mechanism of SCNs and algebraic properties of the hidden output matrix, thereby proposing a new greedy stochastic configuration network (GSCN) for ill-posed problems. First, under the premise of ensuring universal approximation capability, GSCN tries to optimize the randomly assigned input weights and biases based on the hunter–prey optimization (HPO) algorithm. The optimized hidden parameters which can reduce residual error the most are selected for the newly added hidden nodes. Second, singular value decomposition (SVD) and orthogonal–triangular (QR) decomposition with column pivoting are introduced to extract a linearly independent subset of the hidden output matrix for ill-posed problems in SCNs. Finally, one function approximation, six benchmark regressions, two classification datasets, and bearing fault diagnosis are employed to evaluate the performance of GSCN. Experimental results indicate that GSCN shows superior performance with respect to fast convergence, generalization, and stability compared with other contrast methods. This code is available at https://github.com/Gzu-public-bigdata/GSCN.