Greedy deep stochastic configuration networks ensemble with boosting negative correlation learning

Abstract

Deep stochastic configuration networks (DSCNs) employ data-dependent supervision mechanism to randomly assign node parameters and incrementally construct the deep neural network structure, thereby ensuring the model's universal approximation property. To build a random neural networks ensemble model with better generalization performance, we propose a novel greedy deep stochastic configuration networks ensemble model based on boosting negative correlation learning, termed as GDSCNE. Firstly, greedy optimization strategy based on inequality constraints is utilized to generate random parameters of base components with multi-layer architecture, which can accelerate the decline of network residuals when configuring a new node. Additionally, boosting negative correlation learning framework is presented for the base components ensemble process, which uses least square approach with negative correlation learning penalty term to update the ensemble output weights for each base component, subsequently, boosting method is applied to construct a stronger ensemble model by adaptive weighting through the results of base components. Finally, we evaluated GDSCNE on the popular regression benchmark datasets from the KEEL, experimental results demonstrate that GDSCNE outperforms state-of-the-art random learning algorithms in terms of regression accuracy and generalization performance across several regression datasets with varying sizes.