Distributed stochastic configuration networks with cooperative learning paradigm

Abstract

As a new category of randomized neural networks (RNNs), stochastic configuration networks (SCNs) have demonstrated great potential for data analytics. Unlike conventional randomized learning techniques, e.g., random vector functional-link (RVFL) networks, SCNs provide a stochastic configuration mechanism on the assignment of input parameters which guarantees the universal approximation capability of a resulting learner model. In this paper, a distributed version of SCN is developed for decentralized datasets in cooperative learning paradigm. This paper proposes an approach to deal with datasets stored across a network of multiple learning agents without any fusion center. Specifically, we formulate the centralized learning problem as an equivalent form with the decomposition of subproblems coupled in a network and a consensus restriction. Then, a cooperative configuration scheme is proposed for randomly assigning the input weights and bias. Finally, based on the well-known parallel alternating direction method of multipliers (ADMM), the output weights are evaluated iteratively. Simulation studies with comparisons on three benchmark datasets are carried out. The experimental results indicate that our proposed learning scheme performs well and outperforms distributed RVFL networks.