Manifold regularization based distributed semi-supervised learning algorithm using extreme learning machine over time-varying network

Abstract

This paper aims to propose a distributed semi-supervised learning (SSL) algorithm using extreme learning machine (ELM) over time-varying communication network, whose topology changes over time rather than being fixed. In distributed SSL problems, training data including labeled and unlabeled samples is separately stored on each node over the communication network and cannot be centrally processed. In order to solve these problems, we propose an algorithm combining the semi-supervised ELM (SS-ELM) algorithm with the zero-gradient-sum (ZGS) distributed optimization strategy. The SS-ELM algorithm, based on the manifold regularization (MR) framework, is used to approximate the mapping of samples on each node over the communication network. Then, the ZGS strategy is used to train the globally optimal coefficients of the single layer feed-forward neural network (SLFNN) corresponding to the SS-ELM algorithm. Thus, we denote the proposed algorithm as the distributed SS-ELM (DSS-ELM) algorithm. During the training process, all nodes over the communication network exchange updated coefficients rather than raw data with their neighboring nodes. It means that the DSS-ELM algorithm is a fully distributed and privacy-preserving algorithm. The convergence of the proposed DSS-ELM is guaranteed by the Lyapunov method. At last, some simulations are presented to show the efficiency of the proposed algorithm.