Online Supervised Learning with Distributed Features over Multiagent System

Xibin An,Chen Hu,Bing He,Bingqi Liu,Ning Cai

doi:10.1155/2020/8830359

Abstract

Most current online distributed machine learning algorithms have been studied in a data-parallel architecture among agents in networks. We study online distributed machine learning from a different perspective, where the features about the same samples are observed by multiple agents that wish to collaborate but do not exchange the raw data with each other. We propose a distributed feature online gradient descent algorithm and prove that local solution converges to the global minimizer with a sublinear rate O 2 T . Our algorithm does not require exchange of the primal data or even the model parameters between agents. Firstly, we design an auxiliary variable, which implies the information of the global features, and estimate at each agent by dynamic consensus method. Then, local parameters are updated by online gradient descent method based on local data stream. Simulations illustrate the performance of the proposed algorithm.

Highlights

With the development of multiagent system, the observed data are being generated at anywhere, anytime, using different devices and technologies [1,2,3]. ere is a lot of interest in extracting knowledge from this massive amount of data and using it to choose a suitable business strategy [4,5,6], to generate control command [7,8,9] or to make a decision [10,11,12,13]
Many applications are required to process incoming data in online way, e.g., a bank monitors the transactions of its clients to detect frauds [2], wireless sensor networks makes inference [14], and sensor network tracks the uncooperative target [15]. e study of online learning is becoming an important topic of research itself [16,17,18]
Compared with VRD2 [12], feature distributed machine learning (FDML) [1], and the sharing alternating direction method of multipliers (ADMM) algorithm [23], distributed feature online gradient (DFOG) is applicable to online supervised learning with distributed features over networks

Summary

Introduction

With the development of multiagent system, the observed data are being generated at anywhere, anytime, using different devices and technologies [1,2,3]. ere is a lot of interest in extracting knowledge from this massive amount of data and using it to choose a suitable business strategy [4,5,6], to generate control command [7,8,9] or to make a decision [10,11,12,13]. Some distributed machine learning algorithms have been proposed to train a model by letting each agent perform local model updates and exchange some information between neighbors [19,20,21,22]. Ese algorithms in [1, 12, 23] effectively deal with the batch distributed feature learning in a distributed form These algorithms in [1, 12, 23] need to access the entire dataset and cannot be applied in online settings. Compared with VRD2 [12], FDML [1], and the sharing ADMM algorithm [23], DFOG is applicable to online supervised learning with distributed features over networks. Let Rd be the d-dimensional vector space and ‖ω‖22 is the Euclidean norm of a vector ω ∈ Rd

Problem Formulation

Distributed Feature Online Gradient Descent Algorithm

Algorithm Analysis

Simulation

Conclusions