Abstract
Most current online distributed machine learning algorithms have been studied in a data-parallel architecture among agents in networks. We study online distributed machine learning from a different perspective, where the features about the same samples are observed by multiple agents that wish to collaborate but do not exchange the raw data with each other. We propose a distributed feature online gradient descent algorithm and prove that local solution converges to the global minimizer with a sublinear rate O 2 T . Our algorithm does not require exchange of the primal data or even the model parameters between agents. Firstly, we design an auxiliary variable, which implies the information of the global features, and estimate at each agent by dynamic consensus method. Then, local parameters are updated by online gradient descent method based on local data stream. Simulations illustrate the performance of the proposed algorithm.
Highlights
With the development of multiagent system, the observed data are being generated at anywhere, anytime, using different devices and technologies [1,2,3]. ere is a lot of interest in extracting knowledge from this massive amount of data and using it to choose a suitable business strategy [4,5,6], to generate control command [7,8,9] or to make a decision [10,11,12,13]
Many applications are required to process incoming data in online way, e.g., a bank monitors the transactions of its clients to detect frauds [2], wireless sensor networks makes inference [14], and sensor network tracks the uncooperative target [15]. e study of online learning is becoming an important topic of research itself [16,17,18]
Compared with VRD2 [12], feature distributed machine learning (FDML) [1], and the sharing alternating direction method of multipliers (ADMM) algorithm [23], distributed feature online gradient (DFOG) is applicable to online supervised learning with distributed features over networks
Summary
With the development of multiagent system, the observed data are being generated at anywhere, anytime, using different devices and technologies [1,2,3]. ere is a lot of interest in extracting knowledge from this massive amount of data and using it to choose a suitable business strategy [4,5,6], to generate control command [7,8,9] or to make a decision [10,11,12,13]. Some distributed machine learning algorithms have been proposed to train a model by letting each agent perform local model updates and exchange some information between neighbors [19,20,21,22]. Ese algorithms in [1, 12, 23] effectively deal with the batch distributed feature learning in a distributed form These algorithms in [1, 12, 23] need to access the entire dataset and cannot be applied in online settings. Compared with VRD2 [12], FDML [1], and the sharing ADMM algorithm [23], DFOG is applicable to online supervised learning with distributed features over networks. Let Rd be the d-dimensional vector space and ‖ω‖22 is the Euclidean norm of a vector ω ∈ Rd
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
