Distributed Online Algorithm Research Articles

Accelerated distributed expectation-maximization algorithms for the parameter estimation in multivariate Gaussian mixture models

Rapid development for modeling big data requires effective and efficient methods for estimating the parameters involved. Although several accelerated Expectation-Maximization algorithms have been developed, there still exist two major concerns: reducing computational cost and improving model estimation accuracy. We propose three distributed-like algorithms for multivariate Gaussian mixture models, which can accelerate speed and improve estimation accuracy. The first algorithm is distributed algorithm, which is used to speed up the calculation of classic algorithms and improve its estimation accuracy by averaging the one-step estimators obtained from distributed operators. The second algorithm is distributed online algorithm, which is a distributed stochastic approximation procedure that performs online updates when reading online data. The final algorithm is called distributed monotonically over-relaxed algorithm, which uses an over-relaxation factor and a distributing strategy to improve the estimation accuracy of multivariate Gaussian mixture models. We investigate the stability, sensitivity, convergence, and robustness of these algorithms in a numerical study. We also apply these algorithms to three real data sets for validation.

RHDOFS: A Distributed Online Algorithm Towards Scalable Streaming Feature Selection

Feature selection is an important topic in data mining and machine learning, which aims to select an optimal feature subset for building effective and explainable prediction models. This paper introduces Rough Hypercuboid based Distributed Online Feature Selection (RHDOFS) method to tackle two critical challenges of Volume and Velocity associated with Big Data. By exploring the class separability in the boundary region of rough hypercuboid approach, a novel integrated feature evaluation criterion is proposed by examining not only the explicit patterns contained in the positive region but also the useful implicit patterns derived from the boundary region. An efficient online feature selection method for streaming feature scenario is developed to identify relevant and nonredundant features in an incremental iterative fashion. Furthermore, a parallel optimization mechanism by combining both data and computational independence is further employed to accelerate the original sequential implementation. An efficient distributed online feature selection algorithm is presented and implemented on the Apache Spark platform to scale for massive amount of data by exploiting the computational capabilities of multicore clusters. Encouraging results of extensive experiments indicate the superiority and notable advantages of the proposed algorithm over the relevant and representative online feature selection algorithms. Empirical tests on scalability and extensibility also demonstrate our distributed implementation significantly reduces the computational times requirements while maintaining the prediction accuracy, and is capable of scaling well in volume of data and number of computing nodes.

A Distributed Online Algorithm for Promoting Energy Sharing Between EV Charging Stations

In recent years, electric vehicle (EV) charging stations have experienced an increasing supply-demand mismatch due to their fluctuating renewable generation and unpredictable charging demands. To reduce their operating costs, this paper proposes a distributed online algorithm to promote energy sharing between charging stations. We begin with the offline and centralized version of the EV charging stations operation problem, whose objective is to minimize the long-term time-average total cost. Then, we develop an online implementation approach based on the Lyapunov optimization framework. Although the proposed online algorithm runs in a prediction-free manner, we prove that by properly choosing the parameters, the time-coupling constraints remain satisfied. We also provide a theoretical bound for the optimality gap between the offline and online optimums. Furthermore, an improved alternating direction method of multipliers (ADMM) algorithm with an iteration truncation is proposed to enable distributed computation. The proposed algorithm can protect privacy while being suitable for online implementation. Case studies validate the effectiveness of the theoretical results. Performance comparisons are carried out to demonstrate the advantages of the proposed method.

TODG: Distributed Task Offloading With Delay Guarantees for Edge Computing

Edge computing has been an efficient way to provide prompt and near-data computing services for resource-and-delay sensitive IoT applications via computation offloading. Effective computation offloading strategies need to comprehensively cope with several major issues, including 1) the allocation of dynamic communication and computational resources, 2) delay constraints of heterogeneous tasks, and 3) requirements for computationally inexpensive and distributed algorithms. However, most of the existing works mainly focus on part of these issues, which would not suffice to achieve expected performance in complex and practical scenarios. To tackle this challenge, in this paper, we systematically study a distributed computation offloading problem with delay constraints, where heterogeneous computational tasks require continually offloading to a set of edge servers via a limiting number of stochastic communication channels. The task offloading problem is formulated as a delay-constrained long-term stochastic optimization problem under unknown prior statistical knowledge. To solve this problem, we first provide a technical path to transform and decompose it into several slot-level sub-problems. Then, we devise a distributed online algorithm, namely TODG, to efficiently allocate resources and schedule offloading tasks. Further, we present a comprehensive analysis for TODG in terms of the optimality gap, the worst-case delay, and the impact of system parameters. Extensive simulation results demonstrate the effectiveness and efficiency of TODG.

Distributed Online Optimization in Time-Varying Unbalanced Networks Without Explicit Subgradients

This paper studies a distributed online constrained optimization problem over time-varying unbalanced digraphs without explicit subgradients. In sharp contrast to the existing algorithms, we design a novel consensus-based distributed online algorithm with a local randomized zeroth-order oracle and then rescale the oracle by constructing row-stochastic matrices, which aims to address the unbalancedness of time-varying digraphs. Under mild conditions, the average dynamic regret over a time horizon is shown to asymptotically converge at a sublinear rate provided that the accumulated variation grows sublinearly with a specific order. Moreover, the counterpart of the proposed algorithm when subgradients are available is also provided, along with its dynamic regret bound, which reflects that the convergence of our algorithm is essentially not affected by the zeroth-order oracle. Simulations on distributed targets tracking problem and dynamic sparse signal recovery problem in sensor networks are employed to demonstrate the effectiveness of the proposed algorithm.

A distributed real-time pricing strategy based on reinforcement learning approach for smart grid

The real-time pricing (RTP) plays an important role in demand side management of smart grid. In this paper, a distributed RTP strategy which takes the interests of both supply and demand sides into consideration is studied. A holistic model focusing on the interactions between users and the power supplier is proposed in the framework of Markov decision process (MDP). The MDP presentation of smart appliances’ operational processes well embodies their characteristics and the energy correlation of adjacent time slots. Different from existing ones, a novel distributed online algorithm based on a reinforcement learning approach is proposed to solve the MDP model without acquisition of the transition probabilities. Through information exchange between users and the power supplier, the real-time electricity price is decided adaptively, meanwhile, the optimal strategy of power supply and consumption is obtained. The specific information of the utility function for each user does not need to be disclosed to the supplier and other users. Simulation results show that the proposed model and algorithm balance energy supply and demand well, and have a good performance in peak shaving and valley filling.

A scalable distributed online algorithm for optimal power flow in distribution system

In this paper, a scalable distributed online algorithm is proposed for solving optimal power flow (OPF) in real time. The operation of distribution systems can be continuously driven towards given OPF targets only depending on local and some aggregated information of each distribution area. Particularly, the proposed algorithm is applicable to the distribution system where each area operates in arbitrary topologies. The algorithm uses the primal-dual gradient method to update DER set-points recurrently based on the sensitivities of the OPF targets; moreover, we decompose the sensitivities into several separate items and give the method to calculate these separate items in a distributed way. The decomposition and distributed calculation methods are all deduced under the general power flow model that completely specifies the physics of electrical networks with arbitrary topologies. The effectiveness of the algorithm is verified using a modified 502-node distribution system. Compared with the online algorithms requiring to process some operation details in a centralized way, the proposed algorithm can achieve almost the same convergence properties with reducing the calculation time sharply.

Optimal coordination of virtual power plant with photovoltaics and electric vehicles: A temporally coupled distributed online algorithm

This paper presents a temporally coupled distributed online (TDO) algorithm to aggregate and coordinate numerous networked distributed energy resources (DERs) as a virtual power plant (VPP). A centralized stochastic optimization problem is formulated to minimize the long-term social utility loss while satisfying the voltage security, operational requirements of DERs, and VPP service requests. After that, we propose the TDO algorithm to reformulate the primary problem as an adaptation of online convex optimization (OCO). In particular, the temporally coupled constraints are well separated to each timeslot. In real-time operation, the VPP operator updates the incentives according to the measurement feedback. The smart energy gateways (SEGs) equipped at each node maximize their income and utility based on the received incentive signals through adjusting the setpoints of the governed photovoltaics (PV) inverters and electric vehicles (EVs). Unlike conventional distributed optimization algorithms where complicated iterative procedures between agents are unavoidable, the proposed TDO algorithm is computation- and communication-efficient since the SEG can directly employ the closed-form optimal setpoints without iterative communications once receiving the incentives. Furthermore, we design an incentive scheme to coordinate the SEGs based on the privacy protected nonintrusive measurements instead of direct control. Optimality and convergency of TDO are analyzed mathematically. Finally, the proposed method is corroborated numerically on a modified 33-node test feeder. A larger system is tested to validate the computational time performance.

An Efficient Distributed Area Division Method for Cooperative Monitoring Applications with Multiple UAVs

This article addresses the area division problem in a distributed manner providing a solution for cooperative monitoring missions with multiple UAVs. Starting from a sub-optimal area division, a distributed online algorithm is presented to accelerate the convergence of the system to the optimal solution, following a frequency-based approach. Based on the “coordination variables” concept and on a strict neighborhood relation to share information (left, right, above and below neighbors), this technique defines a distributed division protocol to determine coherently the size and shape of the sub-area assigned to each UAV. Theoretically, the convergence time of the proposed solution depends linearly on the number of UAVs. Validation results, comparing the proposed approach with other distributed techniques, are provided to evaluate and analyze its performance following a convergence time criterion.

Distributed Event-Triggered Control for Cooperative Output Regulation of Multiagent Systems With an Online Estimation Algorithm.

In this article, the cooperative output regulation problem of heterogeneous multiagent systems has been investigated. It is assumed that only a few agents could know the system matrix of the exosystem and no agent knows the topological information. Under these conditions, a novel distributed online algorithm is proposed to estimate the information relevant to the topology. Based on this algorithm, a distributed event-triggered adaptive observer is designed such that each agent can observe the exosystem. It is theoretically shown that the proposed distributed controller will make the multiagent system achieve cooperative output regulation asymptotically. Finally, a simulation is presented to show the effectiveness of the result.

Charging Task Scheduling for Directional Wireless Charger Networks

This paper studies the problem of c <u>H</u> arging t <u>A</u> sk <u>S</u> cheduling for direc <u>T</u> ional wireless charg <u>E</u> r networks (HASTE), i.e., given a set of rotatable directional wireless chargers on a 2D area and a series of offline (online) charging tasks, scheduling the orientations of all the chargers with time in a centralized offline (distributed online) fashion to maximize the overall charging utility for all the tasks. We prove that HASTE is NP-hard. Then, we prove that a relaxed version of HASTE falls within the realm of maximizing a submodular function subject to a partition matroid constraint, and propose a centralized offline algorithm that achieves <inline-formula><tex-math notation="LaTeX">$(1-\rho)(1-\frac{1}{e})$</tex-math></inline-formula> approximation ratio to address HASTE where <inline-formula><tex-math notation="LaTeX">$\rho$</tex-math></inline-formula> is the switching delay of chargers. Further, we propose a distributed online algorithm and prove it achieves <inline-formula><tex-math notation="LaTeX">$\frac{1}{2}(1-\rho)(1-\frac{1}{e})$</tex-math></inline-formula> competitive ratio. We conduct simulations and field experiments on a testbed consisting of eight off-the-shelf power transmitters and 8 rechargeable sensor nodes. The results show that our distributed online algorithm achieves 92.97 percent of the optimal charging utility, and outperforms the comparison algorithms by up to 15.28 percent in terms of charging utility.

Efficient DistributedWorkload (Re-)Embedding

Modern networked systems are increasingly reconfigurable, enabling demand-aware infrastructures whose resources can be adjusted according to the workload they currently serve. Such dynamic adjustments can be exploited to improve network utilization and hence performance, by moving frequently interacting communication partners closer, e.g., collocating them in the same server or datacenter. However, dynamically changing the embedding of workloads is algorithmically challenging: communication patterns are often not known ahead of time, but must be learned. During the learning process, overheads related to unnecessary moves (i.e., re-embeddings) should be minimized. This paper studies a fundamental model which captures the tradeoff between the benefits and costs of dynamically collocating communication partners on *#953; servers, in an online manner. Our main contribution is a distributed online algorithm which is asymptotically almost optimal, i.e., almost matches the lower bound (also derived in this paper) on the competitive ratio of any (distributed or centralized) online algorithm.

Asynchronous Online Learning in Multi-Agent Systems With Proximity Constraints

We consider the problem of distributed learning from sequential data via online convex optimization. A multi-agent system is considered where each agent has a private objective but is willing to cooperate in order to minimize the network cost, which is the sum of local cost functions. Different from the classical distributed settings, where the agents coordinate through the use of consensus constraints, we allow the neighboring agent actions to be related via a non-linear proximity function. A decentralized saddle point algorithm is proposed that is capable of handling gradient delays arising from computational issues. The proposed online asynchronous algorithm is analyzed under adversarial settings by developing bounds on the regret of O(√T), which measures the cumulative loss incurred by the online algorithm against a clairvoyant, and network discrepancy of O(T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3/4</sup> ), which measures the cumulative constraint violation or agent disagreement. By allowing the agents to utilize stale gradient information, the proposed algorithm embraces the nuances of distributed learning and serves to be the first distributed online algorithm that can handle adversarial delays. A modified saddle point algorithm is also proposed that explicitly forces the agents to agree as per the constraint function resulting in zero network discrepancy while incurring a slightly higher regret. To showcase the efficacy of the proposed asynchronous algorithm, a spatially correlated random field estimation problem is formulated and solved. Additionally, an application of vision-based target localization with moving cameras demonstrates the benefits of this approach in practice.

Planning While Flying: A Measurement-Aided Dynamic Planning of Drone Small Cells

The deployment of drone small cells has emerged as a promising solution to agile provisioning of Internet backbone access for Internet of Things devices, and many other types of users/devices. In this paper, we consider the problem of deploying a set of drone cells operating on multiple channels in a target area to provide access to the backbone/core network, which is formulated as a combinatorial network utility maximization problem. Since an offline and centralized solution to such a problem is not feasible, a low-complexity and distributed online algorithm is highly desired. Therefore, we propose a measurement-aided dynamic planning (MAD-P) algorithm, where the dispatched drones perform position and channel configurations autonomously on the fly based on the real-time measurement of network throughput to solve the problem in a distributed fashion during flight with minimal centralized control. We prove that the proposed MAD-P algorithm is asymptotically optimal, and investigate how long it takes for the convergence to stationarity under the MAD-P algorithm by giving a mixing time analysis. We also derive an upper bound of the performance gap in presence of measurement errors. Simulation results are provided to validate our analytic results and demonstrate the effectiveness of our algorithm.

Efficient Distributed Workload (Re-)Embedding

Modern networked systems are increasingly reconfigurable, enabling demand-aware infrastructures whose resources can be adjusted according to the workload they currently serve. Such dynamic adjustments can be exploited to improve network utilization and hence performance, by moving frequently interacting communication partners closer, e.g., collocating them in the same server or datacenter. However, dynamically changing the embedding of workloads is algorithmically challenging: communication patterns are often not known ahead of time, but must be learned. During the learning process, overheads related to unnecessary moves (i.e., re-embeddings) should be minimized. This paper studies a fundamental model which captures the tradeoff between the benefits and costs of dynamically collocating communication partners on $\ell$ servers, in an online manner. Our main contribution is a distributed online algorithm which is asymptotically almost optimal, i.e., almost matches the lower bound (also derived in this paper) on the competitive ratio of any (distributed or centralized) online algorithm. As an application, we show that our algorithm can be used to solve a distributed union find problem in which the sets are stored across multiple servers.

ADMM-Based Distributed Online Algorithm for Energy Management in Hybrid Energy Powered Cellular Networks

Energy harvesting has been considered as a promising technique to decrease the conventional grid energy expenditure. However, most renewable energy sources are unreliable and random. To overcome these drawbacks, besides the commonly adopted approaches such as purchasing power from the grid and deploying batteries, energy cooperation is an appealing solution. In this paper, we investigate the energy management problem by jointly optimizing the data admission rate, transmit power, energy sharing among base stations (BSs), battery charging and discharging rate, and the energy purchased from the grid in hybrid energy powered cellular networks. First, the long-term average total cost minimization problem under the constraints on limited battery size and users' data rate requirements is formulated as a stochastic optimization problem. Employing the Lyapunov optimization technique and alternating direction method of multipliers (ADMM), we propose an online distributed algorithm, referred to as distributed online energy management algorithm (DOEMA), where the current system states are needed, without requiring the system statistic or future information. Furthermore, the proposed algorithm can be implemented in a parallel and completely distributed fashion, which could provide more engineering guidelines for practical communication protocols compared with the centralized algorithm. The extensive simulation results are conducted to demonstrate the correctness of the theoretical analysis and validate the performance improvement against other algorithms in terms of system total cost reduction.

Online Learning in Weakly Coupled Markov Decision Processes

We consider multiple parallel Markov decision processes (MDPs) coupled by global constraints, where the time varying objective and constraint functions can only be observed after the decision is made. Special attention is given to how well the decision maker can perform in T slots, starting from any state, compared to the best feasible randomized stationary policy in hindsight. We develop a new distributed online algorithm where each MDP makes its own decision each slot after observing a multiplier computed from past information. While the scenario is significantly more challenging than the classical online learning context, the algorithm is shown to have a tight O(√T) regret and constraint violations simultaneously. To obtain such a bound, we combine several new ingredients including ergodicity and mixing time bound in weakly coupled MDPs, a new regret analysis for online constrained optimization, a drift analysis for queue processes, and a perturbation analysis based on Farkas' Lemma.

Online Learning in Weakly Coupled Markov Decision Processes

We consider multiple parallel Markov decision processes (MDPs) coupled by global constraints, where the time varying objective and constraint functions can only be observed after the decision is made. Special attention is given to how well the decision maker can perform in T slots, starting from any state, compared to the best feasible randomized stationary policy in hindsight. We develop a new distributed online algorithm where each MDP makes its own decision each slot after observing a multiplier computed from past information. While the scenario is significantly more challenging than the classical online learning context, the algorithm is shown to have a tight O (√ T ) regret and constraint violations simultaneously. To obtain such a bound, we combine several new ingredients including ergodicity and mixing time bound in weakly coupled MDPs, a new regret analysis for online constrained optimization, a drift analysis for queue processes, and a perturbation analysis based on Farkas' Lemma.

Interoperator Opportunistic Spectrum Sharing in LTE-Unlicensed

Extending long-term evolution (LTE) to unlicensed bands, termed LTE-unlicensed, is gaining significant interest recently. The coexistence of LTE-unlicensed with the current users of unlicensed bands (such as Wi-Fi) and the interference among LTE-unlicensed users themselves are the two challenges for the success of this new technology. In this paper, we propose a novel distributed online algorithm for opportunistic sharing of unlicensed bands among LTE-unlicensed base stations (BSs) while guaranteeing the quality of service (QoS) of user equipment (UE). We first derive a Lyapunov optimization-based algorithm for BSs to evaluate the true value of unlicensed spectrum, guarantee a maximum delay, and minimize the packet drop rate. We then develop a distributed auction mechanism to maximize the social welfare in each auction and enable optimal spectrum reuse. We prove that BSs bid truthfully with the proposed algorithm, while UEs’ QoS requirements on delay and packet drop rate can be guaranteed with bounded optimality gaps. We also reveal an interesting tradeoff between the delay and packet drop rate. The proposed algorithm is validated with simulations.

Optimal Online Data Dissemination for Resource Constrained Mobile Opportunistic Networks

Delivery delay and communication costs are two conflicting design issues for mobile opportunistic networks with nonreplenishable energy resources. In this paper, we study the optimal data dissemination for resource constrained mobile opportunistic networks, i.e., the delay-constrained least-cost multicasting in mobile opportunistic networks. We formally formulate the problem and introduce a centralized heuristic algorithm which aims to discover a tree for multicasting, in order to meet the delay constraint and achieve low communication cost. While the above algorithm can be implemented by each individual node, it is intrinsically centralized (requiring global information) and, thus, impractical for real-world implementation. However, it offers useful insights for the development of a distributed scheme. The essence of the centralized approach is to first learn the probabilities to deliver the data along different paths to different nodes and then decide the optimal multicast tree by striking the balance between cost and delivery probability. In mobile opportunistic networks, even if the optimal routing tree can be computed by the centralized solution, it is the “best” only on a statistic basis for a large number of data packets. It is not necessarily the best solution for every individual transmission. Based on the above observation, we develop a distributed online algorithm using optimal stopping theory, in which in each meeting event, nodes make adaptive online decisions on whether this communication opportunity should be exploited to deliver data packets. We carry out simulations to evaluate the scalability of the proposed schemes. Furthermore, we prototype the proposed distributed online multicast algorithm using Nexus tablets and conduct an experiment that involves 37 volunteers and lasts for 21 days to demonstrate its effectiveness.