Abstract

This paper studies a fundamental dynamic clustering problem. The input is an online sequence of pairwise communication requests between n nodes (e.g., tasks or virtual machines). Our goal is to minimize the communication cost by partitioning the communicating nodes into ell clusters (e.g., physical servers) of size k (e.g., number of virtual machine slots). We assume that if the communicating nodes are located in the same cluster, the communication request costs 0; if the nodes are located in different clusters, the request is served remotely using inter-cluster communication, at cost 1. Additionally, we can migrate: a node from one cluster to another at cost alpha ge 1. We initiate the study of a stochastic problem variant where the communication pattern follows a fixed distribution, set by an adversary. Thus, the online algorithm needs to find a good tradeoff between benefitting from quickly moving to a seemingly good configuration (of low inter-cluster communication costs), and the risk of prematurely ending up in a configuration which later turns out to be bad, entailing high migration costs. Our main technical contribution is a deterministic online algorithm which is O(log {n})-competitive with high probability (w.h.p.), for a specific but fundamental class of problems: namely on ring graphs. We also provide first insights in slightly more general models, where the adversary is not restricted to a fixed distribution or the ring.

Highlights

  • Modern distributed systems are often highly virtualized and feature unprecedented resource allocation flexibilities

  • Our contributions This paper initiates the study of a natural dynamic clustering problem where communication patterns follow an unknown distribution, chosen by an adversary: the distribution represents the worst-case for the given online algorithm, and communication requests are drawn i.i.d. from this distribution

  • – Local migrations and growing-radius search strategy: In order to avoid high migration costs, our online algorithm is local in the sense that it only moves to nearby cuts/configurations once the condition of the current configuration is met and it needs to be eliminated

Read more

Summary

Introduction

Modern distributed systems are often highly virtualized and feature unprecedented resource allocation flexibilities. The clustering problem is challenging as the detailed communication patterns are often stochastic and the specific distribution unknown ahead of time. Our contributions This paper initiates the study of a natural dynamic clustering problem where communication patterns follow an unknown distribution, chosen by an adversary: the distribution represents the worst-case for the given online algorithm, and communication requests are drawn i.i.d. from this distribution. We are interested in the online problem variant: we assume that the distribution D of the communication pattern (and the σ we observe is generated from) is initially unknown to the online algorithm. This communication pattern is fundamental and captures the aspects and inherent tradeoffs rendering the problem non-trivial In this model, an algorithm changes configurations using rotations (either clockwise or counter-clockwise).

The challenge of dynamic clustering
Deterministic and competitive clustering
Lazy expansion
Analysis
A cost breakdown
Competitive ratio
Elimination costs
Migration cost
Insurance costs
Final costs
Beyond stochastic adversary
Ring communication pattern
More general communication pattern
Related work
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call