Coarse-grained traffic matrix estimation for data center networks

Abstract

We address the problem of estimating the real-time traffic flows in data center networks (DCNs), using the light-weight SNMP data. Unlike the problem of estimating the traffic matrix (TM) across origin–destination (OD) pairs in ISP networks, the traffic flows across servers or Top of Rack (ToR) switch pairs in DCNs are notoriously more irregular and volatile. Although numerous methods have been proposed in past several years to solve the TM estimation problem in ISP networks, none of them could be applied to DCNs directly. In this paper, we make the first step to solve the TM estimation problem in DCNs by leveraging the characteristics of prevailing data center architectures and decomposing the topologies of DCNs, which makes TM estimation problems in DCNs easy to handle. We also state a basic theory to obtain the aggregate traffic characteristics of these clusters unbiasedly. We propose two efficient TM estimation algorithms based on the decomposed topology and the aggregate traffic information, which improves the state-of-the-art tomography methods without requiring any additional instrumentation. Finally, we compare our proposal with a recent representative TM estimation algorithm through both real experiments and extensive simulations, the results show that, (i) the data center TM estimation problem could be well handled after the decomposition step, (ii) our two algorithms outperforms the former one in both speed and accuracy.