Cooperative Monitoring for Internet Data Centers

Abstract

Recent outages in several Web services have demonstrated the potential damage of availability disruptions, since millions of end users rely on these services powered by thousands of servers in large-scale Internet data centers. As Internet data centers continue to grow in scale and complexity, it has become a daunting task to monitor and manage thousands of servers simultaneously. This paper presents a cooperative monitoring framework to continuously monitor availability of thousands of servers in Internet data centers. We propose a simple yet effective algorithm for locating monitor nodes for the purposes of load balancing and resilience, and demonstrate the performance of this method through simulations based on dataset collected from a large Internet content provider. The results show that the monitoring load are well divided among the servers in the data centers, and cooperative monitoring adapts gracefully to the events of monitoring node failures.