Self-Compressive Approach for Distributed System Monitoring

Abstract

Large-Scale distributed hosting infrastructures have become the basic platforms for several real-world production systems. But a challenging task is to achieve both scalability and high precision while monitoring a large number of intra-node attributes that contain information relating to each node and inter-node attribute that denote measurements between different nodes. This paper presents a new distributed monitoring framework Based on video coding techniques of named RBOIC (Replica Based Online Information Compression for Scalable Hosting Infrastructure Monitoring) which uses novel image based approach in which system models snapshots of the monitored distributed system images and applies lightweight online reference block search algorithm to compress monitoring traffic from worker to management node to achieve scalable full coverage monitoring. RBOIC explores the design, implementation of adaptive rood pattern search algorithm to find out optimal or near optimal reference values for each attribute. RBOIC effectively achieve failure-resilient monitoring by restoring monitoring data on replica node. The experimental evaluation of compressive monitoring system has been done using real time monitoring data .The experimental results show that RBOIC can achieve much higher compression ratios with less overhead and adaptive rood pattern search (APRS) achieves better performance by preventing unnecessary intermediate search than diamond search pattern.