Real-time performance monitoring and anomaly detection in the Internet: An adaptive, objective-driven, mix-and-match approach

Abstract

Algorithms for real-time performance management and adaptive fault/anomaly detection in Internet protocol (IP) networks and services have been developed, and a corresponding real-time distributed software platform has been implemented. These algorithms automatically and adaptively detect “soft” faults or anomalies (performance degradations) in IP networks and services, enabling timely correction of network exceptions before failures occur and services are compromised and thereby achieving proactive network and service management. Further, these algorithms are implemented as a reliable, fully distributed real-time software platform — the network/service anomaly detector (NSAD) — with the following features. First, it provides a flexible platform on which preconstructed monitoring and detection components can be mixed, matched, and distributed to form a wide range of application-specific NSADs and performance monitors. Second, NSAD and its components can auto-recover in real time, making it a reliable system for persistent monitoring of networks and their services. Third, anomaly detection is performed on raw network observables — for example, performance data such as management information base-2 (MIB2) and remote monitor-1/2 (RMON1/2) variables — and algebraic functions of these observables (objective functions), thereby enabling objective-driven anomaly detection of wide range and high sensitivity. Fourth, controlled testing (with anomalies injected a priori into a test network) demonstrates that NSAD can detect anomalies reliably in IP networks. Thus, NSAD provides a powerful framework/platform for automatic, proactive fault/anomaly detection and performance management in IP networks and services.