Anomaly-Tolerant Network Traffic Estimation via Noise-Immune Temporal Matrix Completion Model

Abstract

Accurately estimating origin-destination (OD) network traffic is crucial for network management and capacity planning. However, the potential network anomaly and complex noise make this goal difficult to achieve. Existing network traffic estimation methods usually impute network traffic independent of anomaly detection, which ignores the potential relationship between the two tasks to help each other in achieving better performance. Moreover, these approaches can only be suitable for simple Gaussian or outlier noise assumptions, which cannot be applied to more complex noise distributions in practical applications. To address these issues, we propose a novel anomaly-tolerant network traffic estimation approach for simultaneously estimating network traffic and detecting network anomaly. Specifically, by utilizing the inherent low-rank property and temporal characteristic of traffic matrix, we formulate the network traffic estimation problem as a noise-immune temporal matrix completion (NiTMC) model, where the complex noise is fitted by mixture of Gaussian (MoG), and the network anomaly is smoothed by the $L_{2,1}$ -norm regularization. In addition, we also design a convergence-guaranteed optimization algorithm based on the expectation maximization (EM) and block coordinate update (BCU) methods to solve the proposed model. Furthermore, to deal with large-scale network problems, we develop a scalable and memory-efficient algorithm by employing stochastic proximal gradient descent (SPGD) method. Finally, the extensive experiments performed on real datasets demonstrate that our proposed NiTMC model outperforms the previously widely used network traffic estimation methods.