Mining microscopic and macroscopic changes in network data streams

Abstract

Network data streams are unbounded sequences of complex data produced at high rate which represent complex systems that evolve continuously over time. In this scenario, a problem worthy of being studied is the analysis of the changes, which may concern a complex system as a whole or small parts of it. In this paper, these are distinguished into macroscopic changes and microscopic changes: macroscopic changes have impact on a substantial part of the network, whereas microscopic changes concern variations occurring in specific portions of the network. The algorithm we propose, called KARMA, combines the frequent pattern mining framework with an automatic time-window detection approach. In this way, it is able to detect changes on the frequent subnetworks mined from different time-windows: network changes are then represented as variations of structural regularities frequently observed over the stream. KARMA takes an holistic perspective, in which the two kinds of change are related each other. This is the main novelty with respect to the recent studies, which do not simultaneously extract microscopic and macroscopic changes. Experiments on several real-world network data streams show the effectiveness and efficiency of our approach in comparison with competing algorithms and the usefulness of the changes detected.