Network Refinement: Denoising complex networks for better community detection

Abstract

Network models are widely used in many fields for their powerful ability to provide a vivid representation of relationships between variables. Modeling entity relationships as networks can be affected by many factors, such as experimental noise and missing data, making the network structure unclear, which inherently hampers the effectiveness of network-based downstream analyses, especially community detection. Consequently, it is necessary to denoise networks before analyzing networks. However, the significance of the network pre-processing step for downstream analysis has been neglected in most current studies of community detection. On the other hand, existing researches on network denoising are very limited and lack adaptability studies. Specialized denoising methods for improving community detection accuracy are not yet available. In this study, we highlighted the necessity of using network denoising as a pre-processing step to improve the performance of community detection. We proposed a novel network denoising method, called Network Refinement (NR), which used a global diffusion process defined by random walk on graph to enhance the self-organization properties of complex networks. NR took a noisy network as input and output a denoised network with clearer community structure by adjusting the edge weights. We have proved that NR can be understood as a degree normalized version of the Katz index, which renders paths with higher intermediate node degrees less important because their information is dispersed through more adjacent edges. We have showed through sufficient numerical experiments that NR significantly improves the clarity of the network’s mesoscale structure, and NR can be applied as a pre-processing step to substantially boost the performance of various community detection algorithms on both simulated networks and real-world networks.