The impossibility of low-rank representations for triangle-rich complex networks

C Seshadhri,Aneesh Sharma,Andrew Stolman,Ashish Goel

doi:10.1073/pnas.1911030117

Abstract

The study of complex networks is a significant development in modern science, and has enriched the social sciences, biology, physics, and computer science. Models and algorithms for such networks are pervasive in our society, and impact human behavior via social networks, search engines, and recommender systems, to name a few. A widely used algorithmic technique for modeling such complex networks is to construct a low-dimensional Euclidean embedding of the vertices of the network, where proximity of vertices is interpreted as the likelihood of an edge. Contrary to the common view, we argue that such graph embeddings do not capture salient properties of complex networks. The two properties we focus on are low degree and large clustering coefficients, which have been widely established to be empirically true for real-world networks. We mathematically prove that any embedding (that uses dot products to measure similarity) that can successfully create these two properties must have a rank that is nearly linear in the number of vertices. Among other implications, this establishes that popular embedding techniques such as singular value decomposition and node2vec fail to capture significant structural aspects of real-world complex networks. Furthermore, we empirically study a number of different embedding techniques based on dot product, and show that they all fail to capture the triangle structure.

Highlights

The study of complex networks is a significant development in modern science, and has enriched the social sciences, biology, physics, and computer science
The weaknesses of singular value decomposition (SVD) have been empirically observed in recommendation tasks [14,15,16], and our result provides a mathematical validation of these findings
A widely used algorithmic technique for modeling these networks is to construct a low-dimensional Euclidean embedding of the vertices of the network, where proximity of vertices is interpreted as the likelihood of an edge

Summary

COMPUTER SCIENCES

The impossibility of low-rank representations for triangle-rich complex networks. C. We mathematically prove that any embedding (that uses dot products to measure similarity) that can successfully create these two properties must have a rank that is nearly linear in the number of vertices Among other implications, this establishes that popular embedding techniques such as singular value decomposition and node2vec fail to capture significant structural aspects of real-world complex networks. We mathematically prove that low-dimensional embeddings cannot generate graphs with both low average degree and large clustering coefficients, which have been widely established to be empirically true for real-world networks This establishes that popular low-dimensional embedding methods fail to capture significant structural aspects of real-world complex networks. Our main result is that any embedding of graphs that generates graphs with (c, ∆)-triangle foundations, with constant c, ∆, must have near-linear rank This contradicts the belief that low-dimensional embeddings capture the structure of real-world complex networks. Theorem 1 implies that d must be Ω(n/ lg n) to accurately model the low-degree triangle behavior

Empirical Validation

Alternate Models

Broader Context

We choose the

Details of Empirical Results