Latent Position Research Articles

Variational inference for the latent shrinkage position model

AbstractThe latent position model (LPM) is a popular method used in network data analysis where nodes are assumed to be positioned in a ‐dimensional latent space. The latent shrinkage position model (LSPM) is an extension of the LPM which automatically determines the number of effective dimensions of the latent space via a Bayesian nonparametric shrinkage prior. However, the LSPM's reliance on Markov chain Monte Carlo for inference, while rigorous, is computationally expensive, making it challenging to scale to networks with large numbers of nodes. We introduce a variational inference approach for the LSPM, aiming to reduce computational demands while retaining the model's ability to intrinsically determine the number of effective latent dimensions. The performance of the variational LSPM is illustrated through simulation studies and its application to real‐world network data. To promote wider adoption and ease of implementation, we also provide open‐source code.

Clustering by deep latent position model with graph convolutional network

Clustering by deep latent position model with graph convolutional network

Projective, sparse and learnable latent position network models

Projective, sparse and learnable latent position network models

Semisupervised regression in latent structure networks on unknown manifolds

Random graphs are increasingly becoming objects of interest for modeling networks in a wide range of applications. Latent position random graph models posit that each node is associated with a latent position vector, and that these vectors follow some geometric structure in the latent space. In this paper, we consider random dot product graphs, in which an edge is formed between two nodes with probability given by the inner product of their respective latent positions. We assume that the latent position vectors lie on an unknown one-dimensional curve and are coupled with a response covariate via a regression model. Using the geometry of the underlying latent position vectors, we propose a manifold learning and graph embedding technique to predict the response variable on out-of-sample nodes, and we establish convergence guarantees for these responses. Our theoretical results are supported by simulations and an application to Drosophila brain data.

The graph embedded topic model

Most of existing graph neural networks (GNNs) developed for the prevalent text-rich networks typically treat texts as node attributes. This kind of approach unavoidably results in the loss of important semantic structures and restricts the representational power of GNNs. In this work, we introduce a document similarity-based graph convolutional network (DS-GCN) encoder to combine graph topologies and document semantics for text-rich network representation. Then, a graph decoder, based on the latent position model, is used to reconstruct the graph while preserving the network topology. The document matrix is rebuilt by a document decoder, based on the embedded topic model, which considers both topic and word embeddings. By including a cluster membership variable for each node in the network, we thus develop an end-to-end clustering technique relying on a new deep probabilistic model called the graph embedded topic model (GETM). Numerical experiments on three simulated scenarios emphasize the ability of GETM in fusing the graph topology structure and the document embeddings, and highlight its node clustering performance. Moreover, an application on the Cora-enrich citation network is conducted to demonstrate the effectiveness and interest of GETM in practice.

Continuous latent position models for instantaneous interactions

AbstractWe create a framework to analyze the timing and frequency of instantaneous interactions between pairs of entities. This type of interaction data is especially common nowadays and easily available. Examples of instantaneous interactions include email networks, phone call networks, and some common types of technological and transportation networks. Our framework relies on a novel extension of the latent position network model: we assume that the entities are embedded in a latent Euclidean space and that they move along individual trajectories which are continuous over time. These trajectories are used to characterize the timing and frequency of the pairwise interactions. We discuss an inferential framework where we estimate the individual trajectories from the observed interaction data and propose applications on artificial and real data.

Scaling Court Decisions with Citation Networks

AbstractTo compare court decisions in a systematic way, it is typically necessary to first read these decisions and then apply legal methods to them. Measurement models that support analysts in this manual labor usually rely on judges’ voting records. Since these data are often not available, we instead propose a latent-variable model that uses the widely available references in court decisions to measure the decisions’ latent position in their common case-space. We showcase our model in the context of forum shopping and forum selling of Germany’s lower courts.

A Bayesian Nonparametric Latent Space Approach to Modeling Evolving Communities in Dynamic Networks

The evolution of communities in dynamic (time-varying) network data is a prominent topic of interest. A popular approach to understanding these dynamic networks is to embed the dyadic relations into a latent metric space. While methods for clustering with this approach exist for dynamic networks, they all assume a static community structure. This paper presents a Bayesian nonparametric model for dynamic networks that can model networks with evolving community structures. Our model extends existing latent space approaches by explicitly modeling the additions, deletions, splits, and mergers of groups with a hierarchical Dirichlet process hidden Markov model. Our proposed approach, the hierarchical Dirichlet process latent position cluster model (HDP-LPCM), incorporates transitivity, models both individual and group level aspects of the data, and avoids the computationally expensive selection of the number of groups required by most popular methods. We provide a Markov chain Monte Carlo estimation algorithm and demonstrate its ability to detect evolving community structure in a network of military alliances during the Cold War and a narrative network constructed from the Game of Thrones television series.

Latent Space Model for Higher-Order Networks and Generalized Tensor Decomposition

We introduce a unified framework, formulated as general latent space models, to study complex higher-order network interactions among multiple entities. Our framework covers several popular models in recent network analysis literature, including mixture multi-layer latent space model and hypergraph latent space model. We formulate the relationship between the latent positions and the observed data via a generalized multilinear kernel as the link function. While our model enjoys decent generality, its maximum likelihood parameter estimation is also convenient via a generalized tensor decomposition procedure. We propose a novel algorithm using projected gradient descent on Grassmannians. We also develop original theoretical guarantees for our algorithm. First, we show its linear convergence under mild conditions. Second, we establish finite-sample statistical error rates of latent position estimation, determined by the signal strength, degrees of freedom and the smoothness of link function, for both general and specific latent space models. We demonstrate the effectiveness of our method on synthetic data. We also showcase the merit of our method on two real-world datasets that are conventionally described by different specific models in producing meaningful and interpretable parameter estimations and accurate link prediction. Supplementary materials for this article are available online.

Model-based clustering for multidimensional social networks

Abstract Social network data are relational data recorded among a group of actors, interacting in different contexts. Often, the same set of actors can be characterised by multiple social relations, captured by a multidimensional network. A common situation is that of colleagues working in the same institution, whose social interactions can be defined on professional and personal levels. In addition, individuals in a network tend to interact more frequently with similar others, naturally creating communities. Latent space models for network data are useful to recover clustering of the actors, as they allow to represent similarities between them by their positions and relative distances in an interpretable low-dimensional social space. We propose the infinite mixture latent position cluster model for multidimensional network data, which enables model-based clustering of actors interacting across multiple social dimensions. The model is based on a Bayesian non-parametric framework that allows to perform automatic inference on the clustering allocations, the number of clusters, and the latent social space. The method is tested on extensive simulated data experiments. It is also employed to investigate the presence of communities in two multidimensional workplace social networks recording relations of different types among colleagues.

Triangular Concordance Learning of Networks

ABSTRACT Networks are widely used to describe relational data among objects in a complex system. As network data often exhibit clustering structures, research interest often focuses on discovering clusters of nodes. We develop a novel concordance-based method for node clustering in networks, where a linear model is imposed on the latent position of each node with respect to a node-specific center and its covariates via linear transformation. By maximizing a triangular concordance function with a concave pairwise penalty, the latent positions are estimated so that each node would be more likely to be close to its neighbors in contrast to non-neighbors and nodes are clustered by their node-specific centers. We develop an alternating direction method of multipliers algorithm for parameter estimation and an intimacy score between unlinked nodes for link prediction. Our method takes into account common characteristics of network data (i.e., assortativity, link pattern similarity, node heterogeneity and link transitivity), while it does not require the number of clusters to be known. The clustering effectiveness and link prediction accuracy of our method are demonstrated in simulated and real networks. Supplementary materials for this article are available online.

A Statistical Interpretation of Spectral Embedding: The Generalised Random Dot Product Graph

AbstractSpectral embedding is a procedure which can be used to obtain vector representations of the nodes of a graph. This paper proposes a generalisation of the latent position network model known as the random dot product graph, to allow interpretation of those vector representations as latent position estimates. The generalisation is needed to model heterophilic connectivity (e.g. ‘opposites attract’) and to cope with negative eigenvalues more generally. We show that, whether the adjacency or normalised Laplacian matrix is used, spectral embedding produces uniformly consistent latent position estimates with asymptotically Gaussian error (up to identifiability). The standard and mixed membership stochastic block models are special cases in which the latent positions take only K distinct vector values, representing communities, or live in the (K − 1)-simplex with those vertices respectively. Under the stochastic block model, our theory suggests spectral clustering using a Gaussian mixture model (rather than K-means) and, under mixed membership, fitting the minimum volume enclosing simplex, existing recommendations previously only supported under non-negative-definite assumptions. Empirical improvements in link prediction (over the random dot product graph), and the potential to uncover richer latent structure (than posited under the standard or mixed membership stochastic block models) are demonstrated in a cyber-security example.

Faster MCMC for Gaussian latent position network models

AbstractLatent position network models are a versatile tool in network science; applications include clustering entities, controlling for causal confounders, and defining priors over unobserved graphs. Estimating each node’s latent position is typically framed as a Bayesian inference problem, with Metropolis within Gibbs being the most popular tool for approximating the posterior distribution. However, it is well-known that Metropolis within Gibbs is inefficient for large networks; the acceptance ratios are expensive to compute, and the resultant posterior draws are highly correlated. In this article, we propose an alternative Markov chain Monte Carlo strategy—defined using a combination of split Hamiltonian Monte Carlo and Firefly Monte Carlo—that leverages the posterior distribution’s functional form for more efficient posterior computation. We demonstrate that these strategies outperform Metropolis within Gibbs and other algorithms on synthetic networks, as well as on real information-sharing networks of teachers and staff in a school district.

Latent structure blockmodels for Bayesian spectral graph clustering

Spectral embedding of network adjacency matrices often produces node representations living approximately around low-dimensional submanifold structures. In particular, hidden substructure is expected to arise when the graph is generated from a latent position model. Furthermore, the presence of communities within the network might generate community-specific submanifold structures in the embedding, but this is not explicitly accounted for in most statistical models for networks. In this article, a class of models called latent structure block models (LSBM) is proposed to address such scenarios, allowing for graph clustering when community-specific one-dimensional manifold structure is present. LSBMs focus on a specific class of latent space model, the random dot product graph (RDPG), and assign a latent submanifold to the latent positions of each community. A Bayesian model for the embeddings arising from LSBMs is discussed, and shown to have a good performance on simulated and real-world network data. The model is able to correctly recover the underlying communities living in a one-dimensional manifold, even when the parametric form of the underlying curves is unknown, achieving remarkable results on a variety of real data.

Joint latent space models for network data with high-dimensional node variables

Summary Network latent space models assume that each node is associated with an unobserved latent position in a Euclidean , and such latent variables determine the probability of two nodes connecting with each other. In many applications, nodes in the network are often observed along with high-dimensional node variables, and these node variables provide important information for understanding the network structure. However, classical network latent space models have several limitations in incorporating node variables. In this paper, we propose a joint latent space model where we assume that the latent variables not only explain the network structure, but are also informative for the multivariate node variables. We develop a projected gradient descent algorithm that estimates the latent positions using a criterion incorporating both network structure and node variables. We establish theoretical properties of the estimators and provide insights into how incorporating high-dimensional node variables could improve the estimation accuracy of the latent positions. We demonstrate the improvement in latent variable estimation and the improvements in associated downstream tasks, such as missing value imputation for node variables, by simulation studies and an application to a Facebook data example.

A generative node-attribute network model for detecting generalized structure and semantics

A challenge of community detection in attributed networks is how we can design an effective and efficient clustering method that can not only discover a wide of structure types but also have good community semantic annotations. To this end, by sharing the latent position of nodes, a mathematically principled model (named GNAN) that fuses topological information and node-attribute information is developed. Using the expectation–maximization algorithm, the latent position of each node and the model parameters are learned. The new model detects communities more accurately than can be done with topology information alone. And a case study is provided to show the ability of our model in the semantic interpretability of communities. In detail, firstly, inspired by the idea of NMM (Newman’s Mixture Models), a group of parameters that characterize the link behaviors of nodes is introduced into the topological model. In the probabilistic sense, nodes with the same link pattern form a community. Therefore, the combined model can generate not only traditional communities, i.e., groupings of nodes with dense internal connections and sparse external ones, but also a range of other types of structure in networks, such as bipartite structure, core–periphery structure, and their mixture structure, which are collectively referred to as generalized structure. Secondly, based on the homogeneity assumption, another group of parameters describing the distribution of attributes in a community is introduced into the attributed model. Under the control of these parameters, the united model can generate different attributes according to the probability, and automatically discover the critical attributes of the community. Finally, experiments on both synthetic and real-world networks with various network structures show that the new model can detect communities more accurately than the related state-of-the-art models.

Consistent nonparametric estimation for heavy-tailed sparse graphs

We study graphons as a nonparametric generalization of stochastic block models, and show how to obtain compactly represented estimators for sparse networks in this framework. In contrast to previous work, we relax the usual boundedness assumption for the generating graphon and instead assume only integrability, so that we can handle networks that have long tails in their degree distributions. We also relax the usual assumption that the graphon is defined on the unit interval, to allow latent position graphs based on more general spaces. We analyze three algorithms. The first is a least squares algorithm, which gives a consistent estimator for all square-integrable graphons, with errors expressed in terms of the best possible stochastic block model approximation. Next, we analyze an algorithm based on the cut norm, which works for all integrable graphons. Finally, we show that clustering based on degrees works whenever the underlying degree distribution is atomless.

Social position and anthropometric status among adults in the ELSA-Brasil study: a latent class analysis.

The objective of the present study was to evaluate the association between social position and anthropometric status in women and men Brazilian adult. This was a cross-sectional study that used baseline data collected from 2008 to 2010 for the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil, in Portuguese), in the six major Brazilian state capital cities. A total of 15,105 active and retired civil servants aged from 35 to 74 years. Two latent variables were defined by latent class analysis, social position and anthropometric status. Both constructs and the analyses were separately evaluated by sex. Associations were assessed using multivariate logistic regression analysis with adjustment for age, self-reported skin color/race, and marital status. Around 44% of the women and 26% of the men were classified as overweight or obese. Social position tended to be lower in women (43.2%) and higher among men (40.4%). Heavier women were more likely to be black and brown-skinned, whereas slimmer women were more likely to be white. After adjustment, women's weight increased as social position decreased (OR = 1.52; 95%CI: 1.36-1.70), whereas in men weight decreased as social position decreased (OR = 0.87; 95%CI: 0.76-0.99). Social position affected the anthropometric status of women and men differently, with body patterns also being affected by ethnicity/skin color, showing the potentiality of taking the intersectional perspective when investigating the possible social determinants of the phenomenon.

Grammatical and Syntactical Functions of Auxiliaries in English and Urdu

The current study investigates grammatical and syntactical functions of auxiliaries in Urdu and English to draw certain distinctions regarding the behavior, syntactic representation and semantic content auxiliaries add to the main verbs (Vs) through the X-bar theory as a theoretical framework within the minimalist program (MP). The data of the current research were collected from authentic sources (Urdu and English text books and grammars) to consider a diverse range of possible usage of Urdu and English auxiliaries. 200 sentences extracted from textbooks and grammars were analyzed grammatically in terms of morphology, and syntax (through tree-representation and their movement). The findings indicate sharp differences related to the syntactic position of auxiliaries in Urdu and English. In Urdu, auxiliaries always occupy their position after the main verbs in a sentence and their position is fixed and also have the ability to mark number (singular & plural) and gender (masculine & feminine). In contrast, English auxiliaries can only mark number. Another sharp difference is observed in the question formation process. In Urdu, unlike English, auxiliaries cannot be moved from its latent position towards the specifier position at the beginning of the sentence to construct question structures, because, if it happens, the sentence is considered ungrammatical which may result in the violation of MP. The findings of the study would prove helpful for understanding the semantic, grammatical and syntactical nature and behavior of auxiliaries comparatively in Urdu and English.

On Estimation and Inference in Latent Structure Random Graphs

We define a latent structure random graph as a random dot product graph (RDPG) in which the latent position distribution incorporates both probabilistic and geometric constraints, delineated by a family of underlying distributions on some fixed Euclidean space, and a structural support submanifold from which are drawn the latent positions for the graph. For a one-dimensional latent structure model with known structural support, we extend existing results on the consistency of spectral estimates in RDPGs to demonstrate that the parameters of the underlying distribution can be efficiently estimated. We describe how to estimate or learn the structural support in cases where it is unknown, with a focus on graphs with latent positions along the Hardy–Weinberg curve. Finally, we use the latent structural model formulation to address a hitherto-open question in neuroscience on the bilateral homology of the Drosophila left and right hemisphere connectome.