Maximal Entropy Random Walk Research Articles

From unbiased to maximal-entropy random walks on hypergraphs.

Random walks have been intensively studied on regular and complex networks, which are used to represent pairwise interactions. Nonetheless, recent works have demonstrated that many real-world processes are better captured by higher-order relationships, which are naturally represented by hypergraphs. Here we study random walks on hypergraphs. Due to the higher-order nature of these mathematical objects, one can define more than one type of walks. In particular, we study the unbiased and the maximal entropy random walk on hypergraphs with two types of steps, emphasizing their similarities and differences. We characterize these dynamic processes by examining their stationary distributions and associated hitting times. To illustrate our findings, we present a toy example and conduct extensive analyses of artificial and real hypergraphs, providing insights into both their structural and dynamical properties. We hope that our findings motivate further research extending the analysis to different classes of random walks as well as to practical applications.

Stationary Schrödinger Equation and Darwin Term from Maximal Entropy Random Walk

We describe particles in a potential by a special diffusion process, the maximal entropy random walk (MERW) on a lattice. Since MERW originates in a variational problem, it shares the linear algebra of Hilbert spaces with quantum mechanics. The Born rule appears from measurements between equilibrium states in the past and the same equilibrium states in the future. Introducing potentials by the observation that time, in a gravitational field running in different heights with a different speed, MERW respects the rule that all trajectories of the same duration are counted with equal probability. In this way, MERW allows us to derive the Schrödinger equation for a particle in a potential and the Darwin term of the nonrelativistic expansion of the Dirac equation. Finally, we discuss why quantum mechanics cannot be simply a result of MERW, but, due to the many analogies, MERW may pave the way for further understanding.

Path integral based convolution and pooling for graph neural networks* *This article is an updated version of: Ma Z, Xuan J, Wang Y G, Li M and Liò P 2020 Path integral based convolution and pooling for graph neural networks Advances in Neural Information Processing Systems vol 33 ed H Larochelle, M Ranzato, R Hadsell, M F Balcan and H

Graph neural networks (GNNs) extend the functionality of traditional neural networks to graph-structured data. Similar to CNNs, an optimized design of graph convolution and pooling is key to success. Borrowing ideas from physics, we propose path integral-based GNNs (PAN) for classification and regression tasks on graphs. Specifically, we consider a convolution operation that involves every path linking the message sender and receiver with learnable weights depending on the path length, which corresponds to the maximal entropy random walk. It generalizes the graph Laplacian to a new transition matrix that we call the maximal entropy transition (MET) matrix derived from a path integral formalism. Importantly, the diagonal entries of the MET matrix are directly related to the subgraph centrality, thus leading to a natural and adaptive pooling mechanism. PAN provides a versatile framework that can be tailored for different graph data with varying sizes and structures. We can view most existing GNN architectures as special cases of PAN. Experimental results show that PAN achieves state-of-the-art performance on various graph classification/regression tasks, including a new benchmark dataset from statistical mechanics that we propose to boost applications of GNN in physical sciences.

Multiple Infrared Small Targets Detection Based on Hierarchical Maximal Entropy Random Walk

The technique of detecting multiple dim and small targets with low signal-to-clutter ratios (SCR) is essential for infrared search and tracking systems. In this letter, we establish a multiple small targets detection method derived from hierarchical maximal entropy random walk (HMERW). The HMERW revolves the limitation of strong bias to the most salient target of the primal maximal entropy random walk (MERW) based on a proposed graph decomposition theory. To enhance the characteristics of small targets and suppress strong clutters, we design a specific weight matrix for HMERW instead of using the conventional weight matrix in MERW. First, a stationary distribution map is obtained by importing the filtered infrared image into the HMERW. Second, a coefficient map is constructed based on the designed weight matrix to fuse the stationary distribution map. Then, an adaptive threshold is used to segment multiple small targets from the fusion map. Extensive experiments on practical datasets demonstrate that the proposed method is superior to the state-of-the-art methods in terms of target enhancement, background suppression, and multiple small targets detection.

Intermittency as metastability: A predictive approach to evolution in rugged landscapes

Many systems across the sciences evolve through the interaction of multiplicative growth and diffusive transport. In the presence of disorder, these opposing forces can generate localized structures and bursty dynamics, a phenomenon known as “intermittency” in non-equilibrium physics and as “punctuated equilibrium” in evolutionary theory. This behaviour is difficult to forecast; in particular there is no general principle to locate the regions where the system will settle, how long it will stay there, or where it will jump next. Here I introduce a Markovian representation of growth-transport dynamics that closes these gaps. I show that any autonomous positive linear system can be mapped onto a generalization of the “maximal entropy random walk”, a Markov process with non-local transition rates. From this transformation follow a characterization of localization islands as minima of an effective potential, a general scheme to coarse-grain the state space along the basins of attraction of these minima, and an entropic Lyapunov function. These results unify the concepts of linear intermittency and metastability, and provide a generally applicable method to reduce, and predict, the dynamics of disordered linear systems. Applications range from Zeld’dovich's parabolic Anderson model to Eigen's quasispecies model of molecular evolution.

Maximal entropy random walk on heterogenous network for MIRNA-disease Association prediction

The last few decades have verified the vital roles of microRNAs in the development of human diseases and witnessed the increasing interest in the prediction of potential disease-miRNA associations. Owning to the open access of many miRNA-related databases, up until recently, kinds of feasible in silico models have been proposed. In this work, we developed a computational model of Maximal Entropy Random Walk on heterogenous network for MiRNA-disease Association prediction (MERWMDA). MERWMDA integrated known disease-miRNA association, pair-wise functional relation of miRNAs and pair-wise semantic relation of diseases into a heterogenous network comprised of disease and miRNA nodes full of information. As a kind of widely-applied biased walk process with more randomness, MERW was then implemented on the heterogenous network to reveal potential disease-miRNA associations. Cross validation was further performed to evaluate the performance of MERWMDA. As a result, MERWMDA obtained AUCs of 0.8966 and 0.8491 respectively in the aspect of global and local leave-one-out cross validation. What’ more, three different case study strategies on four human complex diseases were conducted to comprehensively assess the quality of the model. Specifically, one kind of case study on Esophageal cancer and Prostate cancer were conducted based on HMDD v2.0 database. 94% and 88% out of the top 50 ranked miRNAs were confirmed by recent literature, respectively. To simulate new disease without known related miRNAs, Lung cancer (confirmed ratio 94%) associated miRNAs were removed for case study. Lymphoma (verified ratio 88%) was adopted to assess the prediction robustness of MERWMDA based on HMDD v1.0 database. We anticipated that MERWMDA could offer valuable candidates for in vitro biomedical experiments in future.

Improved Tampering Localization in Digital Image Forensics Based on Maximal Entropy Random Walk

In this paper we propose to use maximal entropy random walk on a graph for tampering localization in digital image forensics. Our approach serves as an additional post-processing step after conventional sliding-window analysis with a forensic detector. Strong localization property of this random walk will highlight important regions and attenuate the background - even for noisy response maps. Our evaluation shows that the proposed method can significantly outperform both the commonly used threshold-based decision, and the recently proposed optimization-based approach with a Markovian prior.

Maximal entropy random walk for region-based visual saliency.

Visual saliency is attracting more and more research attention since it is beneficial to many computer vision applications. In this paper, we propose a novel bottom-up saliency model for detecting salient objects in natural images. First, inspired by the recent advance in the realm of statistical thermodynamics, we adopt a novel mathematical model, namely, the maximal entropy random walk (MERW) to measure saliency. We analyze the rationality and superiority of MERW for modeling visual saliency. Then, based on the MERW model, we establish a generic framework for saliency detection. Different from the vast majority of existing saliency models, our method is built on a purely region-based strategy, which is able to yield high-resolution saliency maps with well preserved object shapes and uniformly highlighted salient regions. In the proposed framework, the input image is first over-segmented into superpixels, which are taken as the primary units for subsequent procedures, and regional features are extracted. Then, saliency is measured according to two principles, i.e., uniqueness and visual organization, both implemented in a unified approach, i.e., the MERW model based on graph representation. Intensive experimental results on publicly available datasets demonstrate that our method outperforms the state-of-the-art saliency models.

Maximal entropy random walk improves efficiency of trapping in dendrimers.

We use maximal entropy random walk (MERW) to study the trapping problem in dendrimers modeled by Cayley trees with a deep trap fixed at the central node. We derive an explicit expression for the mean first passage time from any node to the trap, as well as an exact formula for the average trapping time (ATT), which is the average of the source-to-trap mean first passage time over all non-trap starting nodes. Based on the obtained closed-form solution for ATT, we further deduce an upper bound for the leading behavior of ATT, which is the fourth power of ln N, where N is the system size. This upper bound is much smaller than the ATT of trapping depicted by unbiased random walk in Cayley trees, the leading scaling of which is a linear function of N. These results show that MERW can substantially enhance the efficiency of trapping performed in dendrimers.

Maximal entropy random walk in community detection

The aim of this paper is to check feasibility of using the maximal-entropy random walk in algorithms finding communities in complex networks. A number of such algorithms exploit an ordinary or a biased random walk for this purpose. Their key part is a (dis)similarity matrix, according to which nodes are grouped. This study en- compasses the use of a stochastic matrix of a random walk, its mean first-passage time matrix, and a matrix of weighted paths count. We briefly indicate the connection between those quantities and propose substituting the maximal-entropy random walk for the previously chosen models. This unique random walk maximises the entropy of ensembles of paths of given length and endpoints, which results in equiprobability of those paths. We compare the performance of the selected algorithms on LFR benchmark graphs. The results show that the change in performance depends very strongly on the particular algorithm, and can lead to slight improvements as well as to significant deterioration.

Maximal-entropy random walk unifies centrality measures

This paper compares a number of centrality measures and several (dis-)similarity matrices with which they can be defined. These matrices, which are used among others in community detection methods, represent quantities connected to enumeration of paths on a graph and to random walks. Relationships between some of these matrices are derived in the paper. These relationships are inherited by the centrality measures. They include measures based on the principal eigenvector of the adjacency matrix, path enumeration, as well as on the stationary state, stochastic matrix, or mean first-passage times of a random walk. As the random walk defining the centrality measure can be arbitrarily chosen, we pay particular attention to the maximal-entropy random walk, which serves as a very distinct alternative to the ordinary (diffusive) random walk used in network analysis. The various importance measures, defined both with the use of ordinary random walk and the maximal-entropy random walk, are compared numerically on a set of benchmark graphs with varying mixing parameter and are grouped with the use of the agglomerative clustering technique. It is shown that centrality measures defined with the two different random walks cluster into two separate groups. In particular, the group of centrality measures defined by the maximal-entropy random walk does not cluster with any other measures on change of graphs' parameters, and members of this group produce mutually closer results than members of the group defined by the ordinary random walk.

Exact solution for statics and dynamics of maximal-entropy random walks on Cayley trees

We provide analytical solutions for two types of random walk: generic random walk (GRW) and maximal-entropy random walk (MERW) on a Cayley tree with arbitrary branching number, root degree, and number of generations. For MERW, we obtain the stationary state given by the squared elements of the eigenvector associated with the largest eigenvalue λ(0) of the adjacency matrix. We discuss the dynamics, depending on the second largest eigenvalue λ(1), of the probability distribution approaching to the stationary state. We find different scaling of the relaxation time with the system size, which is generically shorter for MERW than for GRW. We also signal that depending on the initial conditions, there are relaxations associated with lower eigenvalues which are induced by symmetries of the tree. In general, we find that there are three regimes of a tree structure resulting in different statics and dynamics of MERW; these correspond to strongly, critically, and weakly branched roots.

Maximal Entropy Random Walk: solvable cases of dynamics

We focus on the study of dynamics of two kinds of random walk: generic random walk (GRW) and maximal entropy random walk (MERW) on two model networks: Cayley trees and ladder graphs. The stationary probability distribution for MERW is given by the squared components of the eigenvector associated with the largest eigenvalue \lambda_0 of the adjacency matrix of a graph, while the dynamics of the probability distribution approaching to the stationary state depends on the second largest eigenvalue \lambda_1. Firstly, we give analytic solutions for Cayley trees with arbitrary branching number, root degree, and number of generations. We determine three regimes of a tree structure that result in different statics and dynamics of MERW, which are due to strongly, critically, and weakly branched roots. We show how the relaxation times, generically shorter for MERW than for GRW, scale with the graph size. Secondly, we give numerical results for ladder graphs with symmetric defects. MERW shows a clear exponential growth of the relaxation time with the size of defective regions, which indicates trapping of a particle within highly entropic intact region and its escaping that resembles quantum tunneling through a potential barrier. GRW shows standard diffusive dependence irrespective of the defects.

Maximal-entropy random walks in complex networks with limited information

Maximization of the entropy rate is an important issue to design diffusion processes aiming at a well-mixed state. We demonstrate that it is possible to construct maximal-entropy random walks with only local information on the graph structure. In particular, we show that an almost maximal-entropy random walk is obtained when the step probabilities are proportional to a power of the degree of the target node, with an exponent α that depends on the degree-degree correlations and is equal to 1 in uncorrelated graphs.

Localization of the Maximal Entropy Random Walk

We define a new class of random walk processes which maximize entropy. This maximal entropy random walk is equivalent to generic random walk if it takes place on a regular lattice, but it is not if the underlying lattice is irregular. In particular, we consider a lattice with weak dilution. We show that the stationary probability of finding a particle performing maximal entropy random walk localizes in the largest nearly spherical region of the lattice which is free of defects. This localization phenomenon, which is purely classical in nature, is explained in terms of the Lifshitz states of a certain random operator.