Sparse Regime Research Articles

Detection Thresholds in Very Sparse Matrix Completion

We study the matrix completion problem: an underlying \(m \times n\) matrix P is low rank, with incoherent singular vectors, and a random \(m \times n\) matrix A is equal to P on a (uniformly) random subset of entries of size dn. All other entries of A are equal to zero. The goal is to retrieve information on P from the observation of A. Let \(A_1\) be the random matrix where each entry of A is multiplied by an independent \(\{0,1\}\)-Bernoulli random variable with parameter 1/2. This paper is about when, how and why the non-Hermitian eigen-spectra of the matrices \(A_1 (A - A_1)^*\) and \((A-A_1)^*A_1\) captures more of the relevant information about the principal component structure of A than the eigen-spectra of \(A A^*\) and \(A^* A\). We show that the eigenvalues of the asymmetric matrices \(A_{1} (A - A_{1})^{*}\) and \((A-A_{1})^{*} A_{1}\) with modulus greater than a detection threshold are asymptotically equal to the eigenvalues of \(PP^*\) and \(P^*P\) and that the associated eigenvectors are aligned as well. The central surprise is that by intentionally inducing asymmetry and additional randomness via the \(A_1\) matrix, we can extract more information than if we had worked with the singular value decomposition (SVD) of A. The associated detection threshold is asymptotically exact and is non-universal since it explicitly depends on the element-wise distribution of the underlying matrix P. We show that reliable, statistically optimal but not perfect matrix recovery, via a universal data-driven algorithm, is possible above this detection threshold using the information extracted from the asymmetric eigen-decompositions. Averaging the left and right eigenvectors provably improves estimation accuracy but not the detection threshold. Our results encompass the very sparse regime where d is of order 1 where matrix completion via the SVD of A fails or produces unreliable recovery. We define another variant of this asymmetric principal component analysis procedure that bypasses the randomization step and has a detection threshold that is smaller by a constant factor but with a computational cost that is larger by a polynomial factor of the number of observed entries. Both detection thresholds allow to go beyond the barrier due to the well-known information theoretical limit \(d \asymp \log n\) for exact matrix completion found in the literature.

Aligning random graphs with a sub-tree similarity message-passing algorithm

The problem of aligning Erdős–Rényi random graphs is a noisy, average-case version of the graph isomorphism problem, in which a pair of correlated random graphs is observed through a random permutation of their vertices. We study a polynomial time message-passing algorithm devised to solve the inference problem of partially recovering the hidden permutation, in the sparse regime with constant average degrees. We perform extensive numerical simulations to determine the range of parameters in which this algorithm achieves partial recovery. We also introduce a generalized ensemble of correlated random graphs with prescribed degree distributions, and extend the algorithm to this case.

Community detection and percolation of information in a geometric setting

AbstractWe make the first steps towards generalising the theory of stochastic block models, in the sparse regime, towards a model where the discrete community structure is replaced by an underlying geometry. We consider a geometric random graph over a homogeneous metric space where the probability of two vertices to be connected is an arbitrary function of the distance. We give sufficient conditions under which the locations can be recovered (up to an isomorphism of the space) in the sparse regime. Moreover, we define a geometric counterpart of the model of flow of information on trees, due to Mossel and Peres, in which one considers a branching random walk on a sphere and the goal is to recover the location of the root based on the locations of leaves. We give some sufficient conditions for percolation and for non-percolation of information in this model.

Concentration of maximum degree in random planar graphs

Let P(n,m) be a graph chosen uniformly at random from the class of all planar graphs on vertex set [n]:={1,…,n} with m=m(n) edges. We show that in the sparse regime, when m/n≤1, with high probability the maximum degree of P(n,m) takes at most two different values. In contrast, this is not true anymore in the dense regime, when m/n>1, where the maximum degree of P(n,m) is not concentrated on any subset of [n] with bounded size.

A Long Short-Term Memory for AI Applications in Spike-based Neuromorphic Hardware

Spike-based neuromorphic hardware holds promise for more energy-efficient implementations of deep neural networks (DNNs) than standard hardware such as GPUs. But this requires us to understand how DNNs can be emulated in an event-based sparse firing regime, as otherwise the energy advantage is lost. In particular, DNNs that solve sequence processing tasks typically employ long short-term memory units that are hard to emulate with few spikes. We show that a facet of many biological neurons, slow after-hyperpolarizing currents after each spike, provides an efficient solution. After-hyperpolarizing currents can easily be implemented in neuromorphic hardware that supports multi-compartment neuron models, such as Intel’s Loihi chip. Filter approximation theory explains why after-hyperpolarizing neurons can emulate the function of long short-term memory units. This yields a highly energy-efficient approach to time-series classification. Furthermore, it provides the basis for an energy-efficient implementation of an important class of large DNNs that extract relations between words and sentences in order to answer questions about the text. Deep learning could be less energy intensive when implemented on spike-based neuromorphic chips. An approach inspired by a characteristic feature of biological neurons, the presence of slowly changing internal currents, is developed to emulate long short-term memory units in a sparse spiking regime for neuromorphic implementation.

Bias-Adjusted Spectral Clustering in Multi-Layer Stochastic Block Models

We consider the problem of estimating common community structures in multi-layer stochastic block models, where each single layer may not have sufficient signal strength to recover the full community structure. In order to efficiently aggregate signal across different layers, we argue that the sum-of-squared adjacency matrices contain sufficient signal even when individual layers are very sparse. Our method uses a bias-removal step that is necessary when the squared noise matrices may overwhelm the signal in the very sparse regime. The analysis of our method relies on several novel tail probability bounds for matrix linear combinations with matrix-valued coefficients and matrix-valued quadratic forms, which may be of independent interest. The performance of our method and the necessity of bias removal is demonstrated in synthetic data and in microarray analysis about gene co-expression networks. Supplementary materials for this article are available online.

Rare event asymptotics for exploration processes for random graphs

Large deviations for random graph models has been a topic of significant recent research activity. Much work in this area is focused on the class of dense random graph models (number of edges in the graph scale as n2, where n is the number of vertices) where the theory of graphons has emerged as a principal tool in the study of large deviation properties. These tools do not give a good approach to large deviation problems for random graph models in the sparse regime. The aim of this paper is to study an approach for large deviation problems in this regime by establishing large deviation principles (LDP) on suitable path spaces for certain exploration processes of the associated random graph sequence. Exploration processes are an important tool in the study of sparse random graph models and have been used to understand detailed asymptotics of many functionals of sparse random graphs, such as component sizes, surplus, deviations from trees, etc. In the context of rare event asymptotics of interest here, the point of view of exploration process transforms a large deviation analysis of a static random combinatorial structure to the study of a small noise LDP for certain stochastic dynamical systems with jumps. Our work focuses on one particular class of random graph models, namely the configuration model; however, the general approach of using exploration processes for studying large deviation properties of sparse random graph models has broader applicability. The goal is to study asymptotics of probabilities of nontypical behavior in the large network limit. The first key step for this is to establish a LDP for an exploration process associated with the configuration model. A suitable exploration process here turns out to be an infinite-dimensional Markov process with transition probability rates that diminish to zero in certain parts of the state space. Large deviation properties of such Markovian models is challenging due to poor regularity behavior of the associated local rate functions. Our proof of the LDP relies on a representation of the exploration process in terms of a system of stochastic differential equations driven by Poisson random measures and variational formulas for moments of nonnegative functionals of Poisson random measures. Uniqueness results for certain controlled systems of deterministic equations play a key role in the analysis. Next, using the rate function in the LDP for the exploration process we formulate a calculus of variations problem associated with the asymptotics of component degree distributions. The second key ingredient in our study is a careful analysis of the infinite-dimensional Euler–Lagrange equations associated with this calculus of variations problem. Exact solutions of these systems of nonlinear differential equations are identified which then provide explicit formulas for decay rates of probabilities of nontypical component degree distributions and related quantities.

Inference and Computation for Sparsely Sampled Random Surfaces

Nonparametric inference for functional data over two-dimensional domains entails additional computational and statistical challenges, compared to the one-dimensional case. Separability of the covariance is commonly assumed to address these issues in the densely observed regime. Instead, we consider the sparse regime, where the latent surfaces are observed only at few irregular locations with additive measurement error, and propose an estimator of covariance based on local linear smoothers. Consequently, the assumption of separability reduces the intrinsically four-dimensional smoothing problem into several two-dimensional smoothers and allows the proposed estimator to retain the classical minimax-optimal convergence rate for two-dimensional smoothers. Even when separability fails to hold, imposing it can be still advantageous as a form of regularization. A simulation study reveals a favorable bias-variance tradeoff and massive speed-ups achieved by our approach. Finally, the proposed methodology is used for qualitative analysis of implied volatility surfaces corresponding to call options, and for prediction of the latent surfaces based on information from the entire dataset, allowing for uncertainty quantification. Our cross-validated out-of-sample quantitative results show that the proposed methodology outperforms the common approach of pre-smoothing every implied volatility surface separately. Supplementary materials for this article are available online.

Generation of bimodal solitons in a sapphire whispering-gallery-mode maser at millikelvin temperatures

We present experimental observations of bimodal solitons in a solid state three-level maser cooled to millikelvin temperatures. The maser is built on a highly dilute $\textrm{Fe}^{3+}$ spin ensemble hosted by a high purity $\textrm{Al}_{2}\textrm{O}_{3}$ crystal constituting a high quality factor whispering-gallery-mode resonator. The maser is pumped through one of these modes, near 31 GHz, generating two signals near 12.04 GHz from two distinct modes, 8 MHz apart. The system demonstrates three regimes, namely, a continuous wave regime, a dense soliton regime and a sparse soliton regime. These results open new avenues for studying nonlinear wave phenomena using microwave systems as well as new applications of solitons in this part of the electromagnetic spectrum.

Limiting probabilities of first order properties of random sparse graphs and hypergraphs

AbstractLet be the binomial random graph in the sparse regime, which as is well‐known undergoes a phase transition at . Lynch (, 1992) showed that for every first order sentence , the limiting probability that satisfies as exists, and moreover it is an analytic function of c. In this paper we consider the closure in the interval of the set of all limiting probabilities of first order sentences in . We show that there exists a critical value such that when , whereas misses at least one subinterval when . We extend these results to random sparse d‐uniform hypergraphs, where the probability of a d‐edge is .

Asymptotic distribution of Bernoulli quadratic forms

Consider the random quadratic form Tn=∑ 1≤u<v≤nauvXuXv, where ((auv))1≤u,v≤n is a {0,1}-valued symmetric matrix with zeros on the diagonal, and X1,X2,…,Xn are i.i.d. Ber(pn), with pn∈(0,1). In this paper, we prove various characterization theorems about the limiting distribution of Tn, in the sparse regime, where pn→0 such that E(Tn)=O(1). The main result is a decomposition theorem showing that distributional limits of Tn is the sum of three components: a mixture which consists of a quadratic function of independent Poisson variables; a linear Poisson mixture, where the mean of the mixture is itself a (possibly infinite) linear combination of independent Poisson random variables; and another independent Poisson component. This is accompanied with a universality result which allows us to replace the Bernoulli distribution with a large class of other discrete distributions. Another consequence of the general theorem is a necessary and sufficient condition for Poisson convergence, where an interesting second moment phenomenon emerges.

Analytic solution of the two-star model with correlated degrees.

Exponential random graphs are important to model the structure of real-world complex networks. Here we solve the two-star model with degree-degree correlations in the sparse regime. The model constraints the average correlation between the degrees of adjacent nodes (nearest neighbors) and between the degrees at the end-points of two-stars (next nearest neighbors). We compute exactly the network free energy and show that this model undergoes a first-order transition to a condensed phase. For non-negative degree correlations between next nearest neighbors, the degree distribution inside the condensed phase has a single peak at the largest degree, while for negative degree correlations between next nearest neighbors the condensed phase is characterized by a bimodal degree distribution. We calculate the degree assortativities and show they are nonmonotonic functions of the model parameters, with a discontinuous behavior at the first-order transition. The first-order critical line terminates at a second-order critical point, whose location in the phase diagram can be accurately determined. Our results can help to develop more detailed models of complex networks with correlated degrees.

Study of Liquid-Solid Mass Transfer and Hydrodynamics in Micropacked Bed with Gas-Liquid Flow.

The volumetric liquid–solid (L-S) mass transfer coefficient under gas–liquid (G-L) two-phase flow in a silicon-chip-based micropacked bed reactor (MPBR) was studied using the copper dissolution method and was related to the reactor hydrodynamic behavior. Using a high-speed camera and a robust computational image analysis method that selectively analyzed the bed voidage around the copper particles, the observed hydrodynamics were directly related to the L-S mass transfer rates in the MPBR. This hydrodynamic study revealed different pulsing structures inside the packed copper bed depending on the flow patterns established preceding the packed bed upon increasing gas velocity. A “liquid-dominated slug” flow regime was associated with an upstream slug flow feed. A “sparse slug” flow regime developed with an upstream slug-annular flow feed. At higher gas velocity, a “gas continuous with pulsing” regime developed with an annular flow feed, which had similar features to the pulsing flow in macroscale packed beds, but it was sensitive and easily destabilized by disturbances from upstream or downstream pressure fluctuations. The volumetric L-S mass transfer coefficient decreased with increasing gas velocity under the liquid-dominated slug flow regime and became rather less affected under the sparse slug flow regime. By resolving the transition from the liquid-dominated slug flow to the sparse slug flow and capturing the onset of the gas-continuous with pulsing regime, we gained new insights into the hydrodynamic effects of G-L flows on the L-S mass transfer rates in a MPBR.

Minimax rates in sparse, high-dimensional change point detection

We study the detection of a sparse change in a high-dimensional mean vector as a minimax testing problem. Our first main contribution is to derive the exact minimax testing rate across all parameter regimes for n independent, p-variate Gaussian observations. This rate exhibits a phase transition when the sparsity level is of order ploglog(8n) and has a very delicate dependence on the sample size: in a certain sparsity regime, it involves a triple iterated logarithmic factor in n. Further, in a dense asymptotic regime, we identify the sharp leading constant, while in the corresponding sparse asymptotic regime, this constant is determined to within a factor of 2. Extensions that cover spatial and temporal dependence, primarily in the dense case, are also provided.

Sample-efficient benchmarking of multiphoton interference on a boson sampler in the sparse regime

Verification of a quantum advantage in the presence of noise is a key open problem in the study of near-term quantum devices. In this work, we show how to assess the quality of photonic interference in a linear optical quantum device (boson sampler) by using a maximum likelihood method to measure the strength at which various noise sources are present in the experiment. This allows us to use a sparse set of samples to test whether a given boson sampling experiment meets known upper bounds on the level of noise permissible to demonstrate a quantum advantage. Furthermore, this method allows us monitor the evolution of noise in real time, creating a valuable diagnostic tool. Finally, we observe that sources of noise in the experiment compound, meaning that the observed value of the mutual photon indistinguishability, which is the main imperfection in our study, is an effective value taking into account all sources of error in the experiment.

On the All-or-Nothing Behavior of Bernoulli Group Testing

In this article, we study the problem of non-adaptive group testing, in which one seeks to identify which items are defective given a set of suitably-designed tests whose outcomes indicate whether or not at least one defective item was included in the test. The most widespread recovery criterion seeks to exactly recover the entire defective set, and relaxed criteria such as approximate recovery and list decoding have also been considered. In contrast, we study the fundamental limits of group testing under the significantly relaxed weak recovery criterion, which only seeks to identify a small fraction (e.g., 0.01) of the defective items. Given the near-optimality of i.i.d. Bernoulli testing for exact recovery in sufficiently sparse scaling regimes, it is natural to ask whether this design additionally succeeds with much fewer tests under weak recovery. Our main negative result shows that this is not the case, and in fact, under i.i.d. Bernoulli random testing in the sufficiently sparse regime, an all-or-nothing phenomenon occurs: When the number of tests is slightly below a threshold, weak recovery is impossible, whereas when the number of tests is slightly above the same threshold, high-probability exact recovery is possible. In establishing this result, we additionally prove similar negative results under Bernoulli designs for the weak detection problem (distinguishing between the group testing model vs. completely random outcomes) and the problem of identifying a single item that is definitely defective. On the positive side, we show that all three relaxed recovery criteria can be attained using considerably fewer tests under suitably-chosen non-Bernoulli designs. Thus, our results collectively indicate that when too few tests are available, naively applying i.i.d. Bernoulli testing can lead to catastrophic failure, whereas “cutting one’s losses” and adopting a more carefully-chosen design can still succeed in attaining these less stringent criteria.

Propagation of chaos and the many-demes limit for weakly interacting diffusions in the sparse regime

Propagation of chaos is a well-studied phenomenon and shows that weakly interacting diffusions may become independent as the system size converges to infinity. Most of the literature focuses on the case of exchangeable systems where all involved diffusions have the same distribution and are "of the same size". In this paper, we analyze the case where only a few diffusions start outside of an accessible trap. Our main result shows that in this "sparse regime" the system of weakly interacting diffusions converges in distribution to a forest of excursions from the trap. In particular, initial independence propagates in the limit and results in a forest of independent trees.

Motif-based spectral clustering of weighted directed networks

Clustering is an essential technique for network analysis, with applications in a diverse range of fields. Although spectral clustering is a popular and effective method, it fails to consider higher-order structure and can perform poorly on directed networks. One approach is to capture and cluster higher-order structures using motif adjacency matrices. However, current formulations fail to take edge weights into account, and thus are somewhat limited when weight is a key component of the network under study.We address these shortcomings by exploring motif-based weighted spectral clustering methods. We present new and computationally useful matrix formulae for motif adjacency matrices on weighted networks, which can be used to construct efficient algorithms for any anchored or non-anchored motif on three nodes. In a very sparse regime, our proposed method can handle graphs with a million nodes and tens of millions of edges. We further use our framework to construct a motif-based approach for clustering bipartite networks.We provide comprehensive experimental results, demonstrating (i) the scalability of our approach, (ii) advantages of higher-order clustering on synthetic examples, and (iii) the effectiveness of our techniques on a variety of real world data sets; and compare against several techniques from the literature. We conclude that motif-based spectral clustering is a valuable tool for analysis of directed and bipartite weighted networks, which is also scalable and easy to implement.

Seeded graph matching via large neighborhood statistics

We study a noisy graph isomorphism problem, where the goal is to perfectly recover the vertex correspondence between two edge‐correlated graphs, with an initial seed set of correctly matched vertex pairs revealed as side information. We show that it is possible to achieve the information‐theoretic limit of graph sparsity in time polynomial in the number of vertices n. Moreover, we show the number of seeds needed for perfect recovery in polynomial‐time can be as low as in the sparse graph regime (with the average degree smaller than ) and in the dense graph regime, for a small positive constant . Unlike previous work on graph matching, which used small neighborhoods or small subgraphs with a logarithmic number of vertices in order to match vertices, our algorithms match vertices if their large neighborhoods have a significant overlap in the number of seeds.

Freeness over the diagonal for large random matrices

We prove that independent families of permutation invariant random matrices are asymptotically free with amalgamation over the diagonal, both in expectation and in probability, under a uniform boundedness assumption on the operator norm. We can relax the operator norm assumption to an estimate on sums associated to graphs of matrices, further extending the range of applications (e.g., to Wigner matrices with exploding moments and the sparse regime of the Erdős–Rényi model). The result still holds even if the matrices are multiplied entrywise by random variables satisfying a certain growth condition (e.g., as in the case of matrices with a variance profile and percolation models). Our analysis relies on a modified method of moments based on graph observables.