Asymptotic Gaussian Research Articles

Semi-supervised distribution learning

Abstract This study addresses the challenge of distribution estimation and inference in a semi-supervised setting. In contrast to prior research focusing on parameter inference, this work explores the complexities of semi-supervised distribution estimation, particularly the uniformity problem inherent in functional processes. To tackle this issue, we introduce a versatile framework designed to extract valuable information from unlabelled data by approximating a conditional distribution on covariates. The proposed estimator is derived from the K-fold cross-fitting strategy, exhibiting both consistency and asymptotic Gaussian process properties. Under mild conditions, the proposed estimator outperforms the empirical cumulative distribution function in terms of asymptotic efficiency. Several applications of the methodology are given, including parameter inference and goodness-of-fit tests.

Deterministic equivalent and error universality of deep random features learning*

Abstract This manuscript considers the problem of learning a random Gaussian network function using a fully connected network with frozen intermediate layers and trainable readout layer. This problem can be seen as a natural generalization of the widely studied random features model to deeper architectures. First, we prove Gaussian universality of the test error in a ridge regression setting where the learner and target networks share the same intermediate layers, and provide a sharp asymptotic formula for it. Establishing this result requires proving a deterministic equivalent for traces of the deep random features sample covariance matrices which can be of independent interest. Second, we conjecture the asymptotic Gaussian universality of the test error in the more general setting of arbitrary convex losses and generic learner/target architectures. We provide extensive numerical evidence for this conjecture. In light of our results, we investigate the interplay between architecture design and implicit regularization.

Dimensional crossover in Kardar-Parisi-Zhang growth.

Two-dimensional (2D) Kardar-Parisi-Zhang (KPZ) growth is usually investigated on substrates of lateral sizes L_{x}=L_{y}, so that L_{x} and the correlation length (ξ) are the only relevant lengths determining the scaling behavior. However, in cylindrical geometry, as well as in flat rectangular substrates L_{x}≠L_{y} and, thus, the surfaces can become correlated in a single direction, when ξ∼L_{x}≪L_{y}. From extensive simulations of several KPZ models, we demonstrate that this yields a dimensional crossover in their dynamics, with the roughness scaling as W∼t^{β_{2D}} for t≪t_{c} and W∼t^{β_{1D}} for t≫t_{c}, where t_{c}∼L_{x}^{1/z_{2D}}. The height distributions (HDs) also cross over from the 2D flat (cylindrical) HD to the asymptotic Tracy-Widom Gaussian orthogonal ensemble (Gaussian unitary ensemble) distribution. Moreover, 2D to one-dimensional (1D) crossovers are found also in the asymptotic growth velocity and in the steady-state regime of flat systems, where a family of universal HDs exists, interpolating between the 2D and 1D ones as L_{y}/L_{x} increases. Importantly, the crossover scalings are fully determined and indicate a possible way to solve 2D KPZ models.

On a class of linear regression methods

In this paper, a unified study is presented for the design and analysis of a broad class of linear regression methods. The proposed general framework includes the conventional linear regression methods (such as the least squares regression and the Ridge regression) and some new regression methods (e.g. the Landweber regression and Showalter regression), which have recently been introduced in the fields of optimization and inverse problems. The strong consistency, the reduced mean squared error, the asymptotic Gaussian property, and the best worst case error of this class of linear regression methods are investigated. Various numerical experiments are performed to demonstrate the consistency and efficiency of the proposed class of methods for linear regression.

Asymptotic Gaussian Fluctuations of Eigenvectors in Spectral Clustering

Asymptotic Gaussian Fluctuations of Eigenvectors in Spectral Clustering

NUMERICAL-STATISTICAL INVESTIGATION OF SUPEREXPONENTIAL GROWTH OF THE MEAN PARTICLE FLUX WITH MULTIPLICATION IN A HOMOGENEOUS RANDOM MEDIUM

The new correlative-grid approximation of a homogeneous random field is introduced for the effective numerically-analytical investigation of the superexponential growth of the mean particles flux with multiplication in a random medium. A complexity of particle trajectory realization is not dependent on the correlation scale. The test computations for a critical ball with isotropic scattering showed high accuracy of the corresponding mean flux estimates. For the correlative-grid approximation the possibility of Gaussian asymptotics of the mean particles multiplication rate when the correlation scale decreases is justified.

Detecting the complexity of a functional time series

ABSTRACT Consider a time series that takes values in a general topological space, and suppose that its Small-ball probability is factorised into two terms that play the role of a surrogate density and a volume term. The latter allows us to study the complexity of the underlying process. In some cases, the volume term can be analytically specified in a parametric form as a function of a complexity index. This work presents the study of an estimator for such an index whenever the volume term is monomial. Weak consistency and asymptotic Gaussianity are shown under an appropriate dependence structure, providing theoretical support for the construction of confidence intervals. A Monte Carlo simulation is performed to evaluate the performance of the approach under various conditions. Finally, the method is applied to identify the complexity of two real data sets.

A NOVEL APPROACH TO PREDICTIVE ACCURACY TESTING IN NESTED ENVIRONMENTS

We introduce a new approach for comparing the predictive accuracy of two nested models that bypasses the difficulties caused by the degeneracy of the asymptotic variance of forecast error loss differentials used in the construction of commonly used predictive comparison statistics. Our approach continues to rely on the out of sample mean squared error loss differentials between the two competing models, leads to nuisance parameter-free Gaussian asymptotics, and is shown to remain valid under flexible assumptions that can accommodate heteroskedasticity and the presence of mixed predictors (e.g., stationary and local to unit root). A local power analysis also establishes their ability to detect departures from the null in both stationary and persistent settings. Simulations calibrated to common economic and financial applications indicate that our methods have strong power with good size control across commonly encountered sample sizes.

A harmonic-wavelet-based representation for non-stationary stochastic excitations

Representation of nonstationary stochastic excitations is crucial for stochastic response analyses of (time-varying) linear and nonlinear structural systems. This paper proposes a new representation method of non-stationary stochastic excitations based on the generalized harmonic wavelet (GHW) that takes the phase angles and frequencies as basic random variables. The orthogonal properties of the discrete-form spectral process increments describing non-stationary stochastic processes are formulated. Then the GHW-based representation is derived by using the orthogonal properties. This method can be used to accurately reproduce non-stationary stochastic excitations with the target asymptotic Gaussianity and evolutionary power spectrum density. The effectiveness and accuracy of the proposed method have been validated via numerical examples. This study provides a novel way for the representation of non-stationary processes and deserves to be applied in the stochastic response analyses of structures.

Asymptotic Gaussianity via coalescence probabilities in the Hammond-Sheffield urn

For the renormalised sums of the random $\pm 1$-colouring of the connected components of $\mathbb Z$ generated by the coalescing renewal processes in the "power law P\'olya's urn" of Hammond and Sheffield we prove functional convergence towards fractional Brownian motion, closing a gap in the tightness argument of their paper. In addition, in the regime of the strong renewal theorem we gain insights into the coalescing renewal processes in the Hammond-Sheffield urn (such as the asymptotic depth of most recent common ancestors) and are able to control the coalescence probabilities of two, three and four individuals that are randomly sampled from $[n]$. This allows us to obtain a new, conceptual proof of the asymptotic Gaussianity (including the functional convergence) of the renormalised sums of more general colourings, which can be seen as an invariance principle beyond the main result of Hammond and Sheffield. In this proof, a key ingredient of independent interest is a sufficient criterion for the asymptotic Gaussianity of the renormalised sums in randomly coloured random partitions of $[n]$, based on Stein's method. Along the way we also prove a statement on the asymptotics of the coalescence probabilities in the long-range seedbank model of Blath, Gonz\'alez Casanova, Kurt, and Span\`o.

Robust Inference for Partially Observed Functional Response Data.

Irregular functional data in which densely sampled curves are observed over different ranges pose a challenge for modeling and inference, and sensitivity to outlier curves is a concern in applications. Motivated by applications in quantitative ultrasound signal analysis, this paper investigates a class of robust M-estimators for partially observed functional data including functional location and quantile estimators. Consistency of the estimators is established under general conditions on the partial observation process. Under smoothness conditions on the class of M-estimators, asymptotic Gaussian process approximations are established and used for large sample inference. The large sample approximations justify a bootstrap approximation for robust inferences about the functional response process. The performance is demonstrated in simulations and in the analysis of irregular functional data from quantitative ultrasound analysis.

A Novel Approach to Predictive Accuracy Testing in Nested Environments

We introduce a new approach for comparing the predictive accuracy of two nested models that bypasses the difficulties caused by the degeneracy of the asymptotic variance of forecast error loss differentials used in the construction of commonly used predictive comparison statistics. Our approach continues to rely on the out of sample MSE loss differentials between the two competing models, leads to nuisance parameter free Gaussian asymptotics and is shown to remain valid under flexible assumptions that can accommodate heteroskedasticity and the presence of mixed predictors (e.g. stationary and local to unit root). A local power analysis also establishes its ability to detect departures from the null in both stationary and persistent settings. Simulations calibrated to common economic and financial applications indicate that our methods have strong power with good size control across commonly encountered sample sizes.

Free energy fluctuations of the two-spin spherical SK model at critical temperature

We investigate the fluctuations of the free energy of the two-spin spherical Sherrington–Kirkpatrick model at critical temperature βc = 1. When β = 1, we find asymptotic Gaussian fluctuations with variance 16N2log(N), confirming in the spherical case a physics prediction for the SK model with Ising spins. We, furthermore, prove the existence of a critical window on the scale β=1+αlog(N)N−1/3. For any α∈R, we show that the fluctuations are at most order log(N)/N in the sense of tightness. If α → ∞ at any rate as N → ∞, then, properly normalized, the fluctuations converge to the Tracy–Widom1 distribution. If α → 0 at any rate as N → ∞ or α < 0 is fixed, the fluctuations are asymptotically Gaussian as in the α = 0 case. In determining the fluctuations, we apply a recent result of G. Lambert and E. Paquette [“Strong approximation of Gaussian beta-ensemble characteristic polynomials: The edge regime and the stochastic airy function,” arXiv:2009.05003 (2020)] on the behavior of the Gaussian-β-ensemble at the spectral edge.

Generalized Memory Approximate Message Passing for Generalized Linear Model

For signal reconstruction in a generalized linear model (GLM), generalized approximate message passing (GAMP) is a low-complexity algorithm with many appealing features such as an exact performance characterization in the high-dimensional limit. However, it is viable only when the transformation matrix has independent and identically distributed (IID) entries. Generalized vector AMP (GVAMP) has a wider applicability but with high computational complexity. To overcome the shortcomings of GAMP and GVAMP, we propose a low-complexity and widely applicable generalized memory AMP (GMAMP) framework, including an orthogonal memory linear estimator (MLE) and two orthogonal memory nonlinear estimators (MNLE), which guarantee the asymptotic IID Gaussianity of estimation errors and state evolution (SE) in GMAMP. The proposed GMAMP is universal since the existing AMP, convolutional AMP, orthogonal/vector AMP, GVAMP, and memory AMP (MAMP) are its special instances. More importantly, we provide a principle toward building new advanced AMP-type algorithms based on the proposed GMAMP framework. As an example, we construct a Bayes-optimal GMAMP (BO-GMAMP) algorithm, which adopts a memory match filter estimator to suppress the linear interference, and thus its complexity is comparable to GAMP. Furthermore, we prove that the SE of BO-GMAMP with optimized parameters converges to the same fixed point as that of the high-complexity GVAMP. In other words, BO-GMAMP achieves the replica minimum (i.e., potential Bayes-optimal) mean square error (MSE) if its SE has a unique fixed point. Finally, simulation results are provided to validate the accuracy of the theoretical analysis.

Gaussian limits for subcritical chaos

We present a simple criterion, only based on second moment assumptions, for the convergence of polynomial or Wiener chaos to a Gaussian limit. We exploit this criterion to obtain new Gaussian asymptotics for the partition functions of two-dimensional directed polymers in the sub-critical regime, including a singular product between the partition function and the disorder. These results can also be applied to the KPZ and Stochastic Heat Equation. As a tool of independent interest, we derive an explicit chaos expansion which sharply approximates the logarithm of the partition function.

Asymptotics of sums of regression residuals under multiple ordering of regressors

We prove theorems about the Gaussian asymptotics of an empirical bridge built from linear model regressors with multiple regressor ordering. We study the testing of the hypothesis of a linear model for the components of a random vector: one of the components is a linear combination of the others up to an error that does not depend on the other components of the random vector. The results of observations of independent copies of a random vector are sequentially ordered in ascending order of several of its components. The result is a sequence of vectors of higher dimension, consisting of induced order statistics (concomitants) corresponding to different orderings. For this sequence of vectors, without the assumption of a linear model for the components, we prove a lemma of weak convergence of the distributions of an appropriately centered and normalized process to a centered Gaussian process with almost surely continuous trajectories. Assuming a linear relationship of the components, standard least squares estimates are used to compute regression residuals -- the differences between response values and the predicted ones by the linear model. We prove a theorem of weak convergence of the process of regression residuals under the necessary normalization to a centered Gaussian process.

Gaussian Asymptotics of Jack Measures on Partitions From Weighted Enumeration of Ribbon Paths

Abstract In this paper, we determine two asymptotic results for Jack measures $M(v^{\textrm {out}}, v^{\textrm {in}})$, a measure on partitions defined by two specializations $v^{\textrm {out}}, v^{\textrm {in}}$ of Jack polynomials proposed by Borodin–Olshanski in [10]. Assuming $v^{\textrm {out}} = v^{\textrm {in}}$, we derive limit shapes and Gaussian fluctuations for the anisotropic profiles of these random partitions in three asymptotic regimes associated to vanishing, fixed, and diverging values of the Jack parameter. To do so, we introduce a generalization of Motzkin paths we call “ribbon paths,” show for arbitrary $v^{\textrm {out}}, v^{\textrm {in}}$ that certain Jack measure joint cumulants ${\kappa _n}$ are weighted sums of connected ribbon paths on $n$ sites with $n-1+g$ pairings, and derive our two results from the contributions of $(n,g)=(1,0)$ and $(2,0)$, respectively. Our analysis makes use of Nazarov–Sklyanin’s spectral theory for Jack polynomials. As a consequence, we give new proofs of several results for Schur measures, Plancherel measures, and Jack–Plancherel measures. In addition, we relate our weighted sums of ribbon paths to the weighted sums of ribbon graphs of maps on non-oriented real surfaces recently introduced by Chapuy–Dołęga.

Fluctuation and entropy in spectrally constrained random fields

We investigate the statistical properties of translation invariant random fields (including point processes) on Euclidean spaces (or lattices) under constraints on their spectrum or structure function. An important class of models that motivate our study are hyperuniform and stealthy hyperuniform systems, which are characterised by the vanishing of the structure function at the origin (resp., vanishing in a neighbourhood of the origin). We show that many key features of two classical statistical mechanical measures of randomness - namely, fluctuations and entropy, are governed only by some particular local aspects of their structure function. We obtain exponents for the fluctuations of the local mass in domains of growing size, and show that spatial geometric considerations play an important role - both the shape of the domain and the mode of spectral decay. In doing so, we unveil intriguing oscillatory behaviour of spatial correlations of local masses in adjacent box domains. We describe very general conditions under which we show that the field of local masses exhibit Gaussian asymptotics, with an explicitly described limit. We further demonstrate that stealthy hyperuniform systems with joint densities exhibit degeneracy in their asymptotic entropy per site. In fact, our analysis shows that entropic degeneracy sets in under much milder conditions than stealthiness, as soon as the structure function fails to be logarithmically integrable.

Bayes-Optimal Convolutional AMP

This paper proposes Bayes-optimal convolutional approximate message-passing (CAMP) for signal recovery in compressed sensing. CAMP uses the same low-complexity matched filter (MF) for interference suppression as approximate message-passing (AMP). To improve the convergence property of AMP for ill-conditioned sensing matrices, the so-called Onsager correction term in AMP is replaced by a convolution of all preceding messages. The tap coefficients in the convolution are determined so as to realize asymptotic Gaussianity of estimation errors via state evolution (SE) under the assumption of orthogonally invariant sensing matrices. An SE equation is derived to optimize the sequence of denoisers in CAMP. The optimized CAMP is proved to be Bayes-optimal for all orthogonally invariant sensing matrices if the SE equation converges to a fixed-point and if the fixed-point is unique. For sensing matrices with low-to-moderate condition numbers, CAMP can achieve the same performance as high-complexity orthogonal/vector AMP that requires the linear minimum mean-square error (LMMSE) filter instead of the MF.

From Univariate to Multivariate Coupling Between Continuous Signals and Point Processes: A Mathematical Framework.

Time series data sets often contain heterogeneous signals, composed of both continuously changing quantities and discretely occurring events. The coupling between these measurements may provide insights into key underlying mechanisms of the systems under study. To better extract this information, we investigate the asymptotic statistical properties of coupling measures between continuous signals and point processes. We first introduce martingale stochastic integration theory as a mathematical model for a family of statistical quantities that include the phase locking value, a classical coupling measure to characterize complex dynamics. Based on the martingale central limit theorem, we can then derive the asymptotic gaussian distribution of estimates of such coupling measure that can be exploited for statistical testing. Second, based on multivariate extensions of this result and random matrix theory, we establish a principled way to analyze the low-rank coupling between a large number of point processes and continuous signals. For a null hypothesis of no coupling, we establish sufficient conditions for the empirical distribution of squared singular values of the matrix to converge, as the number of measured signals increases, to the well-known Marchenko-Pastur (MP) law, and the largest squared singular value converges to the upper end of the MP support. This justifies a simple thresholding approach to assess the significance of multivariate coupling. Finally, we illustrate with simulations the relevance of our univariate and multivariate results in the context of neural time series, addressing how to reliably quantify the interplay between multichannel local field potential signals and the spiking activity of a large population of neurons.