Asymptotic Statistics Research Articles

Universality laws for Gaussian mixtures in generalized linear models*

Abstract Let ( x i , y i ) i = 1 , … , n denote independent samples from a general mixture distribution ∑ c ∈ C ρ c P c x , and consider the hypothesis class of generalized linear models y ^ = F ( Θ ⊤ x ) . In this study, we investigate the asymptotic joint statistics of a family of generalized linear estimators ( Θ 1 , … , Θ M ) obtained either from (a) minimizing an empirical risk R ^ n ( Θ ; X , y ) or (b) sampling from the associated Gibbs measure exp ⁡ ( − β n R ^ n ( Θ ; X , y ) ) . Our main contribution is to characterize under which conditions the asymptotic joint statistics of this family depends (in a weak sense) only on the means and covariances of the class conditional features distribution P c x . In particular, this allows us to prove the universality of different quantities of interest, such as the training and generalization errors, redeeming a recent line of work in high-dimensional statistics working under the Gaussian mixture hypothesis. Finally, we discuss the applications of our results in different machine learning tasks of interest, such as ensembling and uncertainty quantification.

Revisiting reverberation formulae

In 1992, the author proposed a generalization of Sabine's formula that develops reverberation time over a series of powers of the reflection coefficient on the boundaries. Some years later, he reduced the development to just two terms that made it possible to monitor reverberation times up to large absorptions. The present paper revisits this development and justifies it with the help of free path statistics in 3D rectangular enclosures. It proves that Kuttruff's reverberation formula is a special case of the general formula, but diverges for large absorption. Reverting to Kuttruff's original integration process leads to a formula that does not diverge. The paper further explains the difference between Eyring-type reverberation times that vanish for total absorption, and Sabine-type reverberation times that never vanish even for total absorption, and proposes a simple scheme for evaluating the asymptotic free path statistics and thus improving reverberation time prediction. Lastly, the approach is extended to non-ergodic enclosures.

Novel two-parameter quadratic exponential distribution: Properties, simulation, and applications

This research paper introduces a novel two-parameter quadratic exponential distribution (NTPQED), thoroughly examining its statistical properties and practical applications. The study delves into essential characteristics of the distribution, including its asymptotic behavior, moments, order statistics, and entropies. Additionally, we present fuzzy reliability, value at risk, mean excess function, limited expected value function, tail value at risk, and tail variance. A comprehensive simulation study evaluates fifteen parameter estimation methods. To illustrate the practical relevance of the proposed distribution, three sets of real-world data are analyzed. The performance of the proposed distribution model is rigorously assessed using various model selection criteria and goodness-of-fit test statistics. Empirical findings consistently demonstrate the proposed distribution's superiority over existing models, highlighting its potential for multiple applications.

Statistical Modeling of Right-Censored Spatial Data Using Gaussian Random Fields

Consider a fixed number of clustered areas identified by their geographical coordinates that are monitored for the occurrences of an event such as a pandemic, epidemic, or migration. Data collected on units at all areas include covariates and environmental factors. We apply a probit transformation to the time to event and embed an isotropic spatial correlation function into our models for better modeling as compared to existing methodologies that use frailty or copula. Composite likelihood technique is employed for the construction of a multivariate Gaussian random field that preserves the spatial correlation function. The data are analyzed using counting process and geostatistical formulation that led to a class of weighted pairwise semiparametric estimating functions. The estimators of model parameters are shown to be consistent and asymptotically normally distributed under infill-type asymptotic spatial statistics. Detailed small sample numerical studies that are in agreement with theoretical results are provided. The foregoing procedures are applied to the leukemia survival data in Northeast England. A comparison to existing methodologies provides improvement.

Vasicek and Van Es entropy‐based spectrum sensing for cognitive radios

AbstractAccurate detection of spectrum holes is a useful requirement for cognitive radios that improves the efficiency of spectrum usage. The authors propose three novel, simple, and entropy‐based detectors for spectrum sensing in cognitive radio. The authors evaluate the probability of detection of these three detectors: Vasicek's entropy detector, truncated Vasicek's entropy detector, and Van Es' entropy detector, over a predefined probability of false‐alarm. In particular, the authors provide the approximate and asymptotic test statistics for these detectors in the presence and absence of Nakagami‐m fading, noise variance uncertainty, and optimised detection threshold. Furthermore, the authors provide a detailed comparison study among all the detectors via Monte Carlo simulations and justify authors results through real‐world data. The authors’ experimental results establish a superior performance of truncated Vasicek's entropy detector over Vasicek's entropy detector, energy detector, differential entropy detector and Van Es' entropy detector in practically viable scenarios.

Joint distribution of two local times for diffusion processes with the application to the construction of various conditioned processes

For a diffusion process X(t) of drift and of diffusion coefficient , we study the joint distribution of the two local times and at positions x = 0 and x = L, as well as the simpler statistics of their sum . Their asymptotic statistics for large time involves two very different cases: (i) when the diffusion process X(t) is transient, the two local times remain finite random variables and we analyze their limiting joint distribution; (ii) when the diffusion process X(t) is recurrent, we describe the large deviations properties of the two intensive local times and and of their intensive sum . These properties are then used to construct various conditioned processes satisfying certain constraints involving the two local times, thereby generalizing our previous work (Mazzolo and Monthus 2022 J. Stat. Mech. 103207) concerning the conditioning with respect to a single local time A(t). In particular for the infinite time horizon , we consider the conditioning towards the finite asymptotic values or , as well as the conditioning towards the intensive values or , that can be compared with the appropriate ‘canonical conditioning’ based on the generating function of the local times in the regime of large deviations. This general construction is then applied to the simplest case where the unconditioned diffusion is the Brownian motion of uniform drift µ.

Homogeneity Test of Many-to-One Relative Risk Ratios in Unilateral and Bilateral Data with Multiple Groups

In medical clinical studies, we often encounter paired organs’ unilateral or bilateral data. For bilateral data, there exists an intraclass correlation between paired organs. Under an intraclass correlation model, this paper proposes asymptotic statistics for testing the equality of many-to-one relative risk ratios in combined unilateral and bilateral data. Furthermore, we calculate the explicit expressions of these statistics. Moreover, these procedures are adequate to solve the hypothesis problems of unilateral or bilateral data. Through comparison, the simulation results show that the score test has a robust empirical type-I error rate and sufficient power. We provide a clinical trial of acute otitis media to illustrate our proposed methods.

Homogeneity Test of the First-Order Agreement Coefficient in a Stratified Design

Gwet's first-order agreement coefficient (AC1) is widely used to assess the agreement between raters. This paper proposes several asymptotic statistics for a homogeneity test of stratified AC1 in large sample sizes. These statistics may have unsatisfactory performance, especially for small samples and a high value of AC1. Furthermore, we propose three exact methods for small pieces. A likelihood ratio statistic is recommended in large sample sizes based on the numerical results. The exact E approaches under likelihood ratio and score statistics are more robust in the case of small sample scenarios. Moreover, the exact E method is effective to a high value of AC1. We apply two real examples to illustrate the proposed methods.

Robust approach for comparing two dependent normal populations through Wald-type tests based on Rényi’s pseudodistance estimators

Since the two seminal papers by Fisher (Biometrika 10:507–521, 1915; Metron 1:1–32, 1921) were published, the test under a fixed value correlation coefficient null hypothesis for the bivariate normal distribution constitutes an important statistical problem. In the framework of asymptotic robust statistics, it remains being a topic of great interest to be investigated. For this and other tests, focused on paired correlated normal random samples, Rényi’s pseudodistance estimators are proposed, their asymptotic distribution is established and an iterative algorithm is provided for their computation. From them the Wald-type test statistics are constructed for different problems of interest and their influence function is theoretically studied. For testing null correlation in different contexts, an extensive simulation study and two real data based examples support the robust properties of our proposal.

A stochastic framework for evolving grain statistics using a neural network model for grain topology transformations

We develop a stochastic framework to evolve grain statistics during 2D isotropic grain growth. Grain microstructures observed in simulations and experiments often demonstrate the existence of asymptotic distributions for descriptors such as grain size and topology (number of sides). Moreover, such grain descriptors/states are used to characterize a microstructure as they influence the mechanical properties of a polycrystal. This motivates us to discover laws that govern the distributions of grain microstructure descriptors. In particular, we focus on the evolution of distributions for grain sizes and the number of sides. Since the surface tension is assumed to be isotropic, the von Neumann–Mullins law provides an evolution equation for grain areas in terms of their topological states—number of edges. However, since grains change their topology as they evolve by interacting with neighbors, we construct a topology transformation model (TTM) that predicts the probability of topology transformation of a grain in terms of its current state and the states of its neighbors. The construction of the TTM relies on a data-driven approach using a fully connected deep neural network. Topology transformations recorded in phase field simulations are used as training data. The resulting neural network model is used in a Monte Carlo framework to evolve grain microstructures in a statistical sense. The stochastic method is validated using the asymptotic and transient grain statistics predicted by large-scale phase field simulations. In addition, our results suggest that the TTM incorporates, in a statistical sense, the geometric constraint that grains fill the space.

Measurement errors models with exponential parametric multiplicative distortions

This paper consider a new exponential parametric distortion measurement errors model for the estimation of correlation coefficient between unobserved variables of interest. These unobservable variables are distorted with exponential parametric distortion measurement errors models, which are exponential functions of an observed confounding variable. The nonlinear least squares estimation and weighted nonlinear least squares estimation are used to estimate parameters in the multiplicative distortion functions. We use the fully parametrical calibration and mixed calibration to estimate the correlation coefficient, and show that the proposed estimators of correlation coefficient are all asymptotically efficient. Moreover, we suggest several asymptotic normal approximation statistics to construct the confidence intervals. We conduct Monte Carlo simulation experiments to examine the performance of the proposed estimators. These methods are applied to analyse a real dataset for an illustration.

Homogeneity test of relative risk ratios for stratified bilateral data under different algorithms

Medical clinical studies about paired body parts often involve stratified bilateral data. The correlation between responses from paired parts should be taken into account to avoid biased or misleading results. This paper aims to test if the relative risk ratios across strata are equal under the optimal algorithms. Based on different algorithms, we obtain the desired global and constrained maximum likelihood estimations (MLEs). Three asymptotic test statistics (i.e. , and ) are proposed. Monte Carlo simulations are conducted to evaluate the performance of these algorithms with respect to mean square errors of MLEs and convergence rate. The empirical results show Fisher scoring algorithm is usually better than other methods since it has effective convergence rate for global MLEs, and makes mean-square error lower for constrained MLEs. Three test statistics are compared in terms of type I error rate (TIE) and power. Among these statistics, is recommended according to its robust TIEs and satisfactory power.

Inference and Estimation for Random Effects in High-Dimensional Linear Mixed Models

We consider three problems in high-dimensional linear mixed models. Without any assumptions on the design for the fixed effects, we construct asymptotic statistics for testing whether a collection of random effects is zero, derive an asymptotic confidence interval for a single random effect at the parametric rate , and propose an empirical Bayes estimator for a part of the mean vector in ANOVA type models that performs asymptotically as well as the oracle Bayes estimator. We support our theoretical results with numerical simulations and provide comparisons with oracle estimators. The procedures developed are applied to the Trends in International Mathematics and Sciences Study (TIMSS) data. Supplementary materials for this article are available online.

On Max-Semistable Laws and Extremes for Dynamical Systems.

Suppose is a measure preserving dynamical system and a measurable observable. Let denote the time series of observations on the system, and consider the maxima process . Under linear scaling of , its asymptotic statistics are usually captured by a three-parameter generalised extreme value distribution. This assumes certain regularity conditions on the measure density and the observable. We explore an alternative parametric distribution that can be used to model the extreme behaviour when the observables (or measure density) lack certain regular variation assumptions. The relevant distribution we study arises naturally as the limit for max-semistable processes. For piecewise uniformly expanding dynamical systems, we show that a max-semistable limit holds for the (linear) scaled maxima process.

Forecasting the U.S. Dollar in the 21st Century

The level of the (log of) the exchange rate seems to have strong forecasting power for dollar exchange rates against major currencies post-2000 at medium- to long-run horizons of 12-, 36- and 60-months. We find that this is true using conventional asymptotic statistics correcting for serial correlation biases. But correcting for small-sample bias using simulation methods, we find little evidence to reject a random walk. This small sample bias arises because of near-spurious correlation when the predictor variable is persistent and the horizon for exchange rate forecasts is long. Similar problems of spurious correlation may arise when other persistent variables are used to forecast changes in the exchange rate. We find, in fact, using asymptotic statistics, the level of the exchange rate provides better forecasts than economic measures of “global risk”, and the measures of global risk do not improve the (possibly spurious) forecasting power of the level of the exchange rate.

A Note on Likelihood Ratio Tests for Models with Latent Variables

The likelihood ratio test (LRT) is widely used for comparing the relative fit of nested latent variable models. Following Wilks’ theorem, the LRT is conducted by comparing the LRT statistic with its asymptotic distribution under the restricted model, a chi ^2 distribution with degrees of freedom equal to the difference in the number of free parameters between the two nested models under comparison. For models with latent variables such as factor analysis, structural equation models and random effects models, however, it is often found that the chi ^2 approximation does not hold. In this note, we show how the regularity conditions of Wilks’ theorem may be violated using three examples of models with latent variables. In addition, a more general theory for LRT is given that provides the correct asymptotic theory for these LRTs. This general theory was first established in Chernoff (J R Stat Soc Ser B (Methodol) 45:404–413, 1954) and discussed in both van der Vaart (Asymptotic statistics, Cambridge, Cambridge University Press, 2000) and Drton (Ann Stat 37:979–1012, 2009), but it does not seem to have received enough attention. We illustrate this general theory with the three examples.

On the rate coding response of peripheral sensory neurons.

The rate coding response of a single peripheral sensory neuron in the asymptotic, near-equilibrium limit can be derived using information theory, asymptotic Bayesian statistics and a theory of complex systems. Almost no biological knowledge is required. The theoretical expression shows good agreement with spike-frequency adaptation data across different sensory modalities and animal species. The approach permits the discovery of a new neurophysiological equation and shares similarities with statistical physics.

Universal statistics of Fisher information in deep neural networks: mean field approach * *This article is an updated version of: Ryo K, Shotaro A and Shun-ichi A 2019 Universal statistics of Fisher information in deep neural networks: mean field approach Proc. Machine Learning Res. 89 3181--92

The Fisher information matrix (FIM) is a fundamental quantity to represent the characteristics of a stochastic model, including deep neural networks (DNNs). The present study reveals novel statistics of FIM that are universal among a wide class of DNNs. To this end, we use random weights and large width limits, which enables us to utilize mean field theories. We investigate the asymptotic statistics of the FIM’s eigenvalues and reveal that most of them are close to zero while the maximum eigenvalue takes a huge value. Because the landscape of the parameter space is defined by the FIM, it is locally flat in most dimensions, but strongly distorted in others. Moreover, we demonstrate the potential usage of the derived statistics in learning strategies. First, small eigenvalues that induce flatness can be connected to a norm-based capacity measure of generalization ability. Second, the maximum eigenvalue that induces the distortion enables us to quantitatively estimate an appropriately sized learning rate for gradient methods to converge.

Statistical tests under Dallal's model: Asymptotic and exact methods.

This paper proposes asymptotic and exact methods for testing the equality of correlations for multiple bilateral data under Dallal's model. Three asymptotic test statistics are derived for large samples. Since they are not applicable to small data, several conditional and unconditional exact methods are proposed based on these three statistics. Numerical studies are conducted to compare all these methods with regard to type I error rates (TIEs) and powers. The results show that the asymptotic score test is the most robust, and two exact tests have satisfactory TIEs and powers. Some real examples are provided to illustrate the effectiveness of these tests.

Moment dynamics for gene regulation with rational rate laws.

This aim of this paper is mainly to investigate the performance of two typical moment closure schemes in gene regulatory master equations of rational rate laws. When the reaction rate is polynomial, the error bounds between the authentic and approximate moments obtained by schemes of Gaussian moment closure and log-normal moment closure are explicitly given. When the reaction rate is not polynomial, it is shown that the two schemes both behave well in the absence of active-inactive state switch, but in the presence of active-inactive state switch the log-normal closure scheme is far superior to the Gaussian closure scheme in capturing the asymptotic ensemble statistics. Moreover, the accuracy of the log-normal closure method is further confirmed by steady-state analytic results and the conditional Gaussian closure method. It is also disclosed that optimal negative feedback exists in suppressing protein noise in the presence of the on-off switch control.