sub-Gaussian Distribution Research Articles

Expanding the clinical application of OPM-MEG using an effective automatic suppression method for the dental brace metal artifact

Optically pumped magnetometer magnetoencephalography (OPM-MEG) holds significant promise for clinical functional brain imaging due to its superior spatiotemporal resolution. However, effectively suppressing metallic artifacts, particularly from devices such as orthodontic braces and vagal nerve stimulators remains a major challenge, hindering the wider clinical application of wearable OPM-MEG devices.A comprehensive analysis of metal artifact characteristics from time, frequency, and time–frequency perspectives was conducted for the first time using an OPM-MEG device in clinical medicine. This study focused on patients with metal orthodontics, examining the modulation of metal artifacts by breath and head movement, the incomplete regular sub-Gaussian distribution, and the high absolute power ratio in the 0.5–8 Hz band. The existing metal artifact suppression algorithms applied to SQUID-MEG, such as fast independent component analysis (FastICA), information maximization (Infomax), and algorithms for multiple unknown signal extraction (AMUSE), exhibit limited efficacy. Consequently, this study introduced the second-order blind identification (SOBI) algorithm, which utilized multiple time delays for the component separation of OPM-MEG measurement signals. We modified the time delays of the SOBI method to improve its efficacy in separating artifact components, particularly those in the ultralow frequency range. This approach employs the frequency-domain absolute power ratio, root mean square (RMS) value, and mutual information methods to automate the artifact component screening process.The effectiveness of this method was validated through simulation experiments involving four subjects in both resting and evoked experiments. In addition, the proposed method was also validated by the actual OPM-MEG evoked experiments of three subjects. Comparative analyses were conducted against the FastICA, Infomax, and AMUSE algorithms. Evaluation metrics included normalized mean square error, normalized delta band power error, RMS error, and signal-to-noise ratio, demonstrating that the proposed method provides optimal suppression of metal artifacts. This advancement holds promise for enhancing data quality and expanding the clinical applications of OPM-MEG.

Minimax Optimal Bandits for Heavy Tail Rewards.

Stochastic multiarmed bandits (stochastic MABs) are a problem of sequential decision-making with noisy rewards, where an agent sequentially chooses actions under unknown reward distributions to minimize cumulative regret. The majority of prior works on stochastic MABs assume that the reward distribution of each action has bounded supports or follows light-tailed distribution, i.e., sub-Gaussian distribution. However, in a variety of decision-making problems, the reward distributions follow a heavy-tailed distribution. In this regard, we consider stochastic MABs with heavy-tailed rewards, whose p th moment is bounded by a constant νp for . First, we provide theoretical analysis on sub-optimality of the existing exploration methods for heavy-tailed rewards where it has been proven that existing exploration methods do not guarantee a minimax optimal regret bound. Second, to achieve the minimax optimality under heavy-tailed rewards, we propose a minimax optimal robust upper confidence bound (MR-UCB) by providing tight confidence bound of a p -robust estimator. Furthermore, we also propose a minimax optimal robust adaptively perturbed exploration (MR-APE) which is a randomized version of MR-UCB. In particular, unlike the existing robust exploration methods, both proposed methods have no dependence on νp . Third, we provide the gap-dependent and independent regret bounds of proposed methods and prove that both methods guarantee the minimax optimal regret bound for a heavy-tailed stochastic MAB problem. The proposed methods are the first algorithm that theoretically guarantees the minimax optimality under heavy-tailed reward settings to the best of our knowledge. Finally, we demonstrate the superiority of the proposed methods in simulation with Pareto and Fréchet noises with respect to regrets.

The least singular value of a random symmetric matrix

Abstract Let A be an $n \times n$ symmetric matrix with $(A_{i,j})_{i\leqslant j}$ independent and identically distributed according to a subgaussian distribution. We show that $$ \begin{align*}\mathbb{P}(\sigma_{\min}(A) \leqslant \varepsilon n^{-1/2} ) \leqslant C \varepsilon + e^{-cn},\end{align*} $$ where $\sigma _{\min }(A)$ denotes the least singular value of A and the constants $C,c>0 $ depend only on the distribution of the entries of A. This result confirms the folklore conjecture on the lower tail of the least singular value of such matrices and is best possible up to the dependence of the constants on the distribution of $A_{i,j}$ . Along the way, we prove that the probability that A has a repeated eigenvalue is $e^{-\Omega (n)}$ , thus confirming a conjecture of Nguyen, Tao and Vu [Probab. Theory Relat. Fields 167 (2017), 777–816].

Nested Simulation for Conditional Value-at-Risk with Discrete Losses

Nested simulation has been an active area of research in recent years, with an important application in portfolio risk measurement. While majority of the literature has been focusing on the continuous case where portfolio loss is assumed to follow a continuous distribution, monetary losses of a portfolio in practice are usually measured in discrete units, oftentimes due to the practical consideration of meaningful decimal places for a given level of precision in risk measurement. In this paper, we study a nested simulation procedure for estimating conditional Value-at-Risk (CVaR), a popular risk measure, in the case where monetary losses of the portfolio take discrete values. Tailored to the discrete nature of portfolio losses, we propose a rounded estimator and show that when the portfolio loss follows a sub-Gaussian distribution or has a sufficiently high-order moment, the mean squared error (MSE) of the resulting CVaR estimator decays to zero at a rate close to [Formula: see text], much faster than the rate of the CVaR estimator in the continuous case which is [Formula: see text], where [Formula: see text] denotes the sampling budget required by the nested simulation procedure. Performance of the proposed estimator is demonstrated using numerical examples.

Shuffle Differential Private Data Aggregation for Random Population

Bridging the advantages of differential privacy in both centralized model (i.e., high accuracy) and local model (i.e., minimum trust), the shuffle privacy model has potential applications in many privacy-sensitive scenarios, such as mobile user data aggregation and federated learning. Since messages from users are anonymized by semi-trusted shufflers (e.g., anonymous channels, edge servers), every user could hide message among other users’ messages and inject only part of noises (a.k.a. privacy amplification). However, existing works assume that the participating user population is known in advance, which is unrealistic for dynamic environments (e.g., mobile computing, vehicular networks). In this work, we study the shuffle privacy model with a random participating population, and give privacy amplification bounds for population size with commonly encountered binomial, Poisson, sub-Gaussian distribution and etc. For further improving accuracy, we formulate and derive optimal dummy sizes for both non-adaptive and adaptive dummies. Finally, to break the error barrier due to the constraint of sending one single message per user, we design a multi-message shuffle private protocol supporting random population. Experiment results show that our approaches reduce more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$60\%$</tex-math></inline-formula> error when compared to the local model and naive approaches. We hope this work provides tailored solutions of shuffle privacy for dynamic mobile/distributed computing.

On the optimality of the Oja’s algorithm for online PCA

In this paper we analyze the behavior of the Oja’s algorithm for online/streaming principal component subspace estimation. It is proved that with high probability it performs an efficient, gap-free, global convergence rate to approximate an principal component subspace for any sub-Gaussian distribution. Moreover, it is the first time to show that the convergence rate, namely the upper bound of the approximation, exactly matches the lower bound of an approximation obtained by the offline/classical PCA up to a constant factor.

Probabilistic and Analytical Aspects of the Symmetric and Generalized Kaiser–Bessel Window Function

The generalized Kaiser–Bessel window function is defined via the modified Bessel function of the first kind and arises frequently in tomographic image reconstruction. In this paper, we study in details the properties of the Kaiser–Bessel distribution, which we define via the symmetric form of the generalized Kaiser–Bessel window function. The Kaiser–Bessel distribution resembles to the Bessel distribution of McKay of the first type, it is a platykurtic or sub-Gaussian distribution, it is not infinitely divisible in the classical sense and it is an extension of the Wigner’s semicircle, parabolic and n-sphere distributions, as well as of the ultra-spherical (or hyper-spherical) and power semicircle distributions. We deduce the moments and absolute moments of this distribution and we find its characteristic and moment generating function in two different ways. In addition, we find its cumulative distribution function in three different ways and we deduce a recurrence relation for the moments and absolute moments. Moreover, by using a formula of Ismail and May on quotient of modified Bessel functions of the first kind, we deduce a closed-form expression for the differential entropy. We also prove that the Kaiser–Bessel distribution belongs to the family of log-concave and geometrically concave distributions, and we study in details the monotonicity and convexity properties of the probability density function with respect to the argument and each of the parameters. In the study of the monotonicity with respect to one of the parameters we complement a known result of Gronwall concerning the logarithmic derivative of modified Bessel functions of the first kind. Finally, we also present a modified method of moments to estimate the parameters of the Kaiser–Bessel distribution, and by using the classical rejection method we present two algorithms for sampling independent continuous random variables of Kaiser–Bessel distribution. The paper is closed with conclusions and proposals for future works.

Random projections of linear and semidefinite problems with linear inequalities

The Johnson-Lindenstrauss Lemma states that there exist linear maps that project a set of points of a vector space into a space of much lower dimension such that the Euclidean distance between these points is approximately preserved. This lemma has been previously used to prove that we can randomly aggregate, using a random matrix whose entries are drawn from a zero-mean sub-Gaussian distribution, the equality constraints of an Linear Program (LP) while preserving approximately the value of the problem. In this paper we extend these results to the inequality case by introducing a random matrix with non-negative entries that allows to randomly aggregate inequality constraints of an LP while preserving approximately the value of the problem. By duality, the approach we propose allows to reduce both the number of constraints and the dimension of the problem while obtaining some theoretical guarantees on the optimal value. We will also show an extension of our results to certain semidefinite programming instances.

Covariance estimation under one-bit quantization

We consider the classical problem of estimating the covariance matrix of a sub-Gaussian distribution from i.i.d. samples in the novel context of coarse quantization, that is, instead of having full knowledge of the samples, they are quantized to one or two bits per entry. This problem occurs naturally in signal processing applications. We introduce new estimators in two different quantization scenarios and derive nonasymptotic estimation error bounds in terms of the operator norm. In the first scenario, we consider a simple, scale-invariant one-bit quantizer and derive an estimation result for the correlation matrix of a centered Gaussian distribution. In the second scenario, we add random dithering to the quantizer. In this case, we can accurately estimate the full covariance matrix of a general sub-Gaussian distribution by collecting two bits per entry of each sample. In both scenarios, our bounds apply to masked covariance estimation. We demonstrate the near optimality of our error bounds by deriving corresponding (minimax) lower bounds and using numerical simulations.

Compression for Multi-Arm Bandits

The multi-armed bandit (MAB) problem is one of the most well-known active learning frameworks. The aim is to select the best among a set of actions by sequentially observing rewards that come from an unknown distribution. Recently, a number of distributed bandit applications have become popular over wireless networks, where agents geographically separated from a learner collect and communicate the observed rewards. In this paper we propose a compression scheme, that compresses the rewards collected by the distributed agents. By providing nearly matching upper and lower bounds, we tightly characterize the number of bits needed per reward for the learner to accurately learn without suffering additional regret. In particular, we establish a generic reward quantization algorithm, that can be applied on top of any (no-regret) MAB algorithm to form a new communication-efficient counterpart. requires only a few (converging to as low as 3 bits as the number of iterations increases) bits to be sent per reward while preserving the same regret bound as uncompressed rewards. Our lower bound is established via constructing hard instances from a subgaussian distribution. Our theory is further corroborated by numerical experiments.

Non-zero constraints in elliptic PDE with random boundary values and applications to hybrid inverse problems

Hybrid inverse problems are based on the interplay of two types of waves, in order to allow for imaging with both high resolution and high contrast. The inversion procedure often consists of two steps: first, internal measurements involving the unknown parameters and some related quantities are obtained, and, second, the unknown parameters have to be reconstructed from the internal data. The reconstruction in the second step requires the solutions of certain PDE to satisfy some non-zero constraints, such as the absence of nodal or critical points, or a non-vanishing Jacobian.In this work, we consider a second-order elliptic PDE and show that it is possible to satisfy these constraints with overwhelming probability by choosing the boundary values randomly, following a sub-Gaussian distribution. The proof is based on a new quantitative estimate for the Runge approximation, a result of independent interest.

A Coherence Parameter Characterizing Generative Compressed Sensing With Fourier Measurements

In 1, a mathematical framework was developed for compressed sensing guarantees in the setting where the measurement matrix is Gaussian and the signal structure is the range of a generative neural network (GNN). The problem of compressed sensing with GNNs has since been extensively analyzed when the measurement matrix and/or network weights follow a subgaussian distribution. We move beyond the subgaussian assumption, to measurement matrices that are derived by sampling uniformly at random rows of a unitary matrix (including subsampled Fourier measurements as a special case). Specifically, we prove the first known restricted isometry guarantee for generative compressed sensing (GCS) with subsampled isometries, and provide recovery bounds with nearly order-optimal sample complexity, addressing an open problem of 2, p. 10. Recovery efficacy is characterized by the coherence, a new parameter, which measures the interplay between the range of the network and the measurement matrix. Our approach relies on subspace counting arguments and ideas central to high-dimensional probability. Furthermore, we propose a regularization strategy for training GNNs to have favourable coherence with the measurement operator. We provide compelling numerical simulations that support this regularized training strategy: our strategy yields low coherence networks that require fewer measurements for signal recovery. This, together with our theoretical results, supports coherence as a natural quantity for characterizing GCS with subsampled isometries.

Comparison of continuity equation and Gaussian mixture model for long-term density propagation using semi-analytical methods

This paper compares the continuum evolution for density equation modelling and the Gaussian mixture model on the 2D phase space long-term density propagation problem in the context of high-altitude and high area-to-mass ratio satellite long-term propagation. The density evolution equation, a pure numerical and pointwise method for the density propagation, is formulated under the influence of solar radiation pressure and Earth’s oblateness using semi-analytical methods. Different from the density evolution equation and Monte Carlo techniques, for the Gaussian mixture model, the analytical calculation of the density is accessible from the first two statistical moments (i.e. the mean and the covariance matrix) corresponding to each sub-Gaussian distribution for an initial Gaussian density distribution. An insight is given into the phase space long-term density propagation problem subject to nonlinear dynamics. The efficiency and validity of the density propagation are demonstrated and compared between the density evolution equation and the Gaussian mixture model with respect to standard Monte Carlo techniques.

Covariance Estimation under One‐bit Quantization

AbstractWe consider the classical problem of estimating the covariance matrix of a subgaussian distribution from i.i.d. samples in the novel context of coarse quantization, i.e., instead of having full knowledge of the samples, they are quantized to one or two bits per entry. This problem occurs naturally in signal processing applications. We introduce new estimators in two different quantization scenarios and derive non‐asymptotic estimation error bounds in terms of the operator norm. In the first scenario we consider a simple, scale‐invariant one‐bit quantizer and derive an estimation result for the correlation matrix of a centered Gaussian distribution. In the second scenario, we add random dithering to the quantizer. In this case we can accurately estimate the full covariance matrix of a general subgaussian distribution by collecting two bits per entry of each sample. In both scenarios, our bounds apply to masked covariance estimation as well. We demonstrate the near‐optimality of our error bounds by deriving corresponding (minimax) lower bounds and using numerical simulations.

Large-deviation asymptotics of condition numbers of random matrices

AbstractLet $\mathbf{X}$ be a $p\times n$ random matrix whose entries are independent and identically distributed real random variables with zero mean and unit variance. We study the limiting behaviors of the 2-normal condition number k(p,n) of $\mathbf{X}$ in terms of large deviations for large n, with p being fixed or $p=p(n)\rightarrow\infty$ with $p(n)=o(n)$ . We propose two main ingredients: (i) to relate the large-deviation probabilities of k(p,n) to those involving n independent and identically distributed random variables, which enables us to consider a quite general distribution of the entries (namely the sub-Gaussian distribution), and (ii) to control, for standard normal entries, the upper tail of k(p,n) using the upper tails of ratios of two independent $\chi^2$ random variables, which enables us to establish an application in statistical inference.

Joint-diagonalizability-constrained multichannel nonnegative matrix factorization based on time-variant multivariate complex sub-Gaussian distribution

Multichannel nonnegative matrix factorization (MNMF) is a common blind source separation technique that employs full-rank spatial covariance matrices (SCMs). The full-rank SCMs can simulate reverberant mixing systems where the sources are spatially spread. In conventional MNMF, spectrograms of observed signals are modeled by some types of distribution, e.g., the Gaussian distribution and Student’s t distribution. However, MNMF based on the sub-Gaussian distribution has not been proposed because its cost function is difficult to minimize. In this paper, we address the statistical model extension of MNMF to the sub-Gaussian distribution to improve the source separation accuracy. In the proposed method, the generalized Gaussian distribution is utilized as the sub-Gaussian model. Moreover, to design an auxiliary function for the proposed cost function, we introduce the joint-diagonalizability constraint to SCMs similarly to FastMNMF. Two types of update rule for the proposed MNMF are derived on the basis of the majorization-minimization (MM) and majorization-equalization (ME) algorithms. Since the optimization speed of each parameter affects the source separation performance, we experimentally analyze the best combination of MM- and ME-algorithm-based update rules in the proposed method. Experiments of blind source separation reveal that the proposed MNMF based on the sub-Gaussian model can outperform conventional methods.

Efficient Lattice Gadget Decomposition Algorithm With Bounded Uniform Distribution

A gadget decomposition algorithm is commonly used in many advanced lattice cryptography applications which support homomorphic operations over ciphertexts to control the noise growth. For a special structure of a gadget, the algorithm is digit decomposition. If such algorithm samples from a subgaussian distribution, that is, the output is randomized, it gives more benefits on output quality. One of the important advantages is Pythagorean additivity which makes the resulting noise contained in a ciphertext grow much less than naive digit decomposition. Therefore, the error analysis becomes cleaner and tighter than the use of other measures like Euclidean norm and infinity norm. Even though such advantage can also be achieved by use of discrete Gaussian sampling, it is not attractive for practical performance due to a large factor in resulting noise and the complex computation of the exponential function, whereas a more relaxed probability condition is required for a subgaussian distribution. Nevertheless, subgaussian sampling has barely received an attention so far, thus no practical algorithms was implemented before an efficient algorithm is presented by Genis et al. , recently. In this paper, we present a practically efficient gadget decomposition algorithm where output follows a subgaussian distribution. We parallelize the existing practical subgaussian gadget decomposition algorithm, using a bounded uniform distribution. Our algorithm is divided into two independent subalgorithms and only one algorithm depends on the input. Therefore, the other algorithm can be considered as precomputation. As an experimental result, our algorithm performs over 50% better than the existing algorithm.

Estimating covariance and precision matrices along subspaces

We study the accuracy of estimating the covariance and the precision matrix of a $D$-variate sub-Gaussian distribution along a prescribed subspace or direction using the finite sample covariance. Our results show that the estimation accuracy depends almost exclusively on the components of the distribution that correspond to desired subspaces or directions. This is relevant and important for problems where the behavior of data along a lower-dimensional space is of specific interest, such as dimension reduction or structured regression problems. We also show that estimation of precision matrices is almost independent of the condition number of the covariance matrix. The presented applications include direction-sensitive eigenspace perturbation bounds, relative bounds for the smallest eigenvalue, and the estimation of the single-index model. For the latter, a new estimator, derived from the analysis, with strong theoretical guarantees and superior numerical performance is proposed.

Estimating linear and quadratic forms via indirect observations

In this paper, we further develop the approach, originating in Juditsky and Nemirovski (Ann. Statist. 37 (2009) 2278–2300), to “computation-friendly” statistical estimation via Convex Programming.Our focus is on estimating a linear or quadratic form of an unknown “signal,” known to belong to a given convex compact set, via noisy indirect observations of the signal. Classical theoretical results on the subject deal with precisely stated statistical models and aim at designing statistical inferences and quantifying their performance in a closed analytic form. In contrast to this traditional (highly instructive) descriptive framework, the approach we promote here can be qualified as operational – the estimation routines and their risks are not available “in a closed form,” but are yielded by an efficient computation. All we know in advance is that under favorable circumstances the risk of the resulting estimate, whether high or low, is provably near-optimal under the circumstances. As a compensation for the lack of “explanatory power,” this approach is applicable to a much wider family of observation schemes than those where “closed form descriptive analysis” is possible. We discuss applications of this approach to classical problems of estimating linear forms of parameters of sub-Gaussian distribution and quadratic forms of parameters of Gaussian and discrete distributions. The performance of the constructed estimates is illustrated by computation experiments in which we compare the risks of the constructed estimates with (numerical) lower bounds for corresponding minimax risks for randomly sampled estimation problems.

Tensor restricted isometry property analysis for a large class of random measurement ensembles

In previous work, theoretical analysis based on the tensor Restricted Isometry Property (t-RIP) established the robust recovery guarantees of a low-tubal-rank tensor. The obtained sufficient conditions depend strongly on the assumption that the linear measurement maps satisfy the t-RIP. In this paper, by exploiting the probabilistic arguments, we prove that such linear measurement maps exist under suitable conditions on the number of measurements in terms of the tubal rank r and the size of third-order tensor n1, n2, n3. And the obtained minimal possible number of linear measurements is nearly optimal compared with the degrees of freedom of a tensor with tubal rank r. Specially, we consider a random sub-Gaussian distribution that includes Gaussian, Bernoulli and all bounded distributions and construct a large class of linear maps that satisfy a t-RIP with high probability. Moreover, the validity of the required number of measurements is verified by numerical experiments.