Jointly Gaussian Research Articles

Zeroth-order methods for noisy Hölder-gradient functions

In this paper, we prove new complexity bounds for zeroth-order methods in non-convex optimization with inexact observations of the objective function values. We use the Gaussian smoothing approach of Nesterov and Spokoiny(Found Comput Math 17(2): 527–566, 2015. https://doi.org/10.1007/s10208-015-9296-2) and extend their results, obtained for optimization methods for smooth zeroth-order non-convex problems, to the setting of minimization of functions with Hölder-continuous gradient with noisy zeroth-order oracle, obtaining noise upper-bounds as well. We consider finite-difference gradient approximation based on normally distributed random Gaussian vectors and prove that gradient descent scheme based on this approximation converges to the stationary point of the smoothed function. We also consider convergence to the stationary point of the original (not smoothed) function and obtain bounds on the number of steps of the algorithm for making the norm of its gradient small. Additionally we provide bounds for the level of noise in the zeroth-order oracle for which it is still possible to guarantee that the above bounds hold. We also consider separately the case of \(\nu = 1\) and show that in this case the dependence of the obtained bounds on the dimension can be improved.

On the use of the local prior on the absolute magnitude of Type Ia supernovae in cosmological inference

ABSTRACT A dark-energy, which behaves as the cosmological constant until a sudden phantom transition at very low redshift (z < 0.1), seems to solve the >4σ disagreement between the local and high-redshift determinations of the Hubble constant, while maintaining the phenomenological success of the Λ cold dark matter model with respect to the other observables. Here, we show that such a hockey-stick dark energy cannot solve the H0 crisis. The basic reason is that the supernova absolute magnitude MB that is used to derive the local H0 constraint is not compatible with the MB that is necessary to fit supernova, baryon acoustic oscillation, and cosmic microwave background data, and this disagreement is not solved by a sudden phantom transition at very low redshift. We make use of this example to show why it is preferable to adopt in the statistical analyses the prior on MB as an alternative to the prior on H0. The three reasons are: (i) one avoids potential double counting of low-redshift supernovae, (ii) one avoids assuming the validity of cosmography, in particular, fixing the deceleration parameter to the standard model value q0 = −0.55, (iii) one includes in the analysis the fact that MB is constrained by local calibration, an information which would otherwise be neglected in the analysis, biasing both model selection and parameter constraints. We provide the priors on MB relative to the recent Pantheon and DES-SN3YR supernova catalogs. We also provide a Gaussian joint prior on H0 and q0 that generalizes the prior on H0 by Supernova H0 for the Equation of State.

Multilook phase difference distribution for slightly rough boundary layered ground

The authors derive the distribution of the phase difference between two multilook scattering signals for a multilayer stack with randomly rough surfaces under a plane wave excitation. First, for infinite slightly rough surfaces described by Gaussian centred stochastic processes, the authors show that the underlying complex scattering signals follow a Gaussian joint distribution. Also, it is demonstrated that this property is within the scope of the first-order perturbation theory. Secondly, the authors use this joint probability law to derive the closed-form expression for the probability density function of the phase difference. The theoretical formula is verified by comparison with Monte-Carlo simulations.

Economic Planning of the Operation of PV-Connected Distribution Network Using a Probabilistic Model for Extending the Network Lifetime

The power sharing of PV sets with battery banks in a low voltage distribution network, is optimized with the aim of extending network lifetime. The network lifetime is analyzed using a probabilistic model, in which each PV-battery set has a certain failure probability of supplying power to any load demand center. This probability is assumed to be of normal distribution, that is related to other factors such as power rating, coverage availability, and battery DoD. To extend network lifetime, redundancies in power sharing are removed by activating different groups of PV sets at different times, with durations depending on their joint Gaussian probabilities in supplying the load demands. The contribution of each PV-battery set, is estimated in an intuitive method according to the evaluated probabilities, in which the network is converted into source nodes and load nodes distributed as an ad hoc network, with formulated Gaussian probabilities. An economic load dispatch is then evaluated among the selected PV sets of probabilities higher than a predefined threshold value, to optimize power sharing of the load. A case study of several PV-battery sets supplying several distributed loads, is analyzed and simulated, with formulated joint probabilities. It is found that lifetime is extended by 190% for three PV sets supplying two load centers.

Propagation of uncertainty of soil hydraulic parameterization in the prediction of water balance components: A stochastic analysis in kaolinitic clay soils

Hydrological models are powerful tools to understand soil–vegetation–atmosphere transport (SVAT) processes. These processes are partially controlled by soil water retention and hydraulic conductivity, in process-based models described by analytical functions (K-θ-h) that define the dependence between pressure head (h), soil water content (θ) and hydraulic conductivity (K). Given the nonlinearity of these process-based SVAT models, stochastic analysis is an interesting tool to get insight in the pattern of model outputs given the uncertainty in the K–θ–h functions. We developed a stochastic framework to evaluate outputs of the SWAP hydrological model according to the uncertainty in the van Genuchten-Mualem K–θ–h analytical functions (VGM). A multivariate Gaussian joint distribution analysis stochastically sampled VGM parameters using two statistical methods, one considering and one disregarding parameter correlations. We evaluated the relations of VGM parameters and simulation outputs (i) for an internal drainage scenario to predict pressure head at flux-based field capacity and (ii) for a cropped scenario to predict water balance components. We also assessed the relative importance of each parameter using linear regression and random forest methods. The hydrological stochastic simulations showed that the developed algorithm allowed to perform multiple SWAP model runs in a quick and unassisted manner on a common personal computer. Although the degree of VGM uncertainty of the studied soils did not affect the SWAP simulation results considerably, model output uncertainty increased when uncorrelated stochastic realizations of VGM parameters were used. The relative propagation of parameter uncertainty showed to be dependent on the simulated scenario (boundary conditions) and on the soil itself. For the correlated VGM stochastic realizations, both criteria used to analyze the parameter importance detected the VGM parameter l as one of the most relevant in determining water balance components variation. This is an interesting outcome, as this parameter is hardly ever determined, and commonly assumed to a predefined value.

Anomaly Detection for Hyperspectral Images Based on Improved Low-Rank and Sparse Representation and Joint Gaussian Mixture Distribution

The background dictionary used in the hyperspectral images anomaly detection based on low-rank and sparse representation (LRASR) contains both target information and background information which will result in low detection accuracy. In response to this problem, this article proposes an improved hyperspectral anomaly detection algorithm that is based on low-rank and sparse representation and joint Gaussian mixture distribution (MOG-LRASR). Modeling the sparse components as a mixture of Gaussian (MOG) distribution in the traditional low-rank and sparse decomposition model can get a purer low-rank background component. Then using the low-rank background component as the input of the dictionary learning to get the sparse matrix to be detected. Since the distribution in the anomalous part is usually sparse and complex, Manhattan distance is used to evaluate anomalous pixels in this article. Using Wilcoxon rank-sum test, the experimental results show that the algorithm proposed in this article has the highest score, which proves the MOG-LRASR has higher detection stability than other algorithms. Also, it has achieved better detection results on other data sets indicated by the experiments.

Enhanced conditional Co-Gibbs sampling algorithm for data imputation

The Gibbs sampler is an iterative algorithm for data imputation of a random vector at locations where values of the variable of interest are missing. In this algorithm, the simulated values converge to a Gaussian random vector distribution with zero mean and a given covariance matrix obtained by solving a simple kriging system through several iterations. In a bivariate dataset, if the principal variable for imputation depends on an auxiliary variable that is more abundant at the sample locations, this algorithm fails to produce the local and spatial cross-correlation structures. To overcome this impediment, a variant of the Gibbs sampler, the conditional Co-Gibbs sampler, has been proposed in this study, where simple kriging is replaced by three alternative cokriging paradigms: multicollocated cokriging, collocated cokriging, and homotopic cokriging. The algorithm was examined for an actual case study to statistically evaluate its performance. The results indicate that the conditional Co-Gibbs sampler with multicollocated cokriging outperformed the alternatives, including simple kriging where data imputation occurred as a consequence of ignoring the influence of the auxiliary variable, partially or totally. In addition, a computer software, provided as an open-source executable file, was used to implement the proposed algorithm for data imputation in bivariate cases.

Densities of distributions of homogeneous functions of Gaussian random vectors

We obtain broad sufficient conditions for the boundedness of distribution densities of homogeneous functions on spaces with Gaussian measures. Estimates for the distribution densities of maxima of quadratic forms are obtained.

On the MMSE estimation of norm of a Gaussian vector under additive white Gaussian noise with randomly missing input entries

This paper considers the task of estimating the l2 norm of a n-dimensional random Gaussian vector from noisy measurements taken after many of the entries of the vector are missed and only K (0 ≤ K ≤ n) entries are retained while the rest of the entries are erased and set to 0. Specifically, we evaluate the minimum mean square error (MMSE) estimator of the l2 norm of the unknown Gaussian vector performing measurements under additive white Gaussian noise (AWGN) on the vector after the data missing and derive expressions for the corresponding mean square error (MSE). We find that the corresponding MSE normalized by n tends to 0 as n → ∞ for any 1 ≤ k ≤ n. Furthermore, expressions for the MSE is derived when the variance of the AWGN noise tends to either 0 or ∞. These results generalize the results of Dytso et al. [1] where the case K=n is considered, i.e. the MMSE estimator of norm of random Gaussian vector is derived from measurements under AWGN noise without considering the data missing phenomenon.

Gaussian variant of Freivalds’ algorithm for efficient and reliable matrix product verification

Abstract In this article, we consider the general problem of checking the correctness of matrix multiplication. Given three n × n n\times n matrices 𝐴, 𝐵 and 𝐶, the goal is to verify that A × B = C A\times B=C without carrying out the computationally costly operations of matrix multiplication and comparing the product A × B A\times B with 𝐶, term by term. This is especially important when some or all of these matrices are very large, and when the computing environment is prone to soft errors. Here we extend Freivalds’ algorithm to a Gaussian Variant of Freivalds’ Algorithm (GVFA) by projecting the product A × B A\times B as well as 𝐶 onto a Gaussian random vector and then comparing the resulting vectors. The computational complexity of GVFA is consistent with that of Freivalds’ algorithm, which is O ⁢ ( n 2 ) O(n^{2}) . However, unlike Freivalds’ algorithm, whose probability of a false positive is 2 - k 2^{-k} , where 𝑘 is the number of iterations, our theoretical analysis shows that, when A × B ≠ C A\times B\neq C , GVFA produces a false positive on set of inputs of measure zero with exact arithmetic. When we introduce round-off error and floating-point arithmetic into our analysis, we can show that the larger this error, the higher the probability that GVFA avoids false positives. Moreover, by iterating GVFA 𝑘 times, the probability of a false positive decreases as p k p^{k} , where 𝑝 is a very small value depending on the nature of the fault on the result matrix and the arithmetic system’s floating-point precision. Unlike deterministic algorithms, there do not exist any fault patterns that are completely undetectable with GVFA. Thus GVFA can be used to provide efficient fault tolerance in numerical linear algebra, and it can be efficiently implemented on modern computing architectures. In particular, GVFA can be very efficiently implemented on architectures with hardware support for fused multiply-add operations.

Iterative algorithms for non-conditional and conditional simulation of Gaussian random vectors

The conditional simulation of Gaussian random vectors is widely used in geostatistical applications to quantify uncertainty in regionalized phenomena that have been observed at finitely many sampling locations. Two iterative algorithms are presented to deal with such a simulation. The first one is a variation of the propagative version of the Gibbs sampler aimed at simulating the random vector without any conditioning data. The novelty of the presented algorithm stems from the introduction of a relaxation parameter that, if adequately chosen, allows quickening the rates of convergence and mixing of the sampler. The second algorithm is meant to convert the non-conditional simulation into a conditional one, based on the successive over-relaxation method. Again, a relaxation parameter allows quickening the convergence in distribution to the desired conditional random vector. Both algorithms are applicable in a very general setting and avoid the pivoting, inversion, square rooting or decomposition of the variance-covariance matrix of the vector to be simulated, thus reduce the computation costs and memory requirements with respect to other discrete geostatistical simulation approaches.

Efficient Importance Sampling for the Left Tail of Positive Gaussian Quadratic Forms

Estimating the left tail of quadratic forms in Gaussian random vectors is of major practical importance in many applications. In this letter, we propose an efficient importance sampling estimator that is endowed with the bounded relative error property. This property significantly reduces the number of simulation runs required by the proposed estimator compared to naive Monte Carlo (MC), especially when the probability of interest is very small. Selected simulation results are presented to illustrate the efficiency of our estimator compared to naive MC as well as some of the well-known approximations.

The Local Geometry of Testing in Ellipses: Tight Control via Localized Kolmogorov Widths

Mean testing over ellipses is a decision problem that arises in several applications, including non-parametric goodness-of-fit testing, signal detection in cognitive radio, and regression function testing in reproducing kernel Hilbert spaces. We study the local geometry of such testing problems with compound alternatives. Given samples of a Gaussian random vector, the goal is to distinguish whether the mean is equal to a known vector within an ellipse, or equal to some other unknown vector in the ellipse. While past work on such problems has focused on the difficulty in a global sense, we study difficulty in a way that is localized to each vector within the ellipse. Our main result is to give sharp upper and lower bounds on the localized minimax testing radius in terms of an explicit formula involving the Kolmogorov width of the ellipse intersected with a Euclidean ball. When applied to particular examples, our general theorems yield interesting rates that were not known before: as a particular case, for testing in Sobolev ellipses of smoothness $\alpha $ , we demonstrate rates that vary from $(\sigma ^{2})^{\frac {4 \alpha }{4 \alpha + 1}}$ , corresponding to the classical global rate, to the faster rate $(\sigma ^{2})^{\frac {8 \alpha }{8 \alpha + 1}}$ , achievable for vectors at favorable locations within the ellipse. We also show that the optimal test for this problem is achieved by a linear projection test that is based on an explicit lower-dimensional projection of the observation vector.

Learning‐based design of random measurement matrix for compressed sensing with inter‐column correlation using copula function

In this work, a novel learning-based approach for the design of a compressed sensing measurement matrix is proposed. In contrast with the state-of-the-art methods, the suggested approach takes into account the correlation within entries of each column of the measurement matrix, namely, the inter-column correlation (ICC). The new method makes use of a rather small number of training sparse signal vectors in a recursive scheme to obtain their corresponding measurement vectors. The latter is exploited to estimate the copula function of measurements which, in turn, is used to generate an arbitrary number of measurement vector ensembles. By using the latter, the autocorrelation of the measurement vectors is estimated precisely and then, the ICC of measurement matrix under design is obtained from the autocorrelation. Given the resulting ICC, the measurement matrix columns are to be generated independently, e.g. by employing a proper random Gaussian vector generator. Performance evaluations using both synthetic and real-world data confirm the superiority of the proposed approach to the less recent methods.

Gaussian copula-based zero-inflated power series joint models to analyze correlated count data

A Gaussian Copula-based regression model is proposed that accounts for associations between count responses with extra zeros. Our approach entails underlying latent variables to indicate the latent mechanisms which generate the count responses where some of the count responses are inflated in a zero point. The model contains, as special sub-models, several important distributions such as the power series distributions with and without extra zeros, for example, Poisson, negative binomial, zero-inflated Poisson and zero-inflated negative binomial distributions. The full likelihood-based inference method is applied for the estimation of parameters to obtain maximum likelihood estimates of the parameters. Modified Pearson residuals, where the correlation between responses is taken into account, are used for finding abnormal observations. To illustrate the utility of the models, some simulations are illustrated. Finally, the proposed models are applied to an insurance data set for insurers, obtained from an observational study, where the number of automobile claims and the number of third party claims are the correlated count responses. The effects of car age and the type of car, driving place on both responses are investigated simultaneously.

Gaussian process optimization with failures: classification and convergence proof

We consider the optimization of a computer model where each simulation either fails or returns a valid output performance. We first propose a new joint Gaussian process model for classification of the inputs (computation failure or success) and for regression of the performance function. We provide results that allow for a computationally efficient maximum likelihood estimation of the covariance parameters, with a stochastic approximation of the likelihood gradient. We then extend the classical improvement criterion to our setting of joint classification and regression. We provide an efficient computation procedure for the extended criterion and its gradient. We prove the almost sure convergence of the global optimization algorithm following from this extended criterion. We also study the practical performances of this algorithm, both on simulated data and on a real computer model in the context of automotive fan design.

A General Version of Price\u2019s Theorem

Assume that X_{Sigma } in mathbb {R}^{n} is a centered random vector following a multivariate normal distribution with positive definite covariance matrix Sigma . Let g : mathbb {R}^{n} rightarrow mathbb {C} be measurable and of moderate growth, say |g(x)| lesssim (1 + |x|)^{N}. We show that the map Sigma mapsto mathbb {E}[g(X_{Sigma })] is smooth, and we derive convenient expressions for its partial derivatives, in terms of certain expectations mathbb {E}[(partial ^{alpha }g)(X_{Sigma })] of partial (distributional) derivatives of g. As we discuss, this result can be used to derive bounds for the expectation mathbb {E}[g(X_{Sigma })] of a nonlinear function g(X_{Sigma }) of a Gaussian random vector X_{Sigma } with possibly correlated entries. For the case when gleft( xright) = g_{1}(x_{1}) cdots g_{n}(x_{n}) has tensor-product structure, the above result is known in the engineering literature as Price’s theorem, originally published in 1958. For dimension n = 2, it was generalized in 1964 by McMahon to the general case g : mathbb {R}^{2} rightarrow mathbb {C}. Our contribution is to unify these results, and to give a mathematically fully rigorous proof. Precisely, we consider a normally distributed random vector X_{Sigma } in mathbb {R}^{n} of arbitrary dimension n in mathbb {N}, and we allow the nonlinearity g to be a general tempered distribution. To this end, we replace the expectation mathbb {E}left[ g(X_{Sigma })right] by the dual pairing leftlangle g,,phi _{Sigma }rightrangle _{mathcal {S}',mathcal {S}}, where phi _{Sigma } denotes the probability density function of X_{Sigma }.

Sampling Hyperspheres via Extreme Value Theory: Implications for Measuring Attractor Dimensions

The attractor Hausdorff dimension is an important quantity bridging information theory and dynamical systems, as it is related to the number of effective degrees of freedom of the underlying dynamical system. By using the link between extreme value theory and Poincare recurrences, it is possible to estimate this quantity from time series of high-dimensional systems without embedding the data. In general $$d \le n$$ , where n is the dimension of the full phase-space, as the dynamics freezes some of the available degrees of freedom. This is equivalent to constraining trajectories on a compact object in phase space, namely the attractor. Information theory shows that the equality $$d=n$$ holds for random systems. However, applying extreme value theory, we show that this result cannot be recovered and that $$d<n$$ . We attribute this effect to the curse of dimensionality, and in particular to the phenomenon of concentration of the norm observed in high-dimensional systems. We derive a theoretical expression for d(n) for Gaussian random vectors, and we show numerically that similar curse of dimensionality effects are found for random systems characterized by non-Gaussian distributions. Finally, we show that the effect of the curse of dimensionality can be quantified using the extreme value theory, thus enabling to retrieve the degree of non-randomness of a system. We provide examples issued from real-world climate and financial datasets.

Characterization of challenges in asymmetric nonlinear vibration energy harvesters subjected to realistic excitation

With an explosion of the internet of things (IoT), vibration energy harvesting provides an environmentally friendly solution to replace consumable batteries in powering IoT wireless sensors. Yet, when implemented in practice, ambient vibration input energy is much less periodic than assumptions adopted in previous studies. This becomes especially important given that asymmetries are inevitable in nonlinear device platforms. This research sheds light on these complex challenges of practical vibration energy harvesting by developing and exploring an analytical model based on equivalent linearization. The modeling approach provides an opportunity to understand influences of asymmetry, nonlinearity, and combined excitation response on the DC power delivery of energy harvesters. In the analytical model, a weighted Gaussian joint distribution is utilized to approximate the influences caused by the random excitation. Combined with numerical and experimental validation, the analysis indicates that with the increase of stochastic base acceleration, two outcomes are possible. A first outcome involves an enhancement of DC power by way of triggering large amplitude nonlinear oscillations. A second outcome corresponds to a loss of high power delivery since the noise interferes with the attainment of the snap-through dynamic. Either reducing asymmetry or increasing harmonic excitation component is found to be favorable to induce the power-enhancing dynamics and inhibit the occurrence of the second case. Although with the simplified Gaussian distribution, the analytical framework cannot reproduce exact details of the dynamic responses in every case, the results show that the statistical trends of the analysis are overall borne out in simulation and experiment. This indicates the new modeling of this research may help guide attention to design and deployment techniques for nonlinear vibration energy harvesters in practical combined excitation environments, where limitations on precise manufacture or placement may introduce structural asymmetry.

Random matrices with prescribed eigenvalues and expectation values for random quantum states

Given a collection λ _ = { λ 1 \underline {\lambda }=\{\lambda _1 , … \dots , λ n } \lambda _n\} of real numbers, there is a canonical probability distribution on the set of real symmetric or complex Hermitian matrices with eigenvalues λ 1 , … , λ n \lambda _1, \ldots , \lambda _n . In this paper, we study various features of random matrices with this distribution. Our main results show that under mild conditions, when n n is large, linear functionals of the entries of such random matrices have approximately Gaussian joint distributions. The results take the form of upper bounds on distances between multivariate distributions, which allows us also to consider the case when the number of linear functionals grows with n n . In the context of quantum mechanics, these results can be viewed as describing the joint probability distribution of the expectation values of a family of observables on a quantum system in a random mixed state. Other applications are given to spectral distributions of submatrices, the classical invariant ensembles, and to a probabilistic counterpart of the Schur–Horn theorem, relating eigenvalues and diagonal entries of Hermitian matrices.