Covariance Operator Research Articles

Information Theory With Kernel Methods

We consider the analysis of probability distributions through their associated covariance operators from reproducing kernel Hilbert spaces. We show that the von Neumann entropy and relative entropy of these operators are intimately related to the usual notions of Shannon entropy and relative entropy, and share many of their properties. They come together with efficient estimation algorithms from various oracles on the probability distributions. We also consider product spaces and show that for tensor product kernels, we can define notions of mutual information and joint entropies, which can then characterize independence perfectly, but only partially conditional independence. We finally show how these new notions of relative entropy lead to new upper-bounds on log partition functions, that can be used together with convex optimization within variational inference methods, providing a new family of probabilistic inference methods.

Two-Sample Tests for Relevant Differences in the Eigenfunctions of Covariance Operators

This paper deals with two-sample tests for functional time series data, which have become widely available in conjunction with the advent of modern complex observation systems. Here, particular interest is in evaluating whether two sets of functional time series observations share the shape of their primary modes of variation as encoded by the eigenfunctions of the respective covariance operators. To this end, a novel testing approach is introduced that connects with, and extends, existing literature in two main ways. First, tests are set up in the relevant testing framework, where interest is not in testing an exact null hypothesis but rather in detecting deviations deemed sufficiently relevant, with relevance determined by the practitioner and perhaps guided by domain experts. Second, the proposed test statistics rely on a self-normalization principle that helps to avoid the notoriously difficult task of estimating the long-run covariance structure of the underlying functional time series. The main theoretical result of this paper is the derivation of the large-sample behavior of the proposed test statistics. Empirical evidence, indicating that the proposed procedures work well in finite samples and compare favorably with competing methods, is provided through a simulation study, and an application to annual temperature data.

Spontaneous conformal symmetry breaking and quantum quadratic gravity

We investigate several quantum phenomena related to quadratic gravity after rewriting the general fourth-order action in a more convenient form that is second order in derivatives and produces only first-class constraints in phase space. We find that a Higgs mechanism may occur in the conformally invariant subset of the general quadratic action if the theory is conformally coupled to a scalar field that acquires a nonzero vacuum expectation value and spontaneously breaks the conformal symmetry. Then, in the broken phase, the originally massless spin-2 ghost may absorb both the scalar and vector fields to become massive. We also perform a Becchi-Rouet-Stora-Tyutin quantization of second-order quadratic gravity in the covariant operator formalism and discuss conditions under which unitarity of the full interacting quantum theory may be established.

On Momentum Operators Given by Killing Vectors Whose Integral Curves Are Geodesics

The paper considers momentum operators on intrinsically curved manifolds. Given that momentum operators are Killing vector fields whose integral curves are geodesics, the corresponding manifold is flat or of the compact type with positive constant sectional curvature and dimensions equal to 1, 3, or 7. Explicit representations of momentum operators and the associated Casimir element are discussed for the 3-sphere S3. It is verified that the structural constants of the underlying Lie algebra are proportional to 2 ℏ/R, where R is the curvature radius of S3 and ℏ is the reduced Planck’s constant. This results in a countable energy and momentum spectrum of freely moving particles in S3. The maximal resolution of the possible momenta is given by the de Broglie wave length, λR=πR, which is identical to the diameter of the manifold. The corresponding covariant position operators are defined in terms of geodesic normal coordinates, and the associated commutator relations of position and momentum are established.

Relative perturbation bounds with applications to empirical covariance operators

The goal of this paper is to establish relative perturbation bounds, tailored for empirical covariance operators. Our main results are expansions for empirical eigenvalues and spectral projectors, leading to concentration inequalities and limit theorems. One of the key ingredients is a specific separation measure for population eigenvalues, which we call the relative rank, giving rise to a sharp invariance principle in terms of limit theorems, concentration inequalities and inconsistency results. Our framework is very general, requiring only p>4 moments and allows for a huge variety of dependence structures.

Testing for the rank of a covariance operator

How can we discern whether the covariance operator of a stochastic process is of reduced rank, and if so, what its precise rank is? And how can we do so at a given level of confidence? This question is central to a great deal of methods for functional data, which require low-dimensional representations whether by functional PCA or other methods. The difficulty is that the determination is to be made on the basis of i.i.d. replications of the process observed discretely and with measurement error contamination. This adds a ridge to the empirical covariance, obfuscating the underlying dimension. We build a matrix-completion inspired test statistic that circumvents this issue by measuring the best possible least square fit of the empirical covariance’s off-diagonal elements, optimised over covariances of given finite rank. For a fixed grid of sufficiently large size, we determine the statistic’s asymptotic null distribution as the number of replications grows. We then use it to construct a bootstrap implementation of a stepwise testing procedure controlling the familywise error rate corresponding to the collection of hypotheses formalising the question at hand. Under minimal regularity assumptions, we prove that the procedure is consistent and that its bootstrap implementation is valid. The procedure circumvents smoothing and associated smoothing parameters, is indifferent to measurement error heteroskedasticity, and does not assume a low-noise regime. An extensive simulation study reveals an excellent practical performance, stably across a wide range of settings and the procedure is further illustrated by means of two data analyses.

Convergence and finite sample approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings

This work studies the convergence and finite sample approximations of entropic regularized Wasserstein distances in the Hilbert space setting. Our first main result is that for Gaussian measures on an infinite-dimensional Hilbert space, convergence in the 2-Sinkhorn divergence is strictly weaker than convergence in the exact 2-Wasserstein distance. Specifically, a sequence of centered Gaussian measures converges in the 2-Sinkhorn divergence if the corresponding covariance operators converge in the Hilbert–Schmidt norm. This is in contrast to the previous known result that a sequence of centered Gaussian measures converges in the exact 2-Wasserstein distance if and only if the covariance operators converge in the trace class norm. In the reproducing kernel Hilbert space (RKHS) setting, the kernel Gaussian–Sinkhorn divergence, which is the Sinkhorn divergence between Gaussian measures defined on an RKHS, defines a semi-metric on the set of Borel probability measures on a Polish space, given a characteristic kernel on that space. With the Hilbert–Schmidt norm convergence, we obtain dimension-independent convergence rates for finite sample approximations of the kernel Gaussian–Sinkhorn divergence, of the same order as the Maximum Mean Discrepancy. These convergence rates apply in particular to Sinkhorn divergence between Gaussian measures on Euclidean and infinite-dimensional Hilbert spaces. The sample complexity for the 2-Wasserstein distance between Gaussian measures on Euclidean space, while dimension-dependent, is exponentially faster than the worst case scenario in the literature.

Tempered functional time series

We propose a broad class of models for time series of curves (functions) that can be used to quantify near long‐range dependence or near unit root behavior. We establish fundamental properties of these models and rates of consistency for the sample mean function and the sample covariance operator. The latter plays a role analogous to sample cross‐covariances for multivariate time series, but is far more important in the functional setting because its eigenfunctions are used in principal component analysis, which is a major tool in functional data analysis. It is used for dimension reduction of feature extraction. We also establish a central limit theorem for functions following our model. Both the consistency rates and the normalizations in the Central Limit Theorem (CLT) are nonstandard. They reflect the local unit root behavior and the long memory structure at moderate lags.

Determinantal Ideals and the Canonical Commutation Relations: Classically or Quantized

We construct homomorphic images of $$su(n,n)^{{\mathbb {C}}}$$ in Weyl Algebras $${{\mathcal {H}}}_{2nr}$$ . More precisely, and using the Bernstein filtration of $${{\mathcal {H}}}_{2nr}$$ , $$su(n,n)^{{\mathbb {C}}}$$ is mapped into degree 2 elements with the negative non-compact root spaces being mapped into second order creation operators. Using the Fock representation of $${{\mathcal {H}}}_{2nr}$$ , these homomorphisms give all unitary highest weight representations of $$su(n,n)^{{\mathbb {C}}}$$ thus reconstructing the Kashiwara–Vergne List for the Segal–Shale–Weil representation. Using an idea from the derivation of the their list, we construct a homomorphism of $$u(r)^{{\mathbb {C}}}$$ into $${{\mathcal {H}}}_{2nr}$$ whose image commutes with the image of $$su(n,n)^{{\mathbb {C}}}$$ , and vice versa. This gives the multiplicities. The construction also gives an easy proof that the ideal of $$(r+1)\times (r+1)$$ minors is prime. Here, of course, $$r\le n-1$$ and for a fixed such r, the space of any irreducible representation of $$su(n,n)^{{\mathbb {C}}}$$ is annihilated by this ideal. As a consequence, these representations can be realized in spaces of solutions to Maxwell type equations. We actually go one step further and determine exactly for which representations from our list there is a non-trivial homomorphism between generalized Verma modules, thereby revealing, by duality, exactly which covariant differential operators have unitary null spaces. We prove the analogous results for $${{\mathcal {U}}}_q(su(n,n)^{{\mathbb {C}}})$$ . The Weyl Algebras are replaced by the Hayashi–Weyl Algebras $${{\mathcal {H}}}{{\mathcal {W}}}_{2nr}$$ and the Fock space by a q-Fock space. Further, determinants are replaced by q-determinants, and a homomorphism of $${{\mathcal {U}}}_q(u(r)^{{\mathbb {C}}})$$ into $${{\mathcal {H}}}{{\mathcal {W}}}_{2nr}$$ is constructed with analogous properties. For this purpose a Drinfeld Double is used. We mention one difference: The quantized negative non-compact root spaces, while still of degree 2, are no longer given entirely by second order creation operators.

Existence, uniqueness, and stability of Fourier series solutions of stochastic wave equations with cubic nonlinearities in 3D

Existence, uniqueness, and stability of approximate Fourier series solutions u to damped and undamped, semi-linear stochastic wave equations of the form (in Itô integral sense) utt=σ2△u+B(u,ut)+G(u,ut)∂W∂t with cubic nonlinearities B(u,ut)=a1u−a2||u||2u−κut and homogeneous boundary conditions (HBCs) on general 3D cubes D=[0,lx]×[0,ly]×[0,lz] is studied. The driving Q-regular space–time noise W with linear-growth bounded, state-dependent diffusion intensities G(u,ut) is supposed to be general in space (x,y,z)∈D, but white in time t≥0. The analysis is carried out on an appropriate, separable Hilbert space (i.e. the space of Fourier coefficients) of all solutions which are driven by the eigenfunctions of Laplace operator △ subject to HBCs on the 3D domain D. The dynamics of related expected total energy functional e(t) plays a central role in our discussion, depending on diverse parameters such as diffusivity constant σ, damping κ≥0, transport coefficient a1, length parameters lx,ly,lz, diffusion constants br of G and eigenvalues αn,m,li,j,k of related covariance operator Q. Several examples illustrate the feasibility of our approach and explain our major results.

Quantifying deviations from separability in space-time functional processes

The estimation of covariance operators of spatio-temporal data is in many applications only computationally feasible under simplifying assumptions, such as separability of the covariance into strictly temporal and spatial factors. Powerful tests for this assumption have been proposed in the literature. However, as real world systems, such as climate data are notoriously inseparable, validating this assumption by statistical tests, seems inherently questionable. In this paper we present an alternative approach: By virtue of separability measures, we quantify how strongly the data’s covariance operator diverges from a separable approximation. Confidence intervals localize these measures with statistical guarantees. This method provides users with a flexible tool, to weigh the computational gains of a separable model against the associated increase in bias. As separable approximations we consider the established methods of partial traces and partial products, and develop weak convergence principles for the corresponding estimators. Moreover, we also prove such results for estimators of optimal, separable approximations, which are arguably of most interest in applications. In particular we present for the first time statistical inference for this object, which has been confined to estimation previously. Besides confidence intervals, our results encompass tests for approximate separability. All methods proposed in this paper are free of nuisance parameters and do not require computationally expensive resampling procedures. A simulation study underlines the advantages of our approach and its applicability is demonstrated by the investigation of German annual temperature data.

Principal component analysis of infinite variance functional data

Principal Components Analysis is a widely used approach of multivariate analysis. Over the past 30 years, it has gained renewed attention in the context of functional data, chiefly as a commonly used tool for dimension reduction or feature extraction. Consequently, a large body of statistical theory has been developed to justify its application in various contexts. This theory focuses on the convergence of sample PCs to their population counterparts in a multitude of statistical models and under diverse data collection assumptions. What such results have in common is the assumption that the population covariance operator exists.This paper is concerned with multivariate and functional data that have infinite variance and, consequently, for which the population covariance operator is not defined. However, the sample covariance operator and its eigenfunctions are always defined. It has been unknown what the asymptotic behavior of these important statistics is in the context of infinite variance multivariate or functional data. We derive suitable large sample theory. In particular, we specify normalizing sequences and conditions for suitably defined consistency. We study multivariate models in which explicit limits can be derived. These examples show that definitions, results and intuition developed for multivariate and functional data with finite variance need not apply in the setting we consider.

Distributed Mean-Field Density Estimation for Large-Scale Systems

This article studies how to estimate the mean-field density of large-scale systems in a distributed manner. Such problems are motivated by the recent swarm control technique that uses mean-field approximations to represent the collective effect of the swarm, wherein the mean-field density (especially its gradient) is usually used in feedback control design. In the first part, we formulate the density estimation problem as a filtering problem of the associated mean-field partial differential equation (PDE), for which we employ Kernel density estimation to construct noisy observations and use filtering theory of PDE systems to design an optimal (centralized) density filter. It turns out that the covariance operator of observation noise depends on the unknown density. Hence, we use approximations for the covariance operator to obtain a suboptimal density filter, and prove that both the density estimates and their gradient are convergent and remain close to the optimal one using the notion of input-to-state stability (ISS). In the second part, we continue to study how to decentralize the density filter such that each agent can estimate the mean-field density based on only its own position and local information exchange with neighbors. We prove that the local density filter is also convergent and remains close to the centralized one in the sense of ISS. Simulation results suggest that the centralized suboptimal density filter is able to generate convergent density estimates, and the local density filter is able to converge and remain close to the centralized filter.

Essential self-adjointness for covariant tri-harmonic operators on manifolds and the separation problem

Consider the tri-harmonic differential expression $L_{V}^{\nabla}u=\left(\nabla^{+}\nabla\right)^{3}u+Vu$, on sections of a hermitian vector bundle over a complete Riemannian manifold $\left(M,g\right)$ with metric $g$, where $\nabla$ is a metric covariant derivative on bundle E over complete Riemannian manifold, $\nabla^{+}$ is the formal adjoint of $\nabla$ and $V$ is a self adjoint bundle on $E$. We will give conditions for $L_{V}^{\nabla}$ to be essential self-adjoint in $L^{2}\left(E\right).$ Additionally, we provide sufficient conditions for $L_{V}^{\nabla}$ to be separated in $L^{2}\left( E\right)$. According to Everitt and Giertz, the differential operator $L_{V}^{\nabla}$ is said to be separated in $L^{2}\left( E\right) $ if for all $u$ $\in L^{2}\left( E\right)$ such that $L_{V}^{\nabla}u\in L^{2}\left( E\right) $, we have $Vu\in L^{2}\left( E\right)$.

Multilevel representations of isotropic Gaussian random fields on the sphere

Abstract Series expansions of isotropic Gaussian random fields on $\mathbb {S}^2$ with independent Gaussian coefficients and localized basis functions are constructed. Such representations with multilevel localized structure provide an alternative to the standard Karhunen–Loève expansions of isotropic random fields in terms of spherical harmonics. The basis functions are obtained by applying the square root of the covariance operator to spherical needlets. Localization of the resulting covariance-dependent multilevel basis is shown under decay conditions on the angular power spectrum of the random field. In addition, numerical illustrations are given and an application to random elliptic PDEs on the sphere is analysed.

An Inverse Random Source Problem for the Biharmonic Wave Equation

This paper is concerned with an inverse source problem for the stochastic biharmonic wave equation. The driven source is assumed to be a microlocally isotropic Gaussian random field with its covariance operator being a classical pseudo-differential operator. The well-posedness of the direct problem is examined in the distribution sense, and the regularity of the solution is discussed for the given rough source. For the inverse problem, the strength of the random source, involved in the principal symbol of its covariance operator, is shown to be uniquely determined by a single realization of the magnitude of the wave field averaged over the frequency band with probability one. Numerical experiments are presented to illustrate the validity and effectiveness of the proposed method for the case that the random source is white noise.

Analysis of unitarity in conformal quantum gravity

We perform a canonical quantization of Weyl’s conformal gravity by means of the covariant operator formalism and investigate the unitarity of the resulting quantum theory. After reducing the originally fourth-order theory to second-order in time derivatives via the introduction of an auxiliary tensor field, we identify the full Fock space of quantum states under a Becchi–Rouet–Stora–Tyutin (BRST) construction that includes Faddeev–Popov ghost fields corresponding to Weyl transformations. This second-order formulation allows the formal tools of operator-based quantum field theory to be applied to quadratic gravity for the first time. Using the Kugo–Ojima quartet mechanism, we identify the physical subspace of quantum states and find that the subspace containing the transverse spin-2 states comes equipped with an indefinite inner product metric and a one-particle Hamiltonian that possesses only a single eigenstate. We construct the Lehmann–Symanzik–Zimmermann (LSZ) reduction formula for the S-matrix in this spin-2 subspace and find that unitarity is violated in scattering events. The explicit way in which this violation occurs represents a new view on the ghost-problem in quadratic theories of quantum gravity.

Continuum Covariance Propagation for Understanding Variance Loss in Advective Systems

Motivated by the spurious variance loss encountered during covariance propagation in atmospheric and other large-scale data assimilation systems, we consider the problem for state dynamics governed by the continuity and related hyperbolic partial differential equations. This loss of variance is often attributed to reduced-rank representations of the covariance matrix, as in ensemble methods for example, or else to the use of dissipative numerical methods. Through a combination of analytical work and numerical experiments, we demonstrate that significant variance loss, as well as gain, typically occurs during covariance propagation, even at full rank. The cause of this unusual behavior is a discontinuous change in the continuum covariance dynamics as correlation lengths become small, for instance in the vicinity of sharp gradients in the velocity field. This discontinuity in the covariance dynamics arises from hyperbolicity: the diagonal of the kernel of the covariance operator is a characteristic surface for advective dynamics. Our numerical experiments demonstrate that standard numerical methods for evolving the state are not adequate for propagating the covariance, because they do not capture the discontinuity in the continuum covariance dynamics as correlations lengths tend to zero. Our analytical and numerical results demonstrate in the context of mass conservation that this leads to significant, spurious variance loss in regions of mass convergence and gain in regions of mass divergence. The results suggest that developing local covariance propagation methods designed specifically to capture covariance evolution near the diagonal may prove a useful alternative to current methods of covariance propagation.

On the domain of non-symmetric and, possibly, degenerate Ornstein–Uhlenbeck operators in separable Banach spaces

Let X be a separable Banach space and let X^* be its topological dual. Let Q:X^*\rightarrow X be a linear, bounded, non-negative, and symmetric operator and let A:D(A)\subseteq X\rightarrow X be the infinitesimal generator of a strongly continuous semigroup of contractions on X . We consider the abstract Wiener space (X,\mu_\infty,H_\infty) , where \mu_\infty is a centered non-degenerate Gaussian measure on X with covariance operator defined, at least formally, as Q_\infty=\int_0^{+\infty} e^{sA}Qe^{sA^*}\,ds, and H_\infty is the Cameron–Martin space associated to \mu_\infty . Let H be the reproducing kernel Hilbert space associated with Q with inner product [\cdot,\cdot]_H . We assume that the operator Q_\infty A^*:D(A^*)\subseteq X^*\rightarrow X extends to a bounded linear operator B\in\mathcal{L}(H) which satisfies B+B^*=-\mathrm{Id}_H , where \mathrm{Id}_H denotes the identity operator on H . Let D and D^2 be the first and second order Fréchet derivative operators. We denote by D_H and (D_H,D^2_H) the closure in L^2(X,\mu_\infty) of the operators QD and (QD,QD^2) , respectively, defined on smooth cylindrical functions, and by W^{1,2}_H(X,\mu_\infty) and W^{2,2}_H(X,\mu_\infty) , respectively, their domains in L^2(X,\mu_\infty) . Furthermore, we denote by D_{A_\infty} the closure of the operator Q_\infty A^*D in L^2(X,\mu_\infty) defined on smooth cylindrical functions, and by W^{1,2}_{A_\infty}(X,\mu_\infty) the domain of D_{A_\infty} in L^2(X,\mu_\infty) . We characterize the domain of the operator L , associated to the bilinear form (u,v)\mapsto-\int_{X}[BD_Hu,D_Hv]_H\,d\mu_\infty, \quad u,v\in W^{1,2}_H(X,\mu_\infty), in L^2(X,\mu_\infty) . More precisely, we prove that D(L) coincides, up to an equivalent renorming, with a subspace of W^{2,2}_H(X,\mu_\infty)\cap W^{1,2}_{A_\infty}(X,\mu_\infty) . We stress that we are able to treat the case when L is degenerate and non-symmetric.

Probabilistic Interpretation of Number Operator Acting on Bernoulli Functionals

Let N be the number operator in the space H of real-valued square-integrable Bernoulli functionals. In this paper, we further pursue properties of N from a probabilistic perspective. We first construct a nuclear space G, which is also a dense linear subspace of H, and then by taking its dual G*, we obtain a real Gel’fand triple G⊂H⊂G*. Using the well-known Minlos theorem, we prove that there exists a unique Gauss measure γN on G* such that its covariance operator coincides with N. We examine the properties of γN, and, among others, we show that γN can be represented as a convolution of a sequence of Borel probability measures on G*. Some other results are also obtained.