Manifold Of Positive Definite Matrices Research Articles

Joint Learning of Full-Structure Noise in Hierarchical Bayesian Regression Models.

We consider the reconstruction of brain activity from electroencephalography (EEG). This inverse problem can be formulated as a linear regression with independent Gaussian scale mixture priors for both the source and noise components. Crucial factors influencing the accuracy of the source estimation are not only the noise level but also its correlation structure, but existing approaches have not addressed the estimation of noise covariance matrices with full structure. To address this shortcoming, we develop hierarchical Bayesian (type-II maximum likelihood) models for observations with latent variables for source and noise, which are estimated jointly from data. As an extension to classical sparse Bayesian learning (SBL), where across-sensor observations are assumed to be independent and identically distributed, we consider Gaussian noise with full covariance structure. Using the majorization-maximization framework and Riemannian geometry, we derive an efficient algorithm for updating the noise covariance along the manifold of positive definite matrices. We demonstrate that our algorithm has guaranteed and fast convergence and validate it in simulations and with real MEG data. Our results demonstrate that the novel framework significantly improves upon state-of-the-art techniques in the real-world scenario where the noise is indeed non-diagonal and full-structured. Our method has applications in many domains beyond biomagnetic inverse problems.

Extrinsic Bayesian Optimization on Manifolds

We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and utilizing the uncertainty in that surrogate by deriving an acquisition function. This acquisition function represents the probability of improvement based on the kernel of the Gaussian process, which guides the search in the optimization process. The critical challenge for designing Bayesian optimization algorithms on manifolds lies in the difficulty of constructing valid covariance kernels for Gaussian processes on general manifolds. Our approach is to employ extrinsic Gaussian processes by first embedding the manifold onto some higher dimensional Euclidean space via equivariant embeddings and then constructing a valid covariance kernel on the image manifold after the embedding. This leads to efficient and scalable algorithms for optimization over complex manifolds. Simulation study and real data analyses are carried out to demonstrate the utilities of our eBO framework by applying the eBO to various optimization problems over manifolds such as the sphere, the Grassmannian, and the manifold of positive definite matrices.

Riemannian Optimization via Frank-Wolfe Methods

We study projection-free methods for constrained Riemannian optimization. In particular, we propose a Riemannian Frank-Wolfe (RFW) method that handles constraints directly, in contrast to prior methods that rely on (potentially costly) projections. We analyze non-asymptotic convergence rates of RFW to an optimum for geodesically convex problems, and to a critical point for nonconvex objectives. We also present a practical setting under which RFW can attain a linear convergence rate. As a concrete example, we specialize RFW to the manifold of positive definite matrices and apply it to two tasks: (i) computing the matrix geometric mean (Riemannian centroid); and (ii) computing the Bures-Wasserstein barycenter. Both tasks involve geodesically convex interval constraints, for which we show that the Riemannian “linear” oracle required by RFW admits a closed form solution; this result may be of independent interest. We complement our theoretical results with an empirical comparison of RFW against state-of-the-art Riemannian optimization methods, and observe that RFW performs competitively on the task of computing Riemannian centroids.

High Resolution Ex Vivo Diffusion Tensor Distribution MRI of Neural Tissue

Neural tissue microstructure plays a key role in developmental, physiological and pathophysiological processes. In the continuing quest to characterize it at ever finer length scales, we use a novel diffusion tensor distribution (DTD) paradigm to probe microstructural features much smaller than the nominal MRI voxel size. We first assume the DTD is a normal tensor variate distribution constrained to lie on the manifold of positive definite matrices, characterized by a mean and covariance tensor. We then estimate the DTD using Monte Carlo signal inversion combined with parsimonious model selection framework that exploits a hierarchy of symmetries of mean and covariance tensors. High resolution multiple pulsed field gradient (mPFG) MRI measurements were performed on a homogeneous isotropic diffusion phantom (PDMS) for control, and excised visual cortex and spinal cord of macaque monkey to investigate the capabilities of DTD MRI in revealing neural tissue microstructural features using strong gradients not typically available in clinical MRI scanners. DTD-derived stains and glyphs, which disentangle size, shape, and orientation heterogeneities of microscopic diffusion tensors, are presented for all samples along with the distribution of the mean diffusivity (MD) within each voxel. We also present a new glyph to visualize the symmetric (kurtosis) and asymmetric parts of the fourth-order covariance tensor. An isotropic mean diffusion tensor and zero covariance tensor was found for the isotropic PDMS phantom, as expected, while the covariance tensor (both symmetric and asymmetric parts) for neural tissue was non-zero indicating that the kurtosis tensor may not be sufficient to fully describe the microstructure. Cortical layers were clearly delineated in the higher moments of the MD spectrum consistent with histology, and microscopic anisotropy was detected in both gray and white matter of neural tissue. DTD MRI captures crossing and splaying white matter fibers penetrating into the cortex, and skewed fiber diameter distributions in the white matter tracts within the cortex and spinal cord. DTD MRI was also shown to subsume diffusion tensor imaging (DTI) while providing additional microstructural information about tissue heterogeneity and microscopic anisotropy within each voxel.

Kriging Riemannian Data via Random Domain Decompositions

Data taking value on a Riemannian manifold and observed over a complex spatial domain are becoming more frequent in applications, for example, in environmental sciences and in geoscience. The analysis of these data needs to rely on local models to account for the nonstationarity of the generating random process, the nonlinearity of the manifold, and the complex topology of the domain. In this article, we propose to use a random domain decomposition approach to estimate an ensemble of local models and then to aggregate the predictions of the local models through Fréchet averaging. The algorithm is introduced in complete generality and is valid for data belonging to any smooth Riemannian manifold but it is then described in details for the case of the manifold of positive definite matrices, the hypersphere and the Cholesky manifold. The predictive performances of the method are explored via simulation studies for covariance matrices and correlation matrices, where the Cholesky manifold geometry is used. Finally, the method is illustrated on an environmental dataset observed over the Chesapeake Bay (USA). Supplementary materials for this article are available online.

Local extrema for Procrustes problems in the set of positive definite matrices

Given two positive definite matrices A and B, a well known result by Gelfand, Naimark and Lidskii establishes a relationship between the eigenvalues of A and B and those of AB by means of majorization inequalities. In this work we make a local study focused on the spectrum of the matrices that achieve the equality in those inequalities. As an application, we complete some previous results concerning Procrustes problems for unitarily invariant norms in the manifold of positive definite matrices.

Convex multivariate operator means

We prove that a trace function, generated by the functional calculus, is geodesically convex in the Riemannian manifold of positive definite matrices, if and only if it is geodesically convex in positive numbers. The analysis of multivariate operator means is facilitated by the study of classes of means that are fix-points under a contraction with respect to the Thompson metric. Although such methods are powerful, they crucially depend on monotonicity. We develop new techniques to prove existence of multivariate operator means that are not necessarily monotone.

Procrustes problems in Riemannian manifolds of positive definite matrices

We consider the manifold of positive definite matrices endowed with the Fisher Riemannian metric and some other distances commonly used in information theory. We show that for each of them the best approximant to A from the unitary orbit of another matrix B commutes with A.

Clustering Brain-Network Time Series by Riemannian Geometry

This paper advocates Riemannian multi-manifold modeling for network-wide time-series analysis: Dynamic brain-network data yield features which are viewed as points in or close to a union of a finite number of submanifolds of a Riemannian manifold. Distinguishing disparate time series amounts then to clustering multiple Riemannian submanifolds. To this end, two feature-generation schemes for network-wide dynamic time series are put forth. The first one is motivated by Granger-causality arguments and uses an auto-regressive moving average model to map low-rank linear vector subspaces, spanned by column vectors of observability matrices, to points into the Grassmann manifold. The second one utilizes (non-linear) dependencies among network nodes by introducing kernel-based partial correlations to generate points in the manifold of positive-definite matrices. Capitalizing on recently developed research on Riemannian-submanifold clustering, an algorithm is provided to differentiate time series based on their Riemannian-geometry properties. Extensive numerical tests on synthetic and real fMRI data demonstrate that the proposed framework outperforms classical and state-of-the-art techniques in clustering brain-network states/structures.

Priors with Coupled First and Second Order Differences for Manifold-Valued Image Processing

Recently variational models with priors involving first and second order derivatives resp. differences were successfully applied for image restoration. There are several ways to incorporate the derivatives of first and second order into the prior, for example additive coupling or using infimal convolution (IC), as well as the more general model of total generalized variation (TGV). The later two methods give also decompositions of the restored images into image components with distinct "smoothness" properties which are useful in applications. This paper is the first attempt to generalize these models to manifold-valued images. We propose both extrinsic and intrinsic approaches. The extrinsic approach is based on embedding the manifold into an Euclidean space of higher dimension. Models following this approach can be formulated within the Euclidean space with a constraint restricting them to the manifold. Then alternating direction methods of multipliers can be employed for finding minima. However, the components within the infimal convolution or total generalized variation decomposition live in the embedding space rather than on the manifold which makes their interpretation difficult. Therefore we also investigate two intrinsic approaches. For manifolds which are Lie groups we propose three priors which exploit the group operation, an additive one, another with IC coupling and a third TGV like one. For computing the minimizers of the intrinsic models we apply gradient descent algorithms. For general Riemannian manifolds we further define a model for infimal convolution based on the recently developed second order differences. We demonstrate by numerical examples that our approaches works well for the circle, the 2-sphere, the rotation group, and the manifold of positive definite matrices with the affine invariant metric.

Repetition invariant geometric means

We introduce a class of repetition invariant geometric means and obtain corresponding contractive barycentric maps of integrable Borel probability measures on the Cartan–Hadamard Riemannian manifold of positive definite matrices. They retain most of the properties of the Cartan barycenter and lead to the conclusion that there are infinitely many distinct contractive barycentric maps. Inequalities from the derived geometric means including the Yamazaki inequality and unitarily invariant norm inequalities are presented.

Geometric mean flows and the Cartan barycenter on the Wasserstein space over positive definite matrices

We introduce a class of flows on the Wasserstein space of probability measures with finite first moment on the Cartan–Hadamard Riemannian manifold of positive definite matrices, and consider the problem of differentiability of the corresponding Cartan barycentric trajectory. As a consequence we have a version of Lie–Trotter formula and a related unitarily invariant norm inequality. Furthermore, a fixed point theorem related to the Karcher equation and the Cartan barycentric trajectory is also presented as an application.

Log-majorization and Lie–Trotter formula for the Cartan barycenter on probability measure spaces

We extend Ando–Hiai's log-majorization for the weighted geometric mean of positive definite matrices into that for the Cartan barycenter in the general setting of probability measures on the Riemannian manifold of positive definite matrices equipped with trace metric. The main key is the settlement of the monotonicity problem of the Cartan barycenteric map on the space of probability measures with finite first moment for the stochastic order induced by the cone. We also derive a version of Lie–Trotter formula and related unitarily invariant norm inequalities for the Cartan barycenter as the main application of log-majorization.

Structure Tensor Riemannian Statistical Models for CBIR and Classification of Remote Sensing Images

This paper deals with parametric techniques for the description of texture on very high resolution (VHR) remote sensing images. These techniques focus on the property of anisotropy as described by the local structure tensor (LST). The novelty of this paper consists in proposing several comprehensive statistical frameworks to handle LST fields for rotation-invariant texture discrimination tasks. These frameworks are all based on probability models defined on the Riemannian manifold of positive definite matrices: a recent Riemannian Gaussian model on the affine-invariant metric space and a multivariate Gaussian distribution on the Log-Euclidean space. A thorough comparison of the proposed methods is performed with respect to some state-of-the-art texture analysis methods. Three experimental protocols are considered based on VHR remote sensing data. The first one consists of a content-based image retrieval (CBIR) protocol for browsing oyster field patches. The second one concerns a supervised classification protocol for grouping maritime pine forest stands in different age classes. The third one is, again, a CBIR protocol performed on the UC Merced land use/land cover patch collection. Tensor-based approaches show similar or even better results than the state-of-the-art texture analysis methods considered for comparison in all the experimental contexts.

Restoration of manifold-valued images by half-quadratic minimization

The paper addresses the generalization of the half-quadratic minimization method for the restoration of images having values in a complete Riemannian manifold. We recall the half-quadratic minimization method using the notation of the c-transform and adapt the algorithm to our special variational setting. We prove the convergence of the method for Hadamard spaces. Extensive numerical examples for images with values on spheres, in the rotation group SO(3) and in the manifold of positive definite matrices demonstrate the excellent performance of the algorithm. In particular, the method with SO(3)-valued data shows promising results for the restoration of images obtained from Electron Backscattered Diffraction which are of interest in material science.

A fixed point mean approximation to the Cartan barycenter of positive definite matrices

In this paper we define a new weighted inductive geometric mean Hn on the Riemannian manifold of positive definite matrices of fixed size and present its fixed point mean approximation to the Cartan barycenter (Karcher mean). We show that for t∈(0,1], the weighted geometric mean equation X=Hn+1(ωt;A1,…,An,X), where ωt=t1+t(w1,…,wn,1/t), has a unique positive definite solution that approaches as t→0+ to the ω-weighted Cartan mean of A1,…,An. Numerical computations for the stochastic convergence in terms of Hn to derive a new contractive barycenter other than the Cartan barycenter are presented.

Conic Geometric Optimization on the Manifold of Positive Definite Matrices

We develop \emph{geometric optimisation} on the manifold of Hermitian positive definite (HPD) matrices. In particular, we consider optimising two types of cost functions: (i) geodesically convex (g-convex); and (ii) log-nonexpansive (LN). G-convex functions are nonconvex in the usual euclidean sense, but convex along the manifold and thus allow global optimisation. LN functions may fail to be even g-convex, but still remain globally optimisable due to their special structure. We develop theoretical tools to recognise and generate g-convex functions as well as cone theoretic fixed-point optimisation algorithms. We illustrate our techniques by applying them to maximum-likelihood parameter estimation for elliptically contoured distributions (a rich class that substantially generalises the multivariate normal distribution). We compare our fixed-point algorithms with sophisticated manifold optimisation methods and obtain notable speedups.

Information Geometry of Positive Measures and Positive-Definite Matrices: Decomposable Dually Flat Structure

Information geometry studies the dually flat structure of a manifold, highlighted by the generalized Pythagorean theorem. The present paper studies a class of Bregman divergences called the (ρ,τ)-divergence. A (ρ,τ) -divergence generates a dually flat structure in the manifold of positive measures, as well as in the manifold of positive-definite matrices. The class is composed of decomposable divergences, which are written as a sum of componentwise divergences. Conversely, a decomposable dually flat divergence is shown to be a (ρ,τ) -divergence. A (ρ,τ) -divergence is determined from two monotone scalar functions, ρ and τ. The class includes the KL-divergence, α-, β- and (α, β)-divergences as special cases. The transformation between an affine parameter and its dual is easily calculated in the case of a decomposable divergence. Therefore, such a divergence is useful for obtaining the center for a cluster of points, which will be applied to classification and information retrieval in vision. For the manifold of positive-definite matrices, in addition to the dually flatness and decomposability, we require the invariance under linear transformations, in particular under orthogonal transformations. This opens a way to define a new class of divergences, called the (ρ,τ) -structure in the manifold of positive-definite matrices.

Weighted deterministic walks for the least squares mean on Hadamard spaces

We show that the Karcher mean of n points in any Hadamard space can be approximated by a natural explicitly constructed sequence. In the special case when the Hadamard space is the Riemannian manifold of positive definite matrices, this has been recently proved by J. Holbrook. A general version, in which the n points are assigned different weights, is established.

NO DICE THEOREM ON SYMMETRIC CONES

The monotonicity of the least squares mean on the Riemannian manifold of positive definite matrices, conjectured by Bhatia and Holbrook and one of key axiomatic properties of matrix geometric means, was recently established based on the Strong Law of Large Number \cite{LL1, BK}. A natural question concerned with the S.L.L.N is so called {\it the no dice conjecture}. It is a problem to make a construction of deterministic sequences converging to the least squares mean without any probabilistic arguments. Very recently, Holbrook \cite{Hol} gave an affirmative answer to the conjecture in the space of positive definite matrices. In this paper, inspired by the work of Holbrook \cite{Hol} and the fact that the convex cone of positive definite matrices is a typical example of a symmetric cone (self-dual homogeneous convex cone), we establish the no dice theorem on general symmetric cones.