Symmetric Positive Definite Matrices Research Articles

Entries of the inverses of large positive definite Toeplitz matrices

AbstractThis is an expository paper embarking on the asymptotic behavior of the entries of the inverses of positive definite symmetric Toeplitz matrices as the matrix dimension goes to infinity. We consider the behavior of the entries in neighborhoods of the four corners as well as the density of the distribution of the entries over all of the inverse matrix.

Pipelined Preconditioned Conjugate Gradient Methods for real and complex linear systems for distributed memory architectures

Preconditioned Conjugate Gradient (PCG) is a popular method for solving large and sparse linear systems of equations. The performance of PCG at scale is affected due to the costly global synchronization steps that arise in dot-products on distributed memory systems. Pipelined PCG (PIPECG) removes the costly global synchronization steps from PCG by only executing a single non-blocking allreduce per iteration and overlapping it with independent computations.In our previous work, we have developed a novel pipelined PCG algorithm called PIPECG-OATI (One Allreduce per Two Iterations) for real linear systems which executes a single non-blocking allreduce per two iterations and provides a large overlap of global communication with independent computations at higher number of cores. Our method achieves this overlap by using iteration combination and by introducing new recurrence and non-recurrence computations. We implement optimizations in the PIPECG-OATI method to use cache memory efficiently.In this work, we present PIPECG-OATI-c method for linear systems with complex Hermitian positive definite and complex symmetric matrices. We compare our method with various pipelined CG methods on a variety of problems and demonstrate that our method always gives the least run times. We performed experiments with our method using 20M and 30M unknowns on up to 16K cores and obtained up to 2.48X performance improvement over PCG and 2.14X performance improvement over PIPECG methods. We also experimented with up to 1-billion unknowns on 16K cores, the largest problem size explored for the CG problem, to our knowledge, and obtained about 25% improvement over PCG.

A new type of CGO solutions and its applications in corner scattering

We consider corner scattering for the operator ∇ ⋅ γ(x)∇ + k 2 ρ(x) in , with γ a positive definite symmetric matrix and ρ a positive scalar function. A corner is referred to one that is on the boundary of the (compact) support of γ(x) − I or ρ(x) − 1, where I stands for the identity matrix. We assume that γ is a scalar function in a small neighborhood of the corner. We show that any admissible incident field will be scattered by such corners, which are allowed to be concave. Moreover, we provide a brief discussion on the existence of non-scattering waves when γ − I has a jump across the corner. In order to prove the results, we construct a new type of complex geometric optics solutions.

A Non-Iterative Method for the Difference of Means on the Lie Group of Symmetric Positive-Definite Matrices

A non-iterative method for the difference of means is presented to calculate the log-Euclidean distance between a symmetric positive-definite matrix and the mean matrix on the Lie group of symmetric positive-definite matrices. Although affine-invariant Riemannian metrics have a perfect theoretical framework and avoid the drawbacks of the Euclidean inner product, their complex formulas also lead to sophisticated and time-consuming algorithms. To make up for this limitation, log-Euclidean metrics with simpler formulas and faster calculations are employed in this manuscript. Our new approach is to transform a symmetric positive-definite matrix into a symmetric matrix via logarithmic maps, and then to transform the results back to the Lie group through exponential maps. Moreover, the present method does not need to compute the mean matrix and retains the usual Euclidean operations in the domain of matrix logarithms. In addition, for some randomly generated positive-definite matrices, the method is compared using experiments with that induced by the classical affine-invariant Riemannian metric. Finally, our proposed method is applied to denoise the point clouds with high density noise via the K-means clustering algorithm.

Seismic Data Denoising With Correlation Feature Optimization Via S-Mean

Random noise elimination acts as an important role in seismic data processing. Moreover, protecting and recovering useful subsurface structure information is also significant. In this study, the S-mean which can obtain the geometric mean of the seismic traces on the symmetric positive definite matrix (SPD) manifold is adopted as a nonlinear filter for seismic denoising. Furthermore, S-mean has the best correlation with other elements based on the S-divergence due to the optimization of finding S-mean on SPD manifold. Therefore, the broken correlation features in noisy seismic data are compensated and maintained well, which can be conducive to describe the subsurface structures. Synthetic example and field data applications qualitatively and quantitatively demonstrate the validity and effectiveness of the proposed workflow.

Epileptic Seizure Detection in EEG Signals Using Discriminative Stein Kernel-Based Sparse Representation

The automatic seizure detection in electroencephalogram (EEG) signals is crucial for the monitoring, diagnosis, and treatment of epilepsy. In this study, an intelligent detection framework with the discriminative Stein kernel-based sparse representation (DSK-SR) is constructed to distinguish epileptic EEG signals. Specifically, in the scheme of DSK-SR, EEG samples are presented by symmetric positive definite (SPD) matrices in the form of covariance descriptors (CovDs). Taking into account the non-Euclidean geometry of the Riemannian manifold of SPD matrices, the traditional SR in Euclidean space cannot be applied in its original form on the manifold. To this end, the DSK defined on the manifold can permit us to embed the manifold into a high-dimensional reproducing kernel Hilbert space (RKHS) to perform SR. Then, test samples are sparsely coded over the training sets, and the classification decision is performed by assessing which class generates the minimal reconstructed residuals. Eventually, competitive experimental results on three widely recognized EEG datasets against the state-of-the-art methods demonstrate the efficacy of the proposed DSK-SR in identifying epileptic EEG signals, indicating its powerful application potential in the automatic seizure detection.

Maximum margin Riemannian manifold-based hyperdisk for fault diagnosis of roller bearing with multi-channel fusion covariance matrix

For rotating machinery, the sudden failure of roller bearing would lead to the downtime of the whole system and even catastrophic accidents. Therefore, multiple accelerometers are usually arranged to comprehensively evaluate the health of roller bearing, enhancing the stability and reliability of monitoring results. This paper proposes a novel fault diagnosis framework by utilizing a multi-channel fusion covariance matrix (MFCM) and Riemannian manifold-based hyperdisk. First, 22 statistical features are acquired from each channel data. Then, MFCM is calculated as the fault feature representation of roller bearing to achieve multi-channel feature fusion, where the element of MFCM represents the correlation information between different channels. Finally, since MFCM is a symmetric positive definite (SPD) matrix, lying on a Riemannian manifold, we design a maximum margin Riemannian manifold-based hyperdisk (MMRMHD) classifier to conduct fault classification, where Log-Euclidean metric (LEM) is introduced to calibrate the distribution of MFCMs. Moreover, to further improve the classification ability of nonlinear SPD data, we map MFCMs into a high-dimensional Hilbert space with the LEM-based kernel function and construct a novel kernelized MMRMHD model. The experimental results on two bearing datasets with multi-channel vibration signals demonstrate the effectiveness and superiority of the proposed fault diagnosis framework.

Computationally Efficient Approximations for Matrix-Based Rényi's Entropy

The recently developed matrix based Renyi's entropy enables measurement of information in data simply using the eigenspectrum of symmetric positive semi definite (PSD) matrices in reproducing kernel Hilbert space, without estimation of the underlying data distribution. This intriguing property makes the new information measurement widely adopted in multiple statistical inference and learning tasks. However, the computation of such quantity involves the trace operator on a PSD matrix $G$ to power $\alpha$(i.e., $tr(G^\alpha)$), with a normal complexity of nearly $O(n^3)$, which severely hampers its practical usage when the number of samples (i.e., $n$) is large. In this work, we present computationally efficient approximations to this new entropy functional that can reduce its complexity to even significantly less than $O(n^2)$. To this end, we leverage the recent progress on Randomized Numerical Linear Algebra, developing Taylor, Chebyshev and Lanczos approximations to $tr(G^\alpha)$ for arbitrary values of $\alpha$ by converting it into matrix-vector multiplications problem. We also establish the connection between the matrix-based Renyi's entropy and PSD matrix approximation, which enables exploiting both clustering and block low-rank structure of $G$ to further reduce the computational cost. We theoretically provide approximation accuracy guarantees and illustrate the properties of different approximations. Large-scale experimental evaluations on both synthetic and real-world data corroborate our theoretical findings, showing promising speedup with negligible loss in accuracy.

Multi-Ellipsoidal Extended Target Tracking With Variational Bayes Inference

In this work, we propose a novel extended target tracking algorithm, which is capable of representing a target or a group of targets with multiple ellipses. Each ellipse is modeled by an unknown symmetric positive-definite random matrix. The proposed model requires solving two challenging problems. First, the data association problem between the measurements and the sub-objects. Second, the inference problem that involves non-conjugate priors and likelihoods which needs to be solved within the recursive filtering framework. We utilize the variational Bayes inference method to solve the association problem and to approximate the intractable true posterior. The performance of the proposed solution is demonstrated in simulations and real-data experiments. The results show that our method outperforms the state-of-the-art methods in terms of accuracy with lower computational complexity.

From symplectic eigenvalues of positive definite matrices to their pseudo-orthogonal eigenvalues

Williamson's theorem states that every real symmetric positive definite matrix $A$ of even order can be brought to diagonal form via a symplectic $T$-congruence transformation. The diagonal entries of the resulting diagonal form are called the symplectic eigenvalues of $A$. We point at an analog of this classical result related to Hermitian positive definite matrices, *-congruences, and another class of transformation matrices, namely, pseudo-unitary matrices. This leads to the concept of pseudo-unitary (or pseudo-orthogonal, in the real case) eigenvalues of positive definite matrices.

MPI+OpenMP реализация метода сопряженных градиентов с предобусловливателем IC(0) на основе использования переупорядочения узлов сетки

The paper proposes methods for using MPI and MPI+OpenMP technologies to construct and invert the preconditioner of an incomplete triangular Cholesky decomposition without filling IC(0) for solving systems of linear algebraic equations with an arbitrary symmetric positive definite matrix. Methods of using MPI and MPI+OpenMP technologies are based on the use of grid node orderings consistent with the division of the calculation area. The use of OpenMP technology in the construction and inversion of the preconditioner is carried out for most rows of the matrix. Comparative timing results for the MPI+OpenMP and MPI implementations of the proposed preconditioning used with the conjugate gradient method for a model problems and the sparse matrix collections SuiteSparse are presented.

Superpixel-Based Brownian Descriptor for Hyperspectral Image Classification

Exploring effective spectral–spatial feature extraction methods is one of the most focused problems in current hyperspectral image (HSI) classification research. However, complex spectral–spatial structure characteristics in HSIs, e.g., shape-variable spatial structure and nonlinear spectral structure, are difficult to be effectively extracted and jointly represented. To overcome this issue, a novel superpixel-based Brownian descriptor (SBD) method for HSI classification is proposed in this article. In specific, superpixel segmentation is first used to extract shape-adaptive spatial structure information from dimension-reduced HSI, leading to generate nonoverlapping homogeneous 3-D image blocks. Then, similar pixels within the 3-D image block are jointly represented by a new local spectral–spatial feature based on the Brownian descriptor (BD). This is the first time that the BD is introduced to measure both linear and nonlinear correlations among different spectral bands in HSI. On one hand, the integration of superpixel and BD helps to provide much richer and more valuable information for better discrimination between different categories. On the other hand, the SBD can effectively represent the internal structure characters within each 3-D image block of different spatial shapes by a symmetric positive definite matrix of a united form. Finally, considering that the SBD lies on the Riemannian manifold space, a log-Euclidean kernel sparse representation (LKSR) classifier is introduced to obtain the classification results. Experimental results on three widely used real hyperspectral datasets indicate the performance superiority of the proposed SBD method over several state-of-the-art techniques.

Geometries on the cone of positive-definite matrices derived from the power potential and their relation to the power means

We study a Riemannian metric on the cone of symmetric positive-definite matrices obtained from the Hessian of the power potential function (1−det⁡(X)β)/β. We give explicit expressions for the geodesics and distance function, under suitable conditions. In the scalar case, the geodesic between two positive numbers coincides with a weighted power mean, while for matrices of size at least two it yields a notion of weighted power mean different from the ones given in the literature. As β tends to zero, the power potential converges to the logarithmic potential, that yields a well-known metric associated with the matrix geometric mean; we show that the geodesic and the distance associated with the power potential converge to the weighted matrix geometric mean and the distance associated with the logarithmic potential, respectively.

Enhanced relaxed physical factorization preconditioner for coupled poromechanics

The relaxed physical factorization (RPF) preconditioner is a recent algorithm allowing for the efficient and robust solution to the block linear systems arising from the three-field displacement-velocity-pressure formulation of coupled poromechanics. For its application, however, it is necessary to invert blocks with the algebraic form Cˆ=(C+βFFT), where C is a symmetric positive definite matrix, FFT a rank-deficient term, and β a real non-negative coefficient. The inversion of Cˆ, performed in an inexact way, can become unstable for large values of β, as it usually occurs at some stages of a full poromechanical simulation. In this work, we propose a family of algebraic techniques to stabilize the inexact solve with Cˆ. This strategy can prove useful in other problems as well where such an issue might arise, such as augmented Lagrangian preconditioning techniques for Navier-Stokes or incompressible elasticity. First, we introduce an iterative scheme obtained by a natural splitting of matrix Cˆ. Second, we develop a technique based on the use of a proper projection operator annihilating the near-kernel modes of Cˆ. Both approaches give rise to a novel class of preconditioners denoted as Enhanced RPF (ERPF). Effectiveness and robustness of the proposed algorithms are demonstrated in both theoretical benchmarks and real-world large-size applications, outperforming the native RPF preconditioner.

Total variation regularized Fréchet regression for metric-space valued data

Non-Euclidean data that are indexed with a scalar predictor such as time are increasingly encountered in data applications, while statistical methodology and theory for such random objects are not well developed yet. To address the need for new methodology in this area, we develop a total variation regularization technique for nonparametric Fréchet regression, which refers to a regression setting where a response residing in a metric space is paired with a scalar predictor and the target is a conditional Fréchet mean. Specifically, we seek to approximate an unknown metric-space valued function by an estimator that minimizes the Fréchet version of least squares and at the same time has small total variation, appropriately defined for metric-space valued objects. We show that the resulting estimator is representable by a piece-wise constant function and establish the minimax convergence rate of the proposed estimator for metric data objects that reside in Hadamard spaces. We illustrate the numerical performance of the proposed method for both simulated and real data, including metric spaces of symmetric positive-definite matrices with the affine-invariant distance, of probability distributions on the real line with the Wasserstein distance, and of phylogenetic trees with the Billera–Holmes–Vogtmann metric.

Alpha Procrustes metrics between positive definite operators: A unifying formulation for the Bures-Wasserstein and Log-Euclidean/Log-Hilbert-Schmidt metrics

This work presents a parametrized family of distances, namely the Alpha Procrustes distances, on the set of symmetric, positive definite (SPD) matrices. The Alpha Procrustes distances provide a unified formulation encompassing both the Bures-Wasserstein and Log-Euclidean distances between SPD matrices. We show that the Alpha Procrustes distances are the Riemannian distances corresponding to a family of Riemannian metrics on the manifold of SPD matrices, which encompass both the Log-Euclidean and Wasserstein Riemannian metrics. This formulation is then generalized to the set of positive definite Hilbert-Schmidt operators on a Hilbert space, unifying the infinite-dimensional Bures-Wasserstein and Log-Hilbert-Schmidt distances. In the setting of reproducing kernel Hilbert spaces (RKHS) covariance operators, we obtain closed form formulas for all the distances via the corresponding kernel Gram matrices. From a statistical viewpoint, the Alpha Procrustes distances give rise to a parametrized family of distances between Gaussian measures on Euclidean space, in the finite-dimensional case, and separable Hilbert spaces, in the infinite-dimensional case, encompassing the 2-Wasserstein distance, with closed form formulas via Gram matrices in the RKHS setting. The presented formulations are new both in the finite and infinite-dimensional settings.

Predicting cognitive scores with graph neural networks through sample selection learning

Analyzing the relation between intelligence and neural activity is of the utmost importance in understanding the working principles of the human brain in health and disease. In existing literature, functional brain connectomes have been used successfully to predict cognitive measures such as intelligence quotient (IQ) scores in both healthy and disordered cohorts using machine learning models. However, existing methods resort to flattening the brain connectome (i.e., graph) through vectorization which overlooks its topological properties. To address this limitation and inspired from the emerging graph neural networks (GNNs), we design a novel regression GNN model (namely RegGNN) for predicting IQ scores from brain connectivity. On top of that, we introduce a novel, fully modular sample selection method to select the best samples to learn from for our target prediction task. However, since such deep learning architectures are computationally expensive to train, we further propose a learning-based sample selection method that learns how to choose the training samples with the highest expected predictive power on unseen samples. For this, we capitalize on the fact that connectomes (i.e., their adjacency matrices) lie in the symmetric positive definite (SPD) matrix cone. Our results on full-scale and verbal IQ prediction outperforms comparison methods in autism spectrum disorder cohorts and achieves a competitive performance for neurotypical subjects using 3-fold cross-validation. Furthermore, we show that our sample selection approach generalizes to other learning-based methods, which shows its usefulness beyond our GNN architecture.

Riemannian Gaussian distributions, random matrix ensembles and diffusion kernels

We show that the Riemannian Gaussian distributions on symmetric spaces, introduced in recent years, are of standard random matrix type. We exploit this to compute analytically marginals of the probability density functions. This can be done fully, using Stieltjes-Wigert orthogonal polynomials, for the case of the space of Hermitian matrices, where the distributions have already appeared in the physics literature. For the case when the symmetric space is the space of m×m symmetric positive definite matrices, we show how to efficiently compute densities of eigenvalues by evaluating Pfaffians at specific values of m. Equivalently, we can obtain the same result by constructing specific skew orthogonal polynomials with regards to the log-normal weight function (skew Stieltjes-Wigert polynomials). Other symmetric spaces are studied and the same type of result is obtained for the quaternionic case. Moreover, we show how the probability density functions are a particular case of diffusion reproducing kernels of the Karlin-McGregor type, describing non-intersecting Brownian motions, which are also diffusion processes in the Weyl chamber of Lie groups.

Hilbert–Schmidt Independence Criterion Regularization Kernel Framework on Symmetric Positive Definite Manifolds

Image recognition tasks involve an increasingly high amount of symmetric positive definite (SPD) matrices data. SPD manifolds exhibit nonlinear geometry, and Euclidean machine learning methods cannot be directly applied to SPD manifolds. The kernel trick of SPD manifolds is based on the concept of projecting data onto a reproducing kernel Hilbert space. Unfortunately, existing kernel methods do not consider the connection of SPD matrices and linear projections. Thus, a framework that uses the correlation between SPD matrices and projections to model the kernel map is proposed herein. To realize this, this paper formulates a Hilbert–Schmidt independence criterion (HSIC) regularization framework based on the kernel trick, where HSIC is usually used to express the interconnectedness of two datasets. The proposed framework allows us to extend the existing kernel methods to new HSIC regularization kernel methods. Additionally, this paper proposes an algorithm called HSIC regularized graph discriminant analysis (HRGDA) for SPD manifolds based on the HSIC regularization framework. The proposed HSIC regularization framework and HRGDA are highly accurate and valid based on experimental results on several classification tasks.

An efficient damped Newton-type algorithm with globalization strategy on Riemannian manifolds

We propose a new globalization strategy of the damped Newton method for finding singularities of a vector field on Riemannian manifolds. We establish its global convergence with a superlinear rate. In particular, this globalization generalizes the known damped Newton’s method for general retraction. The global convergence analysis presented here does not require any hypothesis regarding a singularity of the vector field. We applied the proposed method to solve the truncated singular value problem on the product of two Stiefel manifolds, the dextrous hand grasping problem and an academic problem on the cone of symmetric positive definite matrices, the Rayleigh quotient and a problem of finding zeros of a non-conservative vector field on the sphere. Numerical experiments are presented, showing that the proposed algorithm is more robust than the known damped Newton’s method.