Symmetric Positive Definite Matrices Research Articles

Numerical algorithm for estimating a conditioned symmetric positive definite matrix under constraints

SummaryWe present RCO (regularized Cholesky optimization): a numerical algorithm for finding a symmetric positive definite (PD) matrix with a bounded condition number that minimizes an objective function. This task arises when estimating a covariance matrix from noisy data or due to model constraints, which can cause spurious small negative eigenvalues. A special case is the problem of finding the nearest well‐conditioned PD matrix to a given matrix. RCO explicitly optimizes the entries of the Cholesky factor. This requires solving a regularized non‐linear, non‐convex optimization problem, for which we apply Newton‐CG and exploit the Hessian's sparsity. The regularization parameter is determined via numerical continuation with an accuracy‐conditioning trade‐off criterion. We apply RCO to our motivating educational measurement application of estimating the covariance matrix of an empirical best linear prediction (EBLP) of school growth scores. We present numerical results for two empirical datasets, state and urban. RCO outperforms general‐purpose near‐PD algorithms, obtaining ‐smaller matrix reconstruction bias and smaller EBLP estimator mean‐squared error. It is in fact the only algorithm that solves the right minimization problem, which strikes a balance between the objective function and the condition number. RCO can be similarly applied to the stable estimation of other posterior means. For the task of finding the nearest PD matrix, RCO yields similar condition numbers to near‐PD methods, but provides a better overall near‐null space.

A system-level interface sampling and reduction method for component mode synthesis with varying parameters

This paper presents a new interface sampling and reduction method for varying parameters within the framework of the component mode synthesis (CMS) method. In the conventional interpolation-based parametric reduced-order model (IB-PROM) based on CMS, reduced subsystems are synthesized, and interface degrees-of-freedom are reduced via the secondary eigenvalue analysis in the online stage, which is essential to obtain highly reduced system matrices. However, it inhibits rapid parameter updating and requires extra computational resources in the online stage. In the present study, the pre-reduced interface sampling method is proposed by synthesizing substructures and reducing the interface before beginning the online stage. The interface solution is obtained through offline and online adaptation, likewise the procedure of computing the interior solution of previous studies. Additionally, congruence transformations are applied to each reduced interface, which is mandatory for interpolating symmetric positive definite matrices with respect to a new input parameter. As a result, the online performance of IB-PROM is greatly enhanced, showing rapid and near real-time adaptations to the variations in system parameters.

Modeling Tree-like Heterophily on Symmetric Matrix Manifolds

Tree-like structures, characterized by hierarchical relationships and power-law distributions, are prevalent in a multitude of real-world networks, ranging from social networks to citation networks and protein–protein interaction networks. Recently, there has been significant interest in utilizing hyperbolic space to model these structures, owing to its capability to represent them with diminished distortions compared to flat Euclidean space. However, real-world networks often display a blend of flat, tree-like, and circular substructures, resulting in heterophily. To address this diversity of substructures, this study aims to investigate the reconstruction of graph neural networks on the symmetric manifold, which offers a comprehensive geometric space for more effective modeling of tree-like heterophily. To achieve this objective, we propose a graph convolutional neural network operating on the symmetric positive-definite matrix manifold, leveraging Riemannian metrics to facilitate the scheme of information propagation. Extensive experiments conducted on semi-supervised node classification tasks validate the superiority of the proposed approach, demonstrating that it outperforms comparative models based on Euclidean and hyperbolic geometries.

Imitating via manipulability: Geometry-aware combined DMP with via-point and speed adaptation

Manipulability ellipsoid on the Riemannian manifold provides an effective criterion for guiding the regulation of robot postures in an efficient and natural manner. While many manipulability-based learning by demonstration (LbD) methods achieve favorable performance in human-like manipulation, the adaptation of movement skills encapsulated in Symmetric Positive Definite (SPD) matrices and the implementation of manipulability-relevant skills related to velocity scaling are still largely open. In this paper, we develop a new framework based on Geometry-Aware Combined Dynamic Movement Primitives (GA-CDMP) for learning movement skills from demonstrations. The GA-CDMP model not only adapts learned trajectories to SPD-via-points but also establishes a correlation between SPD-based trajectories and position trajectories, enabling the adaptation of learned SPD-based trajectories across different non-linear motion velocity scales. The experimental evaluations show that our proposed method exhibits remarkable precision in passing through via-points positioned at a distance from the demonstration, while also substantially improving the invariant shape similarity in manipulability by approximately 65% compared to extended Kernelized Movement Primitives (KMPs). Moreover, our velocity adaptive approach achieves impressive success rates of up to 94% and demonstrates a notable reduction in execution time (11.72 ± 0.45 s), representing an almost 10% decrease in comparison to the state-of-the-art method employed in water-carrying tasks.

Foundations of the wald space for phylogenetic trees

AbstractEvolutionary relationships between species are represented by phylogenetic trees, but these relationships are subject to uncertainty due to the random nature of evolution. A geometry for the space of phylogenetic trees is necessary in order to properly quantify this uncertainty during the statistical analysis of collections of possible evolutionary trees inferred from biological data. Recently, the wald space has been introduced: a length space for trees which is a certain subset of the manifold of symmetric positive definite matrices. In this work, the wald space is introduced formally and its topology and structure is studied in detail. In particular, we show that wald space has the topology of a disjoint union of open cubes, it is contractible, and by careful characterisation of cube boundaries, we demonstrate that wald space is a Whitney stratified space of type (A). Imposing the metric induced by the affine invariant metric on symmetric positive definite matrices, we prove that wald space is a geodesic Riemann stratified space. A new numerical method is proposed and investigated for construction of geodesics, computation of Fréchet means and calculation of curvature in wald space. This work is intended to serve as a mathematical foundation for further geometric and statistical research on this space.

A novel two-sample test within the space of symmetric positive definite matrix distributions and its application in finance

A novel two-sample test within the space of symmetric positive definite matrix distributions and its application in finance

Log-Euclidean metric based covariance propagation on SPD manifold for continuous–discrete extended Kalman filtering

For nonlinear systems with continuous dynamic and discrete measurements, a Log-Euclidean metric (LEM) based novel scheme is proposed to refine the covariance integration steps of continuous–discrete Extended Kalman filter (CDEKF). In CDEKF, the covariance differential equation is usually integrated with regular Euclidean matrix operations, which actually ignores the Riemannian structure of underlying space and poses a limit on the further improvement of estimation accuracy. To overcome this drawback, this work proposes to define the covariance variable on the manifold of symmetric positive definite (SPD) matrices and propagate it using the Log-Euclidean metric. To embed the LEM based novel propagation scheme, the manifold integration of the covariance for LEMCDEKF is proposed together with the details of efficient realization, which can integrate the covariance on SPD manifold and avoid the drawback of Euclidean scheme. Numerical simulations certify the new method’s superior accuracy than conventional methods.

Ricci flow-based brain surface covariance descriptors for diagnosing Alzheimer’s disease

Automated feature extraction from MRI brain scans and diagnosis of Alzheimer’s disease are ongoing challenges. With advances in 3D imaging technology, 3D data acquisition is becoming more viable and efficient than its 2D counterpart. Rather than using feature-based vectors, in this paper, for the first time, we suggest a pipeline to extract novel covariance-based descriptors from the cortical surface using the Ricci energy optimization. The covariance descriptors are components of the nonlinear manifold of symmetric positive-definite matrices, thus we focus on using the Gaussian radial basis function to apply manifold-based classification to the 3D shape problem. Applying this novel signature to the analysis of abnormal cortical brain morphometry allows for diagnosing Alzheimer’s disease. Experimental studies performed on about two hundred 3D MRI brain models, gathered from Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset demonstrate the effectiveness of our descriptors in achieving remarkable classification accuracy.

SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space

Symmetric positive definite(SPD) matrices have shown important value and applications in statistics and machine learning, such as FMRI analysis and traffic prediction. Previous works on SPD matrices mostly focus on discriminative models, where predictions are made directly on E(X|y), where y is a vector and X is an SPD matrix. However, these methods are challenging to handle for large-scale data. In this paper, inspired by denoising diffusion probabilistic model(DDPM), we propose a novel generative model, termed SPD-DDPM, by introducing Gaussian distribution in the SPD space to estimate E(X|y). Moreover, our model can estimate p(X) unconditionally and flexibly without giving y. On the one hand, the model conditionally learns p(X|y) and utilizes the mean of samples to obtain E(X|y) as a prediction. On the other hand, the model unconditionally learns the probability distribution of the data p(X) and generates samples that conform to this distribution. Furthermore, we propose a new SPD net which is much deeper than the previous networks and allows for the inclusion of conditional factors. Experiment results on toy data and real taxi data demonstrate that our models effectively fit the data distribution both unconditionally and conditionally.

Task-oriented analysis and visualization of correlation patterns in multi-sensor time series

Time series data are usually collected through instruments equipped with sensors. For multi-sensor time series (MSTS), it is crucial to identify the factors that affect tasks such as classification and regression. To better understand the mechanisms affecting the task, we extracted the correlation patterns between sensors and represented them as symmetric positive definite matrices. The correlation patterns were then transformed into vector form to serve as input features for regression or classification models, and the models were trained for each sensor. Finally, we leveraged the interpretability of the models to analyze and visualize the correlation patterns at both micro and macro scales. By integrating the explanatory power of the models with correlation patterns, we could interpret the task in terms of time and space, providing valuable insights for exploring the underlying data rules. We evaluated our proposed method using both synthetic and real data, and the simulation results confirmed its effectiveness.

Three-stage transfer learning for motor imagery EEG recognition.

Motor imagery (MI) paradigms have been widely used in neural rehabilitation and drowsiness state assessment. The progress in brain-computer interface (BCI) technology has emphasized the importance of accurately and efficiently detecting motor imagery intentions from electroencephalogram (EEG). Despite the recent breakthroughs made in developing EEG-based algorithms for decoding MI, the accuracy and efficiency of these models remain limited by technical challenges posed by cross-subject heterogeneity in EEG data processing and the scarcity of EEG data for training. Inspired by the optimal transport theory, this study aims to develop a novel three-stage transfer learning (TSTL) method, which uses the existing labeled data from a source domain to improve classification performance on an unlabeled target domain. Notably, the proposed method comprises three components, namely, the Riemannian tangent space mapping (RTSM), source domain transformer (SDT), and optimal subspace mapping (OSM). The RTSM maps a symmetric positive definite matrix from the Riemannian space to the tangent space to minimize the marginal probability distribution drift. The SDT transforms the source domain to a target domain by finding the optimal transport mapping matrix to reduce the joint probability distribution differences. The OSM finally maps the transformed source domain and original target domain to the same subspace to further mitigate the distribution discrepancy. The performance of the proposed method was validated on two public BCI datasets, and the average accuracy of the algorithm on two datasets was 72.24% and 69.29%. Our results demonstrated the improved performance of EEG-based MI detection in comparison with state-of-the-art algorithms.

Statistical inference for wavelet curve estimators of symmetric positive definite matrices

In this paper we treat statistical inference for a wavelet estimator of curves of symmetric positive definite (SPD) using the log-Euclidean distance. This estimator preserves positive-definiteness and enjoys permutation-equivariance, which is particularly relevant for covariance matrices. Our second-generation wavelet estimator is based on average-interpolation (AI) and allows the same powerful properties, including fast algorithms, known from nonparametric curve estimation with wavelets in standard Euclidean set-ups. The core of our work is the proposition of confidence sets for our AI wavelet estimator in a non-Euclidean geometry. We derive asymptotic normality of this estimator, including explicit expressions of its asymptotic variance. This opens the door for constructing asymptotic confidence regions which we compare with our proposed bootstrap scheme for inference. Detailed numerical simulations confirm the appropriateness of our suggested inference schemes.

Deep Efficient Continuous Manifold Learning for Time Series Modeling.

Modeling non-euclidean data is drawing extensive attention along with the unprecedented successes of deep neural networks in diverse fields. Particularly, a symmetric positive definite matrix is being actively studied in computer vision, signal processing, and medical image analysis, due to its ability to learn beneficial statistical representations. However, owing to its rigid constraints, it remains challenging to optimization problems and inefficient computational costs, especially, when incorporating it with a deep learning framework. In this paper, we propose a framework to exploit a diffeomorphism mapping between Riemannian manifolds and a Cholesky space, by which it becomes feasible not only to efficiently solve optimization problems but also to greatly reduce computation costs. Further, for dynamic modeling of time-series data, we devise a continuous manifold learning method by systematically integrating a manifold ordinary differential equation and a gated recurrent neural network. It is worth noting that due to the nice parameterization of matrices in a Cholesky space, training our proposed network equipped with Riemannian geometric metrics is straightforward. We demonstrate through experiments over regular and irregular time-series datasets that our proposed model can be efficiently and reliably trained and outperforms existing manifold methods and state-of-the-art methods in various time-series tasks.

Refining DIIS algorithms for Si and GaAs solar cells: incorporation of weight regularization, conjugate gradient, and reverse automatic differentiation techniques.

Pivotal enhancements in electronic minimization algorithms, which are vital for the advancement of computational materials science, are introduced in this research. Our research is dedicated to refining the DIIS algorithm specifically for electronic structure calculations of silicon (Si) and gallium arsenide (GaAs) solar cells, aiming to enhance their efficiency and stability. We have enriched DIIS by integrating a weight regularization factor, significantly bolstering its convergence stability. This modification enhances iteration robustness and curtails the average iteration duration, thus streamlining the convergence process. Furthermore, we have incorporated the conjugate gradient (CG) algorithm to proficiently resolve symmetric positive definite residual matrices. This inclusion substantially accelerates the solution-finding process within the DIIS framework. A novel aspect of our research is the application of reverse automatic differentiation (AD), deployed in two distinct methodologies: the construction of the Jacobian matrix and direct chain rule application for gradient computation. These approaches involve sophisticated mathematical techniques that enhance computational precision and efficiency specifically for Si and GaAs solar cell materials in determining the optimal weights for residual combinations during DIIS iterations. The integration of these advanced methods into the DIIS algorithm not only augments its convergence stability but also ensures a substantial reduction in total computational time. Our findings demonstrate that the enhanced DIIS, CG-enhanced DIIS, and AD-integrated DIIS methods collectively lead to a more efficient electronic minimization process. This balance of stability and efficiency is crucial in high-performance computational materials science, particularly for complex systems analysis. The findings of this research represent a notable advancement in computational strategies for Si and GaAs solar cell materials, providing enhanced methodologies and insights that significantly improve the efficiency and stability of electronic structure calculations in these critical components of renewable energy technologies.

Classification of BCI-EEG Based on the Augmented Covariance Matrix.

Electroencephalography signals are recorded as multidimensional datasets. We propose a new framework based on the augmented covariance that stems from an autoregressive model to improve motor imagery classification. From the autoregressive model can be derived the Yule-Walker equations, which show the emergence of a symmetric positive definite matrix: the augmented covariance matrix. The state-of the art for classifying covariance matrices is based on Riemannian Geometry. A fairly natural idea is therefore to apply this Riemannian Geometry based approach to these augmented covariance matrices. The methodology for creating the augmented covariance matrix shows a natural connection with the delay embedding theorem proposed by Takens for dynamical systems. Such an embedding method is based on the knowledge of two parameters: the delay and the embedding dimension, respectively related to the lag and the order of the autoregressive model. This approach provides new methods to compute the hyper-parameters in addition to standard grid search. The augmented covariance matrix performed ACMs better than any state-of-the-art methods. We will test our approach on several datasets and several subjects using the MOABB framework, using both within-session and cross-session evaluation. The improvement in results is due to the fact that the augmented covariance matrix incorporates not only spatial but also temporal information. As such, it contains information on the nonlinear components of the signal through the embedding procedure, which allows the leveraging of dynamical systems algorithms. These results extend the concepts and the results of the Riemannian distance based classification algorithm.

Efficient estimates for matrix-inverse quadratic forms

In this paper we present two approaches for estimating matrix-inverse quadratic forms xTA−1x, where A is a symmetric positive definite matrix of order n, and x∈Rn. Using the first, analytic approach, we establish two families of estimates which are convenient for matrices with small condition number. Based on the second, heuristic approach, we derive two families of estimates which are suitable for matrices when vector x is close enough to an eigenvector. The low complexity and stability of the estimates is proved. Several numerical results illustrating the effectiveness of the methods are presented.

SPD Manifold Deep Metric Learning for Image Set Classification.

By characterizing each image set as a nonsingular covariance matrix on the symmetric positive definite (SPD) manifold, the approaches of visual content classification with image sets have made impressive progress. However, the key challenge of unhelpfully large intraclass variability and interclass similarity of representations remains open to date. Although, several recent studies have mitigated the two problems by jointly learning the embedding mapping and the similarity metric on the original SPD manifold, their inherent shallow and linear feature transformation mechanism are not powerful enough to capture useful geometric features, especially in complex scenarios. To this end, this article explores a novel approach, termed SPD manifold deep metric learning (SMDML), for image set classification. Specifically, SMDML first selects a prevailing SPD manifold neural network (SPDNet) as the backbone (encoder) to derive an SPD matrix nonlinear representation. To counteract the degradation of structural information during multistage feature embedding, we construct a Riemannian decoder at the end of the encoder, trained by a reconstruction error term (RT), to induce the generated low-dimensional feature manifold of the hidden layer to capture the pivotal information about the visual data describing the imaged scene. We demonstrate through theory and experiments that it is feasible to replace the Riemannian metric with Euclidean distance in RT. Then, the ReCov layer is introduced into the established Riemannian network to regularize the local statistical information within each input feature matrix, which enhances the effectiveness of the learning process. The theoretical analysis of the activation function used in the ReCov layer in terms of continuity and conditions for generating positive definite matrices is beneficial for network design. Inspired by the fact that the single cross-entropy loss used for training is unable to effectively parse the geometric distribution of the deep representations, we finally endow the suggested model with a novel metric learning regularization term. By explicitly incorporating the encoding and processing of the data variations into the network learning process, this term can not only derive a powerful Riemannian representation but also train an effective classifier. The experimental results show the superiority of the proposed approach on three typical visual classification tasks.

Manifolds.jl: An Extensible Julia Framework for Data Analysis on Manifolds

We present the Julia package Manifolds.jl , providing a fast and easy-to-use library of Riemannian manifolds and Lie groups. This package enables working with data defined on a Riemannian manifold, such as the circle, the sphere, symmetric positive definite matrices, or one of the models for hyperbolic spaces. We introduce a common interface, available in ManifoldsBase.jl , with which new manifolds, applications, and algorithms can be implemented. We demonstrate the utility of Manifolds.jl using Bézier splines, an optimization task on manifolds, and principal component analysis on nonlinear data. In a benchmark, Manifolds.jl outperforms all comparable packages for low-dimensional manifolds in speed; over Python and Matlab packages, the improvement is often several orders of magnitude, while over C/C++ packages, the improvement is two-fold. For high-dimensional manifolds, it outperforms all packages except for Tensorflow-Riemopt, which is specifically tailored for high-dimensional manifolds.

Signature methods for brain-computer interfaces

Brain-computer interfaces (BCIs) allow direct communication between one’s central nervous system and a computer without any muscle movement hence by-passing the peripheral nervous system. They can restore disabled people’s ability to interact with their environment, e.g. communication and wheelchair control. However, to this day their performance is still hindered by the non-stationarity of electroencephalography (EEG) signals, as well as their susceptibility to noise from the users’ environment and from their own physiological activity. Moreover, a non-negligible amount of users struggle to use BCI systems based on motor imagery. In this paper, a new method based on the path signature is introduced to tackle this problem by using features which are different from the usual power-based ones. The path signature is a series of iterated integrals computed from a multidimensional path. It is invariant under translation and time reparametrization, which makes it a robust feature for multichannel EEG time series. The performance can be further boosted by combining the path signature with the gold standard Riemannian classifier in the BCI field exploiting the geometric structure of symmetric positive definite (SPD) matrices. The results obtained on publicly available datasets show that the signature method is more robust to inter-user variability than classical ones, especially on noisy and low-quality data. Hence, this study paves the way towards the use of mathematical tools that until now have been neglected, in order to tackle the EEG-based BCI variability issue. It also sheds light on the lead-lag relationship captured by path signature which seems relevant to assess the underlying neural mechanisms.

Optimization Method of the Clamping Force for Large Cabin Parts

In order to realize the stable clamping of large cabin parts, this paper studied the clamping force optimization of the clamping mechanism for large-mass and large-size cabin parts. Firstly, three kinds of contact models are introduced. Then, a clamping matrix is constructed for a particular clamping configuration. The nonlinear friction cone constraint at the contact point is transformed into a linear affine constraint in a smooth Riemannian manifold using the special structure of the positive definite symmetric matrix. Finally, a large cabin part used in the aerospace field is used as an example for calculating and simulating the clamping force optimization. Different optimization algorithms are used to calculate the initial value which is put into the gradient flow optimization method of the clamping force for optimization calculation. Meanwhile, the normal clamping force value of a 2 t object is measured using relevant experimental equipment. The simulation results and experimental results show that the gradient flow optimization method of the clamping force can quickly complete the clamping force optimization of large-mass and large-size cabin parts. The actual measured value of the normal clamping force is close to the simulated convergence value. The distribution of the normal force of the clamping mechanisms and the convergence value of the clamping force for each clamping mechanism can provide some references for determining the output clamping force of the clamping mechanism and confirming a reasonable distribution of the clamping mechanism. It also confirms the feasibility and effectiveness of this method applied to the clamping force optimization for a large axial part.