Symmetric Positive Definite Matrices Research Articles

A note on error bounds for pseudo skeleton approximations of matrices

A new and improved bound for the Chebyshev norm of the error of a maximal volume pseudo-skeleton matrix approximation is presented. This bound uses all the singular values of the approximated matrix up to the r+1st where r is the rank of the approximation. It is shown that this bound is in particular useful for the case of matrices with slowly decaying singular values where the approximation rank is of moderate size. In this case the original bounds for the maximal volume approximation are of the form (r+1)σr+1 and thus are very large and if σr+1=O(1/(r+1)) do not even indicate convergence.A new approximation based on the truncated singular value decomposition of a maximal volume approximation is analysed. It is demonstrated that this approximation is in some cases asymptotically as a good as the truncated singular value approximation of the original matrix. This new approximation is shown to have similar approximation properties as the maximal r-volume method analysed by Osinsky and Zamarashkin (2018) [4] for the case of symmetric positive definite matrices.

On the Computation of Mean and Variance of Spatial Displacements

AbstractThis paper studies the problem of computing an average (or mean) displacement from a set of given spatial displacements using three types of parametric representations: Euler angles and translation vectors, unit quaternions and translation vectors, and dual quaternions. It is shown that the use of Euclidean norm in the space of unit quaternions reduces the problem to that of computing the mean for each quaternion component separately and independently. While the resulting algorithm is simple, a change in the sign of a unit quaternion could lead to an incorrect result. A novel kinematic measure based on dual quaternions is introduced to capture the separation between two spatial displacements. This kinematic measure is used to define the variance of a set of displacements, which is then used to formulate a constrained least squares minimization problem. It is shown that the problem decomposes into that of finding the optimal translation vector and the optimal unit quaternion. The former is simply the centroid of the set of translation vectors and the latter is obtained as the eigenvector corresponding to the least eigenvalue of a 4 × 4 positive definite symmetric matrix. In addition, it is found that the weight factor used in combining rotations and translations in the formulation does not play a role in the final outcome. Examples are provided to show the comparisons of these methods.

Reducing the Dimensionality of SPD Matrices with Neural Networks in BCI

In brain–computer interface (BCI)-based motor imagery, the symmetric positive definite (SPD) covariance matrices of electroencephalogram (EEG) signals with discriminative information features lie on a Riemannian manifold, which is currently attracting increasing attention. Under a Riemannian manifold perspective, we propose a non-linear dimensionality reduction algorithm based on neural networks to construct a more discriminative low-dimensional SPD manifold. To this end, we design a novel non-linear shrinkage layer to modify the extreme eigenvalues of the SPD matrix properly, then combine the traditional bilinear mapping to non-linearly reduce the dimensionality of SPD matrices from manifold to manifold. Further, we build the SPD manifold network on a Siamese architecture which can learn the similarity metric from the data. Subsequently, the effective signal classification method named minimum distance to Riemannian mean (MDRM) can be implemented directly on the low-dimensional manifold. Finally, a regularization layer is proposed to perform subject-to-subject transfer by exploiting the geometric relationships of multi-subject. Numerical experiments for synthetic data and EEG signal datasets indicate the effectiveness of the proposed manifold network.

Nearly Optimal Block-Jacobi Preconditioning

The goal of Jacobi preconditioning of a symmetric positive definite matrix by a diagonal matrix is to choose to minimize the condition number . In 1969, van der Sluis proved that choosing so that the diagonal entries of are all ones reduces to within a factor of the minimum possible, where the factor depends on both the dimension and the norms used to define the condition number. We extend this result in two ways to block-Jacobi preconditioning, where is a block-diagonal matrix with blocks of given sizes, and we consider instead of to maintain the symmetric positive definite (spd) property. First, we extend van der Sluis’s original bound to include block-Jacobi. Second, we define a new norm in which choosing so that the corresponding diagonal blocks of are identity matrices minimizes the condition number. We use this to show that the condition number in the 2-norm of this optimally scaled is at least as large as the condition number in the new norm, and at most a factor larger, where is the number of diagonal blocks. We give an example where the optimal 2-norm condition number nearly attains this new upper bound, which for this example is tighter than van der Sluis’s bound by a factor equal to the matrix dimension. Finally, all these results generalize to the case of one-sided scaling of a full row-rank matrix .

A Modified q-BFGS Algorithm for Unconstrained Optimization

This paper presents a modification of the q-BFGS method for nonlinear unconstrained optimization problems. For this modification, we use a simple symmetric positive definite matrix and propose a new q-quasi-Newton equation, which is close to the ordinary q-quasi-Newton equation in the limiting case. This method uses only first order q-derivatives to build an approximate q-Hessian over a number of iterations. The q-Armijo-Wolfe line search condition is used to calculate step length, which guarantees that the objective function value is decreasing. This modified q-BFGS method preserves the global convergence properties of the q-BFGS method, without the convexity assumption on the objective function. Numerical results on some test problems are presented, which show that an improvement has been achieved. Moreover, we depict the numerical results through the performance profiles.

Phase space reconstruction, geometric filtering based Fisher discriminant analysis and minimum distance to the Riemannian means algorithm for epileptic seizure classification

BackgroundMost clinicians observe the changes of the electroencephalograms (EEGs) via the visual inspection of the recorded signals of the patients. However, the visual inspection is time consuming. Also, it is subjective and empirical. Therefore, designing an accurate and an automatic epilepsy classification system has a profound significance for reducing the workload of the medical officers. MethodFirst, this paper applies the phase space reconstruction (PSR) method for constructing the covariance matrix for characterizing the chaotic states of the EEGs. The covariance matrices are symmetric positive definite (SPD) matrices used as the feature descriptors of the EEGs. This set of the covariance matrices is in fact a kind of the Riemannian manifold. Denote this Riemannian manifold as M. Secondly, the tangent space at the Riemannian geometric mean is computed via the logarithmic mapping operator. Denote this tangent space as TM. This tangent space TM is a kind of the Euclidean space containing a point in the Riemannian manifold M. That is, the SPD matrices in the Riemannian manifold M are mapped to the matrices in the tangent space TM at the Riemannian geometric mean via the logarithmic mapping operator. Then, these matrices in the tangent space TM are further mapped to the tangent feature vectors. The set of the tangent feature vectors is a kind of the vector space. Denote this vector space as VT. Thirdly, the transformation matrix based on the Fisher discrimination analysis (FDA) is used to map the high dimensional vectors in the vector space VT to the low dimensional vectors in another vector space. Then, the discriminative vectors are projected back to the Riemannian manifold M via the exponential mapping operator. Finally, the minimum distance to the Riemannian means (MDRM) algorithm is applied for performing the epileptic seizure classification. ResultsThe computer numerical simulations are conducted based on three well known EEGs epileptic seizure datasets. Overall, our proposed method achieves the best performances for three datasets compared to the state of the art methods. Also, the computational complexity of our proposed method is low. ConclusionOur proposed method is effective and robust for performing the seizure classification. Also, our proposed method can be implemented in the real time.

SSTD: A Novel Spatio-Temporal Demographic Network for EEG-Based Emotion Recognition

Emotion recognition is the key to making machines more intelligent. This study proposes a novel sing-link end-to-end spatio-temporal demographic network (SSTD) that fuses spatial, temporal, and demographic information for electroencephalography (EEG)-based emotion recognition. In the SSTD model, an adaptive time window using single-link hierarchical clustering based on Riemannian metrics was realized for data preprocessing to solve the problem of individual differences. Then, the preprocessed EEG data acted as a gate recurrent unit (GRU) network input to calculate high-level time-domain features. At the same time, the EEG covariance matrices were fed into the symmetric positive definite matrix network (SPDNet) to calculate high-level spatial features. Given the correlation between EEG signals and individual demographic information, gender and age factors were integrated into the spatio-temporal model, resulting in more effective high-level features for EEG-based emotion recognition. Finally, extensive comparative experiments were conducted on two public datasets: DEAP and DREAMER. The average accuracy of valence and arousal on the DEAP dataset are 68.28% and 71.48%, respectively. The average accuracy of valence and arousal on the DREAMER dataset are 76.81% and 81.64%, respectively. Experimental results show that the SSTD model has an excellent recognition performance.

Pole Allocation for Rational Gauss Quadrature Rules for Matrix Functionals Defined by a Stieltjes Function

This paper considers the computation of approximations of matrix functionals of form F(A):=vTf(A)v, where A is a large symmetric positive definite matrix, v is a vector, and f is a Stieltjes function. The functional F(A) is approximated by a rational Gauss quadrature rule with poles on the negative real axis (or part thereof) in the complex plane, and we focus on the allocation of the poles. Specifically, we propose that the poles, when considered positive point charges, be allocated to make the negative real axis (or part thereof) approximate an equipotential curve. This is easily achieved with the aid of conformal mapping.

Deep Domain Adaptation Using Cascaded Learning Networks and Metric Learning

How can we bridge efficiently distribution difference between the source and target domains in an isomorphic latent feature space by using metric learning. This study introduces metric learning on manifolds which combine a cascaded learning network and a metric learning model to form a Unified Domain Adaptation Model. Our approach is based on formulating a transfer from the source to the target as a geometric mean metric learning problem on manifolds. The solution of symmetric positive definite covariance matrices not only reduces the statistical differences between the source and target domains, but also the underlying geometry of the source and target domains using diffusions on the underlying source and target manifolds. By retaining both the nonlinear structure of the Riemannian geometry of the open cone of symmetric positive definite matrices and cascaded learning networks, we improve the state-of-the-art results on the Amazon (A), Caltech256 (C), DSLR (D), Webcam (W) and VisDA benchmark datasets by knowledge transfer, while achieving comparable performances to competing methods on domain adaptation modeling.

A New Quasi-Newton Method with PCG Method for Nonlinear Optimization Problems

The major stationary iterative method used to solve nonlinear optimization problems is the quasi-Newton (QN) method. Symmetric Rank-One (SR1) is a method in the quasi-Newton family. This algorithm converges towards the true Hessian fast and has computational advantages for sparse or partially separable problems [1]. Thus, investigating the efficiency of the SR1 algorithm is significant. It's possible that the matrix generated by SR1 update won't always be positive. The denominator may also vanish or become zero. To overcome the drawbacks of the SR1 method, resulting in better performance than the standard SR1 method, in this work, we derive a new vector <img src=image/13429423_01.gif> depending on the Barzilai-Borwein step size to obtain a new SR1 method. Then using this updating formula with preconditioning conjugate gradient (PCG) method is presented. With the aid of inexact line search procedure by strong Wolfe conditions, the new SR1 method is proposed and its performance is evaluated in comparison to the conventional SR1 method. It is proven that the updated matrix of the new SR1 method, <img src=image/13429423_02.gif>, is symmetric matrix and positive definite matrix, given <img src=image/13429423_03.gif> is initialized to identity matrix. In this study, the proposed method solved 13 problems effectively in terms of the number of iterations (NI) and the number of function evaluations (NF). Regarding NF, the new SR1 method also outperformed the classic SR1 method. The proposed method is shown to be more efficient in solving relatively large-scale problems (5,000 variables) compared to the original method. From the numerical results, the proposed method turned out to be significantly faster, effective and suitable for solving large dimension nonlinear equations.

3M_BANTOR: A regression framework for multitask and multisession brain network distance metrics.

Brain network analyses have exploded in recent years and hold great potential in helping us understand normal and abnormal brain function. Network science approaches have facilitated these analyses and our understanding of how the brain is structurally and functionally organized. However, the development of statistical methods that allow relating this organization to phenotypic traits has lagged behind. Our previous work developed a novel analytic framework to assess the relationship between brain network architecture and phenotypic differences while controlling for confounding variables. More specifically, this innovative regression framework related distances (or similarities) between brain network features from a single task to functions of absolute differences in continuous covariates and indicators of difference for categorical variables. Here we extend that work to the multitask and multisession context to allow for multiple brain networks per individual. We explore several similarity metrics for comparing distances between connection matrices and adapt several standard methods for estimation and inference within our framework: standard F test, F test with scan-level effects (SLE), and our proposed mixed model for multitask (and multisession) BrAin NeTwOrk Regression (3M_BANTOR). A novel strategy is implemented to simulate symmetric positive-definite (SPD) connection matrices, allowing for the testing of metrics on the Riemannian manifold. Via simulation studies, we assess all approaches for estimation and inference while comparing them with existing multivariate distance matrix regression (MDMR) methods. We then illustrate the utility of our framework by analyzing the relationship between fluid intelligence and brain network distances in Human Connectome Project (HCP) data.

Matrix Manifold-Based Performance Monitoring of Automatic Control Systems

A “smart” automatic feedback control system is supposed to be aware of its operational performance throughout the service time. Driven by this desire, this article addresses the problem of monitoring performance variations caused by multiplicative factors, such as components’ faults, repairing, or replacement. Differing from the conventional performance indices (e.g., the quadratic value function), it detects the performance variation information from symmetric positive-definite (SPD) kernel matrices. As a carrier of performance variation information, the SPD kernel matrix is identified online. Furthermore, the performance variation is given a fresh insight in the sense that it drives the sliding of the SPD kernel matrix on a Riemannian manifold. Thus, performance variation monitoring is achieved by quantizing the geodesic between SPD kernel matrices directly on the Riemannian manifold. At last, the performance variation is visualized on the Riemannian manifold and the proposed schemes are verified via a simulation study.

Параллельная реализация метода сопряженных градиентов с предобусловливателем IC1 на основе использования переупорядочения узлов сетки

The paper proposes a method for using MPI+OpenMP technology for constructing and inverting a first-order incomplete triangular Cholesky expansion preconditioner IC1(τ) for solving system of linear algebraic equations with an arbitrary symmetric positive definite matrix on a small number of processors. Methods of using MPI and MPI+OpenMP technologies are based on the use of grid node orderings consistent with the division of the calculation area. When constructing the preconditioning matrix IC1(τ) using MPI, cutting is performed at positions in some of its rows. The use of OpenMP technology in the construction and inversion of the preconditioner is carried out for most rows of the matrix. The time taken to solve problems using the conjugate gradient method with the IC1(τ) preconditioner using MPI and hybrid MPI+OpenMP technology is compared using the example of a model problem and a number of problems from the SuiteSparse collection of sparse matrices.

MPI+OpenMP реализация метода сопряженных градиентов с факторизованным предобусловливателем на основе использования переупорядочения узлов сетки

The paper proposes ways to use MPI and MPI+OpenMP technologies for constructing and inverting a factorized preconditioner of the type incomplete triangular decomposition for solving systems of linear algebraic equations with an arbitrary symmetric positive definite matrix. Methods of using MPI and MPI+OpenMP technologies are based on the use of grid node orderings consistent with the division of the calculation area. The use of OpenMP technology in the construction and inversion of the preconditioner is carried out for most rows of the matrix. Comparative timing results for the MPI+OpenMP and MPI implementations of the proposed preconditioning used with the conjugate gradient method for a model problems and the sparse matrix collections SuiteSparse are presented.

Measure of Similarity between GMMs Based on Geometry-Aware Dimensionality Reduction

Gaussian Mixture Models (GMMs) are used in many traditional expert systems and modern artificial intelligence tasks such as automatic speech recognition, image recognition and retrieval, pattern recognition, speaker recognition and verification, financial forecasting applications and others, as simple statistical representations of underlying data. Those representations typically require many high-dimensional GMM components that consume large computing resources and increase computation time. On the other hand, real-time applications require computationally efficient algorithms and for that reason, various GMM similarity measures and dimensionality reduction techniques have been examined to reduce the computational complexity. In this paper, a novel GMM similarity measure is proposed. The measure is based on a recently presented nonlinear geometry-aware dimensionality reduction algorithm for the manifold of Symmetric Positive Definite (SPD) matrices. The algorithm is applied over SPD representations of the original data. The local neighborhood information from the original high-dimensional parameter space is preserved by preserving distance to the local mean. Instead of dealing with high-dimensional parameter space, the method operates on much lower-dimensional space of transformed parameters. Resolving the distance between such representations is reduced to calculating the distance among lower-dimensional matrices. The method was tested within a texture recognition task where superior state-of-the-art performance in terms of the trade-off between recognition accuracy and computational complexity has been achieved in comparison with all baseline GMM similarity measures.

O(n)-invariant Riemannian metrics on SPD matrices

Symmetric Positive Definite (SPD) matrices are ubiquitous in data analysis under the form of covariance matrices or correlation matrices. Several O(n)-invariant Riemannian metrics were defined on the SPD cone, in particular the kernel metrics introduced by Hiai and Petz. The class of kernel metrics interpolates between many classical O(n)-invariant metrics and it satisfies key results of stability and completeness. However, it does not contain all the classical O(n)-invariant metrics. Therefore in this work, we investigate super-classes of kernel metrics and we study which key results remain true. We also introduce an additional key result called cometric-stability, a crucial property to implement geodesics with a Hamiltonian formulation. Our method to build intermediate embedded classes between O(n)-invariant metrics and kernel metrics is to give a characterization of the whole class of O(n)-invariant metrics on SPD matrices and to specify requirements on metrics one by one until we reach kernel metrics. As a secondary contribution, we synthesize the literature on the main O(n)-invariant metrics, we provide the complete formula of the sectional curvature of the affine-invariant metric and the formula of the geodesic parallel transport between commuting matrices for the Bures-Wasserstein metric.

U-SPDNet: An SPD manifold learning-based neural network for visual classification

With the development of neural networking techniques, several architectures for symmetric positive definite (SPD) matrix learning have recently been put forward in the computer vision and pattern recognition (CV&PR) community for mining fine-grained geometric features. However, the degradation of structural information during multi-stage feature transformation limits their capacity. To cope with this issue, this paper develops a U-shaped neural network on the SPD manifolds (U-SPDNet) for visual classification. The designed U-SPDNet contains two subsystems, one of which is a shrinking path (encoder) making up of a prevailing SPD manifold neural network (SPDNet (Huang and Van Gool, 2017)) for capturing compact representations from the input data. Another is a constructed symmetric expanding path (decoder) to upsample the encoded features, trained by a reconstruction error term. With this design, the degradation problem will be gradually alleviated during training. To enhance the representational capacity of U-SPDNet, we also append skip connections from encoder to decoder, realized by manifold-valued geometric operations, namely Riemannian barycenter and Riemannian optimization. On the MDSD, Virus, FPHA, and UAV-Human datasets, the accuracy achieved by our method is respectively 6.92%, 8.67%, 1.57%, and 1.08% higher than SPDNet, certifying its effectiveness.

Infinitely divisible matrix gamma distribution: Asymptotic behaviour and parameters estimation

In this research paper, we investigate an infinitely divisible p×p matrix gamma distribution AΓp(η,Σ), with parameters η>(p−1)/2 and Σ, concentrated on the cone of symmetric positive definite matrices. The parameter Σ is supposed to be a symmetric positive definite p×p matrix. We also display some of its fundamental properties. Additionally, we identify the link between this multivariate gamma distribution and the Wishart one, which leads us to prove that AΓp(η,Σ) distribution is asymptotically a stochastic linear combination of Wishart matrices. Moreover, we provide an explicit expression of the parameters estimators using the method of moments. Eventually, we exhibit a new simulation algorithm, grounded on the obtained results, in order to illustrate the performance of these estimators.

On computing the symplectic LLT factorization

We analyze two algorithms for computing the symplectic factorization A = LLT of a given symmetric positive definite symplectic matrix A. The first algorithm W1 is an implementation of the HHT factorization from Dopico and Johnson (SIAM J. Matrix Anal. Appl. 31(2):650–673, 2009), see Theorem 5.2. The second one is a new algorithm W2 that uses both Cholesky and Reverse Cholesky decompositions of symmetric positive definite matrices. We present a comparison of these algorithms and illustrate their properties by numerical experiments in MATLAB. A particular emphasis is given on symplecticity properties of the computed matrices in floating-point arithmetic.

SPD matrices representing artery anatomy for first-pass effect prediction by aggregated networks with multi-scale attentions

ABSTRACT First-pass Effect(FPE) for Endovascular Therapy(EVT) is associated with good clinical outcome (mRS2) of patients with Acute Ischaemic Stroke (AIS). A rapid and accurate prediction of FPE before EVT can help neurointerventionists plan the procedure and avoid delays in restoration of cerebral blood flow. However, there are rare studies focused on FPE prediction on arterial vessel anatomy immediately. The intractable difficulty lies in extracting discriminative features to represent a wide variety of vessels with irregular vessel shapes. In this paper, we propose a Symmetric Positive Definite(SPD) matrix-based feature representation extracted from the centerline of arterial vessels, encoding the global discriminative information over the artery for predicting FPE. Subsequently, a collaborative network of multi-scale Convolutional Neural Network(CNN) and Multiple Layer Perception (MLP) with specific attentions is developed. Specifically, the CNN is used for capturing the features among the multi-scale local neighbours of the curve. MLPs are used for capturing more prominent global discriminative features at different scales. The attention mechanism is used to better filter and extract the useful information for feature fusion. Quantitative experimental results demonstrate that our proposed method is able to predict FPE accurately, outperforming the manually defined features and traditional machine learning-based methods in this task, regarding the metrics of AUC, precision, sensitivity, specificity and accuracy.