Symmetric Positive Definite Manifold Research Articles

Log-Euclidean metric based covariance propagation on SPD manifold for continuous–discrete extended Kalman filtering

For nonlinear systems with continuous dynamic and discrete measurements, a Log-Euclidean metric (LEM) based novel scheme is proposed to refine the covariance integration steps of continuous–discrete Extended Kalman filter (CDEKF). In CDEKF, the covariance differential equation is usually integrated with regular Euclidean matrix operations, which actually ignores the Riemannian structure of underlying space and poses a limit on the further improvement of estimation accuracy. To overcome this drawback, this work proposes to define the covariance variable on the manifold of symmetric positive definite (SPD) matrices and propagate it using the Log-Euclidean metric. To embed the LEM based novel propagation scheme, the manifold integration of the covariance for LEMCDEKF is proposed together with the details of efficient realization, which can integrate the covariance on SPD manifold and avoid the drawback of Euclidean scheme. Numerical simulations certify the new method’s superior accuracy than conventional methods.

Deep hybrid manifold for image set classification

The exponential growth of the data volume of image sets, which contain more information than a single image, has attracted increasing attention from researchers. Image set data are often described as covariance matrices or linear subspaces, and the unique geometries they span are symmetric positive definite (SPD) manifolds and Grassmann manifolds, respectively. Image set data are often described as covariance matrices or linear subspaces, and the distinctive geometries they span are symmetric positive definite (SPD) manifold and Grassmann manifold, respectively. However, most studies focus on a single manifold and ignore the useful information of the another manifold. Based on this, we propose a new Deep Hybrid Manifold Network (DHMNet).The DHMNet consists of backbone network, stackable Hybrid Manifold AutoEncoder (HMAE) and,Maximum Fusion Module (MFM). The image set data is modeled through SPD manifold and Grassmann manifold. The modeled data is input into the backbone network composed of SPDNet and GrNet for initial feature extraction, and the output manifold data are input into HMAEs. The HMAE effectively extracts and hybridizes complementary information from different manifolds and has the ability to generate deep representations with rich structural semantic information. For the three image datasets used, DHMNet with two HMAEs improves the classification accuracy by 3.83–5.76% over the classical SPDNet, and even reaches the best when compared to other models, with the best performance on the First Person Hand Action (FPHA) dataset for skeleton-based hand action recognition.

SPD Manifold Deep Metric Learning for Image Set Classification.

By characterizing each image set as a nonsingular covariance matrix on the symmetric positive definite (SPD) manifold, the approaches of visual content classification with image sets have made impressive progress. However, the key challenge of unhelpfully large intraclass variability and interclass similarity of representations remains open to date. Although, several recent studies have mitigated the two problems by jointly learning the embedding mapping and the similarity metric on the original SPD manifold, their inherent shallow and linear feature transformation mechanism are not powerful enough to capture useful geometric features, especially in complex scenarios. To this end, this article explores a novel approach, termed SPD manifold deep metric learning (SMDML), for image set classification. Specifically, SMDML first selects a prevailing SPD manifold neural network (SPDNet) as the backbone (encoder) to derive an SPD matrix nonlinear representation. To counteract the degradation of structural information during multistage feature embedding, we construct a Riemannian decoder at the end of the encoder, trained by a reconstruction error term (RT), to induce the generated low-dimensional feature manifold of the hidden layer to capture the pivotal information about the visual data describing the imaged scene. We demonstrate through theory and experiments that it is feasible to replace the Riemannian metric with Euclidean distance in RT. Then, the ReCov layer is introduced into the established Riemannian network to regularize the local statistical information within each input feature matrix, which enhances the effectiveness of the learning process. The theoretical analysis of the activation function used in the ReCov layer in terms of continuity and conditions for generating positive definite matrices is beneficial for network design. Inspired by the fact that the single cross-entropy loss used for training is unable to effectively parse the geometric distribution of the deep representations, we finally endow the suggested model with a novel metric learning regularization term. By explicitly incorporating the encoding and processing of the data variations into the network learning process, this term can not only derive a powerful Riemannian representation but also train an effective classifier. The experimental results show the superiority of the proposed approach on three typical visual classification tasks.

Tensor-CSPNet: A Novel Geometric Deep Learning Framework for Motor Imagery Classification.

Deep learning (DL) has been widely investigated in a vast majority of applications in electroencephalography (EEG)-based brain-computer interfaces (BCIs), especially for motor imagery (MI) classification in the past five years. The mainstream DL methodology for the MI-EEG classification exploits the temporospatial patterns of EEG signals using convolutional neural networks (CNNs), which have been particularly successful in visual images. However, since the statistical characteristics of visual images depart radically from EEG signals, a natural question arises whether an alternative network architecture exists apart from CNNs. To address this question, we propose a novel geometric DL (GDL) framework called Tensor-CSPNet, which characterizes spatial covariance matrices derived from EEG signals on symmetric positive definite (SPD) manifolds and fully captures the temporospatiofrequency patterns using existing deep neural networks on SPD manifolds, integrating with experiences from many successful MI-EEG classifiers to optimize the framework. In the experiments, Tensor-CSPNet attains or slightly outperforms the current state-of-the-art performance on the cross-validation and holdout scenarios in two commonly used MI-EEG datasets. Moreover, the visualization and interpretability analyses also exhibit the validity of Tensor-CSPNet for the MI-EEG classification. To conclude, in this study, we provide a feasible answer to the question by generalizing the DL methodologies on SPD manifolds, which indicates the start of a specific GDL methodology for the MI-EEG classification.

VSTNet: Robust watermarking scheme based on voxel space transformation for diffusion tensor imaging images

Deep watermarking has been successfully applied to real-value domain (e.g., CT, MRI, DWI) medical images for copyright protection; however, manifold-value domain images, such as diffusion tensor imaging (DTI), have received less academic attention. The non-Euclidean nature of DTI images hinders deep models from generating plausible DTI images because it is difficult to guarantee that each generated DTI image voxel lies on a 3 × 3 symmetric positive definite manifold. In this paper, we propose a robust watermarking scheme based on voxel space transformation for DTI images. First, a voxel space transformation is proposed, which effectively solves the problem of fiber orientation estimation in Riemann networks for DTI images and obtains the eigenvalues and eigenvectors of each voxel using singular value decomposition (SVD) to transform the DTI voxels from the 3 × 3 symmetric positive definite manifold space to Euclidean space. Second, SVD is combined with a deep neural network to extract the high-level features of each voxel eigenvalue in the DTI images and embedding watermarking messages, avoiding the false positive errors problem in traditional watermarking algorithms based on an SVD transform. Finally, using a deep neural network combining multiscale dilated convolution, dense residual connection and channel attention, the algorithm demonstrates high robustness against various intentional or unintentional attacks on DTI images while ensuring the good visual quality and diffusion characteristics of the DTI images embedded with watermarking messages. Experiments show that the watermarking message extraction accuracy reaches 97.8% for Crop (p = 0.7) attacks and 95.4% for Gaussian Blur (k = 7) attacks.

Continuous–Discrete Cubature Kalman Filter With Log-Euclidean Metric-Based Covariance Integration

For continuous-discrete filtering with strong nonlinearity and large measurement intervals, a Log-Euclidean metric (LEM) based novel continuous-discrete cubature Kalman filter (LEMCDCKF) is proposed by shifting the cubature rule-based covariance propagation to Riemannian manifold. In conventional CDCKFs, the covariance differential equation based on cubature points is solved with Euclidean space integration schemes, which inevitably ignore the geometric property and restrict the performance of CDCKF. To remedy this shortage, we propose to define covariance on Riemannian symmetric positive definite (SPD) manifold and integrate the cubature rule-based covariance differential equation with the LEM-based novel scheme, which can successfully account for the manifold property of covariance matrices and provide accurate results. Moreover, by refining CDCKF with the LEM based scheme, the proposed LEMCDCKF shifts the covariance integration process to SPD manifold, which can break through the limitation of Euclidean numerical scheme. Numerical investigations verify the superior performance of the proposed LEMCDCKF in air traffic control scenarios with large measurement intervals.

Learning stable robotic skills on Riemannian manifolds

In this paper, we propose an approach to learn stable dynamical systems that evolve on Riemannian manifolds. Our approach leverages a data-efficient procedure to learn a diffeomorphic transformation, enabling the mapping of simple stable dynamical systems onto complex robotic skills. By harnessing mathematical techniques derived from differential geometry, our method guarantees that the learned skills fulfill the geometric constraints imposed by the underlying manifolds, such as unit quaternions (UQ) for orientation and symmetric positive definite (SPD) matrices for impedance. Additionally, the method preserves convergence towards a given target. Initially, the proposed methodology is evaluated through simulation on a widely recognized benchmark, which involves projecting Cartesian data onto UQ and SPD manifolds. The performance of our proposed approach is then compared with existing methodologies. Apart from that, a series of experiments were performed to evaluate the proposed approach in real-world scenarios. These experiments involved a physical robot tasked with bottle stacking under various conditions and a drilling task performed in collaboration with a human operator. The evaluation results demonstrate encouraging outcomes in terms of learning accuracy and the ability to adapt to different situations.

Generalized Learning Vector Quantization With Log-Euclidean Metric Learning on Symmetric Positive-Definite Manifold.

In many classification scenarios, the data to be analyzed can be naturally represented as points living on the curved Riemannian manifold of symmetric positive-definite (SPD) matrices. Due to its non-Euclidean geometry, usual Euclidean learning algorithms may deliver poor performance on such data. We propose a principled reformulation of the successful Euclidean generalized learning vector quantization (GLVQ) methodology to deal with such data, accounting for the nonlinear Riemannian geometry of the manifold through log-Euclidean metric (LEM). We first generalize GLVQ to the manifold of SPD matrices by exploiting the LEM-induced geodesic distance (GLVQ-LEM). We then extend GLVQ-LEM with metric learning. In particular, we study both 1) a more straightforward implementation of the metric learning idea by adapting metric in the space of vectorized log-transformed SPD matrices and 2) the full formulation of metric learning without matrix vectorization, thus preserving the second-order tensor structure. To obtain the distance metric in the full LEM learning (LEML) approaches, two algorithms are proposed. One method is to restrict the distance metric to be full rank, treating the distance metric tensor as an SPD matrix, and readily use the LEM framework (GLVQ-LEML-LEM). The other method is to cast no such restriction, treating the distance metric tensor as a fixed rank positive semidefinite matrix living on a quotient manifold with total space equipped with flat geometry (GLVQ-LEML-FM). Experiments on multiple datasets of different natures demonstrate the good performance of the proposed methods.

Riemannian Local Mechanism for SPD Neural Networks

The Symmetric Positive Definite (SPD) matrices have received wide attention for data representation in many scientific areas. Although there are many different attempts to develop effective deep architectures for data processing on the Riemannian manifold of SPD matrices, very few solutions explicitly mine the local geometrical information in deep SPD feature representations. Given the great success of local mechanisms in Euclidean methods, we argue that it is of utmost importance to ensure the preservation of local geometric information in the SPD networks. We first analyse the convolution operator commonly used for capturing local information in Euclidean deep networks from the perspective of a higher level of abstraction afforded by category theory. Based on this analysis, we define the local information in the SPD manifold and design a multi-scale submanifold block for mining local geometry. Experiments involving multiple visual tasks validate the effectiveness of our approach.

Reducing the Dimensionality of SPD Matrices with Neural Networks in BCI

In brain–computer interface (BCI)-based motor imagery, the symmetric positive definite (SPD) covariance matrices of electroencephalogram (EEG) signals with discriminative information features lie on a Riemannian manifold, which is currently attracting increasing attention. Under a Riemannian manifold perspective, we propose a non-linear dimensionality reduction algorithm based on neural networks to construct a more discriminative low-dimensional SPD manifold. To this end, we design a novel non-linear shrinkage layer to modify the extreme eigenvalues of the SPD matrix properly, then combine the traditional bilinear mapping to non-linearly reduce the dimensionality of SPD matrices from manifold to manifold. Further, we build the SPD manifold network on a Siamese architecture which can learn the similarity metric from the data. Subsequently, the effective signal classification method named minimum distance to Riemannian mean (MDRM) can be implemented directly on the low-dimensional manifold. Finally, a regularization layer is proposed to perform subject-to-subject transfer by exploiting the geometric relationships of multi-subject. Numerical experiments for synthetic data and EEG signal datasets indicate the effectiveness of the proposed manifold network.

Learning to Optimize on Riemannian Manifolds.

Many learning tasks are modeled as optimization problems with nonlinear constraints, such as principal component analysis and fitting a Gaussian mixture model. A popular way to solve such problems is resorting to Riemannian optimization algorithms, which yet heavily rely on both human involvement and expert knowledge about Riemannian manifolds. In this paper, we propose a Riemannian meta-optimization method to automatically learn a Riemannian optimizer. We parameterize the Riemannian optimizer by a novel recurrent network and utilize Riemannian operations to ensure that our method is faithful to the geometry of manifolds. The proposed method explores the distribution of the underlying data by minimizing the objective of updated parameters, and hence is capable of learning task-specific optimizations. We introduce a Riemannian implicit differentiation training scheme to achieve efficient training in terms of numerical stability and computational cost. Unlike conventional meta-optimization training schemes that need to differentiate through the whole optimization trajectory, our training scheme is only related to the final two optimization steps. In this way, our training scheme avoids the exploding gradient problem, and significantly reduces the computational load and memory footprint. We discuss experimental results across various constrained problems, including principal component analysis on Grassmann manifolds, face recognition, person re-identification, and texture image classification on Stiefel manifolds, clustering and similarity learning on symmetric positive definite manifolds, and few-shot learning on hyperbolic manifolds.

Adaptive Mask Sampling and Manifold to Euclidean Subspace Learning With Distance Covariance Representation for Hyperspectral Image Classification

For the abundant spectral and spatial information recorded in hyperspectral images (HSIs), fully exploring spectral-spatial relationships has attracted widespread attention in hyperspectral image classification (HSIC) community. However, there are still some intractable obstructs. For one thing, in the patch based processing pattern, some spatial neighbor pixels are often inconsistent with the central pixel in land-cover class. For another thing, linear and nonlinear correlations between different spectral bands are vital yet tough for representing and excavating. To overcome these mentioned issues, an adaptive mask sampling and manifold to Euclidean subspace learning (AMS-M2ESL) framework is proposed for HSIC. Specifically, an adaptive mask based intra-patch sampling (AMIPS) module is firstly formulated for intra-patch sampling in an adaptive mask manner based on central spectral vector oriented spatial relationships. Then, based on distance covariance descriptor, a dual channel distance covariance representation (DC-DCR) module is proposed for modeling unified spectral-spatial feature representations and exploring spectral-spatial relationships, especially linear and nonlinear interdependence in spectral domain. Furthermore, considering that distance covariance matrix lies on the symmetric positive definite (SPD) manifold, we implement a manifold to Euclidean subspace learning (M2ESL) module respecting Riemannian geometry of SPD manifold for high-level spectral-spatial feature learning. Additionally, we introduce an approximate matrix square-root (ASQRT) layer for efficient Euclidean subspace projection. Extensive experimental results on three popular HSI data sets with limited training samples demonstrate the superior performance of the proposed method compared with other state-of-the-art methods. The source code is available at https://github.com/lms-07/AMS-M2ESL.

U-SPDNet: An SPD manifold learning-based neural network for visual classification

With the development of neural networking techniques, several architectures for symmetric positive definite (SPD) matrix learning have recently been put forward in the computer vision and pattern recognition (CV&PR) community for mining fine-grained geometric features. However, the degradation of structural information during multi-stage feature transformation limits their capacity. To cope with this issue, this paper develops a U-shaped neural network on the SPD manifolds (U-SPDNet) for visual classification. The designed U-SPDNet contains two subsystems, one of which is a shrinking path (encoder) making up of a prevailing SPD manifold neural network (SPDNet (Huang and Van Gool, 2017)) for capturing compact representations from the input data. Another is a constructed symmetric expanding path (decoder) to upsample the encoded features, trained by a reconstruction error term. With this design, the degradation problem will be gradually alleviated during training. To enhance the representational capacity of U-SPDNet, we also append skip connections from encoder to decoder, realized by manifold-valued geometric operations, namely Riemannian barycenter and Riemannian optimization. On the MDSD, Virus, FPHA, and UAV-Human datasets, the accuracy achieved by our method is respectively 6.92%, 8.67%, 1.57%, and 1.08% higher than SPDNet, certifying its effectiveness.

EEG emotion recognition based on enhanced SPD matrix and manifold dimensionality reduction

Recently, Riemannian geometry-based pattern recognition has been widely employed to brain computer interface (BCI) researches, providing new idea for emotion recognition based on electroencephalogram (EEG) signals. Although the symmetric positive definite (SPD) matrix manifold constructed from the traditional covariance matrix contains large amount of spatial information, these methods do not perform well to classify and recognize emotions, and the high dimensionality problem still unsolved. Therefore, this paper proposes a new strategy for EEG emotion recognition utilizing Riemannian geometry with the aim of achieving better classification performance. The emotional EEG signals of 32 healthy subjects were from an open-source dataset (DEAP). The wavelet packets were first applied to extract the time-frequency features of the EEG signals, and then the features were used to construct the enhanced SPD matrix. A supervised dimensionality reduction algorithm was then designed on the Riemannian manifold to reduce the high dimensionality of the SPD matrices, gather samples of the same labels together, and separate samples of different labels as much as possible. Finally, the samples were mapped to the tangent space, and the K-nearest neighbors (KNN), Random Forest (RF) and Support Vector Machine (SVM) method were employed for classification. The proposed method achieved an average accuracy of 91.86%, 91.84% on the valence and arousal recognition tasks. Furthermore, we also obtained the superior accuracy of 86.71% on the four-class recognition task, demonstrated the superiority over state-of-the-art emotion recognition methods.

A Framework for Short Video Recognition Based on Motion Estimation and Feature Curves on SPD Manifolds

Given the prosperity of video media such as TikTok and YouTube, the requirement of short video recognition is becoming more and more urgent. A significant feature of short video is that there are few switches of scenes in short video, and the target (e.g., the face of the key person in the short video) often runs through the short video. This paper presents a new short video recognition algorithm framework that transforms a short video into a family of feature curves on symmetric positive definite (SPD) manifold as the basis of recognition. Thus far, no similar algorithm has been reported. The results of experiments suggest that our method performs better on three changeling databases than seven other related algorithms published in the top issues.

Riemannian submanifold framework for log-Euclidean metric learning on symmetric positive definite manifolds

This study presents a novel Riemannian submanifold (RS) framework for log-Euclidean metric learning on symmetric positive definite manifolds. Our method identifies the optimal RS without changing the original tangent space. The RS is spanned by multiple bases, and each data point is parameterized using these bases, such that the data can be represented more informatively compared to conventional approaches. In the RS, a new distance function is defined, and its derivative cannot be obtained trivially. We overcome this difficulty and provide a simple analytic form of the derivative. The proposed transformation matrix can take any form, which gives flexibility to our method and enables the creation of several variants. In experiments, our method and its variants surpass state-of-the-art metric learning methods in synthetic, material categorization, and action recognition problems.

Learning a discriminative SPD manifold neural network for image set classification

Performing pattern analysis over the symmetric positive definite (SPD) manifold requires specific mathematical computations, characterizing the non-Euclidian property of the involved data points and learning tasks, such as the image set classification problem. Accompanied with the advanced neural networking techniques, several architectures for processing the SPD matrices have recently been studied to obtain fine-grained structured representations. However, existing approaches are challenged by the diversely changing appearance of the data points, begging the question of how to learn invariant representations for improved performance with supportive theories. Therefore, this paper designs two Riemannian operation modules for SPD manifold neural network. Specifically, a Riemannian batch regularization (RBR) layer is firstly proposed for the purpose of training a discriminative manifold-to-manifold transforming network with a novelly-designed metric learning regularization term. The second module realizes the Riemannian pooling operation with geometric computations on the Riemannian manifolds, notably the Riemannian barycenter, metric learning, and Riemannian optimization. Extensive experiments on five benchmarking datasets show the efficacy of the proposed approach.

Motor imagery EEG decoding using manifold embedded transfer learning

BackgroundBrain computer interface (BCI) utilizes brain signals to help users interact with external devices directly. EEG is one of the most commonly used techniques for brain signal acquisition in BCI. However, it is notoriously difficult to build a generic EEG recognition model due to significant non-stationarity and subject-to-subject variations, and the requirement for long time training. Transfer learning (TL) is particularly useful because it can alleviate the calibration requirement in EEG-based BCI applications by transferring the calibration information from existing subjects to new subject. To take advantage of geometric properties in Riemann manifold and joint distribution adaptation, a manifold embedded transfer learning (METL) framework was proposed for motor imagery (MI) EEG decoding. New methodFirst, the covariance matrices of the EEG trials are first aligned on the SPD manifold. Then the features are extracted from both the symmetric positive definite (SPD) manifold and Grassmann manifold. Finally, the classification model is learned by combining the structural risk minimization (SRM) of source domain and joint distribution alignment of source and target domains. ResultExperimental results on two MI EEG datasets verify the effectiveness of the proposed METL. In particular, when there are a small amount of labeled samples in the target domain, METL demonstrated a more accurate and stable classification performance than conventional methods. Comparison with existing methodsCompared with several state-of-the-art methods, METL has achieved better classification accuracy, 71.81% and 69.06% in single-to-single (STS), 83.14% and 76.00% in multi-to-single (MTS) transfer tasks, respectively. ConclusionsMETL can cope with single source domain or multi-source domains and compared with single-source transfer learning, multi-source transfer learning can improve the performance effectively due to the data expansion. It is effective enough to achieve superior performance for classification of EEG signals.

Learning Non-Euclidean Representations With SPD Manifold for Myoelectric Pattern Recognition.

How to learn informative representations from Electromyography (EMG) signals is of vital importance for myoelectric control systems. Traditionally, hand-crafted features are extracted from individual EMG channels and combined together for pattern recognition. The spatial topological information between different channels can also be informative, which is seldom considered. This paper presents a radically novel approach to extract spatial structural information within diverse EMG channels based on the symmetric positive definite (SPD) manifold. The object is to learn non-Euclidean representations inside EMG signals for myoelectric pattern recognition. The performance is compared with two classical feature sets using accuracy and F1-score. The algorithm is tested on eleven gestures collected from ten subjects, and the best accuracy reaches 84.85%±5.15% with an improvement of 4.04%~20.25%, which outperforms the contrast method, and reaches a significant improvement with the Wilcoxon signed-rank test. Eleven gestures from three public databases involving Ninapro DB2, DB4, and DB5 are also evaluated, and better performance is observed. Furthermore, the computational cost is less than the contrast method, making it more suitable for low-cost systems. It shows the effectiveness of the presented approach and contributes a new way for myoelectric pattern recognition.

Symmetric positive definite manifold learning and its application in fault diagnosis

Locally linear embedding (LLE) is an effective tool to extract the significant features from a dataset. However, most of the relevant existing algorithms assume that the original dataset resides on a Euclidean space, unfortunately nearly all the original data space is non-Euclidean. In addition, the original LLE does not use the discriminant information of the dataset, which will degrade its performance in feature extraction. To address these problems raised in the conventional LLE, we first employ the original dataset to construct a symmetric positive definite manifold, and then estimate the tangent space of this manifold. Furthermore, the local and global discriminant information are integrated into the LLE, and the improved LLE is operated in the tangent space to extract the important features. We introduce Iris dataset to analyze the capability of the proposed method to extract features. Finally, several experiments are performed on five machinery datasets, and experimental results indicate that our proposed method can extract the excellent low-dimensional representations of the original dataset. Compared with the state-of-the-art methods, the proposed algorithm shows a strong capability for fault diagnosis.