Matrix Manifold Research Articles

Reduced order modeling of turbulent reacting flows on low-rank matrix manifolds

A new low-rank approximation, referred to as time-dependent principal component analysis (t-PCA), is developed for reduced-order modeling (ROM) of scalar transport in turbulent reactive flows. In t-PCA, the evolution of the composition matrix is constrained to a low-rank matrix manifold, similar to that in standard PCA. Specifically, the t-PCA approximates the composition matrix through the multiplication of two thin, time-dependent matrices that represent spatial and composition subspaces. The evolution equations for these subspaces are derived by projecting the composition transport equation onto the tangent space of the low-rank matrix manifold. While the evolution equations for the spatial subspace in both PCA and t-PCA are similar, there are differences in how the composition subspace is obtained: (i) In t-PCA, the composition subspace is time-dependent, whereas in PCA, it is static. (ii) The t-PCA does not require any prior data, and an evolution equation for the composition subspace is derived. In PCA, the composition subspace is obtained from data. The t-PCA can be regarded as an on-the-fly low-rank approximation that can adapt to changes in the flow instantaneously. It is shown that the low-rank t-PCA approximations achieve residual levels lower than those obtained via PCA. For demonstrations and a comparative assessment of the ROMs, simulations are conducted of a non-premixed CO/H2 flame in a temporally evolving jet. Two cases are considered, based on the mechanisms previously suggested for combustion kinetics of this flame: (i) the GRI-Mech 3.0 model involving 53 species for a two-dimensional flame, (ii) the skeletal syngas model involving 11 species for a three-dimensional turbulent flame. The results are appraised via a posteriori comparisons against data generated via full-rank direct numerical simulation (DNS) of the same flame, and also with the PCA-reduced data from the DNS. It is shown that t-PCA yields excellent predictions of various features of the thermo-chemistry field.

Computing Euclidean distance and maximum likelihood retraction maps for constrained optimization

Riemannian optimization uses local methods to solve optimization problems whose constraint set is a smooth manifold. A linear step along some descent direction usually leaves the constraints, and hence retraction maps are used to approximate the exponential map and return to the manifold. For many common matrix manifolds, retraction maps are available, with more or less explicit formulas. For implicitly-defined manifolds, suitable retraction maps are difficult to compute. We therefore develop an algorithm which uses homotopy continuation to compute the Euclidean distance retraction for any implicitly-defined submanifold of Rn, and prove convergence results.We also consider statistical models as Riemannian submanifolds of the probability simplex with the Fisher metric. Replacing Euclidean distance with maximum likelihood results in a map which we prove is a retraction. In fact, we prove the retraction is second-order; with the Levi-Civita connection associated to the Fisher metric, it approximates geodesics to second-order accuracy.

From constraints fusion to manifold optimization: A new directional transport manifold metaheuristic algorithm

The ascent of geometry-based models and methodologies, exemplified by geometric deep learning and manifold numerical optimization algorithms, has inaugurated a novel domain across various applications that grapple with geometric data complexities, such as electroencephalogram signals represented by symmetric positive definite matrix manifold, hierarchical data represented by hyperbolic manifold. The imperative fuels this inevitable paradigm shift to encapsulate the intricacies and richness inherent in data, areas where traditional methods prove inadequate. While metaheuristic algorithms are renowned for their versatile adaptability across applications, offering practical solutions within reasonable timeframes. However, the conventional metaheuristic algorithms fail on manifold applications with meaningless solutions. From an extrinsic optimization perspective, we treat manifold optimization problems as general optimization problems with multiple fused constraints that limit the optimization path to the manifold. This study pioneered the proposal and implementation of a metaheuristic manifold optimization, introducing a novel directional transport operator to rectify previously identified issues. Through experimentation across five sets of 25 problems, comparing against five algorithms, including both gradient-free and gradient-dominant counterparts, our proposed algorithm emerges as the optimal performer within the gradient-free category, demonstrating competitiveness even against gradient-dominant algorithms. Furthermore, we applied the proposed algorithm to the robot dynamic manipulation problem, achieving a close-optimal solution that eludes gradient-dominant approaches. This paper delves into the inherent capabilities and establishes the generalization of a metaheuristic algorithm within non-Euclidean functional landscapes. The source code will be available at https://github.com/lingping-fuzzy/metaheuristic-manifold-optimization.

From low-rank retractions to dynamical low-rank approximation and back

In algorithms for solving optimization problems constrained to a smooth manifold, retractions are a well-established tool to ensure that the iterates stay on the manifold. More recently, it has been demonstrated that retractions are a useful concept for other computational tasks on manifold as well, including interpolation tasks. In this work, we consider the application of retractions to the numerical integration of differential equations on fixed-rank matrix manifolds. This is closely related to dynamical low-rank approximation (DLRA) techniques. In fact, any retraction leads to a numerical integrator and, vice versa, certain DLRA techniques bear a direct relation with retractions. As an example for the latter, we introduce a new retraction, called KLS retraction, that is derived from the so-called unconventional integrator for DLRA. We also illustrate how retractions can be used to recover known DLRA techniques and to design new ones. In particular, this work introduces two novel numerical integration schemes that apply to differential equations on general manifolds: the accelerated forward Euler (AFE) method and the Projected Ralston–Hermite (PRH) method. Both methods build on retractions by using them as a tool for approximating curves on manifolds. The two methods are proven to have local truncation error of order three. Numerical experiments on classical DLRA examples highlight the advantages and shortcomings of these new methods.

An accelerated hybrid Riemannian conjugate gradient method for unconstrained optimization

With the goal to deal with a series of optimization problems on general matrix manifolds with differentiable objective functions, we propose an accelerated hybrid Riemannian conjugate gradient technique. Specifically, the acceleration scheme of the proposed method using a modified stepsize which is multiplicatively determined by the Wolfe line search. The search direction of the proposed algorithm is determined by the hybrid conjugate parameter with computationally promising. We showed that the suggested approach converges globally to a stationary point. Our approach performs better than the state of art Riemannian conjugate gradient algorithms, as illustrated by computations on problems such as the orthogonal Procrustes problem and the Brockett-cost-function minimization problem.

Modeling Tree-like Heterophily on Symmetric Matrix Manifolds.

Tree-like structures, characterized by hierarchical relationships and power-law distributions, are prevalent in a multitude of real-world networks, ranging from social networks to citation networks and protein-protein interaction networks. Recently, there has been significant interest in utilizing hyperbolic space to model these structures, owing to its capability to represent them with diminished distortions compared to flat Euclidean space. However, real-world networks often display a blend of flat, tree-like, and circular substructures, resulting in heterophily. To address this diversity of substructures, this study aims to investigate the reconstruction of graph neural networks on the symmetric manifold, which offers a comprehensive geometric space for more effective modeling of tree-like heterophily. To achieve this objective, we propose a graph convolutional neural network operating on the symmetric positive-definite matrix manifold, leveraging Riemannian metrics to facilitate the scheme of information propagation. Extensive experiments conducted on semi-supervised node classification tasks validate the superiority of the proposed approach, demonstrating that it outperforms comparative models based on Euclidean and hyperbolic geometries.

Riemannian quantum circuit optimization for Hamiltonian simulation

Hamiltonian simulation, i.e. simulating the real time evolution of a target quantum system, is a natural application of quantum computing. Trotter-Suzuki splitting methods can generate corresponding quantum circuits; however, a faithful approximation can lead to relatively deep circuits. Here we start from the insight that for translation invariant systems, the gates in such circuit topologies can be further optimized on classical computers to decrease the circuit depth and/or increase the accuracy. We employ tensor network techniques and devise a method based on the Riemannian trust-region algorithm on the unitary matrix manifold for this purpose. For the Ising and Heisenberg models on a one-dimensional lattice, we achieve orders of magnitude accuracy improvements compared to fourth-order splitting methods. The optimized circuits could also be of practical use for the time-evolving block decimation algorithm.

Precoder Design for Massive MIMO Downlink With Matrix Manifold Optimization

Precoder Design for Massive MIMO Downlink With Matrix Manifold Optimization

A cooperative spectrum sensing method based on semi-supervised clustering with variational mode decomposition and information geometry

In practice, obtaining a large number of sample labels for the supervised learning-based cooperative spectrum sensing (CSS) method is challenging. Additionally, the CSS method based on information geometry (IG) and clustering algorithm exhibited low detection performance in low signal-to-noise ratio (SNR) environment. This paper proposes a cooperative spectrum sensing method based on variational mode decomposition and IG with semi-supervised clustering. First, the secondary users (SUs) reconstruct the received signals using variational mode decomposition. Next, the fusion center combines all the reconstructed signals into a single matrix and calculates its corresponding covariance matrix. Finally, a binary classifier is obtained via training offline on the matrix manifold space using a semi-supervised clustering algorithm based on IG. The simulation results show that increasing the number of SUs and sampling points can improve the CSS performance of the proposed algorithm. Furthermore, the proposed algorithm is compared with other CSS algorithms, demonstrating its superior spectrum sensing performance regardless of whether the SUs are in the same very low SNR environment.

Tangent Space Based Alternating Projections for Nonnegative Low Rank Matrix Approximation

In this paper, we develop a new alternating projection method to compute nonnegative low rank matrix approximation for nonnegative matrices. In the nonnegative low rank matrix approximation method, the projection onto the manifold of fixed rank matrices can be expensive as the singular value decomposition is required. We propose to use the tangent space of the point in the manifold to approximate the projection onto the manifold in order to reduce the computational cost. We show that the sequence generated by the alternating projections onto the tangent spaces of the fixed rank matrices manifold and the nonnegative matrix manifold, converge linearly to a point in the intersection of the two manifolds where the convergent point is sufficiently close to optimal solutions. This convergence result based inexact projection onto the manifold is new and is not studied in the literature. Numerical examples in data clustering, pattern recognition and hyperspectral data analysis are given to demonstrate that the performance of the proposed method is better than that of nonnegative matrix factorization methods in terms of computational time and accuracy.

DRIPS: A framework for dimension reduction and interpolation in parameter space

Reduced-order models are often used to describe the behavior of complex systems, whose simulation with a full model is too expensive, or to extract salient features from the full model's output. We introduce a new model-reduction framework DRIPS (dimension reduction and interpolation in parameter space) that combines the offline local model reduction with the online parameter interpolation of reduced-order bases (ROBs). The offline step of this framework relies on dynamic mode decomposition (DMD) to build a low-rank linear surrogate model, equipped with a local ROB, for quantities of interest derived from the training data generated by repeatedly solving the (nonlinear) high-fidelity model for multiple parameter points. The online step consists of the construction of a parametric reduced-order model for each target/test point in the parameter space, with the interpolation of ROBs done on a Grassman manifold and the interpolation of reduced-order operators done on a matrix manifold. The DMD component enables DRIPS to model (typically low-dimensional) quantities of interest directly, without having to access the (typically high-dimensional and possibly nonlinear) operators in a high-fidelity model that governs the dynamics of the underlying high-dimensional state variables, as required in projection-based reduced-order modeling. A series of numerical experiments suggests that DRIPS yields a model reduction, which is computationally more efficient than the commonly used projection-based proper orthogonal decomposition; it does so without requiring a prior knowledge of the governing equation for quantities of interest. Moreover, for the nonlinear systems considered, DRIPS is more accurate than Gaussian-process interpolation (Kriging).

Tensor rank reduction via coordinate flows

Recently, there has been a growing interest in efficient numerical algorithms based on tensor networks and low-rank techniques to approximate high-dimensional functions and solutions to high-dimensional PDEs. In this paper, we propose a new tensor rank reduction method based on coordinate transformations that can greatly increase the efficiency of high-dimensional tensor approximation algorithms. The idea is simple: given a multivariate function, determine a coordinate transformation so that the function in the new coordinate system has smaller tensor rank. We restrict our analysis to linear coordinate transformations, which gives rise to a new class of functions that we refer to as tensor ridge functions. Leveraging Riemannian gradient descent on matrix manifolds we develop an algorithm that determines a quasi-optimal linear coordinate transformation for tensor rank reduction. The results we present for rank reduction via linear coordinate transformations open the possibility for generalizations to larger classes of nonlinear transformations.

Ship Detection in SAR Images via Kullback-Leibler Divergence-Based Matrix Information Geometry Detector

Ship detection in synthetic aperture radar (SAR) imagery is an important application in the field of marine remote sensing. As the most common method in ship detection, classical constant false alarm rate (CFAR) methods rely on the target-to-clutter contrast and accurate estimates of distribution models, making it difficult to adapt to complex and variable sea surface backgrounds. Considering the limitations of the CFAR methods, this paper proposes a new detector for ship detection based on matrix information geometry (MIG) theory. The proposed detector models each SAR sample data as a Hermitian positive-definite (HPD) matrix, and uses the geometric mean of all HPD matrices corresponding to the background window to estimate the clutter, thus transforming the target detection problem into a measure of the Kullback-Leibler divergence (KLD) between two points on the matrix manifold. Different from the traditional CFAR detectors which use intensity information to differentiate between targets and clutter, the proposed detector utilizes the intrinsic geometry of the matrix manifold and employs KLD with good differentiation to measure the variance from targets to clutter.

Cluster-Permutohedra and Submanifolds of Flag Varieties with Torus Actions

Abstract In this paper, we describe a relation between the notion of graphicahedron, introduced by Araujo-Pardo, Del Río-Francos, López-Dudet, Oliveros, and Schulte in 2010 and toric topology of manifolds of sparse isospectral Hermitian matrices. More precisely, we recall the notion of a cluster-permutohedron, a certain finite poset defined for a simple graph $\Gamma $. This poset is build as a combination of cosets of the symmetric group, and the geometric lattice of the graphical matroid of $\Gamma $. This poset is similar to the graphicahedron of $\Gamma $, in particular, 1-skeleta of both posets are isomorphic to Cayley graphs of the symmetric group. We describe the relation between cluster-permutohedron and graphicahedron using Galois connection and the notion of a core of a finite topology. We further prove that the face poset of the natural torus action on the manifold of isospectral $\Gamma $-shaped Hermitian matrices is isomorphic to the cluster-permutohedron. Using recent results in toric topology, we show that homotopy properties of graphicahedra may serve an obstruction to equivariant formality of isospectral matrix manifolds. We introduce a generalization of a cluster-permutohedron and describe the combinatorial structure of a large family of manifolds with torus actions, including Grassmann manifolds and partial flag manifolds.

External Identification of a Reciprocal Lossy Multiport Circuit under Measurement Uncertainties by Riemannian Gradient Descent

The present paper deals with the external identification of a reciprocal, special passive, 2n-port network under measurement uncertainties. In the present context, the multiport model is represented by an admittance matrix and the condition that the network is ‘reciprocal special passive’ refers to the assumption that the real part of the admittance matrix is symmetric and positive-definite. The key point is to reformulate the identification problem as a matrix optimization program over the matrix manifold S+(2n)×S(2n). The optimization problem requires a least-squares criterion function designed to cope with over-determinacy due to the incoherent data pairs whose cardinality exceeds the problem’s number of degrees of freedom. The present paper also proposes a numerical solution to such an optimization problem based on the Riemannian-gradient steepest descent method. The numerical results show that the proposed method is effective as long as reasonable measurement error levels and problem sizes are being dealt with.

Physics-informed dynamic mode decomposition

In this work, we demonstrate how physical principles—such as symmetries, invariances and conservation laws—can be integrated into thedynamic mode decomposition(DMD). DMD is a widely used data analysis technique that extracts low-rank modal structures and dynamics from high-dimensional measurements. However, DMD can produce models that are sensitive to noise, fail to generalize outside the training data and violate basic physical laws. Our physics-informed DMD (piDMD) optimization, which may be formulated as a Procrustes problem, restricts the family of admissible models to a matrix manifold that respects the physical structure of the system. We focus on five fundamental physical principles—conservation, self-adjointness, localization, causality and shift-equivariance—and derive several closed-form solutions and efficient algorithms for the corresponding piDMD optimizations. With fewer degrees of freedom, piDMD models are less prone to overfitting, require less training data, and are often less computationally expensive to build than standard DMD models. We demonstrate piDMD on a range of problems, including energy-preserving fluid flow, the Schrödinger equation, solute advection-diffusion and three-dimensional transitional channel flow. In each case, piDMD outperforms standard DMD algorithms in metrics such as spectral identification, state prediction and estimation of optimal forcings and responses.

Low-Rank Matrix Completion via QR-Based Retraction on Manifolds

Low-rank matrix completion aims to recover an unknown matrix from a subset of observed entries. In this paper, we solve the problem via optimization of the matrix manifold. Specially, we apply QR factorization to retraction during optimization. We devise two fast algorithms based on steepest gradient descent and conjugate gradient descent, and demonstrate their superiority over the promising baseline with the ratio of at least 24%.

Image Stitching With Manifold Optimization

Image stitching usually relies on spatial transformations to do the overlap alignment and distortion mitigation. This paper presents a novel manifold optimization method to seek for the optimal transformations. Especially, the transformations involved in the stitching are treated as elements of a prescribed matrix manifold. Then, the spatially varying homographies are computed by efficient second-order minimization (ESM) on the geometric error of aligning feature correspondences, but with their intrinsic manifold parameterization. To mitigate the distortion, the interpolation between homography and similarity transformation is performed on a general matrix manifold. These on-manifold operations improve the stitching quality with less artifacts of ghosting and distortion. The experiments show the effectiveness of our manifold optimization for image stitching.

LDA-MIG Detectors for Maritime Targets in Nonhomogeneous Sea Clutter

This paper deals with the problem of detecting maritime targets embedded in nonhomogeneous sea clutter, where limited number of secondary data is available due to the heterogeneity of sea clutter. A class of linear discriminant analysis (LDA)-based matrix information geometry (MIG) detectors is proposed in the supervised scenario. As customary, Hermitian positive-definite (HPD) matrices are used to model the observational sample data, and the clutter covariance matrix of received dataset is estimated as geometric mean of the secondary HPD matrices. Given a set of training HPD matrices with class labels, that are elements of a higher-dimensional HPD matrix manifold, the LDA manifold projection learns a mapping from the higher-dimensional HPD matrix manifold to a lower-dimensional one subject to maximum discrimination. In the current study, the LDA manifold projection, with the cost function maximizing between-class distance while minimizing within-class distance, is formulated as an optimization problem in the Stiefel manifold. Four robust LDA-MIG detectors corresponding to different geometric measures are proposed. Numerical results based on both simulated radar clutter with interferences and real IPIX radar data show the advantage of the proposed LDA-MIG detectors against their counterparts without using LDA as well as the state-of-art maritime target detection methods in nonhomogeneous sea clutter.

A Quantum-Behaved Particle Swarm Optimization Algorithm on Riemannian Manifolds

The Riemannian manifold optimization algorithms have been widely used in machine learning, computer vision, data mining, and other technical fields. Most of these algorithms are based on the geodesic or the retracement operator and use the classical methods (i.e., the steepest descent method, the conjugate gradient method, the Newton method, etc.) to solve engineering optimization problems. However, they lack the ability to solve non-differentiable mathematical models and ensure global convergence for non-convex manifolds. Considering this issue, this paper proposes a quantum-behaved particle swarm optimization (QPSO) algorithm on Riemannian manifolds named RQPSO. In this algorithm, the quantum-behaved particles are randomly distributed on the manifold surface and iteratively updated during the whole search process. Then, the vector transfer operator is used to translate the guiding vectors, which are not in the same Euclidean space, to the tangent space of the particles. Through the searching of these guiding vectors, we can achieve the retracement and update of points and finally obtain the optimized result. The proposed RQPSO algorithm does not depend on the expression form of a problem and could deal with various engineering technical problems, including both differentiable and non-differentiable ones. To verify the performance of RQPSO experimentally, we compare it with some traditional algorithms on three common matrix manifold optimization problems. The experimental results show that RQPSO has better performance than its competitors in terms of calculation speed and optimization efficiency.