Matrix Manifold Research Articles

Real-time solution of linear computational problems using databases of parametric reduced-order models with arbitrary underlying meshes

A comprehensive approach for real-time computations using a database of parametric, linear, projection-based reduced-order models (ROMs) based on arbitrary underlying meshes is proposed. In the offline phase of this approach, the parameter space is sampled and linear ROMs defined by linear reduced operators are pre-computed at the sampled parameter points and stored. Then, these operators and associated ROMs are transformed into counterparts that satisfy a certain notion of consistency. In the online phase of this approach, a linear ROM is constructed in real-time at a queried but unsampled parameter point by interpolating the pre-computed linear reduced operators on matrix manifolds and therefore computing an interpolated linear ROM. The proposed overall model reduction framework is illustrated with two applications: a parametric inverse acoustic scattering problem associated with a mockup submarine, and a parametric flutter prediction problem associated with a wing-tank system. The second application is implemented on a mobile device, illustrating the capability of the proposed computational framework to operate in real-time.

Interpretable domain adaptation via optimization over the Stiefel manifold

In domain adaptation, the goal is to find common ground between two, potentially differently distributed, data sets. By finding common concepts present in two sets of words pertaining to different domains, one could leverage the performance of a classifier for one domain for use on the other domain. We propose a solution to the domain adaptation task, by efficiently solving an optimization problem through Stochastic Gradient Descent. We provide update rules that allow us to run Stochastic Gradient Descent directly on a matrix manifold: the steps compel the solution to stay on the Stiefel manifold. This manifold encompasses projection matrices of word vectors onto low-dimensional latent feature representations, which allows us to interpret the results: the rotation magnitude of the word vector projection for a given word corresponds to the importance of that word towards making the adaptation. Beyond this interpretability benefit, experiments show that the Stiefel manifold method performs better than state-of-the-art methods.

Computing Laser Beam Paths in Optical Cavities: An Approach Based on Geometric Newton Method

In the last decade, increasing attention has been drawn to high-precision optical experiments, which push resolution and accuracy of the measured quantities beyond their current limits. This challenge requires to place optical elements (e.g., mirrors, lenses) and to steer light beams with subnanometer precision. Existing methods for beam direction computing in resonators, e.g., iterative ray tracing or generalized ray transfer matrices, are either computationally expensive or rely on overparameterized models of optical elements. By exploiting Fermat's principle, we develop a novel method to compute the steady-state beam configurations in resonant optical cavities formed by spherical mirrors, as a function of mirror positions and curvature radii. The proposed procedure is based on the geometric Newton method on matrix manifold, a tool with second-order convergence rate, that relies on a second-order model of the cavity optical length. As we avoid coordinates to parametrize the beam position on mirror surfaces, the computation of the second-order model does not involve the second derivatives of the parametrization. With the help of numerical tests, we show that the convergence properties of our procedure hold for non-planar polygonal cavities, and we assess the effectiveness of the geometric Newton method in determining their configurations with high degree of accuracy and negligible computational effort.

A Riemannian rank-adaptive method for low-rank optimization

This paper presents an algorithm that solves optimization problems on a matrix manifold M⊆Rm×n with an additional rank inequality constraint. The algorithm resorts to well-known Riemannian optimization schemes on fixed-rank manifolds, combined with new mechanisms to increase or decrease the rank. The convergence of the algorithm is analyzed and a weighted low-rank approximation problem is used to illustrate the efficiency and effectiveness of the algorithm.

Online SSVEP-based BCI using Riemannian geometry

Challenges for the next generation of Brain Computer Interfaces (BCI) are to mitigate the common sources of variability (electronic, electrical, biological) and to develop online and adaptive systems following the evolution of the subject׳s brain waves. Studying electroencephalographic (EEG) signals from their associated covariance matrices allows the construction of a representation which is invariant to extrinsic perturbations. As covariance matrices should be estimated, this paper first presents a thorough study of all estimators conducted on real EEG recording. Working in Euclidean space with covariance matrices is known to be error-prone, one might take advantage of algorithmic advances in Riemannian geometry and matrix manifold to implement methods for Symmetric Positive-Definite (SPD) matrices. Nonetheless, existing classification algorithms in Riemannian spaces are designed for offline analysis. We propose a novel algorithm for online and asynchronous processing of brain signals, borrowing principles from semi-unsupervised approaches and following a dynamic stopping scheme to provide a prediction as soon as possible. The assessment is conducted on real EEG recording: this is the first study on Steady-State Visually Evoked Potential (SSVEP) experimentations to exploit online classification based on Riemannian geometry. The proposed online algorithm is evaluated and compared with state-of-the-art SSVEP methods, which are based on Canonical Correlation Analysis (CCA). It is shown to improve both the classification accuracy and the information transfer rate in the online and asynchronous setup.

Riemannian Preconditioning

This paper exploits a basic connection between sequential quadratic programming and Riemannian gradient optimization to address the general question of selecting a metric in Riemannian optimization, in particular when the Riemannian structure is sought on a quotient manifold. The proposed method is shown to be particularly insightful and efficient in quadratic optimization with orthogonality and/or rank constraints, which covers most current applications of Riemannian optimization in matrix manifolds.

A Riemannian Fletcher--Reeves Conjugate Gradient Method for Doubly Stochastic Inverse Eigenvalue Problems

We consider the inverse eigenvalue problem of reconstructing a doubly stochastic matrix from the given spectrum data. We reformulate this inverse problem as a constrained nonlinear least squares problem over several matrix manifolds, which minimizes the distance between isospectral matrices and doubly stochastic matrices. Then a Riemannian Fletcher--Reeves conjugate gradient method is proposed for solving the constrained nonlinear least squares problem, and its global convergence is established. An extra gain is that a new Riemannian isospectral flow method is obtained. Our method is also extended to the case of prescribed entries. Finally, some numerical tests are reported to illustrate the efficiency of the proposed method.

A Geometric Nonlinear Conjugate Gradient Method for Stochastic Inverse Eigenvalue Problems

In this paper, we focus on the stochastic inverse eigenvalue problem of reconstructing a stochastic matrix from the prescribed spectrum. We directly reformulate the stochastic inverse eigenvalue problem as a constrained optimization problem over several matrix manifolds to minimize the distance between isospectral matrices and stochastic matrices. Then we propose a geometric Polak--Ribière--Polyak-based nonlinear conjugate gradient method for solving the constrained optimization problem. The global convergence of the proposed method is established. Our method can also be extended to the stochastic inverse eigenvalue problem with prescribed entries. An extra advantage is that our models yield new isospectral flow methods. Finally, we report some numerical tests to illustrate the efficiency of the proposed method for solving the stochastic inverse eigenvalue problem and the case of prescribed entries.

Clustering Tree-Structured Data on Manifold.

Tree-structured data usually contain both topological and geometrical information, and are necessarily considered on manifold instead of euclidean space for appropriate data parameterization and analysis. In this study, we propose a novel tree-structured data parameterization, called Topology-Attribute matrix (T-A matrix), so the data clustering task can be conducted on matrix manifold. We incorporate the structure constraints embedded in data into the non-negative matrix factorization method to determine meta-trees from the T-A matrix, and the signature vector of each single tree can then be extracted by meta-tree decomposition. The meta-tree space turns out to be a cone space, in which we explore the distance metric and implement the clustering algorithm based on the concepts like Fréchet mean. Finally, the T-A matrix based clustering (TAMBAC) framework is evaluated and compared using both simulated data and real retinal images to illustrate its efficiency and accuracy.

Method of Gearbox Fault Diagnosis Based on Covariance Matrix Manifold Entropy Feature

Method of Gearbox Fault Diagnosis Based on Covariance Matrix Manifold Entropy Feature

A geometrical numerical linear algebra approach to residual‐enhanced parametric model reduction

AbstractThe most prominent approaches to model reduction share the general principle, that a given large‐scale system is projected onto a suitable subspace spanned by a low‐dimensional basis. The projection basis essentially determines the approximation quality of the resulting reduced order system. Nonlinear parametric dependencies may be taken into account by applying interpolation techniques on matrix manifolds for computing projection bases and/or reduced systems at arbitrary parameter conditions, see [1] for a recent survey. In this short note, an original approach to enhance the prediction capabilities of the interpolated reduced‐order projection bases for parametrized linear systems is proposed. To this end, a suitable residual‐based goal function is introduced, which measures the approximation quality of a given projection subspace. As this function is invariant under changes of basis, it is to be considered as a mapping on the Grassmann‐Manifold Gr(n, p) of p‐dimensional subspaces of ℝn. As such, it may be optimized using the conjugate gradient method specifically adapted to the manifold structure of the Grassmannian Gr(n, p), [2,3]. (© 2014 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Combinatorial Optimization with Information Geometry: The Newton Method

We discuss the use of the Newton method in the computation of max(p → Εp [f]), where p belongs to a statistical exponential family on a finite state space. In a number of papers, the authors have applied first order search methods based on information geometry. Second order methods have been widely used in optimization on manifolds, e.g., matrix manifolds, but appear to be new in statistical manifolds. These methods require the computation of the Riemannian Hessian in a statistical manifold. We use a non-parametric formulation of information geometry in view of further applications in the continuous state space cases, where the construction of a proper Riemannian structure is still an open problem.

Auto-regressive moving-average discrete-time dynamical systems and autocorrelation functions on real-valued Riemannian matrix manifolds

The present research paper proposes an extension of the classical scalar Auto-Regressive Moving-Average (ARMA) model to real-valued Riemannian matrix manifolds. The resulting ARMA model on matrix manifolds is expressed as a non-linear discrete-time dynamical system in state-space form whose state evolves on the tangent bundle associated with the underlying manifold. A number of examples are discussed within the present contribution that aim at illustrating the numerical behavior of the proposed ARMA model. In order to measure the degree of temporal dependency between the state-values of the ARMA model, an extension of the classical autocorrelation function for scalar sequences is suggested on the basis of the geometrical features of the underlying real-valued matrix manifold.

Low-Rank Matrix Completion by Riemannian Optimization

The matrix completion problem consists of finding or approximating a low-rank matrix based on a few samples of this matrix. We propose a new algorithm for matrix completion that minimizes the least-square distance on the sampling set over the Riemannian manifold of fixed-rank matrices. The algorithm is an adaptation of classical nonlinear conjugate gradients, developed within the framework of retraction-based optimization on manifolds. We describe all the necessary objects from differential geometry necessary to perform optimization over this low-rank matrix manifold, seen as a submanifold embedded in the space of matrices. In particular, we describe how metric projection can be used as retraction and how vector transport lets us obtain the conjugate search directions. Finally, we prove convergence of a regularized version of our algorithm under the assumption that the restricted isometry property holds for incoherent matrices throughout the iterations. The numerical experiments indicate that our approach scales very well for large-scale problems and compares favorably with the state-of-the-art, while outperforming most existing solvers.

SIGEST

This issue's SIGEST paper, “The Trace Ratio Optimization Problem,” by T. T. Ngo, M. Bellalij, and Y. Saad, is from the SIAM Journal on Matrix Analysis and Applications. The mathematical problem is to maximize a nonlinear function over a matrix manifold. The difficult formulation, tackled in this paper, has as its data two $p \times p$ positive definite matrices $A$ and $B$. The problem is to maximize the quotient of \[ \fracTr(V^T A V)Tr(V^T B V) \] over the manifold of $n \times p$ matrices with orthonormal columns. A different and simpler formulation adds the constraint $\mbox{Tr}(V^T B V) = 1$. Many previous papers considered this form more tractable. However, the results are not as useful as those from the quotient formulation. In this paper the authors show how to maximize the quotient more efficiently than was possible even for the simplified formulation. The (very elegant) algorithm's two important parts are a Lanczos process to find the $p$ largest eigenvalues of an $n \times n$ matrix and a Newton iteration for a scalar nonlinear equation. The application is to supervised learning. A classic example is Fischer linear discriminant analysis, where one seeks to identify a hyperplane that separates two or more datasets. The solution $V$ is a projection of the data onto an “optimal” lower dimensional space of dimension $p$. The matrices $A$ and $B$ are built from the data. The numerator in the fraction represents the quality of separation of the classes, and the denominator how well each class is clustered in the projected space. These authors clearly quantify these concepts in the introduction, which is a generous expansion of the introduction of the original paper. We teach the Lanczos process and Newton's method in introductory courses. These two ideas are the basis for the results in this paper. This issue's SIGEST paper is a compelling example of how methods from elementary courses can be applied to very nontrivial problems and produce surprising results.

Projection-like Retractions on Matrix Manifolds

This paper deals with constructing retractions, a key step when applying optimization algorithms on matrix manifolds. For submanifolds of Euclidean spaces, we show that the operation consisting of taking a tangent step in the embedding Euclidean space followed by a projection onto the submanifold is a retraction. We also show that the operation remains a retraction if the projection is generalized to a projection-like procedure that consists of coming back to the submanifold along “admissible” directions, and we give a sufficient condition on the admissible directions for the generated retraction to be second order. This theory offers a framework in which previously proposed retractions can be analyzed, as well as a toolbox for constructing new ones. Illustrations are given for projection-like procedures on some specific manifolds for which we have an explicit, easy-to-compute expression.

Advances in matrix manifolds for computer vision

The attention paid to matrix manifolds has grown considerably in the computer vision community in recent years. There are a wide range of important applications including face recognition, action recognition, clustering, visual tracking, and motion grouping and segmentation. The increased popularity of matrix manifolds is due partly to the need to characterize image features in non-Euclidean spaces. Matrix manifolds provide rigorous formulations allowing patterns to be naturally expressed and classified in a particular parameter space. This paper gives an overview of common matrix manifolds employed in computer vision and presents a summary of related applications. Researchers in computer vision should find this survey beneficial due to the overview of matrix manifolds, the discussion as well as the collective references.

DINDSCAL: direct INDSCAL

The well-known INDSCAL model for simultaneous metric multidimensional scaling (MDS) of three-way data analyzes doubly centered matrices of squared dissimilarities. An alternative approach, called for short DINDSCAL, is proposed for analyzing directly the input matrices of squared dissimilarities. An important consequence is that missing values can be easily handled. The DINDSCAL problem is solved by means of the projected gradient approach. First, the problem is transformed into a gradient dynamical system on a product matrix manifold (of Stiefel sub-manifold of zero-sum matrices and non-negative diagonal matrices). The constructed dynamical system can be numerically integrated which gives a globally convergent algorithm for solving the DINDSCAL. The DINDSCAL problem and its solution are illustrated by well-known data routinely used in metric MDS and INDSCAL. Alternatively, the problem can also be solved by iterative algorithm based on the conjugate (projected) gradient method, which MATLAB implementation is enclosed as an appendix.

Isospectral flow method for nonnegative inverse eigenvalue problem with prescribed structure

The nonnegative inverse eigenvalue problem is that given a family of complex numbers λ = { λ 1 , … , λ n } , find a nonnegative matrix of order n with spectrum λ . This problem is difficult and remains unsolved partially. In this paper, we focus on its generalization that the reconstructed nonnegative matrices should have some prescribed entries. It is easy to see that this new problem will come back to the common nonnegative inverse eigenvalue problem if there is no constraint of the locations of entries. A numerical isospectral flow method which is developed by hybridizing the optimization theory and steepest descent method is used to study the reconstruction. Moreover, an error estimate of the numerical iteration for ordinary differential equations on the matrix manifold is presented. After that, a numerical method for the nonnegative symmetric inverse eigenvalue problem with prescribed entries and its error estimate are considered. Finally, the approaches are verified by the numerical test results.

An Online Method for Interpolating Linear Parametric Reduced-Order Models

A two-step online method is proposed for interpolating projection-based linear parametric reduced-order models (ROMs) in order to construct a new ROM for a new set of parameter values. The first step of this method transforms each precomputed ROM into a consistent set of generalized coordinates. The second step interpolates the associated linear operators on their appropriate matrix manifold. Real-time performance is achieved by precomputing inner products between the reduced-order bases underlying the precomputed ROMs. The proposed method is illustrated by applications in mechanical and aeronautical engineering. In particular, its robustness is demonstrated by its ability to handle the case where the sampled parameter set values exhibit a mode veering phenomenon.