Procrustes Problem Research Articles

On Procrustes Analysis in Hyperbolic Space

Congruent Procrustes analysis aims to find the best matching between two point sets through rotation, reflection and translation. We formulate the Procrustes problem for hyperbolic spaces, review the canonical definition of the center of point sets, and give a closed form solution for the optimal isometry for noise-free measurements. We also analyze the performance of the proposed method under measurement noise.

Generalized Embedding Regression: A Framework for Supervised Feature Extraction.

Sparse discriminative projection learning has attracted much attention due to its good performance in recognition tasks. In this article, a framework called generalized embedding regression (GER) is proposed, which can simultaneously perform low-dimensional embedding and sparse projection learning in a joint objective function with a generalized orthogonal constraint. Moreover, the label information is integrated into the model to preserve the global structure of data, and a rank constraint is imposed on the regression matrix to explore the underlying correlation structure of classes. Theoretical analysis shows that GER can obtain the same or approximate solution as some related methods with special settings. By utilizing this framework as a general platform, we design a novel supervised feature extraction approach called jointly sparse embedding regression (JSER). In JSER, we construct an intrinsic graph to characterize the intraclass similarity and a penalty graph to indicate the interclass separability. Then, the penalty graph Laplacian is used as the constraint matrix in the generalized orthogonal constraint to deal with interclass marginal points. Moreover, the L2,1 -norm is imposed on the regression terms for robustness to outliers and data's variations and the regularization term for jointly sparse projection learning, leading to interesting semantic interpretability. An effective iterative algorithm is elaborately designed to solve the optimization problem of JSER. Theoretically, we prove that the subproblem of JSER is essentially an unbalanced Procrustes problem and can be solved iteratively. The convergence of the designed algorithm is also proved. Experimental results on six well-known data sets indicate the competitive performance and latent properties of JSER.

MERACLE: Constructive Layer-Wise Conversion of a Tensor Train into a MERA

In this article, two new algorithms are presented that convert a given data tensor train into either a Tucker decomposition with orthogonal matrix factors or a multi-scale entanglement renormalization ansatz (MERA). The Tucker core tensor is never explicitly computed but stored as a tensor train instead, resulting in both computationally and storage efficient algorithms. Both the multilinear Tucker-ranks as well as the MERA-ranks are automatically determined by the algorithm for a given upper bound on the relative approximation error. In addition, an iterative algorithm with low computational complexity based on solving an orthogonal Procrustes problem is proposed for the first time to retrieve optimal rank-lowering disentangler tensors, which are a crucial component in the construction of a low-rank MERA. Numerical experiments demonstrate the effectiveness of the proposed algorithms together with the potential storage benefit of a low-rank MERA over a tensor train.

The quaternion-based spatial-coordinate and orientation-frame alignment problems.

The general problem of finding a global rotation that transforms a given set of spatial coordinates and/or orientation frames (the `test' data) into the best possible alignment with a corresponding set (the `reference' data) is reviewed. For 3D point data, this `orthogonal Procrustes problem' is often phrased in terms of minimizing a root-mean-square deviation (RMSD) corresponding to a Euclidean distance measure relating the two sets of matched coordinates. This article focuses on quaternion eigensystem methods that have been exploited to solve this problem for at least five decades in several different bodies of scientific literature, where they were discovered independently. While numerical methods for the eigenvalue solutions dominate much of this literature, it has long been realized that the quaternion-based RMSD optimization problem can also be solved using exact algebraic expressions based on the form of the quartic equation solution published by Cardano in 1545; focusing on these exact solutions exposes the structure of the entire eigensystem for the traditional 3D spatial-alignment problem. The structure of the less-studied orientation-data context is then explored, investigating how quaternion methods can be extended to solve the corresponding 3D quaternion orientation-frame alignment (QFA) problem, noting the interesting equivalence of this problem to the rotation-averaging problem, which also has been the subject of independent literature threads. The article concludes with a brief discussion of the combined 3D translation-orientation data alignment problem. Appendices are devoted to a tutorial on quaternion frames, a related quaternion technique for extracting quaternions from rotation matrices and a review of quaternion rotation-averaging methods relevant to the orientation-frame alignment problem. The supporting information covers novel extensions of quaternion methods to the 4D Euclidean spatial-coordinate alignment and 4D orientation-frame alignment problems, some miscellaneous topics, and additional details of the quartic algebraic eigenvalue problem.

Linear constraint problem of Hermitian unitary symplectic matrices

ABSTRACT In this paper, we consider a linear constraint problem of Hermitian unitary symplectic matrices and its approximation. By constructing a simple unitary matrix U, we verify that Hermitian unitary symplectic matrices are unitary similar to block diagonal Hermitian unitary matrices via U, which simplifies and is crucial to solving the linear constraint problem, and is a special feature of this paper. Then, we solve the linear constraint problem completely, that is deriving the sufficient and necessary conditions of it and inducing Hermitian unitary symplectic solutions to it. We also obtain its optimal approximate solutions. Furthermore, the Procrustes problem of Hermitian unitary symplectic matrices is considered when the linear constraint problem has no solution.

Spherical Regression Under Mismatch Corruption With Application to Automated Knowledge Translation

Motivated by a series of applications in data integration, language translation, bioinformatics, and computer vision, we consider spherical regression with two sets of unit-length vectors when the data are corrupted by a small fraction of mismatch in the response-predictor pairs. We propose a three-step algorithm in which we initialize the parameters by solving an orthogonal Procrustes problem to estimate a translation matrix ignoring the mismatch. We then estimate a mapping matrix aiming to correct the mismatch using hard-thresholding to induce sparsity, while incorporating potential group information. We eventually obtain a refined estimate for by removing the estimated mismatched pairs. We derive the error bound for the initial estimate of in both fixed and high-dimensional setting. We demonstrate that the refined estimate of achieves an error rate that is as good as if no mismatch is present. We show that our mapping recovery method not only correctly distinguishes one-to-one and one-to-many correspondences, but also consistently identifies the matched pairs and estimates the weight vector for combined correspondence. We examine the finite sample performance of the proposed method via extensive simulation studies, and with application to the unsupervised translation of medical codes using electronic health records data. Supplementary materials for this article are available online.

Orthogonal Procrustes and norm-dependent optimality

This note revisits the classical orthogonal Procrustes problem and investigates the norm-dependent geometric behavior underlying Procrustes alignment for subspaces. It presents generic, deterministic bounds quantifying the performance of a specified Procrustes-based choice of subspace alignment. Numerical examples illustrate the theoretical observations and offer additional, empirical findings which are discussed in detail. This note complements recent advances in statistics involving Procrustean matrix perturbation decompositions and eigenvector estimation.

Counting Stationary Points of the Loss Function in the Simplest Constrained Least-square Optimization

We use Kac-Rice method to analyze statistical features of an landscape of the loss function in a random version of the Oblique Procrustes Problem, one of the simplest optimization problems of the least-square type on a sphere.

Square Root-Based Multi-Source Early PSD Estimation and Recursive RETF Update in Reverberant Environments by Means of the Orthogonal Procrustes Problem

Multi-channel short-time Fourier transform (STFT) domain-based processing of reverberant microphone signals commonly relies on power-spectral-density (PSD) estimates of early source images, where early refers to reflections contained within the same STFT frame. State-of-the-art approaches to multi-source early PSD estimation, given an estimate of the associated relative early transfer functions (RETFs), conventionally minimize the approximation error defined with respect to the early correlation matrix, requiring non-negative inequality constraints on the PSDs. Instead, we here propose to factorize the early correlation matrix and minimize the approximation error defined with respect to the early-correlation-matrix square root. The proposed minimization problem -- constituting a generalization of the so-called orthogonal Procrustes problem -- seeks a unitary matrix and the square roots of the early PSDs up to an arbitrary complex argument, making non-negative inequality constraints redundant. A solution is obtained iteratively, requiring one singular value decomposition (SVD) per iteration. The estimated unitary matrix and early PSD square roots further allow to recursively update the RETF estimate, which is not inherently possible in the conventional approach. An estimate of the said early-correlation-matrix square root itself is obtained by means of the generalized eigenvalue decomposition (GEVD), where we further propose to restore non-stationarities by desmoothing the generalized eigenvalues in order to compensate for inevitable recursive averaging. Simulation results indicate fast convergence of the proposed multi-source early PSD estimation approach in only one iteration if initialized appropriately, and better performance as compared to the conventional approach.

An Eigenvalue-Based Method for the Unbalanced Procrustes Problem

In this paper, we propose a novel eigenvalue-based approach to solving the unbalanced orthogonal Procrustes problem. By making effective use of the necessary condition for the global minimizer and ...

Solution of symmetric positive semidefinite Procrustes problem

In this paper, the symmetric positive semidefinite Procrustes problem is considered. By using matrix inner product and matrix decomposition theory, an explicit expression of the solution is given. Based on the explicit expression given in this paper, it is easy to derive the explicit expression of the solution given by Woodgate [K.G. Woodgate. Least-squares solution of F = PG over positive semidefinite symmetric P . Linear Algebra Appl., 245:171–190, 1996.] and by Liao [A.P. Liao. On the least squares problem of a matrix equation. J. Comput. Math., 17:589–594, 1999.] for the Procrustes problem in some special cases. The explicit expression given in this paper also shows that the solution of the special inverse eigenvalue problem considered by Zhang [L. Zhang. A class of inverse eigenvalue problem for symmetric nonnegative definite matrices. J. Hunan Educational Inst., 2:11–17, 1995 (in Chinese).] can be computed exactly. Examples to illustrate the correctness of the theory results are given.

Solution of symmetric positive semidefinite Procrustes problem

In this paper, the symmetric positive semidefinite Procrustes problem is considered. By using matrix inner product and matrix decomposition theory, an explicit expression of the solution is given. Based on the explicit expression given in this paper, it is easy to derive the explicit expression of the solution given by Woodgate [K.G. Woodgate. Least-squares solution of F = PG over positive semidefinite symmetric P . Linear Algebra Appl., 245:171–190, 1996.] and by Liao [A.P. Liao. On the least squares problem of a matrix equation. J. Comput. Math., 17:589–594, 1999.] for the Procrustes problem in some special cases. The explicit expression given in this paper also shows that the solution of the special inverse eigenvalue problem considered by Zhang [L. Zhang. A class of inverse eigenvalue problem for symmetric nonnegative definite matrices. J. Hunan Educational Inst., 2:11–17, 1995 (in Chinese).] can be computed exactly. Examples to illustrate the correctness of the theory results are given.

Characterization and Computation of Matrices of Maximal Trace over Rotations

The constrained orthogonal Procrustes problem is the least-squares problem that calls for a rotation matrix that optimally aligns two corresponding sets of points in d-dimensional Euclidean space. This problem generalizes to the so-called Wahba's problem which is the same problem with nonnegative weights. Given a dxd matrix M, solutions to these problems are intimately related to the problem of finding a dxd rotation matrix U that maximizes the trace of UM, i.e., that makes UM a matrix of maximal trace over rotations, and it is well known this can be achieved with a method based on the computation of the singular value decomposition (SVD) of M. As the main goal of this paper, we characterize dxd matrices of maximal trace over rotation matrices in terms of their eigenvalues, and for d = 2, 3, we show how this characterization can be used to determine whether a matrix is of maximal trace over rotation matrices. Finally, although depending only slightly on the characterization, as a secondary goal of the paper, for d = 2, 3, we identify alternative ways, other than the SVD, of obtaining solutions to the aforementioned problems.

A Novel Orthogonal Extreme Learning Machine for Regression and Classification Problems

An extreme learning machine (ELM) is an innovative algorithm for the single hidden layer feed-forward neural networks and, essentially, only exists to find the optimal output weight so as to minimize output error based on the least squares regression from the hidden layer to the output layer. With a focus on the output weight, we introduce the orthogonal constraint into the output weight matrix, and propose a novel orthogonal extreme learning machine (NOELM) based on the idea of optimization column by column whose main characteristic is that the optimization of complex output weight matrix is decomposed into optimizing the single column vector of the matrix. The complex orthogonal procrustes problem is transformed into simple least squares regression with an orthogonal constraint, which can preserve more information from ELM feature space to output subspace, these make NOELM more regression analysis and discrimination ability. Experiments show that NOELM has better performance in training time, testing time and accuracy than ELM and OELM.

A Purely Algebraic Justification of the Kabsch-Umeyama Algorithm.

The constrained orthogonal Procrustes problem is the least-squares problem that calls for a rotation matrix that optimally aligns two matrices of the same order. Over past decades, the algorithm of choice for solving this problem has been the Kabsch-Umeyama algorithm, which is effectively no more than the computation of the singular value decomposition of a particular matrix. Its justification, as presented separately by Kabsch and Umeyama, is not totally algebraic since it is based on solving the minimization problem via Lagrange multipliers. In order to provide a more transparent alternative, it is the main purpose of this paper to present a purely algebraic justification of the algorithm through the exclusive use of simple concepts from linear algebra. For the sake of completeness, a proof is also included of the well known and widely used fact that the orientation-preserving rigid motion problem, i.e., the least-squares problem that calls for an orientation-preserving rigid motion that optimally aligns two corresponding sets of points in d-dimensional Euclidean space, reduces to the constrained orthogonal Procrustes problem.

On the equivariance properties of self-adjoint matrices

We investigate self-adjoint matrices with respect to their equivariance properties. We show in particular that a matrix is self-adjoint if and only if it is equivariant with respect to the action of a group which is isomorphic to . If the self-adjoint matrix possesses multiple eigenvalues – this may, for instance, be induced by symmetry properties of an underlying dynamical system – then A is even equivariant with respect to the action of a group where are the multiplicities of the eigenvalues of A. We discuss implications of this result for equivariant bifurcation problems, and we briefly address further applications for the Procrustes problem, graph symmetries and Taylor expansions.

Hand-Eye Calibration: 4-D Procrustes Analysis Approach

We give a universal analytical solution to the hand-eye calibration problem ${AX} = {XB}$ with known matrices ${A}$ and ${B}$ and unknown variable ${X}$ , all in the set of special Euclidean group SE(3). The developed method relies on the 4-D Procrustes analysis. A unit-octonion representation is proposed for the first time to solve such a Procrustes problem through which an optimal closed-form eigendecomposition solution is derived. By virtue of such a solution, the uncertainty description of ${X}$ , being a sophisticated problem previously, can be solved in a simpler manner. The proposed approach is then verified using simulations and real-world experimentations on an industrial robotic arm. The results indicate that it owns better accuracy and better description of uncertainty and consumes much less computation time.

Cross-resolution face recognition with pose variations via multilayer locality-constrained structural orthogonal procrustes regression

In real video surveillance scenes, the extracted face regions generally have low-resolution (LR) and are sensitive to pose and illumination variations; these flaws undoubtedly degrade the subsequent recognition task. To overcome these challenges, we propose an approach named multilayer locality-constrained structural orthogonal Procrustes regression (MLCSOPR). The proposed MLCSOPR not only learns the pose-robust discriminative representation features to reduce the resolution gap between the LR image space and the high-resolution (HR) one but also strengthens the consistency between the LR and HR image space. In particular, several contributions are made in this paper: (i) Inspired by the orthogonal Procrustes problem (OPP), a matrix approximation is exploited to find an optimal correction between two data matrices. (ii) The nuclear norm constraint is applied to the reconstruction error to maintain the structural property. (iii) Based on the abovementioned learned resolution-robust representation features, a linear regression-based classification strategy is adopted to recognize the LR input face images. Experiments on commonly used face databases have shown the effectiveness of the proposed method on cross-resolution face matching with pose variations.

Characterization and Computation of Matrices of Maximal Trace Over Rotations

The constrained orthogonal Procrustes problem is the least-squares problem that calls for a rotation matrix that optimally aligns two corresponding sets of points in d-dimensional Euclidean space. This problem generalizes to the so-called Wahba's problem which is the same problem with nonnegative weights. Given a dxd matrix M, solutions to these problems are intimately related to the problem of finding a dxd rotation matrix U that maximizes the trace of UM, i.e., that makes UM a matrix of maximal trace over rotations, and it is well known this can be achieved with a method based on the computation of the singular value decomposition (SVD) of M. As the main goal of this paper, we characterize dxd matrices of maximal trace over rotation matrices in terms of their eigenvalues, and for d = 2, 3, we show how this characterization can be used to determine whether a matrix is of maximal trace over rotation matrices. Finally, although depending only slightly on the characterization, as a secondary goal of the paper, for d = 2, 3, we identify alternative ways, other than the SVD, of obtaining solutions to the aforementioned problems.

Semi-sparse PCA.

It is well known that the classical exploratory factor analysis (EFA) of data with more observations than variables has several types of indeterminacy. We study the factor indeterminacy and show some new aspects of this problem by considering EFA as a specific data matrix decomposition. We adopt a new approach to the EFA estimation and achieve a new characterization of the factor indeterminacy problem. A new alternative model is proposed, which gives determinate factors and can be seen as a semi-sparse principal component analysis (PCA). An alternating algorithm is developed, where in each step a Procrustes problem is solved. It is demonstrated that the new model/algorithm can act as a specific sparse PCA and as a low-rank-plus-sparse matrix decomposition. Numerical examples with several large data sets illustrate the versatility of the new model, and the performance and behaviour of its algorithmic implementation.