Orthogonal Procrustes Problem Research Articles

An accelerated hybrid Riemannian conjugate gradient method for unconstrained optimization

With the goal to deal with a series of optimization problems on general matrix manifolds with differentiable objective functions, we propose an accelerated hybrid Riemannian conjugate gradient technique. Specifically, the acceleration scheme of the proposed method using a modified stepsize which is multiplicatively determined by the Wolfe line search. The search direction of the proposed algorithm is determined by the hybrid conjugate parameter with computationally promising. We showed that the suggested approach converges globally to a stationary point. Our approach performs better than the state of art Riemannian conjugate gradient algorithms, as illustrated by computations on problems such as the orthogonal Procrustes problem and the Brockett-cost-function minimization problem.

Computing the completely positive factorization via alternating minimization

AbstractIn this article, we propose a novel alternating minimization scheme for finding completely positive factorizations. In each iteration, our method splits the original factorization problem into two optimization subproblems, the first one being a orthogonal procrustes problem, which is taken over the orthogoal group, and the second one over the set of entrywise positive matrices. We present both a convergence analysis of the method and favorable numerical results.

Orthogonal Procrustes and Machine Learning: Predicting Bill of Materials errors on time

In an industrial product development process, the Bill of Materials (BOM) is a hierarchical, multi-level representation of all components, parts and quantities of a product. With increasing complexity of industrial products, also BOMs become more complex and thus prone to errors, for example when the individual parts of a product are changed during the product development process. Frequently, these Bill of Materials errors have to be identified manually or by using simple, rule-based schemes. In this paper, we provide a technical background of BOMs, showing the intricacy of temporal BOMs errors in an industrial product development process. The work of other authors, which focused on association mining and tree reconciliation to detect Bill of Materials errors, is analysed. We found that there is currently no system being able to prescribe where in a Bill of Materials and when in the product development process, errors are probable to occur. Also, Machine Learning (ML) methods have not been applied yet. Based on these findings, we formalize the notions Bill of Materials and Bill of Materials errors. Furthermore, we present a deterministic distance measure for BOMS. We provide an answer to the main question of how to represent a Bill of Materials for Machine Learning tasks by solving the orthogonal Procrustes problem for dynamic, hierarchical datasets. Then, we describe an isolation forest based approach to temporal anomaly detection, which points at potential errors in a Bill of Materials at a specific timestamp. Furthermore, we apply Machine Learning and present a multi-output Multi Layer Perceptron for the prediction of temporal Bill of Materials errors. The model predicts where and at which point of time Bill of Materials errors are probable to occur, which renders it a prescriptive system. Eventually, we optimize the performance of our model using contextualization via k-means clustering. Finally, we apply our prescriptive pipeline to a real world dataset and show its superiority to existing methods using a qualitative comparison.

Near-optimal bounds for generalized orthogonal Procrustes problem via generalized power method

Given multiple point clouds, how to find the rigid transform (rotation, reflection, and shifting) such that these point clouds are well aligned? This problem, known as the generalized orthogonal Procrustes problem (GOPP), has found numerous applications in statistics, computer vision, and imaging science. While one commonly-used method is finding the least squares estimator, it is generally an NP-hard problem to obtain the least squares estimator exactly due to the notorious nonconvexity. In this work, we apply the semidefinite programming (SDP) relaxation and the generalized power method to solve this generalized orthogonal Procrustes problem. In particular, we assume the data are generated from a signal-plus-noise model: each observed point cloud is a noisy copy of the same unknown point cloud transformed by an unknown orthogonal matrix and also corrupted by additive Gaussian noise. We show that the generalized power method (equivalently alternating minimization algorithm) with spectral initialization converges to the unique global optimum to the SDP relaxation, provided that the signal-to-noise ratio is high. Moreover, this limiting point is exactly the least squares estimator and also the maximum likelihood estimator. Our theoretical bound is near-optimal in terms of the information-theoretic limit (only loose by a factor of the dimension and a log factor). Our results significantly improve the state-of-the-art results on the tightness of the SDP relaxation for the generalized orthogonal Procrustes problem, an open problem posed by Bandeira et al. (2014) [8].

The mpEDMD Algorithm for Data-Driven Computations of Measure-Preserving Dynamical Systems

Koopman operators globally linearize nonlinear dynamical systems and their spectral information is a powerful tool for the analysis and decomposition of nonlinear dynamical systems. However, Koopman operators are infinite dimensional, and computing their spectral information is a considerable challenge. We introduce measure-preserving extended dynamic mode decomposition (mpEDMD), the first Galerkin method whose eigendecomposition converges to the spectral quantities of Koopman operators for general measure-preserving dynamical systems. mpEDMD is a data-driven algorithm based on an orthogonal Procrustes problem that enforces measure-preserving truncations of Koopman operators using a general dictionary of observables. It is flexible and easy to use with any preexisting dynamic mode decomposition (DMD)-type method, and with different types of data. We prove convergence of mpEDMD for projection-valued and scalar-valued spectral measures, spectra, and Koopman mode decompositions. For the case of delay embedding (Krylov subspaces), our results include the first convergence rates of the approximation of spectral measures as the size of the dictionary increases. We demonstrate mpEDMD on a range of challenging examples, its increased robustness to noise compared to other DMD-type methods, and its ability to capture the energy conservation and cascade of a turbulent boundary layer flow with Reynolds number and state-space dimension .

Error Calibration of Cross Magnetic Gradient Tensor System with Total Least‐Squares Method

The magnetic gradient tensor system configured by fluxgate magnetometers is subjected to different scale factors, three‐axis nonorthogonality, bias, and misalignment errors. All those errors above will influence the measurement precision directly, so the magnetic gradient tensor system must be calibrated before use. In this paper, an error calibration method of the magnetic gradient tensor system is proposed. The procedure of the proposed method is as follows. Firstly, the error calibration model of the single fluxgate magnetometer is established and generated an ellipsoid mathematical expression. To simplify the calculation, the ellipsoid mathematical expression is transformed into a linear model of intermediate variables, and then, error parameters are estimated by the total least‐squares method. Secondly, the orthogonal procrustes problem is adopted to calibrate misalignment errors. Finally, simulations and experiments with a cross magnetic gradient tensor system are carried out for verification of the proposed error calibration method. Results show that compared with the original least‐squares method, the proposed method can increase the measurement accuracy of the cross magnetic gradient tensor system greatly.

Numerical strategies for recursive least squares solutions to the matrix equation AX = B

The recursive solution to the Procrustes problem -with or without constraints- is thoroughly investigated. Given known matrices and , the proposed solution minimizes the square of the Frobenius norm of the difference when rows or columns are added to and . The proposed method is based on efficient strategies which reduce the computational cost by utilizing previous computations when new data are acquired. This is particularly useful in the iterative solution of an unbalanced orthogonal Procrustes problem. The results show that the computational efficiency of the proposed recursive algorithms is more significant when the dimensions of the matrices are large. This demonstrates the usefulness of the proposed algorithms in the presence of high-dimensional data sets. The practicality of the new method is demonstrated through an application in machine learning, namely feature extraction for image processing.

A Convex Relaxation Approach for Learning the Robust Koopman Operator

Although data-driven models, especially deep learning, have achieved astonishing results on many prediction tasks for nonlinear sequences, challenges still remain in finding an appropriate way to embed prior knowledge of physical dynamics in these models. In this work, we introduce a convex relaxation approach for learning robust Koopman operators of nonlinear dynamical systems, which are intended to construct approximate space spanned by eigenfunctions of the Koopman operator. Different from the classical dynamic mode decomposition, we use the layer weights of neural networks as eigenfunctions of the Koopman operator, providing intrinsic coordinates that globally linearize the dynamics. We find that the approximation of space can be regarded as an orthogonal Procrustes problem on the Stiefel manifold, which is highly sensitive to noise. The key contribution of this paper is to demonstrate that strict orthogonal constraint can be replaced by its convex relaxation, and the performance of the model can be improved without increasing the complexity when dealing with both clean and noisy data. After that, the overall model can be optimized via backpropagation in an end-to-end manner. The comparisons of the proposed method against several state-of-the-art competitors are shown on nonlinear oscillators and the lid-driven cavity flow.

Attitude Uncertainty Analysis of a Three-Vehicle Constrained Formation.

The uncertainty analysis of attitude estimates enables the comparison between different methods, and, thus, it is important for practical applications. This work studies the uncertainty for the attitude determination of a three-vehicle constrained formation. Moreover, the existing solution is improved by including the uncertainty results in a weighted orthogonal Procrustes problem. In the formation considered herein, the vehicles measure inertial references and relative line-of-sight vectors. Nonetheless, the line of sight between two elements of the formation is restricted. The uncertainty analysis uses perturbation theory and, consequently, considers a small first-order perturbation in the measurements. The covariance matrices are obtained for all relative and inertial attitude candidates from the linearization of the solution using a first-order Taylor expansion. Then, the uncertainty is completed by considering the covariance for the weighted orthogonal Procrustes problem, from the literature, and the definition of covariance for the remaining attitudes. The uncertainty characterization is valid for configurations with a unique solution. Finally, the theoretical results are validated by applying Monte Carlo simulations, which show that the predicted errors are statistically consistent with the numerical implementation of the solution with noise. Furthermore, the theoretical uncertainty predicts the accuracy changes near special configurations where there is loss of information.

Detecting Lexical Semantic Change across Corpora with Smooth Manifolds (Student Abstract)

Comparing two bodies of text and detecting words with significant lexical semantic shift between them is an important part of digital humanities. Traditional approaches have relied on aligning the different embeddings using the Orthogonal Procrustes problem in the Euclidean space. This study presents a geometric framework that leverages smooth Riemannian manifolds for corpus-specific orthogonal rotations and a corpus-independent scaling metric to project the different vector spaces into a shared latent space. This enables us to capture any affine relationship between the embedding spaces while utilising the rich geometry of smooth manifolds.

The generalized orthogonal Procrustes problem in the high noise regime

AbstractWe consider the problem of estimating a cloud of points from numerous noisy observations of that cloud after unknown rotations and possibly reflections. This is an instance of the general problem of estimation under group action, originally inspired by applications in three-dimensional imaging and computer vision. We focus on a regime where the noise level is larger than the magnitude of the signal, so much so that the rotations cannot be estimated reliably. We propose a simple and efficient procedure based on invariant polynomials (effectively: the Gram matrices) to recover the signal, and we assess it against fundamental limits of the problem that we derive. We show our approach adapts to the noise level and is statistically optimal (up to constants) for both the low and high noise regimes. In studying the variance of our estimator, we encounter the question of the sensivity of a type of thin Cholesky factorization, for which we provide an improved bound which may be of independent interest.

MERACLE: Constructive Layer-Wise Conversion of a Tensor Train into a MERA

In this article, two new algorithms are presented that convert a given data tensor train into either a Tucker decomposition with orthogonal matrix factors or a multi-scale entanglement renormalization ansatz (MERA). The Tucker core tensor is never explicitly computed but stored as a tensor train instead, resulting in both computationally and storage efficient algorithms. Both the multilinear Tucker-ranks as well as the MERA-ranks are automatically determined by the algorithm for a given upper bound on the relative approximation error. In addition, an iterative algorithm with low computational complexity based on solving an orthogonal Procrustes problem is proposed for the first time to retrieve optimal rank-lowering disentangler tensors, which are a crucial component in the construction of a low-rank MERA. Numerical experiments demonstrate the effectiveness of the proposed algorithms together with the potential storage benefit of a low-rank MERA over a tensor train.

The quaternion-based spatial-coordinate and orientation-frame alignment problems.

The general problem of finding a global rotation that transforms a given set of spatial coordinates and/or orientation frames (the `test' data) into the best possible alignment with a corresponding set (the `reference' data) is reviewed. For 3D point data, this `orthogonal Procrustes problem' is often phrased in terms of minimizing a root-mean-square deviation (RMSD) corresponding to a Euclidean distance measure relating the two sets of matched coordinates. This article focuses on quaternion eigensystem methods that have been exploited to solve this problem for at least five decades in several different bodies of scientific literature, where they were discovered independently. While numerical methods for the eigenvalue solutions dominate much of this literature, it has long been realized that the quaternion-based RMSD optimization problem can also be solved using exact algebraic expressions based on the form of the quartic equation solution published by Cardano in 1545; focusing on these exact solutions exposes the structure of the entire eigensystem for the traditional 3D spatial-alignment problem. The structure of the less-studied orientation-data context is then explored, investigating how quaternion methods can be extended to solve the corresponding 3D quaternion orientation-frame alignment (QFA) problem, noting the interesting equivalence of this problem to the rotation-averaging problem, which also has been the subject of independent literature threads. The article concludes with a brief discussion of the combined 3D translation-orientation data alignment problem. Appendices are devoted to a tutorial on quaternion frames, a related quaternion technique for extracting quaternions from rotation matrices and a review of quaternion rotation-averaging methods relevant to the orientation-frame alignment problem. The supporting information covers novel extensions of quaternion methods to the 4D Euclidean spatial-coordinate alignment and 4D orientation-frame alignment problems, some miscellaneous topics, and additional details of the quartic algebraic eigenvalue problem.

Spherical Regression Under Mismatch Corruption With Application to Automated Knowledge Translation

Motivated by a series of applications in data integration, language translation, bioinformatics, and computer vision, we consider spherical regression with two sets of unit-length vectors when the data are corrupted by a small fraction of mismatch in the response-predictor pairs. We propose a three-step algorithm in which we initialize the parameters by solving an orthogonal Procrustes problem to estimate a translation matrix ignoring the mismatch. We then estimate a mapping matrix aiming to correct the mismatch using hard-thresholding to induce sparsity, while incorporating potential group information. We eventually obtain a refined estimate for by removing the estimated mismatched pairs. We derive the error bound for the initial estimate of in both fixed and high-dimensional setting. We demonstrate that the refined estimate of achieves an error rate that is as good as if no mismatch is present. We show that our mapping recovery method not only correctly distinguishes one-to-one and one-to-many correspondences, but also consistently identifies the matched pairs and estimates the weight vector for combined correspondence. We examine the finite sample performance of the proposed method via extensive simulation studies, and with application to the unsupervised translation of medical codes using electronic health records data. Supplementary materials for this article are available online.

Orthogonal Procrustes and norm-dependent optimality

This note revisits the classical orthogonal Procrustes problem and investigates the norm-dependent geometric behavior underlying Procrustes alignment for subspaces. It presents generic, deterministic bounds quantifying the performance of a specified Procrustes-based choice of subspace alignment. Numerical examples illustrate the theoretical observations and offer additional, empirical findings which are discussed in detail. This note complements recent advances in statistics involving Procrustean matrix perturbation decompositions and eigenvector estimation.

Square Root-Based Multi-Source Early PSD Estimation and Recursive RETF Update in Reverberant Environments by Means of the Orthogonal Procrustes Problem

Multi-channel short-time Fourier transform (STFT) domain-based processing of reverberant microphone signals commonly relies on power-spectral-density (PSD) estimates of early source images, where early refers to reflections contained within the same STFT frame. State-of-the-art approaches to multi-source early PSD estimation, given an estimate of the associated relative early transfer functions (RETFs), conventionally minimize the approximation error defined with respect to the early correlation matrix, requiring non-negative inequality constraints on the PSDs. Instead, we here propose to factorize the early correlation matrix and minimize the approximation error defined with respect to the early-correlation-matrix square root. The proposed minimization problem -- constituting a generalization of the so-called orthogonal Procrustes problem -- seeks a unitary matrix and the square roots of the early PSDs up to an arbitrary complex argument, making non-negative inequality constraints redundant. A solution is obtained iteratively, requiring one singular value decomposition (SVD) per iteration. The estimated unitary matrix and early PSD square roots further allow to recursively update the RETF estimate, which is not inherently possible in the conventional approach. An estimate of the said early-correlation-matrix square root itself is obtained by means of the generalized eigenvalue decomposition (GEVD), where we further propose to restore non-stationarities by desmoothing the generalized eigenvalues in order to compensate for inevitable recursive averaging. Simulation results indicate fast convergence of the proposed multi-source early PSD estimation approach in only one iteration if initialized appropriately, and better performance as compared to the conventional approach.

An Eigenvalue-Based Method for the Unbalanced Procrustes Problem

In this paper, we propose a novel eigenvalue-based approach to solving the unbalanced orthogonal Procrustes problem. By making effective use of the necessary condition for the global minimizer and ...

Characterization and Computation of Matrices of Maximal Trace over Rotations

The constrained orthogonal Procrustes problem is the least-squares problem that calls for a rotation matrix that optimally aligns two corresponding sets of points in d-dimensional Euclidean space. This problem generalizes to the so-called Wahba's problem which is the same problem with nonnegative weights. Given a dxd matrix M, solutions to these problems are intimately related to the problem of finding a dxd rotation matrix U that maximizes the trace of UM, i.e., that makes UM a matrix of maximal trace over rotations, and it is well known this can be achieved with a method based on the computation of the singular value decomposition (SVD) of M. As the main goal of this paper, we characterize dxd matrices of maximal trace over rotation matrices in terms of their eigenvalues, and for d = 2, 3, we show how this characterization can be used to determine whether a matrix is of maximal trace over rotation matrices. Finally, although depending only slightly on the characterization, as a secondary goal of the paper, for d = 2, 3, we identify alternative ways, other than the SVD, of obtaining solutions to the aforementioned problems.

A Novel Orthogonal Extreme Learning Machine for Regression and Classification Problems

An extreme learning machine (ELM) is an innovative algorithm for the single hidden layer feed-forward neural networks and, essentially, only exists to find the optimal output weight so as to minimize output error based on the least squares regression from the hidden layer to the output layer. With a focus on the output weight, we introduce the orthogonal constraint into the output weight matrix, and propose a novel orthogonal extreme learning machine (NOELM) based on the idea of optimization column by column whose main characteristic is that the optimization of complex output weight matrix is decomposed into optimizing the single column vector of the matrix. The complex orthogonal procrustes problem is transformed into simple least squares regression with an orthogonal constraint, which can preserve more information from ELM feature space to output subspace, these make NOELM more regression analysis and discrimination ability. Experiments show that NOELM has better performance in training time, testing time and accuracy than ELM and OELM.

A Purely Algebraic Justification of the Kabsch-Umeyama Algorithm.

The constrained orthogonal Procrustes problem is the least-squares problem that calls for a rotation matrix that optimally aligns two matrices of the same order. Over past decades, the algorithm of choice for solving this problem has been the Kabsch-Umeyama algorithm, which is effectively no more than the computation of the singular value decomposition of a particular matrix. Its justification, as presented separately by Kabsch and Umeyama, is not totally algebraic since it is based on solving the minimization problem via Lagrange multipliers. In order to provide a more transparent alternative, it is the main purpose of this paper to present a purely algebraic justification of the algorithm through the exclusive use of simple concepts from linear algebra. For the sake of completeness, a proof is also included of the well known and widely used fact that the orientation-preserving rigid motion problem, i.e., the least-squares problem that calls for an orientation-preserving rigid motion that optimally aligns two corresponding sets of points in d-dimensional Euclidean space, reduces to the constrained orthogonal Procrustes problem.