Simultaneous Matrix Diagonalization Research Articles

A new SOCP relaxation of nonconvex quadratic programming problems with a few negative eigenvalues

We present a new second order cone programming (SOCP) relaxation of nonconvex quadratic programs with a few negative eigenvalues (NQP-r-NE) by employing the difference of convex (DC) decomposition and simultaneous matrix diagonalization together. The auxiliary variables are bounded from above by one more convex quadratic constraint in the proposed SOCP relaxation than that in the classical SOCP relaxation provided by Kim and Kojima (2001). We prove that the proposed SOCP relaxation is strictly tighter than the classical SOCP relaxation under certain circumstances, especially when there exists one constraint matrix being positive definite in the primal problem. Three types of numerical experiments including the large-scale NQP-r-NE problem with the number of negative eigenvalues r≤20, the optimal spectrum sharing problem in MIMO cognitive radio networks and the two-trust-region subproblem are provided to illustrate that the proposed SOCP relaxation could achieve a better lower bound than that of the classical SOCP relaxation. Moreover, when the number of variables is larger than 250, the proposed SOCP relaxation shows its great superiority both in the bound quality and computing efficiency compared to the classical SOCP relaxation.

Newton-type methods for simultaneous matrix diagonalization

This paper proposes a Newton-type method to solve numerically the eigenproblem of several diagonalizable matrices, which pairwise commute. A classical result states that these matrices are simultaneously diagonalizable. From a suitable system of equations associated to this problem, we construct a sequence that converges quadratically towards the solution. This construction is not based on the resolution of a linear system as is the case in the classical Newton method. Moreover, we provide a theoretical analysis of this construction and exhibit a condition to get a quadratic convergence. We also propose numerical experiments, which illustrate the theoretical results.

A Global Optimization Algorithm for Solving Linearly Constrained Quadratic Fractional Problems

This paper first proposes a new and enhanced second order cone programming relaxation using the simultaneous matrix diagonalization technique for the linearly constrained quadratic fractional programming problem. The problem has wide applications in statics, economics and signal processing. Thus, fast and effective algorithm is required. The enhanced second order cone programming relaxation improves the relaxation effect and computational efficiency compared to the classical second order cone programming relaxation. Moreover, although the bound quality of the enhanced second order cone programming relaxation is worse than that of the copositive relaxation, the computational efficiency is significantly enhanced. Then we present a global algorithm based on the branch and bound framework. Extensive numerical experiments show that the enhanced second order cone programming relaxation-based branch and bound algorithm globally solves the problem in less computing time than the copositive relaxation approach.

Fast Hilbert–Wavelet Simulation of Nonstationary Wind Field Using Noniterative Simultaneous Matrix Diagonalization

AbstractThe frequency content and spatial correlation of nonstationary wind fields during extreme events typically present time-variant characteristics. Several effective schemes [e.g., using evolu...

Coupled CP Decomposition of Simultaneous MEG-EEG Signals for Differentiating Oscillators During Photic Driving.

Magnetoencephalography (MEG) and electroencephalography (EEG) are contemporary methods to investigate the function and organization of the brain. Simultaneously acquired MEG-EEG data are inherently multi-dimensional and exhibit coupling. This study uses a coupled tensor decomposition to extract the signal sources from MEG-EEG during intermittent photic stimulation (IPS). We employ the Coupled Semi-Algebraic framework for approximate CP decomposition via SImultaneous matrix diagonalization (C-SECSI). After comparing its performance with alternative methods using simulated benchmark data, we apply it to MEG-EEG recordings of 12 participants during IPS with fractions of the individual alpha frequency between 0.4 and 1.3. In the benchmark tests, C-SECSI is more accurate than SECSI and alternative methods, especially in ill-conditioned scenarios, e.g., involving collinear factors or noise sources with different variances. The component field-maps allow us to separate physiologically meaningful oscillations of visually evoked brain activity from background signals. The frequency signatures of the components identify either an entrainment to the respective stimulation frequency or its first harmonic, or an oscillation in the individual alpha band or theta band. In the group analysis of both, MEG and EEG data, we observe a reciprocal relationship between alpha and theta band oscillations. The coupled tensor decomposition using C-SECSI is a robust, powerful method for the extraction of physiologically meaningful sources from multidimensional biomedical data. Unsupervised signal source extraction is an essential solution for rendering advanced multi-modal signal acquisition technology accessible to clinical diagnostics, pre-surgical planning, and brain computer interface applications.

A New Spatial Branch and Bound Algorithm for Quadratic Program with One Quadratic Constraint and Linear Constraints

This paper proposes a novel second-order cone programming (SOCP) relaxation for a quadratic program with one quadratic constraint and several linear constraints (QCQP) that arises in various real-life fields. This new SOCP relaxation fully exploits the simultaneous matrix diagonalization technique which has become an attractive tool in the area of quadratic programming in the literature. We first demonstrate that the new SOCP relaxation is as tight as the semidefinite programming (SDP) relaxation for the QCQP when the objective matrix and constraint matrix are simultaneously diagonalizable. We further derive a spatial branch-and-bound algorithm based on the new SOCP relaxation in order to obtain the global optimal solution. Extensive numerical experiments are conducted between the new SOCP relaxation-based branch-and-bound algorithm and the SDP relaxation-based branch-and-bound algorithm. The computational results illustrate that the new SOCP relaxation achieves a good balance between the bound quality and computational efficiency and thus leads to a high-efficiency global algorithm.

A low-dimensional SDP relaxation based spatial branch and bound method for nonconvex quadratic programs

In this paper, we propose a novel low-dimensional semidefinite programming (SDP) relaxation for convex quadratically constrained nonconvex quadratic programming problems. This new relaxation is derived by simultaneous matrix diagonalization method under the difference of convex decomposition scheme. The highlight of the relaxation is the low dimensionality of the positive semidefinite constraint, which only depends on the number of negative eigenvalues in the objective function. We prove that a mixed SOCP and SDP relaxation appeared in the literature is equivalent to the proposed relaxation, while the proposed relaxation has fewer constraints. We also provide conditions under which the proposed relaxation is as tight as the classical SDP relaxation and provides an optimal value for the original problem. For general cases, a spatial branch-and-bound algorithm is designed for finding a global optimal solution. Extensive numerical experiments support that the proposed algorithm outperforms two cutting-edge algorithms when the problem size is medium or large and the number of negative eigenvalues in the nonconvex objective function is relatively small.

Tensor-based time-delay estimation for second and third generation global positioning system

Global Navigation Satellite Systems (GNSS), such as GPS, Galileo, GLONASS, and BeiDou, are vital for safety-critical applications such as civilian aviation and autonomous vehicles. These applications demand very accurate positioning, especially in scenarios in which multipath components are present. In this paper, we propose a semi-algebraic framework for approximate Canonical Polyadic Decomposition via simultaneous matrix diagonalization (SECSI) with Higher Order Singular Value Decomposition (HOSVD) low-rank approximation for time-delay estimation for antenna array based GNSS receivers. The HOSVD-SECSI based approach outperforms all the state-of-the-art tensor-based algorithms for time-delay estimation, mainly, for scenarios with highly correlated or coherent signals and in case of more than two multipath components.

Multidimensional Harmonic Retrieval via Coupled Canonical Polyadic Decomposition—Part II: Algorithm and Multirate Sampling

In Part I of this paper, we have presented a link between multidimensional harmonic retrieval (MHR) and the recently proposed coupled canonical polyadic decomposition (CPD), which implies new uniqueness conditions for MHR that are more relaxed than the existing results based on a Vandermonde constrained CPD. In Part II, we explain that the coupled CPD also provides a computational framework for MHR. In particular, we present an algebraic method for MHR based on simultaneous matrix diagonalization that is guaranteed to find the exact solution in the noiseless case, under conditions discussed in Part I. Since the simultaneous matrix diagonalization method reduces the MHR problem into an eigenvalue problem, the proposed algorithm can be interpreted as an MHR generalization of the classical ESPRIT method for one-dimensional harmonic retrieval. We also demonstrate that the presented coupled CPD framework for MHR can algebraically support multirate sampling. We develop an efficient implementation which has about the same computational complexity for single-rate and multirate sampling. Numerical experiments demonstrate that by simultaneously exploiting the harmonic structure in all dimensions and making use of multirate sampling, the coupled CPD framework for MHR can lead to an improved performance compared to the conventional Vandermonde constrained CPD approaches.

A semi-algebraic framework for approximate CP decompositions via simultaneous matrix diagonalizations (SECSI)

In this paper, we propose a framework to compute approximate CANDECOMP / PARAFAC (CP) decompositions. Such tensor decompositions are viable tools in a broad range of applications, creating the need for versatile tools to compute such decompositions with an adjustable complexity-accuracy trade-off.To this end, we propose a novel SEmi-algebraic framework that allows the computation of approximate C P decompositions via SImultaneous Matrix Diagonalizations (SECSI). In contrast to previous Simultaneous Matrix Diagonalization (SMD)-based approaches, we use the tensor structure to construct not only one but the full set of possible SMDs. Solving all SMDs, we obtain multiple estimates of the factor matrices and present strategies to choose the best estimate in a subsequent step. This SECSI framework retains the option to choose the number of SMDs to solve and to adopt various strategies for the selection of the final solution out of the multiple estimates. A best matching scheme based on an exhaustive search as well as heuristic selection schemes are devised to flexibly adapt to specific applications. Four example algorithms with different accuracy-complexity trade-off points are compared to state-of-the-art algorithms. We obtain more reliable estimates and a reduced computational complexity.

Canonical Polyadic Decomposition with a Columnwise Orthonormal Factor Matrix

Canonical polyadic decomposition (CPD) of a higher-order tensor is an important tool in mathematical engineering. In many applications at least one of the matrix factors is constrained to be columnwise orthonormal. We first derive a relaxed condition that guarantees uniqueness of the CPD under this constraint. Second, we give a simple proof of the existence of the optimal low-rank approximation of a tensor in the case that a factor matrix is columnwise orthonormal. Third, we derive numerical algorithms for the computation of the constrained CPD. In particular, orthogonality-constrained versions of the CPD methods based on simultaneous matrix diagonalization and alternating least squares are presented. Numerical experiments are reported.

Blind Identification of Underdetermined Mixtures by Simultaneous Matrix Diagonalization

In this paper, we study simultaneous matrix diagonalization-based techniques for the estimation of the mixing matrix in underdetermined independent component analysis (ICA). This includes a generalization to underdetermined mixtures of the well-known SOBI algorithm. The problem is reformulated in terms of the parallel factor decomposition (PARAFAC) of a higher-order tensor. We present conditions under which the mixing matrix is unique and discuss several algorithms for its computation.

Fourth-Order Cumulant-Based Blind Identification of Underdetermined Mixtures

In this paper we study two fourth-order cumulant-based techniques for the estimation of the mixing matrix in underdetermined independent component analysis. The first method is based on a simultaneous matrix diagonalization. The second is based on a simultaneous off-diagonalization. The number of sources that can be allowed is roughly quadratic in the number of observations. For both methods, explicit expressions for the maximum number of sources are given. Simulations illustrate the performance of the techniques

Quadratic optimization for simultaneous matrix diagonalization

Simultaneous diagonalization of a set of matrices is a technique that has numerous applications in statistical signal processing and multivariate statistics. Although objective functions in a least-squares sense can be easily formulated, their minimization is not trivial, because constraints and fourth-order terms are usually involved. Most known optimization algorithms are, therefore, subject to certain restrictions on the class of problems: orthogonal transformations, sets of symmetric, Hermitian or positive definite matrices, to name a few. In this paper, we present a new algorithm called QDIAG that splits the overall optimization problem into a sequence of simpler second order subproblems. There are no restrictions imposed on the transformation matrix, which may be nonorthogonal, indefinite, or even rectangular, and there are no restrictions regarding the symmetry and definiteness of the matrices to be diagonalized, except for one of them. We apply the new method to second-order blind source separation and show that the algorithm converges fast and reliably. It allows for an implementation with a complexity independent of the number of matrices and, therefore, is particularly suitable for problems dealing with large sets of matrices

A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization

Canonical decomposition is a key concept in multilinear algebra. In this paper we consider the decomposition of higher‐order tensors which have the property that the rank is smaller than the greatest dimension. We derive a new and relatively weak deterministic sufficient condition for uniqueness. The proof is constructive. It shows that the canonical components can be obtained from a simultaneous matrix diagonalization by congruence, yielding a new algorithm. From the deterministic condition we derive an easy‐to‐check dimensionality condition that guarantees generic uniqueness.

Dimensionality reduction in higher-order signal processing and rank-( R1, R2,…, RN) reduction in multilinear algebra

In this paper we review a multilinear generalization of the singular value decomposition and the best rank-( R 1, R 2,…, R N ) approximation of higher-order tensors. We show that they are important tools for dimensionality reduction in higher-order signal processing. We discuss applications in independent component analysis, simultaneous matrix diagonalization and subspace variants of algorithms based on higher-order statistics.