Symmetric Matrix Factorization Research Articles

Enforced Block Diagonal Graph Learning for Multikernel Clustering

The existing multikernel graph clustering (MKGC) methods have emerged with notable success on nonlinear clustering tasks since the graph learning can effectively capture graph structure similarity between sample pair. Ideally, a high-quality graph should enjoy the good block diagonal property, i.e. the intercluster similarities correspond to zeros, while intracluster similarities represent nonzeros. Meanwhile, the number of diagonal blocks for a graph equals to the number of clusters on the dataset. However, most of the existing MKGC methods design a corpulent three-parts graph leaning process that poses challenges for hyperparameter tuning, time cost, and clustering performance. To overcome these challenging issues, we propose an enforced block diagonal graph learning for multikernel clustering (EBDGL-MKC) method, where we pursue a high-quality block diagonal graph via well-designed one-part graph leaning scheme rather than three parts. Inspired by symmetric matrix factorization (SMF), we first design a one-part block diagonal graph learning scheme to learn multiple block diagonal graphs, by exploring an explicit theoretical connection between the clustering partition of kernel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$</tex-math> </inline-formula> -means and the excellent block diagonal graph. Then, these block diagonal graphs are stacked into a low-rank tensor for exploiting the high-order structure information hidden in the nonlinear data. After that, an effective alternate algorithm with convergence proof is performed on extensive experiments to demonstrate the superiority of EBDGL compared with the state-of-the-art multikernel clustering (MKC) methods.

The decimation scheme for symmetric matrix factorization

Matrix factorization is an inference problem that has acquired importance due to its vast range of applications that go from dictionary learning to recommendation systems and machine learning with deep networks. The study of its fundamental statistical limits represents a true challenge, and despite a decade-long history of efforts in the community, there is still no closed formula able to describe its optimal performances in the case where the rank of the matrix scales linearly with its size. In the present paper, we study this extensive rank problem, extending the alternative ‘decimation’ procedure that was recently introduced by Camilli and Mézard, and carry out a thorough study of its performance. Decimation aims at recovering one column/line of the factors at a time, by mapping the problem into a sequence of neural network models of associative memory at a tunable temperature. The main advantage of decimation is that its theoretical performance can be studied using statistical physics techniques. In this paper we provide the general analysis applying to a large class of compactly supported priors and we show that the replica symmetric free entropy of the neural network models takes a universal form in the low temperature limit. We then study the decimation phase diagrams in two concrete cases, the sparse Ising prior and the uniform prior, for which we show that matrix factorization is theoretically possible when the ration P/N of rank to variable is below a certain threshold. This suggest that a possible route to solving algorithmically the matrix factorization problem could be to find efficient decimation algorithms; in this respect, we show that simple simulated annealing, though being effective for limited signal sizes, is not scalable.

Fast discrete cross-modal hashing with semantic consistency

Supervised cross-modal hashing has attracted widespread concentrations for large-scale retrieval task due to its promising retrieval performance. However, most existing works suffer from some of following issues. Firstly, most of them only leverage the pair-wise similarity matrix to learn hash codes, which may result in class information loss. Secondly, the pair-wise similarity matrix generally lead to high computing complexity and memory cost. Thirdly, most of them relax the discrete constraints during optimization, which generally results in large cumulative quantization error and consequent inferior hash codes. To address above problems, we present a Fast Discrete Cross-modal Hashing method in this paper, FDCH for short. Specifically, it firstly leverages both class labels and the pair-wise similarity matrix to learn a sharing Hamming space where the semantic consistency can be better preserved. Then we propose an asymmetric hash codes learning model to avoid the challenging issue of symmetric matrix factorization. Finally, an effective and efficient discrete optimal scheme is designed to generate discrete hash codes directly, and the computing complexity and memory cost caused by the pair-wise similarity matrix are reduced from O(n2) to O(n), where n denotes the size of training set. Extensive experiments conducted on three real world datasets highlight the superiority of FDCH compared with several cross-modal hashing methods and demonstrate its effectiveness and efficiency.

Collaborative crystal structure prediction

The prediction of crystal structures is one of the most essential challenges in designing novel functional materials. A data-driven prediction technique that uses the database of known crystal structures and substitutes ions among materials of known crystal structures to concoct new crystal structures has been proposed. This technique has been applied to generate crystal-structure candidates for the purpose of first-principles-calculation-based high-throughput computational screening. However, this technique has a functional limitation that the ion substitution tendencies are available only for typical ions such that their associated crystal structures appear in well-known materials. To overcome such a limitation, this work introduces an idea of collaborative filtering to the calculation of the ionic substitution tendencies. Based on this idea, we develop symmetric matrix factorization (SMF) method to model underlying substitution conditions. In addition, we present a symmetric matrix co-factorization (SMCF) method to incorporate additional knowledge pertaining to chemical properties in estimating the substitution tendencies among ions with extremely small amount of previous knowledge in the database. The performance of the prediction is investigated along with existing techniques through in silico experiments using real crystal-structure database. The numerical results show that the proposed SMF- and SMCF-based prediction outperform existing techniques in terms of the prediction accuracy.

Computing matrix symmetrizers, part 2: New methods using eigendata and linear means; a comparison

Over any field F every square matrix A can be factored into the product of two symmetric matrices as A=S1⋅S2 with Si=SiT∈Fn,n and either factor can be chosen nonsingular, as was discovered by Frobenius in 1910.Frobenius' symmetric matrix factorization has been lying almost dormant for a century. The first successful method for computing matrix symmetrizers, i.e., symmetric matrices S such that SA is symmetric, was inspired by an iterative linear systems algorithm of Huang and Nong (2010) in 2013 [29,30]. The resulting iterative algorithm has solved this computational problem over R and C, but at high computational cost.This paper develops and tests another linear equations solver, as well as eigen- and principal vector or Schur Normal Form based algorithms for solving the matrix symmetrizer problem numerically. Four new eigendata based algorithms use, respectively, SVD based principal vector chain constructions, Gram–Schmidt orthogonalization techniques, the Arnoldi method, or the Schur Normal Form of A in their formulations. They are helped by Datta's 1973 method that symmetrizes unreduced Hessenberg matrices directly.The eigendata based methods work well and quickly for generic matrices A and create well conditioned matrix symmetrizers through eigenvector dyad accumulation. But all of the eigen based methods have differing deficiencies with matrices A that have ill-conditioned or complicated eigen structures with nontrivial Jordan normal forms. Our symmetrizer studies for matrices with ill-conditioned eigensystems lead to two open problems of matrix optimization.

Regularized symmetric positive definite matrix factorizations for linear systems arising from RBF interpolation and differentiation

Scattered data interpolation using Radial Basis Functions involves solving an ill-conditioned symmetric positive definite (SPD) linear system (with appropriate selection of basis function) when the direct method is used to evaluate the problem. The standard algorithm for solving a SPD system is a Cholesky factorization. Severely ill-conditioned theoretically SPD matrices may not be numerically SPD (NSPD) in which case a Cholesky factorization fails. An alternative symmetric matrix factorization, the square root free Cholesky factorization, has the same flop count as a Cholesky factorization and is successful even when a matrix ceases to be NSPD. A regularization method can be used to prevent the failure of the Cholesky factorization and to improve the accuracy of both SPD matrix factorizations when the matrices are severely ill-conditioned. The specification of the regularization parameter is discussed as well as convergence/stopping criteria for the algorithm. The formation of differentiation matrices with the regularized SPD factorizations is demonstrated to improve eigenvalue stability properties of RBF methods for hyperbolic PDEs.

IHelp: An Intelligent Online Helpdesk System

Due to the importance of high-quality customer service, many companies use intelligent helpdesk systems (e.g., case-based systems) to improve customer service quality. However, these systems face two challenges: 1) Case retrieval measures: most case-based systems use traditional keyword-matching-based ranking schemes for case retrieval and have difficulty to capture the semantic meanings of cases and 2) result representation: most case-based systems return a list of past cases ranked by their relevance to a new request, and customers have to go through the list and examine the cases one by one to identify their desired cases. To address these challenges, we develop iHelp, an intelligent online helpdesk system, to automatically find problem-solution patterns from the past customer-representative interactions. When a new customer request arrives, iHelp searches and ranks the past cases based on their semantic relevance to the request, groups the relevant cases into different clusters using a mixture language model and symmetric matrix factorization, and summarizes each case cluster to generate recommended solutions. Case and user studies have been conducted to show the full functionality and the effectiveness of iHelp.

High performance sparse solver for unsymmetrical linear equations with out-of-core strategies and its application on meshless methods

A new direct method for solving unsymmetrical sparse linear systems(USLS) arising from meshless methods was introduced. Computation of certain meshless methods such as meshless local Petrov-Galerkin (MLPG) method need to solve large USLS. The proposed solution method for unsymmetrical case performs factorization processes symmetrically on the upper and lower triangular portion of matrix, which differs from previous work based on general unsymmetrical process, and attains higher performance. It is shown that the solution algorithm for USLS can be simply derived from the existing approaches for the symmetrical case. The new matrix factorization algorithm in our method can be implemented easily by modifying a standard JKI symmetrical matrix factorization code. Multi-blocked out-of-core strategies were also developed to expand the solution scale. The approach convincingly increases the speed of the solution process, which is demonstrated with the numerical tests.

The ${\cal {D Q R}}$ algorithm, basic theory, convergence, and conditional stability

We describe a fast matrix eigenvalue algorithm that uses a matrix factorization and reverse order multiply technique involving three factors and that is based on the symmetric matrix factorization as well as on $D$ –orthogonal reduction techniques where $D= \mbox{ diag}(\pm1)$ is computed from the given matrix $A$ . It operates on a similarity reduction of a real matrix $A_{nn}$ to general tridiagonal form $T$ and computes all of $A$ 's eigenvalues in $\frac{4}{3} n^3 + O(n^2)$ operations, where the $O(n^2)$ part of the operations is possibly performed over ${\Bbb C}$ , instead of the 7–8 $n^3$ real flops required by the $QR$ eigenvalue algorithm. Potential breakdo wn of the $DQR$ algorithm can occur in the reduction to tridiagonal form and in the $D$ –orthogonal $DQR$ reductions. Both, however, can be monitored during the computations. The former occurs rather rarely for dimensions $n \leq 300$ and can essentially be bypassed, while the latter is extremely rare and can be bypassed as well in our conditionally stable implementation of the $DQR$ steps. We prove an implicit $DQ$ theorem which allows implicit shifts, give a convergence proof for the $DQR$ algorithm and show that $DQR$ is conditionally stable for general balanced tridiagonal matrices $T_{\rm b}$ .

Implementation of Linear Multistep Methods for Solving Constrained Equations of Motion

A class of multistep methods for numerical integration of the equations of motion of constrained mechanical systems is considered. They are based on different local parametrizations of the differentiable manifold defined by the kinematic constraints. During integration, the kinematic constraints are preserved at all levels: position, velocity, and acceleration. The numerical implementations described in the paper require only one symmetric matrix factorization at each integration step.

On threshold pivoting in the multifrontal method for sparse indefinite systems

A simple modification to the numerical pivot selection criteria in the multifrontal scheme of Duff and Reid for sparse symmetric matrix factorization is presented. For a given threshold value, the modification allows a broader choice of block 2 X 2 pivots owing to a less restrictive pivoting condition. It also extends the range of permissible threshold values from [0, 1/2) to [0, 0.6404). Moreover, the bound on element growth for stability consideration in the modified scheme is nearly the same as that of the original strategy.