Column Pivoting Research Articles

A new solution to one sample problem in face recognition using FLDA

Fisher linear discriminant analysis (FLDA) is a very popular method in face recognition. But FLDA fails when one image per person is available. This is due to the fact that the within-class scatter matrices cannot be calculated. An image decomposition method that uses QR-decomposition with column pivoting (QRCP) is proposed in this paper to overcome one image per person problem. At first, the image and its two approximations that are evaluated using QRCP-decomposition are all placed in the training set. Then 2D-FLDA method becomes applicable with these new data. The performance of the proposed image decomposition algorithm is tested on five different face databases, namely ORL, FERET, YALE, UMIST, and PolyU-NIR using 2D-FLDA. Our image decomposition algorithm performs better than the SVD based method mentioned by Gao et al. (2008) [1] in terms of recognition rate and training time in all of the above databases.

Optimal selection of artificial boundary conditions for model update and damage detection

Sensitivity-based model error localization and damage detection is hindered by the relative differences in modal sensitivity magnitude among updating parameters. The method of artificial boundary conditions is shown to directly address this limitation, resulting in the increase of the number of updating parameters at which errors can be accurately localized. Using a single set of FRF data collected from a modal test, the artificial boundary conditions (ABC) method identifies experimentally the natural frequencies of a structure under test for a variety of different boundary conditions, without having to physically apply the boundary conditions, hence the term “artificial”. The parameter-specific optimal ABC sets applied to the finite element model will produce increased sensitivities in the updating parameter, yielding accurate error localization and damage detection solutions. A method is developed for identifying the parameter-specific optimal ABC sets for updating or damage detection, and is based on the QR decomposition with column pivoting. Updating solution residuals, such as magnitude error and false error location, are shown to be minimized when the updating parameter set is limited to those corresponding to the QR pivot columns. The existence of an optimal ABC set for a given updating parameter is shown to be dependent on the number of modes used, and hence the method developed provides a systematic determination of the minimum number of modes required for localization in a given updating parameter. These various concepts are demonstrated on a simple model with simulated test data.

Stable solutions of linear systems involving long chain of matrix multiplications

This paper is concerned with solving linear system ( I n + B L ⋯ B 2 B 1 ) x = b arising from the Green’s function calculation in the quantum Monte Carlo simulation of interacting electrons. The order of the system and integer L are adjustable. Also adjustable is the conditioning of the coefficient matrix to give rise an extreme ill-conditioned system. Two numerical methods based on the QR decomposition with column pivoting and the singular value decomposition, respectively, are studied in this paper. It is proved that the computed solution x ∼ by each of the methods is weakly backward stable in the sense that the computed x ∼ is close to the exact solution of a nearby linear system [ I n + ( B L + Δ B L ) ⋯ ( B 2 + Δ B 2 ) ( B 1 + Δ B 1 ) ] x ∼ = b with each Δ B i small in norm relatively to B i .

Feature selection using singular value decomposition and QR factorization with column pivoting for text-independent speaker identification

Selection of features is one of the important tasks in the application like Speaker Identification (SI) and other pattern recognition problems. When multiple features are extracted from the same frame of speech, it is expected that a feature vector would contain redundant features. Redundant features confuse the speaker model in multidimensional space resulting in degraded performance by the system. Careful selection of potential features can remove this redundancy while helping to achieve the higher rate of accuracy at lower computational cost. Although the selection of features is difficult without having exhaustive search, this paper proposes an alternative and straight forward technique for feature selection using Singular Value Decomposition (SVD) followed by QR Decomposition with Column Pivoting (QRcp). The idea is to capture the most salient part of the information from the speakers’ data by choosing those features that can explain different dimensions showing minimal similarities (or maximum acoustic variability) among them in orthogonal sense. The performances after selection of features using proposed criterion have been compared with using Mel-frequency Cepstral Coefficients (MFCC), Linear Frequency (LF) Cepstral Coefficients (LFCC) and a new feature proposed in this paper that is based on Gaussian shaped filters on mel-scale. It is shown that proposed SVD-QRcp based feature selection outperforms F-Ratio based method and the proposed feature extraction tool is superior to baseline MFCC & LFCC.

Computing Multivariate Fekete and Leja Points by Numerical Linear Algebra

We discuss and compare two greedy algorithms that compute discrete versions of Fekete-like points for multivariate compact sets by basic tools of numerical linear algebra. The first gives the so-called approximate Fekete points by QR factorization with column pivoting of Vandermonde-like matrices. The second computes discrete Leja points by LU factorization with row pivoting. Moreover, we study the asymptotic distribution of such points when they are extracted from weakly admissible meshes.

A new and fast implementation for null space based linear discriminant analysis

In this paper we present a new implementation for the null space based linear discriminant analysis. The main features of our implementation include: (i) the optimal transformation matrix is obtained easily by only orthogonal transformations without computing any eigendecomposition and singular value decomposition (SVD), consequently, our new implementation is eigendecomposition-free and SVD-free; (ii) its main computational complexity is from a economic QR factorization of the data matrix and a economic QR factorization of a n× n matrix with column pivoting, here n is the sample size, thus our new implementation is a fast one. The effectiveness of our new implementation is demonstrated by some real-world data sets.

Fast and Stable Polynomial Equation Solving and Its Application to Computer Vision

This paper presents several new results on techniques for solving systems of polynomial equations in computer vision. Gröbner basis techniques for equation solving have been applied successfully to several geometric computer vision problems. However, in many cases these methods are plagued by numerical problems. In this paper we derive a generalization of the Gröbner basis method for polynomial equation solving, which improves overall numerical stability. We show how the action matrix can be computed in the general setting of an arbitrary linear basis for ℂ[x]/I. In particular, two improvements on the stability of the computations are made by studying how the linear basis for ℂ[x]/I should be selected. The first of these strategies utilizes QR factorization with column pivoting and the second is based on singular value decomposition (SVD). Moreover, it is shown how to improve stability further by an adaptive scheme for truncation of the Gröbner basis. These new techniques are studied on some of the latest reported uses of Gröbner basis methods in computer vision and we demonstrate dramatically improved numerical stability making it possible to solve a larger class of problems than previously possible.

Computing approximate Fekete points by QR factorizations of Vandermonde matrices

We propose a numerical method (implemented in Matlab) for computing approximate Fekete points on compact multivariate domains. It relies on the search of maximum volume submatrices of Vandermonde matrices computed on suitable discretization meshes, and uses a simple greedy algorithm based on QR factorization with column pivoting. The method gives also automatically an algebraic cubature formula, provided that the moments of the underlying polynomial basis are known. Numerical tests are presented for the interval and the square, which show that approximate Fekete points are well suited for polynomial interpolation and cubature.

Using Perturbed $QR$ Factorizations to Solve Linear Least-Squares Problems

We propose and analyze a new tool to help solve sparse linear least-squares problems $\min_{x}\|Ax-b\|_{2}$. Our method is based on a sparse $QR$ factorization of a low-rank perturbation $\hat{A}$ of $A$. More precisely, we show that the $R$ factor of $\hat{A}$ is an effective preconditioner for the least-squares problem $\min_{x}\|Ax-b\|_{2}$, when solved using LSQR. We propose applications for the new technique. When $A$ is rank deficient, we can add rows to ensure that the preconditioner is well conditioned without column pivoting. When $A$ is sparse except for a few dense rows, we can drop these dense rows from $A$ to obtain $\hat{A}$. Another application is solving an updated or downdated problem. If $R$ is a good preconditioner for the original problem $A$, it is a good preconditioner for the updated/downdated problem $\hat{A}$. We can also solve what-if scenarios, where we want to find the solution if a column of the original matrix is changed/removed. We present a spectral theory that analyzes the generalized spectrum of the pencil $(A^{*}A,R^{*}R)$ and analyze the applications.

On the Failure of Rank-Revealing QR Factorization Software -- A Case Study

This article reports an unexpected and rather erratic behavior of the LAPACK software implementation of the QR factorization with Businger-Golub column pivoting. It is shown that, due to finite precision arithmetic, the software implementation of the factorization can catastrophically fail to produce a properly structured triangular factor, thus leading to a potentially severe underestimate of a matrix's numerical rank. The 30-year old problem, dating back to LINPACK, has (undetectedly) badly affected many computational routines and software packages, as well as the study of rank-revealing QR factorizations. We combine computer experiments and numerical analysis to isolate, analyze, and fix the problem. Our modification of the current LAPACK xGEQP3 routine is already included in the LAPACK 3.1.0 release. The modified routine is numerically more robust and with a negligible overhead. We also provide a new, equally efficient, and provably numerically safe partial-column norm-updating strategy.

In search of an optimization technique for Artificial Neural Network to classify abnormal heart sounds

Artificial Neural Network (ANN) finds use in classification of heart sounds for its discriminative training ability and easy implementation. The selection of number of nodes for an ANN remains an important issue as an overparameterized ANN gets trained along with the redundant information that results in poor validation. Also a larger network means more computational expense, resulting more hardware and time related cost. Therefore, a compact and optimum design of neural network is needed towards real-time detection of pathological patterns, if any from heart sound signals. This work attempts to (i) design a compact form of output layer with less number of nodes than output classes, (ii) select a set of input features that are effective for identification of heart sound signals using Singular Value Decomposition (SVD), QR factorization with column pivoting (QRcp) and Fisher's F-ratio, (iii) make certain optimum selection of nodes in the hidden layer for a more effective ANN structure using SVD and (iv) select and prune weights based on the concept of local relative sensitivity index (LRSI) for empirically chosen overparameterized ANN structure for phonocardiogram (PCG) classification. It is observed that the proposed techniques perform better in terms of reduction of model residues and time complexity for classifying 12 different pathological cases and normal heart sound.

Simple and Efficient Nonparametric Method for Estimating the Number of Signals Without Eigendecomposition

Inspired by the computational simplicity and numerical stability of QR decomposition, a nonparametric method for estimating the number of signals without eigendecomposition (MENSE) is proposed for the coherent narrowband signals impinging on a uniform linear array (ULA). By exploiting the array geometry and its shift invariance property to decorrelate the coherency of signals through subarray averaging, the number of signals is revealed in the rank of the QR upper-trapezoidal factor of the autoproduct of a combined Hankel matrix formed from the cross correlations between some sensor data. Since the infection of additive noise is defused, signal detection capability is improved. A new detection criterion is then formulated in terms of the row elements of the QR upper-triangular factor when finite array data are available, and the number of signals is determined as a value of the running index for which this ratio criterion is maximized, where the QR decomposition with column pivoting is also used to improve detection performance. The statistical analysis clarifies that the MENSE detection criterion is asymptotically consistent. Furthermore, the proposed MENSE algorithm is robust against the array uncertainties including sensor gain and phase errors and mutual coupling and against the deviations from the spatial homogeneity of noise model. The effectiveness of the MENSE is verified through numerical examples, and the simulation results show that the MENSE is superior in detecting closely spaced signals with a small number of snapshots and/or at relatively low signal-to-noise ratio (SNR)

Numerical Stability of the Parallel Jacobi Method

In this paper we study numerical stability of the parallel Jacobi method for computing the singular values and singular subspaces of an invertible upper triangular matrix that is obtained from QR decomposition with column pivoting. We show that in this case the parallel Jacobi method locates singular values and singular subspaces to full machine accuracy.

Roundoff Error Estimates of the Modified Gram–Schmidt Algorithm with Column Pivoting

In this paper we provide backward and forward roundoff error estimates of the modified Gram–Schmidt algorithm with column pivoting. It turns out that the row-wise growth factors of this algorithm are bounded. Furthermore, if the coefficient matrix is well conditioned, then with properly chosen tolerance η, this algorithm can also correctly determine the numerical rank of the coefficient matrix. We also derive a forward roundoff error estimate of this algorithm for the solution of the least squares problem.

A LOW-COMPLEXITY DIVIDE-AND-CONQUER METHOD FOR COMPUTING EIGENVALUES AND EIGENVECTORS OF SYMMETRIC BAND MATRICES ∗

A framework for an efficient low-complexity divide-and-conquer algorithm for computing eigenvalues and eigenvectors of an n × n symmetric band matrix with semibandwidth b ≪ n is proposed and its arithmetic complexity analyzed. The distinctive feature of the algorithm—after subdivision of the original problem into p subproblems and their solution—is a separation of the eigenvalue and eigenvector computations in the central synthesis problem. The eigenvalues are computed recursively by representing the corresponding symmetric rank b(p−1) modification of a diagonal matrix as a series of rank-one modifications. Each rank-one modifications problem can be solved using techniques developed for the tridiagonal divide-and-conquer algorithm. Once the eigenvalues are known, the corresponding eigenvectors can be computed efficiently using modified QR factorizations with restricted column pivoting. It is shown that the complexity of the resulting divide-and-conquer algorithm is O (n2b2) (in exact arithmetic).

A modified Gram–Schmidt algorithm with iterative orthogonalization and column pivoting

Iterative orthogonalization is aimed to ensure small deviation from orthogonality in the Gram–Schmidt process. Former applications of this technique are restricted to classical Gram–Schmidt (CGS) and column-oriented modified Gram–Schmidt (MGS). The major aim of this paper is to explain how iterative orthogonalization is incorporated into row-oriented MGS. The interest that we have in a row-oriented iterative MGS comes from the observation that this method is capable of performing column pivoting. The use of column pivoting delays the deteriorating effects of rounding errors and helps to handle rank-deficient least-squares problems. A second modification proposed in this paper considers the use of Gram–Schmidt QR factorization for solving linear least-squares problems. The standard solution method is based on one orthogonalization of the r.h.s. vector b against the columns of Q. The outcome of this process is the residual vector, r ∗ , and the solution vector, x ∗ . The modified scheme is a natural extension of the standard solution method that allows it to apply iterative orthogonalization. This feature ensures accurate computation of small residuals and helps in cases when Q has some deviation from orthogonality.

On the Computation of the Restricted Singular Value Decomposition via the Cosine-Sine Decomposition

In this paper, we show that the restricted singular value decomposition of a matrix triplet $A\in \R^{n \times m}, B\in \R^{n \times l}, C\in \R^{p \times m}$ can be computed by means of the cosine-sine decomposition. In the first step, the matrices A, B, C are reduced to a lower-dimensional matrix triplet ${\cal A}, {\cal B}, {\cal C}$, in which ${\cal B}$ and ${\cal C}$ are nonsingular, using orthogonal transformations such as the QR-factorization with column pivoting and the URV decomposition. In the second step, the components of the restricted singular value decomposition of A, B, C are derived from the singular value decomposition of ${\cal B}^{-1}{\cal A}{\cal C}^{-1}$. Instead of explicitly forming the latter product, a link with the cosine-sine decomposition, which can be computed by Van Loan's method, is exploited. Some numerical examples are given to show the performance of the presented method.

New Accurate Algorithms for Singular Value Decomposition of Matrix Triplets

This paper presents a new algorithm for accurate floating-point computation of the singular value decomposition (SVD) of the product $A = B^{\tau} S C,$ where $ B\in {\hbox{{\bf R}$^{p\times m}$}},$ $C\in {\hbox{{\bf R}$^{q\times n}$}},$ $S\in {\hbox{{\bf R}$^{p\times q}$}},$ and $p\leq m,$ $q\leq n$. The new algorithm uses diagonal scalings, the LU factorization with complete pivoting, the QR factorization with column pivoting, and matrix multiplication to replace $A$ by $A' = B'^{\tau}S'C',$ where A and A' have the same singular values and the matrix A' is computed explicitly. The singular values of A' are computed using the Jacobi SVD algorithm. It is shown that the accuracy of the new algorithm is determined by (i) the accuracy of the QR factorizations of $B^{\tau}$ and $C^{\tau}$; (ii) the accuracy of the LU factorization with complete pivoting of S; and (iii) the accuracy of the computation of the SVD of a matrix A'$ with moderate $\min_{D=\diag}\kappa_2(A'D)$. Theoretical analysis and numerical evidence show that, in the case of rank(B)= rank(C)=p and full rank S, the accuracy of the new algorithm is unaffected by replacing B, S, C with, respectively, D1 B, D2SD3,D4 C, where Di, i=1, . . .,4, are arbitrary diagonal matrices. As an application, the paper proposes new accurate algorithms for computing the (H,K)--SVD and (H-1,K)--SVD of S.

An algorithm for symmetric generalized inverse eigenvalue problems

Using QR-like decomposition with column pivoting and least squares techniques, we propose a new and efficient algorithm for solving symmetric generalized inverse eigenvalue problems, and give its locally quadratic convergence analysis. We also present some numerical experiments which illustrate the behaviour of our algorithm.

Simplifying fuzzy rule-based models using orthogonal transformation methods

An important issue in fuzzy-rule-based modeling is how to select a set of important fuzzy rules from a given rule base. Even though it is conceivable that removal of redundant or less important fuzzy rules from the rule base can result in a compact fuzzy model with better generalizing ability, the decision as to which rules are redundant or less important is not an easy exercise. In this paper, we introduce several orthogonal transformation-based methods that provide new or alternative tools for rule selection. These methods include an orthogonal least squares (OLS) method, an eigenvalue decomposition (ED) method, a singular value decomposition and QR with column pivoting (SVD-QR) method, a total least squares (TLS) method, and a direct singular value decomposition (D-SVD) method. A common attribute of these methods is that they all work on a firing strength matrix and employ some measure index to detect the rules that should be retained and eliminated. We show the performance of these methods by applying them to solving a nonlinear plant modeling problem. Our conclusions based on analysis and simulation can be used as a guideline for choosing a proper rule selection method for a specific application.