Modified Gram-Schmidt Research Articles

On growth factors of the modified Gram–Schmidt algorithm

AbstractGrowth factors play a central role in studying the stability properties and roundoff estimates of matrix factorizations; therefore, they have attracted many numerical analysts to study upper bounds of these growth factors.In this article, we derive several upper bounds of row‐wise growth factors of the modified Gram–Schmidt (MGS) algorithm to solve the least squares (LS) problem and the weighted LS problem. We also extend the analysis to the MGS‐like algorithm to solve the constrained LS problem. Copyright © 2008 John Wiley & Sons, Ltd.

Propagation property analysis of metamaterial constructed by conductive SRRs and wires using the MGS-based algorithm

An efficient numerical algorithm based on the modified Gram-Schmidt (MGS) procedure is developed in this paper to solve the electrical-field integral equation for characterizing metamaterials. The method of moments (MoM) with rooftop basis functions is implemented in the integral-equation solver. The MGS-based algorithm is implemented into the solver for decomposing the local MoM dense matrix without prior knowledge of the matrix elements. Although each element of the metamaterials is electrically small in size, the number of elements is very large so that the number of unknowns for characterizing the metamaterials is very large. Even so, this algorithm in the MoM solver has demonstrated via examples to be efficient and accurate. Numerical results on the number N of unknowns show that the CPU time per iteration and the memory requirements are both reduced from O(N/sup 2/) to O(N/sup 1.5/). After implementing this algorithm in the solver, propagation characteristics are finally presented when electromagnetic waves pass through a metamaterial prism that is synthesized using square split-ring resonators and wires in free space.

A modified Gram–Schmidt algorithm with iterative orthogonalization and column pivoting

Iterative orthogonalization is aimed to ensure small deviation from orthogonality in the Gram–Schmidt process. Former applications of this technique are restricted to classical Gram–Schmidt (CGS) and column-oriented modified Gram–Schmidt (MGS). The major aim of this paper is to explain how iterative orthogonalization is incorporated into row-oriented MGS. The interest that we have in a row-oriented iterative MGS comes from the observation that this method is capable of performing column pivoting. The use of column pivoting delays the deteriorating effects of rounding errors and helps to handle rank-deficient least-squares problems. A second modification proposed in this paper considers the use of Gram–Schmidt QR factorization for solving linear least-squares problems. The standard solution method is based on one orthogonalization of the r.h.s. vector b against the columns of Q. The outcome of this process is the residual vector, r ∗ , and the solution vector, x ∗ . The modified scheme is a natural extension of the standard solution method that allows it to apply iterative orthogonalization. This feature ensures accurate computation of small residuals and helps in cases when Q has some deviation from orthogonality.

An accurate parallel block Gram-Schmidt algorithm without reorthogonalization

The modified Gram–Schmidt (MGS) orthogonalization process—used for example in the Arnoldi algorithm—often constitutes the bottleneck that limits parallel efficiencies. Indeed, a number of communications, proportional to the square of the problem size, are required to compute the dot-products. A block formulation is attractive but it suffers from potential numerical instability. In this paper, we address this issue and propose a simple procedure that allows the use of a block Gram—Schmidt algorithm while guaranteeing a numerical accuracy close to that of MGS. The main idea is to determine the size of the blocks dynamically. The main advantages of this dynamic procedure are two-fold: first, high performance matrix–vector multiplications can be used to decrease the execution time. Next, in a parallel environment, the number of communications is reduced. Performance comparisons with the alternative Iterated CGS also show an improvement for a moderate number of processors. Copyright © 2000 John Wiley & Sons, Ltd.

Ordered modified Gram-Schmidt orthogonalization revised

Nour-Omid et al. (1991) proposed an ordered modified Gram-Schmidt (MGS) algorithm, which was supposed to improve the orthogonality state of the solution. Thorough analysis, even of a simple, planar case, shows (Štuller, 1994) that, yet in the exact arithmetic, one cannot expect to obtain — independently of the forward, reverse, or any other type, including “ordered”, orthogonalization — the desired solution: a vector in the orthogonal complement of the given vectors. For the planar case, some simple rules of the thumb can be given to minimize the error, but we are not sure they can be directly generalized to higher dimensions as has been done by Nour-Omid et al. Naturally, in finite precision, the situation is even worse. The unique way out we see in the iterative (Bjorck, 1994) Gram-Schmidt methods, for which we (1994) presented several algorithms with different criterions of efficiency applied to.

A NEW MODIFIED GRAM-SCHMIDT ORTHOGONAL MATRIX FACTORIZATION BASED ALGORITHM FOR PARALLEL SOLUTION OF LINEAR EQUATIONS

In this paper, we present a new algorithm based on Modified Gram-Schmidt (MGS) orthogonal matrix factorization for parallel solution of linear equations Ax = b. Unlike the existing methods, the proposed algorithm using a new technique called two-sided elimination unifies both the triangularization and back-substitution phases to produce the complete solution vector x. The new algorithm replaces the back-substitution phase in the existing methods, which requires O(N) steps using O(N) processors or O(log2 2N) steps using O(N3) processors, by only one step division. Being based on the MGS method, the new algorithm is numerically stable. Finally, we study the performance of the algorithm on hypercube multiprocessor systems.

Real-time design of an adaptive nonlinear predictive controller

Abstract Based on real-time identification and using the concept of NARX (Nonlinear AutoRegressive with exogenous inputs) models, a new adaptive nonlinear predictive controller (ANPC) design is proposed. NARX models represent a natural way to describe the input-output relationship of severely nonlinear systems. From an initial batch of input-output data, a parsimonious NARX model is obtained using the Modified Gram-Schmidt (MGS) orthogonalization algorithm. Following this initial off-line identification and model reduction procedure, the control loop is closed. The ANPC directly uses the obtained structure and initial parameter estimates, which are updated each time step using recursive identification. The controller is designed similar to a typical linear predictive controller based on solving a nonlinear programming (NLP) problem. This paper shows how to solve this NLP problem on-line without the knowledge of the NARX model structure. The design is given for the multi-input multi-output (MIMO) case.

Solution of augmented linear systems using orthogonal factorizations

We consider solvingx+Ay=b andATx=c for givenb, c andm ×n A of rankn. This is called the augmented system formulation (ASF) of two standard optimization problems, which include as special cases the minimum 2-norm of a linear underdetermined system (b=0) and the linear least squares problem (c=0), as well as more general problems. We examine the numerical stability of methods (for the ASF) based on the QR factorization ofA, whether by Householder transformations, Givens rotations, or the modified Gram-Schmidt (MGS) algorithm, and consider methods which useQ andR, or onlyR. We discuss the meaning of stability of algorithms for the ASF in terms of stability of algorithms for the underlying optimization problems.

Recursive least-squares algorithms of modified Gram-Schmidt type for parallel weight extraction

This paper presents some new algorithms for parallel weight extraction in the recursive least-squares (RLS) estimation based on the modified Gram-Schmidt (MGS) method. These are the counterparts of the algorithms using an inverse QR decomposition based on the Givens rotations and do not contain the square root operation. Systolic-array implementations of the algorithms are considered on a 2-D rhombic array. Simulation results are also presented to compare the finite word-length effect of these new algorithms and existing algorithms.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A Newton basis GMRES implementation

The GMRES method by Saad and Schultz is one of the most popular iterative methods for the solution of large sparse non-symmetric linear systems of equations. The implementation proposed by Saad and Schultz uses the Arnoldi process and the modified Gram-Schmidt (MGS) method to compute orthonormal bases of certain Krylov subspaces. The MGS method requires many vector-vector operations, which can be difficult to implement efficiently on vector and parallel computers due to the low granularity of these operations. We present a new implementation of the GMRES method in which, for each Krylov subspace used, we first determine a Newton basis, and then orthogonalize it by computing a QR factorization of the matrix whose columns are the vectors of the Newton basis. In this way we replace the vector-vector operations of the MGS method by the task of computing a QR factorization of a dense matrix. This makes the implementa tion more flexible, and provides a possibility to adapt the computations to the computer at hand in order to achieve better performance.

A Recursive Modified Gram-Schmidt Identification with Directional Tracking of Parameters

Abstract The new version of the Recursive Modified Gram- Schmidt algorithm with directional parameter tracking technique is presented for the input-output regression models. The directional forgetting improves significantly the numerical behavior of algorithm when data are not persistently exciting. The algorithm allows a highly modular and numerically safe implementation even if an extremely fast parameter tracking speed is desired. The algorithm works with a modular triangular structure of reflection coefficients. A comparative study demonstrates the improved numerical performance over the standard square root algorithms with the exponential parameter tracking.

An efficient algorithm and systolic architecture for multiple channel adaptive filtering

A multiple-input-multiple-output orthogonalization algorithm and its efficient systolic implementation are presented. The processing architecture is developed using a basic two-input-two-output decorrelation processing element as the primitive building block. Its features are discussed and compared to the approach of K. Gerlach and F.A. Studer (see ibid., vol.AP-34, no.3, p.458-462, 1986) which is based on the modified Gram-Schmidt (MGS) orthogonalization procedure. For simplicity of illustration in the development, batch processing is emphasized. The main features of the newly developed multiple-channel orthogonalization architecture are: (1) it requires no broadcasting of data and any given processing node in the structure only communicates with its neighboring nodes in pipelining fashion; (2) in terms of the total number of arithmetic operations, it is at least as efficient as the MGS approach; (3) the new architecture is developed in a systematic and bottom-up fashion; (4) it is an extremely regular and compact processing structure; (5) no unscrambling of the output channels is needed; and (6) the architecture presented places no restriction on the number of input channels. >

Linear Least Squares by Elimination and MGS

An algorithm combining Gaussian elimination with the modified Gram-Schmidt (MGS) procedure is given for solving the linear least squares problem. The method is based on the operational efficiency of Gaussian elimination for LU decompositions and the numerical stability of MGS for unitary decompositions and is designed for slightly overdetermined linear systems.

Experiments on Error Growth Associated with Some Linear Least-Squares Procedures

Some numerical experiments were performed to compare the performance of procedures for solving the linear least-squares problem based on GramSchmidt, Modified Gram-Schmidt, and Householder transformations, as well as the classical method of forming and solving the normal equations. In addition, similar comparisons were made of the first three procedures and a procedure based on Gaussian elimination for solving an $n \times n$ system of equations. The results of these experiments suggest that: (1) the Modified Gram-Schmidt procedure is best for the least-squares problem and that the procedure based on Householder transformations performed competitively; (2) all the methods for solving least-squares problems suffer the effects of the condition number of $\begin {array}{*{20}{c}} A & {^T} & A \\ \end {array}$, although in a different manner for the first three procedures than for the fourth; and (3) the procedure based on Gaussian elimination is the most economical and essentially, the most accurate for solving $n \times n$ systems of linear equations. Some effects of pivoting in each of the procedures are included.

Experiments on error growth associated with some linear least-squares procedures

Some numerical experiments were performed to compare the performance of procedures for solving the linear least-squares problem based on GramSchmidt, Modified Gram-Schmidt, and Householder transformations, as well as the classical method of forming and solving the normal equations. In addition, similar comparisons were made of the first three procedures and a procedure based on Gaussian elimination for solving an n × n n \times n system of equations. The results of these experiments suggest that: (1) the Modified Gram-Schmidt procedure is best for the least-squares problem and that the procedure based on Householder transformations performed competitively; (2) all the methods for solving least-squares problems suffer the effects of the condition number of A a m p ; T a m p ; A \begin {array}{*{20}{c}} A &amp; {^T} &amp; A \\ \end {array} , although in a different manner for the first three procedures than for the fourth; and (3) the procedure based on Gaussian elimination is the most economical and essentially, the most accurate for solving n × n n \times n systems of linear equations. Some effects of pivoting in each of the procedures are included.