Fast Least-squares Algorithms Research Articles

Random phase retrieval approach using Euclidean matrix norm of sum and difference map and fast least-squares algorithm

In order to meet the requirements of the in-situ optical measurement, a random phase retrieval approach using the Euclidean matrix norm of sum and difference map and fast least-squares algorithm (EMNSD&FLSA) is proposed. It can obtain the high accuracy as the iterative algorithm and cost less computational time as the non-iterative algorithm simultaneously. No pre-filtering, iteration and relatively accurate initial phase lead to the high accuracy, fast iterative method and only two phase shifted interferograms lead to the high speed. It can obtain absolutely accurate result when the background intensity and modulation amplitude are perfect. Moreover, it is suitable for different levels of noises. If the high accuracy is required, it would be best to suppress the noise before using EMNSD&FLSA, and the phase shift would be best to far away from 0 rad and π rad, furthermore, the numbers of patterns are best to be more than 2. Last but not least, it is robust for the circular, straight or complex fringes. The simulations and experiments verify the correctness and feasibility of EMNSD&FLSA.

A fast least-squares algorithm for population inference.

BackgroundPopulation inference is an important problem in genetics used to remove population stratification in genome-wide association studies and to detect migration patterns or shared ancestry. An individual’s genotype can be modeled as a probabilistic function of ancestral population memberships, Q, and the allele frequencies in those populations, P. The parameters, P and Q, of this binomial likelihood model can be inferred using slow sampling methods such as Markov Chain Monte Carlo methods or faster gradient based approaches such as sequential quadratic programming. This paper proposes a least-squares simplification of the binomial likelihood model motivated by a Euclidean interpretation of the genotype feature space. This results in a faster algorithm that easily incorporates the degree of admixture within the sample of individuals and improves estimates without requiring trial-and-error tuning.ResultsWe show that the expected value of the least-squares solution across all possible genotype datasets is equal to the true solution when part of the problem has been solved, and that the variance of the solution approaches zero as its size increases. The Least-squares algorithm performs nearly as well as Admixture for these theoretical scenarios. We compare least-squares, Admixture, and FRAPPE for a variety of problem sizes and difficulties. For particularly hard problems with a large number of populations, small number of samples, or greater degree of admixture, least-squares performs better than the other methods. On simulated mixtures of real population allele frequencies from the HapMap project, Admixture estimates sparsely mixed individuals better than Least-squares. The least-squares approach, however, performs within 1.5% of the Admixture error. On individual genotypes from the HapMap project, Admixture and least-squares perform qualitatively similarly and within 1.2% of each other. Significantly, the least-squares approach nearly always converges 1.5- to 6-times faster.ConclusionsThe computational advantage of the least-squares approach along with its good estimation performance warrants further research, especially for very large datasets. As problem sizes increase, the difference in estimation performance between all algorithms decreases. In addition, when prior information is known, the least-squares approach easily incorporates the expected degree of admixture to improve the estimate.

Past input reconstruction in fast least-squares algorithms

This paper solves the following problem: given the computed variables in a fast least-squares prediction algorithm, determine all past input sequences that would have given rise to the variables in question. This problem is motivated by the backward consistency approach to numerical stability in this algorithm class; the set of reachable variables in exact arithmetic is known to furnish a stability domain. Our problem is equivalent to a first- and second-order interpolation problem introduced by Mullis and Roberts (1976) and studied by others. Our solution differs in two respects. First, relations to classical interpolation theory are brought out, which allows us to parametrize all solutions. By-products of our formulation are correct necessary and sufficient conditions for the problem to be solvable, in contrast to previous works, whose claimed sufficient conditions are shown to fall short. Second, our solution obtains any valid past input as the impulse response of an appropriately constrained orthogonal filter, whose rotation parameters derive in a direct manner from the computed variables in a fast least-squares prediction algorithm. Formulas showing explicitly the form of all valid past inputs should facilitate the study of what past input perturbation is necessary to account for accumulated arithmetic errors in this algorithm class. This, in turn, is expected to have an impact in studying accuracy aspects in fast least-squares algorithms.

A fast least-squares algorithm for linearly constrained adaptive filtering

An extension of the field of fast least-squares techniques is presented. It is shown that the adaptation gain, which is updated with a number of operations proportional to the number of transversal filter coefficients, can be used to update the coefficients of a linearly constrained adaptive filter. An algorithm that is robust to round-off errors is derived. It is general and flexible. It can handle multiple constraints and multichannel signals. Its performance is illustrated by simulations and compared with the classical LMS-based Frost (1972) algorithm.

Displacement Structures of Covariance Matrices, Lossless Systems, and Numerical Algorithm Design

Low displacement rank theory underlies many fast algorithms designed for structured covariance matrices. Some of these have gained notoriety for their numerical instability problems, particularly fast least-squares algorithms. Recent studies have shown that instability is not inherent to fast algorithms, but rather comes from the violation of backward consistency constraints. This paper thus details the connection between covariance matrices of a given displacement inertia and lossless rational matrices, as well as the role of this connection in numerically consistent algorithms. This basic connection allows displacement structures to be parametrized via a sequence of rotation angles obtained from a lossless system. The utility of this approach is that, irrespective of errors in the rotation parameter set, they remain consistent with a positive definite matrix of a prescribed displacement inertia. This property in turn may be rephrased as meaningful forms of backward consistency in numerical algorithms. The rotation parameters then take the form of Givens or Jacobi angles applied to data, in contrast to classical approaches which directly manipulate dyadic decompositions of the displacement structure. The concepts are illustrated in popular signal processing applications. In particular, these connections lend clear insight into the stable computation of reflection coefficients of Toeplitz systems, and also serve to resolve the numerical instability problem of fast least-squares algorithms.

Numerical stability properties of a QR-based fast least squares algorithm

The numerical stability of a recent QR-based fast least-squares algorithm is established from a backward stability perspective. A stability domain approach applicable to any least-squares algorithm, constructed from the set of reachable states in exact arithmetic, is presented. The error propagation question is shown to be subordinate to a backward consistency constraint, which requires that the set of numerically reachable variables be contained within the stability domain associated to the algorithm. This leads to a conceptually lucid approach to the numerical stability question which frees the analysis of stationary assumptions on the filtered sequences and obviates the tedious linearization methods of previous approaches. Moreover, initialization phenomena and considerations related to poorly exciting inputs admit clear interpretations from this perspective. The algorithm under study is proved, in contrast to many fast algorithms, to be minimal.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Numerical stability issues in fast least-squares adaptation algorithms

The concepts of backward consistency and minimality are shown to be the essential tools in studying error propagation in fast least-squares adaptive filters. A conceptual framework identifying the principal error propagation mechanisms is developed, allowing the numerical stability of new or existing algorithms to be ascertained withoutthe technical labor previously thought necessary. The key findings show that arithmetic instability is <i>not</i> a fundamental feature of fast least-squares algorithms. Rather, unstable propagation can only derive from the violation of backward consistency constraints. The absence of minimality (in the system theory sense) is identified as the principal culprit in this regard, although the issue of stability for predictable sequences also plays an intriguing role. The concepts are illustrated for some popularly used algorithms, and order-recursive algorithms are confirmed to have intrinsic advantages over their transversal counterparts.

A fast multichannel approach to adaptive image estimation

A computationally efficient method for adaptive image estimation is developed, based on the multichannel form of the one-dimensional fast least-squares algorithms. Extended forms of various two-dimensional autoregressive image models are derived and used for this purpose. It is shown that the method, named the fast multichannel space recursive estimation technique (FAMSRET), can overcome image boundary discontinuity problems by processing consecutive image lines in opposite directions. Two-dimensional instrumental variables are introduced and used by the FAMSRET algorithm for the efficient estimation of images, which are degraded by additive white noise. Examples are given that illustrate the performance of the proposed techniques.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Simplified fast least-squares algorithms for speech predictors

The letter deals with the development of a simplified fast least-squares algorithm which is free of roundoff error accumulation. The simplified algorithm requires 9N MADPR (multiplications and divisions per recursion) rather than 10N MADPR as in the fast least squares or fast Kalman (FLS) case where N is the order of predictor. The superiority of SFLS over FLS and LMS approaches is illustrated by prediction gain performance curves for various speech signals.

Error propagation properties of recursive least-squares adaptation algorithms

The numerical properties of implementations of the recursive least-squares identification algorithm are of great importance for their continuous use in various adaptive schemes. Here we investigate how an error that is introduced at an arbitrary point in the algorithm propagates. It is shown that conventional LS algorithms, including Bierman's UD-factorization algorithm are exponentially stable with respect to such errors, i.e. the effect of the error decays exponentially. The base of the decay is equal to the forgetting factor. The same is true for fast lattice algorithms. The fast least-squares algorithm, sometimes known as the ‘fast Kalman algorithm’ is however shown to be unstable with respect to such errors.