Large-scale Least-squares Problems Research Articles

Efficient variance component estimation for large-scale least-squares problems in satellite geodesy

Efficient variance component estimation for large-scale least-squares problems in satellite geodesy

Weighted Monte Carlo with Least Squares and Randomized Extended Kaczmarz for Option Pricing

We propose a methodology for computing single and multi-asset European option prices, and more generally expectations of scalar functions of (multivariate) random variables. This new approach combines the ability of Monte Carlo simulation to handle high-dimensional problems with the efficiency of function approximation. Specifically, we first generalize the recently developed method for multivariate integration in [arXiv:1806.05492] to integration with respect to probability measures. The method is based on the principle “approximate and integrate” in three steps i) sample the integrand at points in the integration domain, ii) approximate the integrand by solving a least-squares problem, iii) integrate the approximate function. In high-dimensional applications we face memory limitations due to large storage requirements in step ii). Combining weighted sampling and the randomized extended Kaczmarz algorithm we obtain a new efficient approach to solve large-scale least-squares problems. Our convergence and cost analysis along with numerical experiments show the effectiveness of the method in both low and high dimensions, and under the assumption of a limited number of available simulations.

An Efficient Algorithm for Real-Time Estimation and Prediction of Dynamic OD Tables

The problem of estimating and predicting Origin-Destination (OD) tables is known to be important and difficult. In the specific context of Intelligent Transportation Systems (ITS), the dynamic nature of the problem and the real-time requirements make it even more intricate. We consider here a least-square modeling approach for solving the OD estimation and prediction problem, which seems to offer convenient and flexible algorithms. The dynamic nature of the problem is represented by an autoregressive process, capturing the serial correlations of the state variables. Our formulation is inspired from Cascetta et al. (1993) and Ashok and Ben-Akiva (1993). We compare the Kalman filter algorithm to LSQR, an iterative algorithm proposed by Paige and Saunders (1982) for the solution of large-scale least-squares problems. LSQR explicitly exploits matrix sparsity, allowing to consider larger problems likely to occur in real applications. We show that the LSQR algorithm significantly decreases the computation effort needed by the Kalman filter approach for large-scale problems. We also provide a theoretical number of flops for both algorithms to predict which algorithm will perform better on a specific instance of the problem.

The solution of large-scale least-squares problems on supercomputers

We discuss methods for solving medium to large-scale sparse least-squares problems on supercomputers, illustrating our remarks by experiments on the CRAY-2 supercomputer at Harwell. The method we are primarily concerned with is an augmented system approach which has the merit of both robustness and accuracy, in addition to a kernel operation that is just the solution of a symmetric indefinite system. We consider extensions to handle weighted and constrained problems, and include experiments on systems similar to those arising in the Karmarkar algorithm for linear programming. We indicate how recent improvements to the kernel software could greatly improve the performance of the least-squares code.

Block AOR Iterative Schemes for Large-Scale Least-Squares Problems

The problem of accelerating the convergence rate of iterative schemes, as they apply to the solution of large-scale least-squares problems by means of different splittings, is addressed here. New convergence results, together with explicit expressions for the optimal factors and their corresponding spectral radii, are derived for certain schemes from the block Accelerated Overrelaxation (AOR) family. It is shown that, for a class of least-squares problems, the proposed optimal block iterative scheme converges unconditionally and asymptotically faster than the optimal block Successive Overrelaxation (SOR) method, while in all other cases the two schemes are competitive. Numerical examples are used to illustrate our results.

An algorithmic approach for the analysis of extrapolated iterative schemes applied to least-squares problems

The problem of determining the optimal values of extrapolated iterative schemes, as they apply to the solution of large-scale least-squares problems, is addressed here. Based on algebraic and geometric eigenvalue properties of the Accelerated Gauss—Seidel (AGS), we devise a simple algorithmic procedure, which successfully yields the optimal values of the Extrapolated AGS (EAGS). Comparisons with the optimal SOR scheme reveal that the two optimal schemes strongly compete. Numerical examples are used to demonstrate our results.

Convergence of a direct-iterative method for large-scale least-squares problems

In 1975 Chen and Gentleman suggested a 3-block SOR method for solving least-squares problems, based on a partitioning scheme for the observation matrix A into A= A 1 A 2 where A 1 is square and nonsingular. In many cases A 1 is obvious from the nature of the problem. This combined direct-iterative method was discussed further and applied to angle adjustment problems in geodesy, where A 1 is easily formed and is large and sparse, by Plemmons in 1979. Recently, Niethammer, de Pillis, and Varga have rekindled interest in this method by correcting and extending the SOR convergence interval. The purpose of our paper is to discuss an alternative formulation of the problem leading to a 2-block SOR method. For this formulation it is shown that the resulting direct-iterative method always converges for sufficiently small SOR parameter, in contrast to the 3-block formulation. Formulas for the optimum SOR parameter and the resulting asymptotic convergence factor, based upon ‖ A 2 A -1 1‖ 2, are given. Furthermore, it is shown that this 2-cyclic block SOR method always gives better convergence results than the 3-cyclic one for the same amount of work per iteration. The direct part of the algorithm requires only a sparse-matrix factorization of A 1. Our purpose here is to establish theoretical convergence results, in line with the purpose of the recent paper by Niethammer, de Pillis, and Varga. Practical considerations of choosing A 1 in certain applications and of estimating the resulting ‖ A 2 A -1 1‖ 2 will be addressed elsewhere.