Large Linear Problems Research Articles

Reconstruction of electron precipitation characteristics from a set of multiwavelength digital all‐sky auroral images

We present a method for reconstructing the precipitating electron flux from a set of multiwavelength digital all‐sky auroral images. The method involves solving a large linear inversion problem, and it works for any number of all‐sky imager stations and wavelengths. The idea in our method is to invert the energy‐dependent precipitating electron flux directly and not the three‐dimensional volume emission rate as an intermediate step. In this way, the inversion is automatically restricted to physical emission rate profiles. We show the effectiveness of the method in the all‐sky camera case by simulated examples. The method is also applied to real data. The method is capable of correctly reconstructing not only auroral arc positions but also the precipitating electron energy and number fluxes, although not necessarily the full differential flux. A natural limitation is that arcs that are significantly off‐zenith and are viewed sideways by only one all‐sky camera station are smeared out. However, even for these arcs the average arc position and energy characteristics come out with reasonable accuracy.

Preface

The workshop will be held in conjunction with the www.di.unipi.it/cp98/, Fourth International Conference on Principles and Practice of Constraint Programming (CP98), October 26 - 30, 1998 This workshop will explore hybrid algorithms for complex optimisation problems. These will be based on the integration of different approaches, viz. constraint programming, mathematical programming and stochastic repair techniques. These approaches have sometimes been perceived as competitors, but more recently they have come to be seen as complementary. The need for such algorithms in solving optimisation problems arises from the fact that different techniques are suited to solving different aspects of the problem. Among the key techniques are constraint propagation, Lagrangean relaxation, linear programming, randomised search and local optimisation. Constraint propagation is ideal for customising an algorithm with domain-dependent constraints. Lagrangean relaxation is the technique of choice for obtaining lower bounds in a branch and bound approach. Linear programming is the only technique that can solve to optimiality large linear problems. Randomised search is necessary for problems that are too large for branch and bound. Local optimisation is the preferred technique for problems that have a specific neighbourhood structure, such as routing problems. Many optimisation problems are hybrid, embracing several different requirements. For example real-life dispatching problems involve both matching, routing and scheduling. For such problems there is a clear need to mix techniques. This mixing can take different forms. On one hand generic methods, such as tabu search or constraint propagation can be significantly enhanced by importing techniques developed for branch and bound (such as edge-finding) or for local optimisation (such as neighbourhood structure). On the other hand different techniques can be applied at different stages of the search. For instance initial solutions can be found using constructive search with constraint propagation, and the solution can then be improved through local optimisation. Mark Wallace IC-Parc William Penney Laboratory Imperial College London SW7 2BZ UK Tel: (+44)171.594834 Fax: (+44)171.5948432 [email protected] Yves Caseau Bouygues Challenger, 1 Av Eugene Freyssinet, 78061 St-Quentin-Yvelines Cedex FRANCE Tel: (+33)1 30 60 51 57 Fax: (+33)1 30 60 27 27 [email protected] Eric Jacquet-Lagreze EuroDecision Helmut Simonis Cosytec Gilles Pesant Centre for Research on Transportation, Univ. Montreal Additional information will be available at the CP98 web site: http://www.di.unipi.it/cp98/

Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems

{The conjugate gradient method applied to the normal equations ATAx=ATb (CGLS) is often used for solving large sparse linear least squares problems. The mathematically equivalent algorithm LSQR based on the Lanczos bidiagonalization process is an often recommended alternative. In this paper, the achievable accuracy of different conjgate gradient and Lanczos methods in finite precision is studied. It is shown that an implementation of algorithm CGLS in which the residual sk=AT(b-Axk) of the normal equations is recurred will not in general achieve accurate solutions. The same conclusion holds for the method based on Lanczos bidiagonalization with starting vector ATb. For the preferred implementation of CGLS we bound the error ||r-rk|| of the computed residual rk. Numerical tests are given that confirm a conjecture of backward stability. The achievable accuracy of LSQR is shown to be similar. The analysis essentially also covers the preconditioned case.

A stationary iterative pseudoinverse algorithm

Iterative methods applied to the normal equationsA T Ax=A T b are sometimes used for solving large sparse linear least squares problems. However, when the matrix is rank-deficient many methods, although convergent, fail to produce the unique solution of minimal Euclidean norm. Examples of such methods are the Jacobi and SOR methods as well as the preconditioned conjugate gradient algorithm. We analyze here an iterative scheme that overcomes this difficulty for the case of stationary iterative methods. The scheme combines two stationary iterative methods. The first method produces any least squares solution whereas the second produces the minimum norm solution to a consistent system.

Sparse Linear Least Squares Problems in Optimization

Numerical and computational aspects of direct methods for large and sparse least squares problems are considered. After a brief survey of the most often used methods, we summarize the important conclusions made from a numerical comparison in matlab. Significantly improved algorithms have during the last 10-15 years made sparse QR factorization attractive, and competitive to previously recommended alternatives. Of particular importance is the multifrontal approach, characterized by low fill-in, dense subproblems and naturally implemented parallelism. We describe a Householder multifrontal scheme and its implementation on sequential and parallel computers. Available software has in practice a great influence on the choice of numerical algorithms. Less appropriate algorithms are thus often used solely because of existing software packages. We briefly survey software packages for the solution of sparse linear least squares problems. Finally, we focus on various applications from optimization, leading to the solution of large and sparse linear least squares problems. In particular, we concentrate on the important case where the coefficient matrix is a fixed general sparse matrix with a variable diagonal matrix below. Inner point methods for constrained linear least squares problems give, for example, rise to such subproblems. Important gains can be made by taking advantage of structure. Closely related is also the choice of numerical method for these subproblems. We discuss why the less accurate normal equations tend to be sufficient in many applications.

Reply to comment by M. M. Deal and G. Nolet on ‘Estimation of resolution and covariance for large matrix inversions'

We appreciate the comments made by Deal & Nolet (1996, hereafter referred to as DN Berryman 19941. For large linear problems, the number of iterations needed to obtain an acceptable solution is smaller than the full SVD rank of the original matrix. Nonetheless, the extremal Ritz values are often good approximations to the corresponding singular values of the original matrix (Golub & Van Loan 1989, p.479), as shown by the examples in both ZM the computed Ritz values are spaced across the range of the singular values of the original matrix with a distribution defined by zeros of Jacobi polynomials (van der Sluis & van der Vorst 1990). To explore the solution subspace more fully, additional iterations are performed beyond that at which an acceptable solution is defined. After a sufficient number of iterations, Ritz values converge to singular values but with the consequences that Lanczos and Ritz vectors are no longer orthogonal; some duplicate Ritz values and vectors are generated. The selective orthogonalization method of Parlett & Scott (1979) addresses this loss of orthogonality at the cost of increased algorithm complexity and reduced numerical efficiency. We choose to address this problem by identifying and eliminating the duplicate Ritz values and Ritz vectors (also called Ritz pairs) (Scales 1989; Parlett 1980, p. 272), and to use the remaining (nearly) orthogonal Ritz pairs to construct our LSQRA solution. It should be noted that it is not necessary to approximate an LSQRA solution to get the resolution and covariance matrices; the Lanczos decomposition is sufficient.

Full waveform inversion of field sonar returns for a visco-acoustic Earth; a comparison of linearized and fully nonlinear methods

The techniques of linearized least squares inversion (LLSI) and simulated annealing (SA) are both used to invert a series of synthetic and real normal-incidence, geo-acoustic sonar returns for estimates of impedance versus two-way travel time in the top several meters of ocean floor sediment. The objective is to determine the better (faster, more accurate) method for inverting this class of data. LLSI uses an over parameterized earth, i.e., one composed of layers whose thickness corresponds to a travel time equal to the sample interval. This makes the inverse problem quite large, but also makes it nearly linear. SA uses a more efficient parameterization, one whose layers have variable thickness as well as variable impedance. Because of the relatively narrow frequency band (/spl sim/1 octave at 20 dB down from the peak) the time domain signal is oscillatory and inversion for layer thickness is nonlinear. Results show greater time efficiency in solving the large linear problem (LLSI) than in solving the small nonlinear problem (SA). However, in both cases almost all of the waveform energy was modeled, indicating that essentially all the information in the data had been successfully recovered. The inversions are applied to 10-20 kHz field data acquired offshore Florida, and several techniques are employed to enhance the effectiveness of each inversion method.

On Row Relaxation Methods for Large Constrained Least Squares Problems

This paper addresses the question of how to construct a row relaxation method for solving large unstructured linear least squares problems, with or without linear constraints. The proposed approach combines the Herman–Lent–Hurwitz scheme for solving regularized least squares problems with the Lent–Censor–Hildreth method for solving linear constraints. However, numerical experiments show that the Herman–Lent–Hurwitz scheme has difficulty reaching a least squares solution. This difficulty is resolved by applying the Riley–Golub iterative improvement process.

Parallel subspace method for non-Hermitian eigenproblems on the Connection Machine (CM2)

In this paper we present a parallel implementation of Arnoldi's subspace method on the Connection Machine. With a 16K-processor CM2, we obtained performances of a few hundred Megaflops for a matrix size of several thousands when computing a small number of eigenvalues and eigenvectors. The extrapolated performance on a 64K-processor CM2 indicates that the asymptotic speed will be greater than 1 Gigaflop for very large matrices. We show that it is possible to use the subspace method with a good throughput or speed-up on massively-parallel architectures like the CM2. We remark that other classical methods for linear algebra problems such as, for example, backsubstitution and the QR method, cannot exploit all the potential power of massively-parallel machines. Next, we propose using the subspace method as a programming methodology for massively-parallel machines in order to obtain a good performance when solving some large linear algebra problems, especially eigenproblems. This method is the most frequently one used for very large eigenproblems and it is also well adapted to massively-parallel architectures. The choice of the subspace size is very important both for numerical stability and speed-up. We conclude with a discussion on the effects of the orders chosen for the subspaces on performance.

On the uniqueness of the solution to a large linear assignment problem

The core of a linear assignment problem contains a continuum of allocations. A model is presented where this core shrinks to a unique solution of the limit problem as the number of agents grows.

Solving large linear inverse problems by projection

SUMMARY As originally formulated by Backus & Gilbert (1970), ill-posed linear inverse problems possess a unique minimum norm solution, and a locally averaged property of the model may be estimated with a resolution that is a monotonic function of its variance. Application of Backus-Gilbert theory requires the inversion of an N X N matrix, where N is the number of data, and therefore becomes cumbersome for large N. In this paper we show how Lanczos iteration may be used to project the original linear problem on a problem of much smaller size in order to obtain an approximation to the Backus-Gilbert solution without the need of matrix inversion. To calculate the resolution in the projected system one only needs to invert a symmetric tridiagonal matrix.

Solving large and sparse linear least-squares problems by conjugate gradient algorithms

Let A ε ℛm × n(with m ⩾ n and rank (A) = n) and b ε ℛm × 1 be given. Assume that an approximation to x = A†b = (ATA)−1ATb is to be calculated. This problem is transformed into an equivalent problem with a symmetric and positive definite matrix C = D−1 (RT)−1ATAR−1D−1 where D is a diagonal matrix and R is an upper triangular matrix. The conjugate gradient (CG) algorithm is applied in the solution of the system of linear algebraic equations Cy = d (y = DRx, d = D−1(RT)−1ATb). If D = R = I, then the CG algorithm is in fact applied to the system of normal equations ATAx = ATb and the speed of convergence could be very slow. If D and R are obtained by some kind of orthogonalization DR = QTA with Q εℛm × n satisfying QTQ = I; D is often equal to I and Q is never used in the CG algorithm) and if the calculations are performed without rounding errors, then C = I and the CG algorithm converges in one iteration only. Even if the orthogonalization process is carried out with rounding errors, the matrix C is normally close to the identity matrix I and the CG algorithm is quickly convergent. However, for large m and n the orthogonal decomposition is an expensive process (both in regard to storage and in regard to computing time). Therefore it may be profitable to calculate an incomplete orthogonal decomposition. This is achieved by introducing a special parameter T, a drop-tolerance, such that all elements which in the course of the computations become smaller in absolute value than T are removed. Numerical examples are given to illustrate that the CG algorithm applied to system Cy = d and used with incomplete factors D and R is very efficient for some classes of problems. It should be emphasized that matrix C is never calculated explicitly; the whole work is carried out by the use of A, D and R only.

A Comparison Between Some Direct and Iterative Methods for Certain Large Scale Geodetic Least Squares Problems

The purpose of this paper is to describe and compare some numerical methods for solving large dimensional linear least squares problems that arise in geodesy and, more specially, from Doppler positioning. The methods that are considered are the direct orthogonal decomposition, and the combination of conjugate gradient type algorithms with projections as well as the exploitation of “Property A”. Numerical results are given and the respective advantage of the methods are discussed with respect to such parameters as CPU time, input/output and storage requirements. Extensions of the results to more general problemsare also discussed.

Errors in reduction methods

A mathematical basis is given for comparing the relative merits of various techniques used to reduce the order of large linear and non-linear dynamics problems during their numerical integration. In such techniques as Guyan-Irons, path derivatives, selected eigenvectors, Ritz vectors, etc., the nth order initial value problem of [ /.y = f(y) for t > 0, y(0) given] is typically reduced to the mth order ( m ⪡ n) problem of z ̇ = g(z) for t > 0, z(0) given] by the transformation y = Pz where P changes from technique to technique. This paper gives an explicit approximate expression for the reduction error e i in terms of P and the Jacobian of f. It is shown that: (a) reduction techniques are more accurate when the time rate of change of the response y is relatively small; (b) the change in response between two successive stations contributes to the errors at future stations after the change in response is transformed by a filtering matrix H, defined in terms of P; (c) the error committed at a station propagates to future stations by a mixing and scaling matrix G, defined in terms of P, Jacobian of f, and time increment h. The paper discusses the conditions under which the reduction errors may be minimized and gives guidelines for selecting the reduction basis vector, i.e. the columns of P.

The role of modern control theory in the design of controls for aircraft turbine engines

ONTROL systems for early aircraft turbine engines performed a rather simple task: metering the fuel to the combustor at the proper fuel-to-air ratio for both transient and steady-state operating conditions. More recently, however, things have changed significantly. To achieve higher thrust-to-weight ratios and to improve specific fuel consumption, many additional manipulated engine inputs have been added. Figure 1 shows the trend in complexity in terms of the number of controlled engine variables. Noted are a number of operational engines that have been put into service. Typical of the added engine manipulated inputs are afterburner fuel flow, variable compressor-inlet guide vanes, variable compressor stators, and variable exhaust-nozzle area. The task of designing a control algorithm for an engine having a number of inputs and outputs now becomes a formidable problem. Traditional single-input/single-output techniques can be used, but are often inadequate and require many judgmental interactions to get even close to a suitable engine control law. The designer would really like a direct, straightforward method for handling the multivariable problem. This procedure should be able to eliminate unwanted interactions between the different variables, while bringing into play those interactions that are favorable. Faced with these needs, in the early 1970s the engine control community began to investigate what new design methodologies were available. As a result, a number of researchers began to apply to the engine control design problem tools from the evolving methodology generally termed control theory (MCT). This paper reviews what has been accomplished in applying MCT to the aircraft turbine engine control design problem over the last 10 or so years and what work yet remains to be done. This review is organized as follows: first is a brief discussion of the evolution of control design methodology, followed by a description of the problems that must be faced in applying MCT to the engine control design task. The past accomplishments in applying MCT to engine control is the subject of the third and most detailed section. The paper concludes with a discussion of the future requirements for advanced engine control design. Evolution of Control Design Methodology In the early 1960s, control theoreticians began to use linear state-space methods to design closed-loop controls for large complex physical systems. At the same time, computers that could easily solve the numerical problems associated with large linear system problems were rapidly evolving and becoming readily accessible. Thus began the era of modern control theory. This terminology was used to differentiate the new methods from the traditional linear system, single

Numerical Methods for Large Sparse Linear Least Squares Problems

Large sparse least squares problems arise in many applications, including geodetic network adjustments and finite element structural analysis. Although geodesists and engineers have been solving such problems for years, it is only relatively recently that numerical analysts have turned attention to them. In this paper we present a survey of numerical methods for large sparse linear least squares problems, focusing mainly on developments since the last comprehensive surveys of the subject published in 1976. We consider direct methods based on elimination and on orthogonalization, as well as various iterative methods. The ramifications of rank deficiency, constraints, and updating are also discussed.

A Comparison of Some Methods for Solving Sparse Linear Least-Squares Problems

The method of normal equations, the Peters–Wilkinson algorithm and an algorithm based on Givens rotations for solving large sparse linear least squares problems are discussed and compared. Numerical experiments show that the method of normal equations should be considered when the observation matrix is sparse and well conditioned. For ill-conditioned problems, the algorithm based on Givens rotations is preferable.

Solution of Large-Scale Sparse Least Squares Problems Using Auxiliary Storage

Very large sparse linear least squares problems arise in a variety of applications, such as geodetic network adjustments, photogrammetry, earthquake studies, and certain types of finite element analysis. Many of these problems are so large that it is impossible to solve them without using auxiliary storage devices. Some problems are so massive that the storage needed for their solution exceeds the virtual address space of the largest machines. In this paper we describe a method for solving such problems on a typical (large) computer and provide the results of some experiments illustrating the effectiveness of our approach. The method includes an automatic partitioning scheme which is essential to the efficient management of the data on auxiliary files.

A testing scheme for subroutines solving large linear problems

Abstract The mathematical models used in different fields of engineering and science often lead to the solution of large linear problems. These problems may be

Solution of Spatial Equilibrium Problems with Benders Decomposition

Spatial equilibrium problems are frequently formulated as large scale quadratic programming problems or linear complementarity problems. We show that these problems can be reduced to two or more smaller problems with Generalized Benders Decomposition. The procedure then becomes iterative with the repetitive solution of the smaller problems. In practice, the iterative procedure has converged rapidly.