Survey and Review

Abstract

What do these algorithms have in common? The answer is that they are all infinite algorithms for finite problems. We could solve (1) by Gaussian elimination, (2) by adding up the pairwise forces, (3) by nested dissection, or (4) by the simplex method, and in each case we would get the exact answer, apart from rounding errors, in a finite number of steps. But it may be quicker to use methods that take an infinite number of steps! These algorithms are approximate and iterative, but they converge so fast that they are often the best way to solve large-scale problems if you want "only," say, 15 digits of accuracy. (1)--(4) and their relatives have changed scientific computing profoundly since the 1970s and 1980s. The following article by Forsgren, Gill, and Wright has its roots in (4). But once you depart from finite algorithms and embrace iteration, the finiteness of the underlying problem loses its importance: you gain robustness as well as speed. Suddenly linear programming is not so different from nonlinear, and the subject of this article is the huge subject of interior methods for nonlinear optimization that has blossomed since Karmarkar proposed his famous algorithm in 1984. The move to fast continuous algorithms has other aspects. Computer scientists have found that although a discrete combinatorial problem may be NP-hard if you want an exact solution, near-optimal solutions may be computable via continuous algorithms such as those reviewed here. The problem of finding large cuts in graphs, for example, can be solved in polynomial time if you'll settle for a cut that might be 13% smaller than the largest possible. Thus interior methods are playing a role in drawing together discrete and continuous applied mathematicians. SIAM Review draws mathematicians together too, and it has been a privilege to serve as Section Editor for four years. In January, I will be succeeded in this post by Randy LeVeque of the University of Washington.