Dykstra's Algorithm Research Articles

Finding the projection of a point onto the intersection of convex sets via projections onto half-spaces

We present a modification of Dykstra's algorithm which allows us to avoid projections onto general convex sets. Instead, we calculate projections onto either a half-space or onto the intersection of two half-spaces. Convergence of the algorithm is established and special choices of the half-spaces are proposed. The option to project onto half-spaces instead of general convex sets makes the algorithm more practical. The fact that the half-spaces are quite general enables us to apply the algorithm in a variety of cases and to generalize a number of known projection algorithms. The problem of projecting a point onto the intersection of closed convex sets receives considerable attention in many areas of mathematics and physics as well as in other fields of science and engineering such as image reconstruction from projections. In this work we propose a new class of algorithms which allow projection onto certain super half-spaces, i.e., half-spaces which contain the convex sets. Each one of the algorithms that we present gives the user freedom to choose the specific super half-space from a family of such half-spaces. Since projecting a point onto a half-space is an easy task to perform, the new algorithms may be more useful in practical situations in which the construction of the super half-spaces themselves is not too difficult.

Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization.

The phase retrieval problem is of paramount importance in various areas of applied physics and engineering. The state of the art for solving this problem in two dimensions relies heavily on the pioneering work of Gerchberg, Saxton, and Fienup. Despite the widespread use of the algorithms proposed by these three researchers, current mathematical theory cannot explain their remarkable success. Nevertheless, great insight can be gained into the behavior, the shortcomings, and the performance of these algorithms from their possible counterparts in convex optimization theory. An important step in this direction was made two decades ago when the error reduction algorithm was identified as a nonconvex alternating projection algorithm. Our purpose is to formulate the phase retrieval problem with mathematical care and to establish new connections between well-established numerical phase retrieval schemes and classical convex optimization methods. Specifically, it is shown that Fienup's basic input-output algorithm corresponds to Dykstra's algorithm and that Fienup's hybrid input-output algorithm can be viewed as an instance of the Douglas-Rachford algorithm. We provide a theoretical framework to better understand and, potentially, to improve existing phase recovery algorithms.

Primal and polar approach for computing the symmetric diagonally dominant projection

AbstractWe solve the problem of minimizing the distance from a given matrix to the cone of symmetric and diagonally dominant matrices with positive diagonal (SDD+). Using the extreme rays of the polar cone we project onto the supporting hyperplanes of the faces of SDD+ and then, applying the cyclic Dykstra's algorithm, we solve the problem. Similarly, using the extreme rays of SDD+ we characterize the projection onto the polar cone, which also allows us to obtain the projection onto SDD+ by means of the orthogonal decomposition theorem for convex cones. In both cases the symmetry and the sparsity pattern of the given matrix are preserved. Preliminary numerical experiments indicate that the polar approach is a promising one. Copyright © 2002 John Wiley & Sons, Ltd.

A Convergence Analysis of Dykstra's Algorithm for Polyhedral Sets

Let H be a nonempty closed convex set in a Hilbert space X determined by the intersection of a finite number of closed half spaces. It is well known that given an $x_0 \in X$, Dykstra's algorithm applied to this collection of closed half spaces generates a sequence of iterates that converge to PH(x0 ), the orthogonal projection of x0 onto H. The iterates, however, do not necessarily lie in H. We propose a combined Dykstra--conjugate-gradient method such that, given an $\varepsilon > 0$, the algorithm computes an $x \in H$ with $\|x - P_H(x_0)\| < \varepsilon$. Moreover, for each iterate xm of Dykstra's algorithm we calculate a bound for $\|x_m - P_H(x_0)\|$ that approaches zero as m tends to infinity. Applications are made to computing bounds for $\|x_m - P_H(x_0)\|$ where H is a polyhedral cone. Numerical results are presented from a sample isotone regression problem.

Estimation of weighted multinomial probabilities under log-convex constraints

Many problems in Statistics involve maximizing a multinomial likelihood over a restricted region. In this paper, we consider instead maximizing a weighted multinomial likelihood. We show that a dual problem always exits which is frequently more tractable and that a solution to the dual problem leads directly to a solution of the primal problem. Moreover, the form of the dual problem suggests an iterative algorithm for solving the MLE problem when the constraint region can be written as a finite intersection of cones. We show that this iterative algorithm is guaranteed to converge to the true solution and show that when the cones are isotonic, this algorithm is a version of Dykstra's algorithm (Dykstra, J. Amer. Statist. Assoc. 78 (1983) 837–842) for the special case of least squares projection onto the intersection of isotonic cones. We give several meaningful examples to illustrate our results. In particular, we obtain the nonparametric maximum likelihood estimator of a monotone density function in the presence of selection bias.

Fenchel Duality and the Strong Conical Hull Intersection Property

We study a special dual form of a convex minimization problem in a Hilbert space, which is formally suggested by Fenchel dualityand is useful for the Dykstra algorithm. For this special duality problem, we prove that strong duality holds if and only if the collection of underlying constraint sets {C1,...,Cm} has the strong conical hull intersection property. That is, $$\left( {\mathop \cap \limits_1^m C_i - x)^ \circ = \sum\limits_1^m {(C_1 - x} } \right)^ \circ {\text{, for each }}x \in \mathop \cap \limits_1^m C_1$$ where D° denotes the dual cone of D. In general, we can establish weak duality for a convex minimization problem in a Hilbert space by perturbing the constraint sets so that the perturbed sets have the strong conical hull intersection property. This generalizes a result of Gaffke and Mathar.

Successive Approximate Algorithm for Best Approximation from a Polyhedron

SupposeKis the intersection of a finite number of closed half-spaces {Ki} in a Hilbert spaceX, andx∈X\\K. Dykstra's cyclic projections algorithm is a known method to determine an approximate solution of the best approximation ofxfromK, which is denoted byPK(x). Dykstra's algorithm reduces the problem to an iterative scheme which involves computing the best approximation from the individualKi. It is known that the sequence {xj} generated by Dykstra's method converges to the best approximationPK(x). But since it is difficult to find the definite value of an upper bound of the error ‖xj−PK(x)‖, the applicability of the algorithm is restrictive. This paper introduces a new method, called thesuccessive approximate algorithm, by which one can generate a finite sequencex0, x1, …, xkwithxk=PK(x). In addition, the error ‖xj−PK(x)‖ is monotone decreasing and has a definite upper bound easily to be determined. So the new algorithm is very applicable in practice.

Dykstra's algorithm for constrained least-squares rectangular matrix problems

In a recent paper, the authors applied Dykstra's alternating projection algorithm to solve constrained least-squares n × n matrix problems. We extend these results in two different directions. First, we make use of the singular value decomposition to solve now constrained least-squares rectangular m × n matrix problems that arise in several applications. Second, we propose a new and improved implementation of the projection algorithm onto the ε-positive definite set of matrices. This implementation does not require the computation of all eigenvalues and eigenvectors of a matrix per iteration, and still guarantees convergence. Finally, encouraging preliminary numerical results are discussed.

A Fenchel Duality Aspect of Iterative I-Projection Procedures

In this paper we interpret Dykstra's iterative procedure for finding anI-projection onto the intersection of closed, convex sets in terms of itsFenchel dual. Seen in terms of its dual formulation, Dykstra's algorithm isintuitive and can be shown to converge monotonically to the correctsolution. Moreover, we show that it is possible to sharply bound thelocation of the constrained optimal solution.