Projection-type Algorithm Research Articles

A Projection Algorithm for Non-Monotone Variational Inequalities

We introduce a projection-type algorithm for solving the variational inequality problem for point-to-set operators, and establish its convergence properties. Namely, we assume that the operator of the variational inequality is continuous in the point-to-set sense, i.e., inner- and outer-semicontinuous. Under the assumption that the dual solution set is not empty, we prove that our method converges to a solution of the variational inequality. Instead of the monotonicity assumption, we require the non-emptiness of the solution set of the dual formulation of the variational inequality. We provide numerical experiments illustrating the behaviour of our iterates. Moreover, we compare our new method with a recent similar one.

Solving policy design problems: Alternating direction method of multipliers-based methods for structured inverse variational inequalities

Inverse variational inequalities have broad applications in various disciplines, and some of them have very appealing structures. There are several algorithms (e.g., proximal point algorithms and projection-type algorithms) for solving the inverse variational inequalities in general settings, while few of them have fully exploited the special structures. In this paper, we consider a class of inverse variational inequalities that has a separable structure and linear constraints, which has its root in spatial economic equilibrium problems. To design an efficient algorithm, we develop an alternating direction method of multipliers (ADMM) based method by utilizing the separable structure. Under some mild assumptions, we prove its global convergence. We propose an improved variant that makes the subproblems much easier and derive the convergence result under the same conditions. Finally, we present the preliminary numerical results to show the capability and efficiency of the proposed methods.

Gradient projection-type algorithms for solving equilibrium problems and its applications

In this paper, we study strongly pseudo-monotone equilibrium problems in real Hilbert and introduce two simple subgradient-type methods for solving it. The advantages of our schemes are the simplicity of their algorithmic structure which consists of only one projection onto the feasible set and there is no need to solve any strongly convex programming problem, which is often used in related methods in the literature. Under mild and standard assumptions, strong convergence theorems of the proposed algorithms are established. We test and compare the performances of our schemes with some related methods in the literature for solving the Nash–Cournot oligopolistic equilibrium model.

A new projection-type method for solving multi-valued mixed variational inequalities without monotonicity

ABSTRACT In this paper, a new projection-type algorithm for solving multi-valued mixed variational inequalities without monotonicity is presented. Under some suitable assumptions, it is showed that the sequence generated by the proposed algorithm converges globally to a solution of the multi-valued mixed variational inequality considered. The algorithm exploited in this paper is based on the generalized f -projection operator due to Wu and Huang [The generalized f-projection operator with an application. Bull Austral Math Soc. 2006;73:307–317] rather than the well-known resolvent operator. Preliminary computational experience is also reported. The results presented in this paper generalize and improve some known results given in the literature.

A linearly convergent algorithm for sparse signal reconstruction

For the sparse signal reconstruction problem in compressive sensing, we propose a projection-type algorithm without any backtracking line search based on a new formulation of the problem. Under suitable conditions, global convergence and its linear convergence of the designed algorithm are established. The efficiency of the algorithm is illustrated through some numerical experiments on some sparse signal reconstruction problem.

Design of Self-Organizing Intelligent Controller Using Fuzzy Neural Network

In this paper, a self-organizing intelligent controller (SOIC) is proposed for a class of nonlinear systems. The basic idea of this study is to use a self-organizing fuzzy neural network to imitate control law directly, and then, appeal to obtain a compact structure of controller to further reduce the computational burden and enhance the control performance. First, an effective criterion, using the tracking performance and structure risk of controller, is developed to self-organize the control rules online for SOIC to improve the tracking performance. Second, the structure and parameters of SOIC are updated by an adaptive projection-type algorithm to reduce the heavy computational burden to speed up the control response. Third, the stability of SOIC is proved in the sense of Lyapunov and the guidelines for selecting the control parameters are given. Finally, the effectiveness of SOIC is illustrated with three nonlinear systems. It is shown that the proposed SOIC can achieve better control performance in comparison with some other control schemes.

Self-adaptive gradient projection algorithms for variational inequalities involving non-Lipschitz continuous operators

In this paper, we introduce a self-adaptive inertial gradient projection algorithm for solving monotone or strongly pseudomonotone variational inequalities in real Hilbert spaces. The algorithm is designed such that the stepsizes are dynamically chosen and its convergence is guaranteed without the Lipschitz continuity and the paramonotonicity of the underlying operator. We will show that the proposed algorithm yields strong convergence without being combined with the hybrid/viscosity or linesearch methods. Our results improve and develop previously discussed gradient projection-type algorithms by Khanh and Vuong (J. Global Optim. 58, 341–350 2014).

Strong convergence of a double projection-type method for monotone variational inequalities in Hilbert spaces

We introduce a projection-type algorithm for solving monotone variational inequality problems in real Hilbert spaces without assuming Lipschitz continuity of the corresponding operator. We prove that the whole sequence of iterates converges strongly to a solution of the variational inequality. The method uses only two projections onto the feasible set in each iteration in contrast to other strongly convergent algorithms which either require plenty of projections within a step size rule or have to compute projections on possibly more complicated sets. Some numerical results illustrate the behavior of our method.

A Projected Subgradient Algorithm for Bilevel Equilibrium Problems and Applications

In this paper, we propose a new algorithm for solving a bilevel equilibrium problem in a real Hilbert space. In contrast to most other projection-type algorithms, which require to solve subproblems at each iteration, the subgradient method proposed in this paper requires only to calculate, at each iteration, two subgradients of convex functions and one projection onto a convex set. Hence, our algorithm has a low computational cost. We prove a strong convergence theorem for the proposed algorithm and apply it for solving the equilibrium problem over the fixed point set of a nonexpansive mapping. Some numerical experiments and comparisons are given to illustrate our results. Also, an application to Nash–Cournot equilibrium models of a semioligopolistic market is presented.

Solvability of the Minty Variational Inequality

We consider the existence of solutions to the Minty variational inequality, as it plays a key role in a projection-type algorithm for solving the variational inequality. It is shown that, if the underlying mapping has a separable structure with each component of the mapping being quasimonotone, then the Minty variational inequality has a solution. An example shows that the underlying mapping itself is not necessarily quasimonotone, although each of its components is.

A projection-type algorithm for solving generalized mixed variational inequalities

We propose a projection-type algorithm for generalized mixed variational inequality problem in Euclidean space ℝn. We establish the convergence theorem for the proposed algorithm, provided the multi-valued mapping is continuous and f-pseudomonotone with nonempty compact convex values on dom(f), where f: ℝn → ℝU{+∞} is a proper function. The algorithm presented in this paper generalize and improve some known algorithms in literatures. Preliminary computational experience is also reported.

A Projection-Type Method for Set Valued Variational Inequality Problems on Hadamard Manifolds

In this paper, we develop a projection-type algorithm for set-valued variational inequalities on Hadamard manifolds. The proposed method is well defined whether the solution set of the problem is non-empty or not. Under pseudomonotonicity assumptions on the underlying vector field, our method is convergent to a solution of the given set-valued variational inequality. The results presented in this paper generalize and improve some known results introduced by Tang et al. (Optimization 64(5):1081–1096, 2015).

Accelerated fast iterative shrinkage thresholding algorithms for sparsity-regularized cone-beam CT image reconstruction.

The development of iterative image reconstruction algorithms for cone-beam computed tomography (CBCT) remains an active and important research area. Even with hardware acceleration, the overwhelming majority of the available 3D iterative algorithms that implement nonsmooth regularizers remain computationally burdensome and have not been translated for routine use in time-sensitive applications such as image-guided radiation therapy (IGRT). In this work, two variants of the fast iterative shrinkage thresholding algorithm (FISTA) are proposed and investigated for accelerated iterative image reconstruction in CBCT. Algorithm acceleration was achieved by replacing the original gradient-descent step in the FISTAs by a subproblem that is solved by use of the ordered subset simultaneous algebraic reconstruction technique (OS-SART). Due to the preconditioning matrix adopted in the OS-SART method, two new weighted proximal problems were introduced and corresponding fast gradient projection-type algorithms were developed for solving them. We also provided efficient numerical implementations of the proposed algorithms that exploit the massive data parallelism of multiple graphics processing units. The improved rates of convergence of the proposed algorithms were quantified in computer-simulation studies and by use of clinical projection data corresponding to an IGRT study. The accelerated FISTAs were shown to possess dramatically improved convergence properties as compared to the standard FISTAs. For example, the number of iterations to achieve a specified reconstruction error could be reduced by an order of magnitude. Volumetric images reconstructed from clinical data were produced in under 4 min. The FISTA achieves a quadratic convergence rate and can therefore potentially reduce the number of iterations required to produce an image of a specified image quality as compared to first-order methods. We have proposed and investigated accelerated FISTAs for use with two nonsmooth penalty functions that will lead to further reductions in image reconstruction times while preserving image quality. Moreover, with the help of a mixed sparsity-regularization, better preservation of soft-tissue structures can be potentially obtained. The algorithms were systematically evaluated by use of computer-simulated and clinical data sets.

Taylor Projection: A New Solution Method to Dynamic General Equilibrium Models

This paper presents a new solution method for dynamic equilibrium models. The proposed method approximates the solution by polynomials that zero the residual function and its derivatives at a given point x0. It is essentially a projection-type algorithm, but is significantly faster than standard projection, since the problem is highly sparse and can be easily solved by a Newton solver. The obtained solution is accurate locally in the neighbourhood of x_0. Importantly, a local solution can be obtained at any point of the state space. This makes it possible to solve models at points that are further away from the steady state. As an illustration, the paper solves a multi-country growth model and simulates the Solow convergence from an initial point of high cross-country inequality.

Weaker hypotheses for the general projection algorithm with corrections

Abstract In an earlier paper [J. of Appl. Math. and Informatics, 29(3-4)(2011), 697-712] we proposed a general projection-type algorithm with corrections and proved its convergence under a set of special assumptions. In this paper we prove convergence of this algorithm under a much weaker set of assumptions. This new framework gives us the possibility to obtain as a particular case of our method the two-step algorithm analysed in [B I T, 38(2)(1998), 275-282].

A fast exact filtering approach to a family of affine projection-type algorithms

The affine projection (AP)-type algorithms produce a good tradeoff between convergence speed and complexity. As the projection order increases, the convergence rate of the AP algorithm is improved at a relatively high complexity. Many efforts have been made to reduce the complexity. However, most of the efficient versions of the AP-type algorithms are based on the fast approximate filtering (FAF) scheme originally proposed in the fast AP (FAP) algorithm. The approximation leads to degraded convergence performance. Recently, a fast exact filtering (FEF) AP (FEAP) algorithm was proposed by Y. Zakharov. In this paper, we propose a new FEF approach to further reduce the complexity of the FEAP algorithm given that the calculation of the weight vector is not the primary objective for the application at hand. The proposed FEF scheme is then extended to the dichotomous coordinate descent (DCD)-AP, affine projection sign (APS), and modified filtered-x affine projection (MFxAP) algorithms. The complexity of AP-type algorithms based on the proposed FEF approach is comparable to that based on the FAF scheme. Moreover, analysis results show that the complexity reduction of the new algorithms is achieved without any performance degradation.

A polynomial projection-type algorithm for linear programming

We propose a simple O([n5/logn]L) algorithm for linear programming feasibility, that can be considered as a polynomial-time implementation of the relaxation method. Our work draws from Chubanov’s “Divide-and-Conquer” algorithm (Chubanov, 2012), with the recursion replaced by a simple and more efficient iterative method. A similar approach was used in a more recent paper of Chubanov (2013).

Iterative method for optimal design of flat-spectral-response arrayed waveguide gratings

A novel iterative projection-type optimal design algorithm of arrayed waveguide gratings (AWGs) with a flat spectral response is proposed based on the Fourier optics model of AWG. The enhancement of the spectral-response flatness of the AWG is demonstrated, with an analysis on the trade-off relationship between band flatness and crosstalk.

Software for weighted structured low-rank approximation

A software package is presented that computes locally optimal solutions to low-rank approximation problems with the following features: •mosaic Hankel structure constraint on the approximating matrix,•weighted 2-norm approximation criterion,•fixed elements in the approximating matrix,•missing elements in the data matrix, and•linear constraints on an approximating matrix’s left kernel basis. It implements a variable projection type algorithm and allows the user to choose standard local optimization methods for the solution of the parameter optimization problem. For an m×n data matrix, with n>m, the computational complexity of the cost function and derivative evaluation is O(m2n). The package is suitable for applications with n≫m. In statistical estimation and data modeling–the main application areas of the package–n≫m corresponds to modeling of large amount of data by a low-complexity model. Performance results on benchmark system identification problems from the database DAISY and approximate common divisor problems are presented.

ERROR ESTIMATES FOR THE FULLY DISCRETE STABILIZED GAUGE-UZAWA METHOD -PART I: THE NAVIER-STOKES EQUATIONS

The stabilized Gauge-Uzawa method (SGUM), which is a second order projection type algorithm to solve the time-dependent Navier-Stokes equations, has been newly constructed in 2013 Pyo's paper. The accuracy of SGUM has been proved only for time discrete scheme in the same paper, but it is crucial to study for fully discrete scheme, because the numerical errors depend on discretizations for both space and time, and because discrete spaces between velocity and pressure can not be chosen arbitrary. In this paper, we find out properties of the fully discrete SGUM and estimate its errors and stability to solve the evolution Navier-Stokes equations. The main difficulty in this estimation arises from losing some cancellation laws due to failing divergence free condition of the discrete velocity function. This result will be extended to Boussinesq equations in the continuous research (part II) and is essential in the study of part II.