Global Convergence Results Research Articles

A Self-adaptive Algorithm for Solving Basis Pursuit Denoising Problem

In this paper, we further consider a method for solving the basis pursuit denoising problem (BPDP), which has received considerable attention in signal processing and statistical inference. To this end, a new self-adaptive algorithm is proposed, its global convergence results is established. Furthermore, we also show that the method is sublinearly convergent rate of O( 1/k). Finally, the availability of given method is shown via somek numerical examples.

Coupled versus decoupled penalization of control complementarity constraints

This paper deals with the numerical solution of optimal control problems with control complementarity constraints. For that purpose, we suggest the use of several penalty methods which differ with respect to the handling of the complementarity constraint which is either penalized as a whole with the aid of NCP-functions or decoupled in such a way that non-negativity constraints as well as the equilibrium condition are penalized individually. We first present general global and local convergence results which cover several different penalty schemes before two decoupled methods which are based on a classical ℓ1- and ℓ2-penalty term, respectively, are investigated in more detail. Afterwards, the numerical implementation of these penalty methods is discussed. Based on some examples, where the optimal boundary control of a parabolic partial differential equation is considered, some quantitative properties of the resulting algorithms are compared.

First-Order Methods for Constrained Convex Programming Based on Linearized Augmented Lagrangian Function

First-order methods (FOMs) have been popularly used for solving large-scale problems. However, many existing works only consider unconstrained problems or those with simple constraint. In this paper, we develop two FOMs for constrained convex programs, where the constraint set is represented by affine equations and smooth nonlinear inequalities. Both methods are based on the classical augmented Lagrangian function. They update the multipliers in the same way as the augmented Lagrangian method (ALM) but use different primal updates. The first method, at each iteration, performs a single proximal gradient step to the primal variable, and the second method is a block update version of the first one. For the first method, we establish its global iterate convergence and global sublinear and local linear convergence, and for the second method, we show a global sublinear convergence result in expectation. Numerical experiments are carried out on the basis pursuit denoising, convex quadratically constrained quadratic programs, and the Neyman-Pearson classification problem to show the empirical performance of the proposed methods. Their numerical behaviors closely match the established theoretical results.

Decentralized Autonomous Navigation of Swarms of Non-Holonomic Robots for Sweep Coverage of Unsteady Environmental Boundaries

Abstract Several non-holonomic constant-speed under-actuated robots with lower-limited turning radii travel in 3D. Every robot has access only to its own coordinate and the coordinates of close neighbors along a certain specific space direction (typically, vertical), as well as to the current value of an unknown dynamic scalar field. Starting from occasional locations, all robots should arrive at the isosurface with the given field value. Then they should evenly distribute themselves over a pre-specified range of the “altitudes” and repeatedly encircle the isosurface while remaining on it, each at its own “altitude”. A decentralized control law is presented that solves this mission. This law is justified by a rigorous global convergence result; its performance is confirmed by computer simulation tests.

ADMM-Type Methods for Generalized Nash Equilibrium Problems in Hilbert Spaces

We consider the generalized Nash equilibrium problem (GNEP) with $ N $ players in a Hilbert space setting. The joint constraints are eliminated by an augmented Lagrangian-type approach, leading to an ADMM (alternating direction method of multipliers) algorithm. In contrast to standard optimization problems, however, the direct extension of ADMM to GNEPs is not necessarily convergent even for $N = 2$ players. We therefore use a regularized version of ADMM and present a global convergence result for $ N \geq 2 $ players under a partial strong monotonicity and a partial Lipschitz condition. Furthermore, also different from the optimization context, it turns out that the corresponding regularization parameters have to be sufficiently large in order to guarantee global convergence. We therefore also discuss a second ADMM-type method with an adaptive choice of the regularization parameters, with the aim of keeping the regularization parameters smaller and, hence, getting faster convergence. Numerical results are presented for some examples arising in infinite-dimensional Hilbert spaces.

On the global convergence of a new spectral residual algorithm for nonlinear systems of equations

We present a derivative-free method for solving systems of nonlinear equations that belongs to the class of spectral residual methods. We will show that by endowing a previous version of the algorithm with a suitable new linesearch strategy, standard global convergence results can be attained under mild general assumptions. The robustness of the new method is therefore potentially improved with respect to the previous version as shown by the reported numerical experiments.

A new conjugate gradient method for acceleration of gradient descent algorithms

Abstract An accelerated of the steepest descent method for solving unconstrained optimization problems is presented. which propose a fundamentally different conjugate gradient method, in which the well-known parameter βk is computed by an new formula. Under common assumptions, by using a modified Wolfe line search, descent property and global convergence results were established for the new method. Experimental results provide evidence that our proposed method is in general superior to the classical steepest descent method and has a potential to significantly enhance the computational efficiency and robustness of the training process.

A partially proximal S-ADMM for separable convex optimization with linear constraints

A classical approach to solving two-block separable convex optimization could be the symmetric alternating direction method of multipliers (S-ADMM). However, its convergence may not be guaranteed for a general multi-block case without additional assumptions. Bai et al. proposed a variant of S-ADMM entitled the generalized symmetric ADMM (GS-ADMM), in which the variables are regrouped into two groups firstly. The two groups of variables are updated in a Gauss-Seidel scheme, while the variables within each group are updated in a Jacobi scheme and the Lagrangian multipliers are updated two times. In order to derive its convergence property, the authors add a special proximal term to each subproblem. In this paper, inspired by the partial PPA block-wise ADMM (PPBADMM) [32] proposed by Shen et al., we propose a partially proximal S-ADMM (PPSADMM). In PPSADMM, the special proximal term is only added to the subproblems in the first group as PPBADMM. We perform an extension step on all variables with a fixed step size at the end of each iteration. Without stringent assumptions, we establish the global convergence result and the O(1/t) convergence rate in the ergodic sense for PPSADMM. Its numerical performance is justified on two types of problems.

Linearly preconditioned nonlinear conjugate gradient acceleration of the PX-EM algorithm

The EM algorithm is a widely applicable algorithm for modal estimation but often criticized for its slow convergence. A new hybrid accelerator named APX-EM is proposed for speeding up the convergence of EM algorithm, which is based on both Linearly Preconditioned Nonlinear Conjugate Gradient (PNCG) and PX-EM algorithm. The intuitive idea is that, each step of the PX-EM algorithm can be viewed approximately as a generalized gradient just like the EM algorithm, then the linearly PNCG method can be used to accelerate the EM algorithm. Essentially, this method is an adjustment of the AEM algorithm, and it usually achieves a faster convergence rate than the AEM algorithm by sacrificing a little simplicity. The convergence of the APX-EM algorithm, includes a global convergence result for this method under suitable conditions, is discussed. This method is illustrated for factor analysis and a random-effects model.

On the complexity of a hybrid proximal extragradient projective method for solving monotone inclusion problems

In a series of papers (Solodov and Svaiter in J Convex Anal 6(1):59–70, 1999; Set-Valued Anal 7(4):323–345, 1999; Numer Funct Anal Optim 22(7–8):1013–1035, 2001) Solodov and Svaiter introduced new inexact variants of the proximal point method with relative error tolerances. Point-wise and ergodic iteration-complexity bounds for one of these methods, namely the hybrid proximal extragradient method (1999) were established by Monteiro and Svaiter (SIAM J Optim 20(6):2755–2787, 2010). Here, we extend these results to a more general framework, by establishing point-wise and ergodic iteration-complexity bounds for the inexact proximal point method studied by Solodov and Svaiter (2001). Using this framework we derive global convergence results and iteration-complexity bounds for a family of projective splitting methods for solving monotone inclusion problems, which generalize the projective splitting methods introduced and studied by Eckstein and Svaiter (SIAM J Control Optim 48(2):787–811, 2009).

On the solution of linearly constrained optimization problems by means of barrier algorithms

Many practical problems require the solution of large-scale constrained optimization problems for which preserving feasibility is a key issue, and the evaluation of the objective function is very expensive. In these cases it is mandatory to start with a feasible approximation of the solution, the obtention of which should not require objective function evaluations. The necessity of solving this type of problems motivated us to revisit the classical barrier approach for nonlinear optimization, providing a careful implementation of a modern version of this method. This is the main objective of the present paper. For completeness, we provide global convergence results and comparative numerical experiments with one of the state-of-the-art interior-point solvers for continuous optimization.

Descent Perry conjugate gradient methods for systems of monotone nonlinear equations

In this paper, we present a family of Perry conjugate gradient methods for solving large-scale systems of monotone nonlinear equations. The methods are developed by combining modified versions of Perry (Oper. Res. Tech. Notes 26(6), 1073–1078, 1978) conjugate gradient method with the hyperplane projection technique of Solodov and Svaiter (1998). Global convergence and numerical results of the methods are established and preliminary numerical results shows that the proposed methods are promising and more effective compared to some existing methods in the literature.

A new modification nonlinear conjugate gradient method with strong wolf-powell line search

<span>The conjugate gradient method has played a special role in solving large-scale unconstrained Optimization problems. In this paper, we propose a new family of CG coefficients that possess sufficient descent conditions and global convergence properties this CG method is similar to (Wei et al) [7]. Global convergence result is established under Strong Wolf-Powell line search. Numerical results to find the optimum solution of some test functions show the new proposed formula has the best result in CPU time and the number of iterations, and the number of gradient evaluations when it comparing with FR, PRP, DY, and WYL </span>

A complementarity model and algorithm for direct multi-commodity flow supply chain network equilibrium problem

In this paper, a three-level supply chain network equilibrium problem with direct selling and multi-commodity flow is considered. To this end, we first present equilibrium conditions which satisfy decision-making behaviors for manufacturers, retailers and consumer markets, respectively. Based on this, a nonlinear complementarity model of supply chain network equilibrium problem is established. In addition, we propose a new projection-type algorithm to solve this model without the backtracking line search, and global convergence result and \begin{document}$ R- $\end{document} linearly convergence rate for the new algorithm are established under weaker conditions, respectively. We also illustrate the efficiency of given algorithm through some numerical examples.

A hybrid conjugate gradient algorithm for constrained monotone equations with application in compressive sensing

Combining the projection method of Solodov and Svaiter with the Liu-Storey and Fletcher Reeves conjugate gradient algorithm of Djordjević for unconstrained minimization problems, a hybrid conjugate gradient algorithm is proposed and extended to solve convex constrained nonlinear monotone equations. Under some suitable conditions, the global convergence result of the proposed method is established. Furthermore, the proposed method is applied to solve the ℓ1-norm regularized problems to restore sparse signal and image in compressive sensing. Numerical comparisons of the proposed algorithm versus some other conjugate gradient algorithms on a set of benchmark test problems, sparse signal reconstruction and image restoration in compressive sensing show that the proposed scheme is computationally more efficient and robust than the compared schemes.

Two fast vector-wise update algorithms for orthogonal nonnegative matrix factorization with sparsity constraint

Recently, orthogonal nonnegative matrix factorization (ONMF) has been introduced and shown to work remarkably well for clustering tasks. Because of the nonnegativity and the orthogonality constraints, the orthogonal factor matrix of ONMF is naturally sparse. Based on this fact, by introducing sparsity constraints on the orthogonal coefficient matrix, we propose two vector-wise algorithms based on Hierarchical Alternating Least Squares (HALS) and Block Prox-linear (BPL) methods to the approximately sparse orthogonal nonnegative matrix factorization (SONMF). Some global convergence results are established under the mild conditions. Numerical results including synthetic and real-world datasets are given to illustrate that the proposed algorithms compute highly accurate values and perform better than the other testing ONMF methods.

A prediction-correction inexact alternating direction method for convex nonlinear second-order cone programming with linear constraints

The convex nonlinear second-order cone programming with linear constraints is equivalent to a separate structure convex programming. A prediction-correction inexact alternating direction method is proposed for the separate structure convex programming. In the proposed method, the convex objective function is not required to be Lipschitz continuous and only needs satisfy an inequality. The global convergence result is given. Numerical results demonstrate that our method is efficient for some random second-order cone programming problems in lower accuracy. In addition, our method can be extended to the convex nonlinear circular cone programming with linear constraints. We also give the simulation results of the three-fingered grasping force optimization problems.

Cooperative decentralized reactive circumnavigation of unpredictably moving and deforming speedy extended objects

Abstract A team of speed- and acceleration-limited robots travel in a plane that hosts an unpredictably moving and deforming extended targeted object. In its local frame, every robot has access to its own velocity and is able to identify the relative coordinates of the objects within a given finite visibility range, as well as the nearest point of the object. A sliding mode communication-free sensor-based strategy is presented that drives the robots to a desired distance from the targeted object and ensures its subsequent circumnavigation with maintaining this distance and effective self-distribution around the object. The proposed control law individually operates at any robot and is reactive, i.e., it directly converts the current sensory data into the current control in a reflex-like fashion. The performance of the proposed navigation law is rigorously justified by a global convergence result and is confirmed via computer simulation tests.

Gradient-Free Tracking of Unsteady Environmental Level Sets in Dynamic Environments by a Nonholonomic Robot

A non-holonomic under-actuated planar robot is propelled via interaction of its actuators with a dynamic surrounding medium; a control input is the angular velocity of robot’s self-rotation relative to the medium. Meanwhile, the motion of the medium is unknown and unpredictable; the relative surge speed is time-varying and treated as uncertain. There is an unpredictably varying scalar environmental field. From a remote initial location, the robot should reach the isoline where the field assumes a pre-specified value, and then should repeatedly run its length. Robot measures only the field value at the current location and has no access to the field gradient or parameters of the medium motion. To solve this task, at first conditions are established that are necessary for the mission to be feasible with the given limitations on the robot’s dynamics. Then a navigation law is presented that solves the mission under slight and partly unavoidable enhancement of these conditions. This law is computationally inexpensive and directly converts the current sensory data into the current control in a reflex-like fashion. The performance of the law is rigorously justified by a global convergence result and is confirmed via computer simulation tests.

Sliding mode strategies for monitoring and compensation of cyber-attacks to bCyber-Physical Systems

In this paper, the problem of detection and reconstruction of cyber-attacks in linear cyber-physical systems is considered. The class of cyber-attacks described in this paper can corrupt the states or the outputs of a cyber-physical system. An attack monitor based on High-Order sliding mode is proposed to reconstruct the cyber-attack. A First Order Approximation Filter is proposed to ensure global stability and convergence results. Using sliding mode techniques, an attack compensation is developed for square plants, guarateeing finite time convergence to the output tracking while rejecting the effects of the cyber-attack.