Sequence Of Function Values Research Articles

A first-order method for solving bilevel convex optimization problems in Banach space

ABSTRACT We consider a general class of convex optimization problems in which one seeks to minimize a strongly convex function over a closed and convex set which is by itself an optimal set of another convex problem in Banach space. Regularized forward–backward splitting method is applied to find the minimum like-norm solution of the minimization problem under investigation. We also introduce a gradient-based method, called the minimal like-norm gradient method, for solving this class of problems and establish the convergence of the sequence generated by the algorithm as well as a rate of convergence of the sequence of function values.

On the Linear Convergence of Forward–Backward Splitting Method: Part I—Convergence Analysis

In this paper, we study the complexity of the forward–backward splitting method with Beck–Teboulle’s line search for solving convex optimization problems, where the objective function can be split into the sum of a differentiable function and a nonsmooth function. We show that the method converges weakly to an optimal solution in Hilbert spaces, under mild standing assumptions without the global Lipschitz continuity of the gradient of the differentiable function involved. Our standing assumptions is weaker than the corresponding conditions in the paper of Salzo (SIAM J Optim 27:2153–2181, 2017). The conventional complexity of sublinear convergence for the functional value is also obtained under the local Lipschitz continuity of the gradient of the differentiable function. Our main results are about the linear convergence of this method (in the quotient type), in terms of both the function value sequence and the iterative sequence, under only the quadratic growth condition. Our proof technique is direct from the quadratic growth conditions and some properties of the forward–backward splitting method without using error bounds or Kurdya-Łojasiewicz inequality as in other publications in this direction.

Global complexity analysis of inexact successive quadratic approximation methods for regularized optimization under mild assumptions

Successive quadratic approximations (SQA) are numerically efficient for minimizing the sum of a smooth function and a convex function. The iteration complexity of inexact SQA methods has been analyzed recently. In this paper, we present an algorithmic framework of inexact SQA methods with four types of line searches, and analyze its global complexity under milder assumptions. First, we show its well-definedness and some decreasing properties. Second, under the quadratic growth condition and a uniform positive lower bound condition on stepsizes, we show that the function value sequence and the iterate sequence are linearly convergent. Moreover, we obtain a o(1/k) complexity without the quadratic growth condition, improving existing O(1/k) complexity results. At last, we show that a local gradient-Lipschitz-continuity condition could guarantee a uniform positive lower bound for the stepsizes.

A discrete multi-objective fireworks algorithm for flowshop scheduling with sequence-dependent setup times

Multi-objective flow shop scheduling problem with sequence-dependent setup times (MOFSP-SDST) is a class of important production scheduling problem with strong industry background. In this paper, a MOFSP-SDST mathematic model with the objectives of total production cost, makespan, mean flow time and mean idle time of machines is developed. To solve this multi-objective model, a novel multi-objective approach based on fuzzy correlation entropy analysis is proposed firstly. In this multi-objective approach, two types of objective function value sequences, namely the referenced function value sequence and comparable function value sequence, are constructed and mapped into two types of fuzzy sets by a modified relative membership function. The fuzzy correlation entropy coefficient between the two types of fuzzy sets is used to select better solutions in a multi-objective problem. A discrete multi-objective fireworks algorithm (DMOFWA) is proposed to address the MOFSP-SDST. In the DMOFWA, a new multi-objective approach is adopted to handle the multiple objectives and guide the search of the algorithm. Two kinds of machine learning strategies are adopted, namely opposition-based learning (OBL) and clustering analysis (CA). The OBL is employed to learn from the current search space and improve the exploration ability of DMOFWA, and the CA based on fuzzy correlation entropy coefficient is proposed to cluster firework individuals. Computational and statistical results show that the novel multi-objective approach, OBL and CA strategies can effectively improve the performance of DMOFWA. Furthermore, the results indicate that DMOFWA performs better than four state-of-the-art comparison algorithms.

Nonconvex Proximal Incremental Aggregated Gradient Method with Linear Convergence

In this paper, we study the proximal incremental aggregated gradient(PIAG) algorithm for minimizing the sum of L-smooth nonconvex component functions and a proper closed convex function. By exploiting the L-smooth property and with the help of an error bound condition, we can show that the PIAG method still enjoys some nice linear convergence properties even for nonconvex minimization. To illustrate this, we first demonstrate that the generated sequence globally converges to the stationary point set. Then, there exists a threshold such that the objective function value sequence and the iterate point sequence are R-linearly convergent when the stepsize is chosen below this threshold.

The 2-Coordinate Descent Method for Solving Double-Sided Simplex Constrained Minimization Problems

This paper considers the problem of minimizing a continuously differentiable function with a Lipschitz continuous gradient subject to a single linear equality constraint and additional bound constraints on the decision variables. We introduce and analyze several variants of a 2-coordinate descent method: a block descent method that performs an optimization step with respect to only two variables at each iteration. Based on two new optimality measures, we establish convergence to stationarity points for general nonconvex objective functions. In the convex case, when all the variables are lower bounded but not upper bounded, we show that the sequence of function values converges at a sublinear rate. Several illustrative numerical examples demonstrate the effectiveness of the method.

A first order method for finding minimal norm-like solutions of convex optimization problems

We consider a general class of convex optimization problems in which one seeks to minimize a strongly convex function over a closed and convex set which is by itself an optimal set of another convex problem. We introduce a gradient-based method, called the minimal norm gradient method, for solving this class of problems, and establish the convergence of the sequence generated by the algorithm as well as a rate of convergence of the sequence of function values. The paper ends with several illustrating numerical examples.

A modified Fletcher-Reeves-Type derivative-free method for symmetric nonlinear equations

In this paper, we propose a descent derivative-free method for solving symmetric nonlinear equations. The method is an extension of the modified Fletcher-Reeves (MFR) method proposed by Zhang, Zhou and Li [25] to symmetric nonlinear equations. It can be applied to solve large-scale symmetric nonlinear equations due to lower storage requirement. An attractive property of the method is that the directions generated by the method are descent for the residual function. By the use of some backtracking line search technique, the generated sequence of function values is decreasing. Under appropriate conditions, we show that the proposed method is globally convergent. The preliminary numerical results show that the method is practically effective.

A linearly convergent derivative-free descent method for the second-order cone complementarity problem

We consider a class of derivative-free descent methods for solving the second-order cone complementarity problem (SOCCP). The algorithm is based on the Fischer–Burmeister (FB) unconstrained minimization reformulation of the SOCCP, and utilizes a convex combination of the negative partial gradients of the FB merit function ψFB as the search direction. We establish the global convergence results of the algorithm under monotonicity and the uniform Jordan P-property, and show that under strong monotonicity the merit function value sequence generated converges at a linear rate to zero. Particularly, the rate of convergence is dependent on the structure of second-order cones. Numerical comparisons are also made with the limited BFGS method used by Chen and Tseng (An unconstrained smooth minimization reformulation of the second-order cone complementarity problem, Math. Program. 104(2005), pp. 293–327), which confirm the theoretical results and the effectiveness of the algorithm.

Solving unconstrained optimization problem with a filter-based nonmonotone pattern search algorithm

This paper proposes a new frame-based pattern search algorithm for unconstrained optimization. The implementation of filter leads the sequence of function values at iterates { x ( k ) } nonmonotonically decreasing and thus motivates us to consider nonmonotone technique. Furthermore, the use of these two techniques can improve the efficiency of general pattern search algorithms. The numerical results show that the new algorithm is practical and efficient.

Convergence property of a class of variable metric methods

We investigate convergence property of the restricted Broyden class of variable metric methods. We show that when these methods with unit step are applied to a strictly convex quadratic objective function, the generated iterative sequence converges to the unique solution of the problem globally and superlinearly. Moreover, the distance between the iterative matrix and the Hessian matrix of the objective function decreases with iterations. The sequence of function values also exhibits descent property when the iteration is sufficiently large.

Nonmonotone trust region method for solving optimization problems

In this paper a trust region (TR) method with nonmonotone technique for optimization is proposed. We construct a new ratio of actual descent and predicted descent which is a simple and natural generalization of the modified Armijo line search rule. The paper exposes the relationship between the trust region method and line search approach. Since this method possesses the robust properties of trust region subproblem, it is globally convergent although we employ the nonmonotone sequence of function values instead of the monotone sequence. In addition, the proof of convergence is obviously simpler than one of Newton-type method with nonmonotone line search. Finally, applications of the nonmonotone TR algorithm to some optimization problems are discussed.

On Minimizing and Stationary Sequences of a New Class of Merit Functions for Nonlinear Complementarity Problems

Motivated by the work of Fukushima and Pang (Ref. 1), we study the equivalent relationship between minimizing and stationary sequences of a new class of merit functions for nonlinear complementarity problems (NCP). These merit functions generalize that obtained via the squared Fischer–Burmeister NCP function, which was used in Ref. 1. We show that a stationary sequence {xk} ⊂ /Ren is a minimizing sequence under the condition that the function value sequence {F(xk)} is bounded above or the Jacobian matrix sequence {F′(xk)} is bounded, where F is the function involved in NCP. The latter condition is also assumed by Fukushima and Pang. The converse is true under the assumption of {F′(xk)} bounded. As an example shows, even for a bounded function F, the boundedness of the sequence {F′(xk)} is necessary for a minimizing sequence to be a stationary sequence.

On the rate of convergence of two minimax algorithms

We show that the sequences of function values constructed by two versions of a minimax algorithm converge linearly to the minimum values. Both versions use the Pshenichnyi-Pironneau-Polak search direction subprocedure; the first uses an exact line search to determine the stepsize, while the second one uses an Armijo-type stepsize rule. The proofs depend on a second-order sufficiency condition, but not on strict complementary slackness. Minimax problems in which each function appearing in the max is a composition of a twice continuously differentiable function with a linear function typically do not satisfy second-order sufficiency conditions. Nevertheless, we show that, on such minimax problems, the two algorithms do converge linearly when the outer functions are convex and strict complementary slackness holds at the solutions.