Inexact Restoration Research Articles

An inexact restoration direct multisearch filter approach to multiobjective constrained derivative-free optimization

Direct Multisearch (DMS) is a well-established class of methods for multiobjective derivative-free optimization, where constraints are addressed by an extreme barrier approach, only evaluating feasible points. In this work, we propose the replacement of this extreme barrier approach by a filter strategy, combined with an inexact feasibility restoration step, to address constraints in the DMS framework. The filter approach treats feasibility as an additional component of the objective function that needs to be minimized. The inexact restoration step attempts to generate new feasible points, contributing to prioritize this feasibility, a requirement for the good performance of any filter approach. Theoretical results are provided, analysing the different types of sequences of points generated by the new algorithm, and numerical experiments on a set of nonlinearly constrained biobjective problems are reported, stating the good algorithmic performance of the proposed approach.

Improving the Global Convergence of Inexact Restoration Methods for Constrained Optimization Problems

Improving the Global Convergence of Inexact Restoration Methods for Constrained Optimization Problems

Inexact restoration for minimization with inexact evaluation both of the objective function and the constraints

In a recent paper an Inexact Restoration method for solving continuous constrained optimization problems was analyzed from the point of view of worst-case functional complexity and convergence. On the other hand, the Inexact Restoration methodology was employed, in a different research, to handle minimization problems with inexact evaluation and simple constraints. These two methodologies are combined in the present report, for constrained minimization problems in which both the objective function and the constraints, as well as their derivatives, are subject to evaluation errors. Together with a complete description of the method, complexity and convergence results will be proved.

On the Convergence Properties of a Stochastic Trust-Region Method with Inexact Restoration

We study the convergence properties of SIRTR, a stochastic inexact restoration trust-region method suited for the minimization of a finite sum of continuously differentiable functions. This method combines the trust-region methodology with random function and gradient estimates formed by subsampling. Unlike other existing schemes, it forces the decrease of a merit function by combining the function approximation with an infeasibility term, the latter of which measures the distance of the current sample size from its maximum value. In a previous work, the expected iteration complexity to satisfy an approximate first-order optimality condition was given. Here, we elaborate on the convergence analysis of SIRTR and prove its convergence in probability under suitable accuracy requirements on random function and gradient estimates. Furthermore, we report the numerical results obtained on some nonconvex classification test problems, discussing the impact of the probabilistic requirements on the selection of the sample sizes.

Inexact Restoration Methods for Semivectorial Bilevel Programming Problem on Riemannian Manifolds

For a better understanding of the bilevel programming on Riemannian manifolds, a semivectorial bilevel programming scheme is proposed in this paper. The semivectorial bilevel programming is firstly transformed into a single-level programming problem by using the Karush–Kuhn–Tucker (KKT) conditions of the lower-level problem, which is convex and satisfies the Slater constraint qualification. Then, the single-level programming is divided into two stages: restoration and minimization, based on which an Inexact Restoration algorithm is developed. Under certain conditions, the stability and convergence of the algorithm are analyzed.

An inexact restoration-nonsmooth algorithm with variable accuracy for stochastic nonsmooth convex optimization problems in machine learning and stochastic linear complementarity problems

We study unconstrained optimization problems with nonsmooth and convex objective function in the form of a mathematical expectation. The proposed method approximates the expected objective function with a sample average function using Inexact Restoration-based adapted sample sizes. The sample size is chosen in an adaptive manner based on Inexact Restoration. The algorithm uses line search and assumes descent directions with respect to the current approximate function. We prove the a.s. convergence under standard assumptions. Numerical results for two types of problems, machine learning loss function for training classifiers and stochastic linear complementarity problems, prove the efficiency of the proposed scheme.

A stochastic first-order trust-region method with inexact restoration for finite-sum minimization

We propose a stochastic first-order trust-region method with inexact function and gradient evaluations for solving finite-sum minimization problems. Using a suitable reformulation of the given problem, our method combines the inexact restoration approach for constrained optimization with the trust-region procedure and random models. Differently from other recent stochastic trust-region schemes, our proposed algorithm improves feasibility and optimality in a modular way. We provide the expected number of iterations for reaching a near-stationary point by imposing some probability accuracy requirements on random functions and gradients which are, in general, less stringent than the corresponding ones in literature. We validate the proposed algorithm on some nonconvex optimization problems arising in binary classification and regression, showing that it performs well in terms of cost and accuracy, and allows to reduce the burdensome tuning of the hyper-parameters involved.

Inexact restoration for derivative-free expensive function minimization and applications

The Inexact Restoration approach has proved to be an adequate tool for handling the problem of minimizing an expensive function within an arbitrary feasible set by using different degrees of precision. This framework allows one to obtain suitable convergence and complexity results for an approach that rationally combines low- and high-precision evaluations. In this paper we consider the case where the domain of the optimization problem is an abstract metric space. Assumptions about differentiability or even continuity will not be used in the general algorithm based on Inexact Restoration. Although optimization phases that rely on smoothness cannot be used in this case, basic convergence and complexity results are recovered. A new derivative-free optimization phase is defined and the subproblems that arise at this phase are solved using a regularization approach that takes advantage of different notions of stationarity. The new methodology is applied to the problem of reproducing a controlled experiment that mimics the failure of a dam.

Feasible Sequential Convex Programming With Inexact Restoration for Multistage Ascent Trajectory Optimization

This paper presents a feasible sequential convex programming method to solve the multistage ascent trajectory optimization problem. The proposed method is based on the inexact restoration technique, in which a more feasible intermediate iterate is first produced by solving a constrained least-squares problem, and then a more optimal iterate is generated by solving a convex programming problem constructed around the newly found feasible solution. By virtue of the inexact restoration idea, the proposed method prevents the common artificial infeasibility issue and can provide the intermediate iterate as a feasible suboptimal solution if the algorithmic procedure terminates before convergence. In addition, the Picard iteration-based convexification and Chebyshev polynomial-based discretization methods are employed in the proposed method, given their benefits in terms of robustness, efficiency, and solution accuracy. Numerical simulations for a minimum-time launch ascent problem are conducted, and the results show that the proposed method exhibits better practical performance than other sequential convex programming methods.

The demand adjustment problem via inexact restoration method

In this work, the demand adjustment problem (DAP) associated with urban traffic planning is studied. The framework for the formulation of the DAP is mathematical programming with equilibrium constraints. In particular, if the optimization program associated with the equilibrium constraint is considered, the DAP results in a bilevel optimization problem. In this approach, the DAP via the inexact restoration method is treated.

Nonmonotone inexact restoration approach for minimization with orthogonality constraints

Minimizing a differentiable function restricted to the set of matrices with orthonormal columns finds applications in several fields of science and engineering. In this paper, we propose to solve this problem through a nonmonotone variation of the inexact restoration method consisting of two phases: restoration phase, aimed to improve feasibility, and minimization phase, aimed to decrease the function value in the tangent set of the constraints. For this, we give a suitable characterization of the tangent set of the orthogonality constraints, allowing us to (i) deal with the minimization phase efficiently and (ii) employ the Cayley transform to bring a point in the tangent set back to feasibility, leading to a SVD-free restoration phase. Based on previous global convergence results for the nonmonotone inexact restoration algorithm, it follows that any limit point of the sequence generated by the new algorithm is stationary. Moreover, numerical experiments on three different classes of the problem indicate that the proposed method is reliable and competitive with other recently developed methods.

Inexact restoration with subsampled trust-region methods for finite-sum minimization

Convex and nonconvex finite-sum minimization arises in many scientific computing and machine learning applications. Recently, first-order and second-order methods where objective functions, gradients and Hessians are approximated by randomly sampling components of the sum have received great attention. We propose a new trust-region method which employs suitable approximations of the objective function, gradient and Hessian built via random subsampling techniques. The choice of the sample size is deterministic and ruled by the inexact restoration approach. We discuss local and global properties for finding approximate first- and second-order optimal points and function evaluation complexity results. Numerical experience shows that the new procedure is more efficient, in terms of overall computational cost, than the standard trust-region scheme with subsampled Hessians.

On the Complexity of an Inexact Restoration Method for Constrained Optimization

Recent papers indicate that some algorithms for constrained optimization may exhibit worst-case complexity bounds that are very similar to those of unconstrained optimization algorithms. A natural ...

Non-monotone inexact restoration method for nonlinear programming

This paper deals with a new variant of the inexact restoration method of Fischer and Friedlander (Comput Optim Appl 46:333–346, 2010) for nonlinear programming. We propose an algorithm that replaces the monotone line search performed in the tangent phase by a non-monotone one, using the sharp Lagrangian as merit function. Convergence to feasible points satisfying the convex approximate gradient projection condition is proved under mild assumptions. Numerical results on representative test problems show that the proposed approach outperforms the monotone version when a suitable non-monotone parameter is chosen and is also competitive against other globalization strategies for inexact restoration.

Bilevel optimization with a multiobjective problem in the lower level

Bilevel problems model instances with a hierarchical structure. Aiming at an efficient solution of a constrained multiobjective problem according with some pre-defined criterion, we reformulate this semivectorial bilevel optimization problem as a classic bilevel one. This reformulation intents to encompass all the objectives, so that the properly efficient solution set is recovered by means of a convenient weighted-sum scalarization approach. Inexact restoration strategies potentially take advantage of the structure of the problem under consideration, being employed as an alternative to the Karush-Kuhn-Tucker reformulation of the bilevel problem. Genuine multiobjective problems possess inequality constraints in their modeling, and these constraints generate theoretical and practical difficulties to our lower level problem. We handle these difficulties by means of a perturbation strategy, providing the convergence analysis, together with enlightening examples and illustrative numerical tests.

On the employment of inexact restoration for the minimization of functions whose evaluation is subject to errors

On the employment of inexact restoration for the minimization of functions whose evaluation is subject to errors

Global convergence of a derivative-free inexact restoration filter algorithm for nonlinear programming

In this work, we present an algorithm for solving constrained optimization problems that does not make explicit use of the objective function derivatives. The algorithm mixes an inexact restoration framework with filter techniques, where the forbidden regions can be given by the flat or slanting filter rule. Each iteration is decomposed into two independent phases: a feasibility phase which reduces an infeasibility measure without evaluations of the objective function, and an optimality phase which reduces the objective function value. As the derivatives of the objective function are not available, the optimality step is computed by derivative-free trust-region internal iterations. Any technique to construct the trust-region models can be used since the gradient of the model is a reasonable approximation of the gradient of the objective function at the current point. Assuming this and classical assumptions, we prove that the full steps are efficient in the sense that near a feasible nonstationary point, the decrease in the objective function is relatively large, ensuring the global convergence results of the algorithm. Numerical experiments show the effectiveness of the proposed method.

Inexact Restoration approach for minimization with inexact evaluation of the objective function

A new method is introduced for minimizing a function that can be computed only inexactly, with different levels of accuracy. The challenge is to evaluate the (potentially very expensive) objective function with low accuracy as far as this does not interfere with the goal of getting high accuracy minimization at the end. For achieving this goal the problem is reformulated in terms of constrained optimization and handled with an Inexact Restoration technique. Convergence is proved and numerical experiments motivated by Electronic Structure Calculations are presented, which indicate that the new method overcomes current approaches for solving large-scale problems.

An inexact restoration approach to optimization problems with multiobjective constraints under weighted-sum scalarization

We apply a flexible inexact-restoration (IR) algorithm to optimization problems with multiobjective constraints under the weighted-sum scalarization approach. In IR methods each iteration has two phases. In the first phase one aims to improve the feasibility and, in the second phase, one minimizes a suitable objective function. We show that with the IR framework there is a natural way to explore the structure of the problem in both IR phases. Numerical experiments are conducted on Portfolio optimization, the More–Garbow–Hillstrom collection, and random fourth-degree polynomials, where we show the advantages of exploiting the structure of the problem.

An inexact restoration derivative-free filter method for nonlinear programming

An inexact restoration derivative-free filter method for nonlinear programming is introduced in this paper. Each iteration is composed of a restoration phase, which reduces a measure of infeasibility, and an optimization phase, which reduces the objective function. The restoration phase is solved using a derivative-free method for solving underdetermined nonlinear systems with bound constraints, developed previously by the authors. An alternative for solving the optimization phase is considered. Theoretical convergence results and some preliminary numerical experiments are presented.