Inexact Method Research Articles

Inexact methods for the low rank solution to large scale Lyapunov equations

The rational Krylov subspace method (RKSM) and the low-rank alternating directions implicit (LR-ADI) iteration are established numerical tools for computing low-rank solution factors of large-scale Lyapunov equations. In order to generate the basis vectors for the RKSM, or extend the low-rank factors within the LR-ADI method, the repeated solution to a shifted linear system of equations is necessary. For very large systems this solve is usually implemented using iterative methods, leading to inexact solves within this inner iteration (and therefore to “inexact methods”). We will show that one can terminate this inner iteration before full precision has been reached and still obtain very good accuracy in the final solution to the Lyapunov equation. In particular, for both the RKSM and the LR-ADI method we derive theory for a relaxation strategy (e.g. increasing the solve tolerance of the inner iteration, as the outer iteration proceeds) within the iterative methods for solving the large linear systems. These theoretical choices involve unknown quantities, therefore practical criteria for relaxing the solution tolerance within the inner linear system are then provided. The theory is supported by several numerical examples, which show that the total amount of work for solving Lyapunov equations can be reduced significantly.

On the complexity of a hybrid proximal extragradient projective method for solving monotone inclusion problems

In a series of papers (Solodov and Svaiter in J Convex Anal 6(1):59–70, 1999; Set-Valued Anal 7(4):323–345, 1999; Numer Funct Anal Optim 22(7–8):1013–1035, 2001) Solodov and Svaiter introduced new inexact variants of the proximal point method with relative error tolerances. Point-wise and ergodic iteration-complexity bounds for one of these methods, namely the hybrid proximal extragradient method (1999) were established by Monteiro and Svaiter (SIAM J Optim 20(6):2755–2787, 2010). Here, we extend these results to a more general framework, by establishing point-wise and ergodic iteration-complexity bounds for the inexact proximal point method studied by Solodov and Svaiter (2001). Using this framework we derive global convergence results and iteration-complexity bounds for a family of projective splitting methods for solving monotone inclusion problems, which generalize the projective splitting methods introduced and studied by Eckstein and Svaiter (SIAM J Control Optim 48(2):787–811, 2009).

A partially inexact ADMM with o(1/n) asymptotic convergence rate, 𝒪(1/n) complexity, and immediate relative error tolerance

In this work, we propose a new partially inexact Alternating Direction Method of Multipliers (ADMM) with relative error tolerance. This method departs from previous semi-inexact variants of the ADMM by allowing the second subproblem to be solved inexactly, instead of the first one. Relative error tolerance criteria for inexact ADMM methods require, to be verified, the solution of the two proximal subproblems in each iteration; therefore, by allowing for inexactness in the approximate solutions of the second subproblem, the error criterion is immediately testable during the computation of that approximate solution and no backtracking is required. In this sense, the proposed method uses an immediate error criterion. We also establish iteration complexity and asymptotic convergence rate for a modified ergodic mean, with respect to the residuals in Karush–Kuhn–Tucker conditions. As far as we know, this is the first asymptotic convergence rate, with respect to this error measure, for ADMM applied to a general problem.

A Global Linear and Local Superlinear (Quadratic) Inexact Non-Interior Continuation Method for Variational Inequalities Over General Closed Convex Sets

We use the concept of barrier-based smoothing approximations to extend the non-interior continuation method, which was proposed by B. Chen and N. Xiu for nonlinear complementarity problems based on Chen-Mangasarian smoothing functions, to an inexact non-interior continuation method for variational inequalities over general closed convex sets. Newton equations involved in the method are solved inexactly to deal with high dimension problems. The method is proved to have global linear and local superlinear/quadratic convergence under suitable assumptions. We apply the method to non-negative orthants, positive semidefinite cones, polyhedral sets, epigraphs of matrix operator norm cone and epigraphs of matrix nuclear norm cone.

Weaker conditions for inexact mutitpoint Newton-like methods

In this paper we study the problem of analyzing the convergence both local and semilocal of inexact Newton-like methods for approximating the solution of an equation in which there exists nondifferentiability. We will impose conditions, to ensure that the method converges, are weaker than in the ones imposed in previous results. The theoretical results shown in this study are applied to a chemical application in order to be proven.

An algorithm for the minimization of nonsmooth nonconvex functions using inexact evaluations and its worst-case complexity

An adaptive regularization algorithm using inexact function and derivatives evaluations is proposed for the solution of composite nonsmooth nonconvex optimization. It is shown that this algorithm needs at most $$O(|\log (\epsilon )|\,\epsilon ^{-2})$$ evaluations of the problem’s functions and their derivatives for finding an $$\epsilon $$ -approximate first-order stationary point. This complexity bound therefore generalizes that provided by Bellavia et al. (Theoretical study of an adaptive cubic regularization method with dynamic inexact Hessian information. arXiv:1808.06239 , 2018) for inexact methods for smooth nonconvex problems, and is within a factor $$|\log (\epsilon )|$$ of the optimal bound known for smooth and nonsmooth nonconvex minimization with exact evaluations. A practically more restrictive variant of the algorithm with worst-case complexity $$O(|\log (\epsilon )|+\epsilon ^{-2})$$ is also presented.

Discretization of Linear Problems in Banach Spaces: Residual Minimization, Nonlinear Petrov--Galerkin, and Monotone Mixed Methods

This work presents a comprehensive discretization theory for abstract linear operator equations in Banach spaces. The fundamental starting point of the theory is the idea of residual minimization in dual norms, and its inexact version using discrete dual norms. It is shown that this development, in the case of strictly-convex reflexive Banach spaces with strictly-convex dual, gives rise to a class of nonlinear Petrov-Galerkin methods and, equivalently, abstract mixed methods with monotone nonlinearity. Crucial in the formulation of these methods is the (nonlinear) bijective duality map. Under the Fortin condition, we prove discrete stability of the abstract inexact method, and subsequently carry out a complete error analysis. As part of our analysis, we prove new bounds for best-approximation projectors, which involve constants depending on the geometry of the underlying Banach space. The theory generalizes and extends the classical Petrov-Galerkin method as well as existing residual-minimization approaches, such as the discontinuous Petrov-Galerkin method.

Local Convergence of Inexact Newton-Like Method under Weak Lipschitz Conditions

The paper develops the local convergence of Inexact Newton-Like Method (INLM) for approximating solutions of nonlinear equations in Banach space setting. We employ weak Lipschitz and center-weak Lipschitz conditions to perform the error analysis. The obtained results compare favorably with earlier ones such as [7, 13, 14, 18, 19]. A numerical example is also provided.

A unified convergence framework for nonmonotone inexact decomposition methods

In this work we propose a general framework that provides a unified convergence analysis for nonmonotone decomposition algorithms. The main motivation to embed nonmonotone strategies within a decomposition approach lies in the fact that enforcing the reduction of the objective function could be unnecessarily expensive, taking into account that groups of variables are individually updated. We define different search directions and line searches satisfying the conditions required by the presented nonmonotone decomposition framework to obtain global convergence. We employ a set of large-scale network equilibrium problems as a computational example to show the advantages of a nonmonotone algorithm over its monotone counterpart. In conclusion, a new smart implementation for decomposition methods has been derived to solve numerical issues on large-scale partially separable functions.

Research of step-length estimation methods for full waveform inversion in time domain

ABSTRACTFull waveform inversion (FWI) is a powerful tool for reconstructing high-resolution subsurface parameters estimated by iteratively minimising the difference between calculated and observed data. Step-length estimation is a key step in the successful implementation of the FWI algorithm. An optimal step-length value rapidly causes the FWI algorithm to reach the global minimum with reduced iterations and fewer extra forward modelling simulations during each iteration. Step-length can typically be calculated using an inexact or an exact line-search method. The backtracking line-search method (BLSM) is a typical inexact method. Initial methods of guessing the step-length and evaluation conditions determine the efficiency of a BLSM scheme. Here, we propose a quadratic extrapolation value as the initial guess in a BLSM scheme, and then compare it with other initial-guess approaches by using the first Wolfe condition to evaluate step-length values. Exact line-search methods include a parabolic fitting search method through three points (PFSM-3), a parabolic fitting search method through two points (PFSM-2) and the analytical step-length method (ASLM). To find optimal and stable step-length estimation methods for FWI, we compare four step-length estimation methods: BLSM, PFSM-3, PFSM-2 and ASLM. Numerical examples using synthetic data demonstrate that quadratic extrapolation values perform better than first-order change and adaptive values in the BLSM scheme, in terms of resolution of the reconstructed model and computational costs. Of the four step-length estimation methods, ASLM and BLSM are both efficient for noise-free data, and robust ASLM is more efficient for noisy data. However, PFSM-2 and PFSM-3 are less efficient because of low accuracy and high computational cost.

An inexact l2-norm penalty method for cardinality constrained portfolio optimization

We analyze and solve a single-period portfolio optimization problem with non-convex constraints, which address practical concerns of investment such as the active share weights of sectors and the number of stocks held in a portfolio. We reformulate the problem to simplify the computation and propose an inexact l2-norm penalty method to solve the problem.

An Inexact Modified Newton Method for Viscc and Application in Grasping Force

For the variational inequality with symmetric cone constraints problem, we consider using the inexact modified Newton method to efficiently solve it. It provides a unified framework for dealing with the variational inequality with nonlinear constraints, variational inequality with the second-order cone constraints, and the variational inequality with semi-definite cone constraints. We show that each stationary point of the unconstrained minimization reformulation based on the Fischer-Burmeister merit function is a solution to the problem. It is proved that the proposed algorithm is globally convergent under suitable conditions. The computation results show that the feasibility and efficiency of our algorithm.

A New Method to Solve Fuzzy Interval Flexible Linear Programming Using a Multi-Objective Approach

ABSTRACT Recently fuzzy interval flexible linear programs have attracted many interests. These models are an extension of the classical linear programming which deal with crisp parameters. However, in most of the real-world applications, the nature of the parameters of the decision-making problems are generally imprecise. Such uncertainties can lead to increased complexities in the related optimisation efforts. Simply ignoring these uncertainties is considered undesired as it may result in inferior or wrong decisions. Therefore, inexact linear programming methods are desired under uncertainty. In this paper, we concentrate a fuzzy flexible linear programming model with flexible constraints and the interval objective function and then propose a new solving approach based on solving an associated multi-objective model. Finally, a numerical example is included to illustrate the mentioned solving process.

Comparing the accuracy of estimated terrain elevations across spatial resolution

ABSTRACTTerrain is modelled in Geographic Information Science on a grid, assuming that elevation values are constant within any single pixel of a Digital Elevation Model (DEM). Pixels are considered flat and rigid, for computational simplicity (a ‘rigid pixel’ paradigm). This paradigm does not account for the slope and curvature of terrain within each pixel, generating imprecise measurements, particularly as pixel size increases or in uneven terrain. This paper relaxes the rigid pixel assumption, allowing for possible sub-pixel variations in slope and curvature (a ‘surface-adjusted’ paradigm). This paper compares different interpolation methods to investigate sub-pixel variations for estimating elevation of arbitrary points given a regular grid. Tests interpolating elevation values for 20,000 georeferenced off-centroid random points from a regular grid DEM are presented, using a variety of exact and inexact local deterministic interpolation methods within contiguity configurations incorporating first and second order neighbours. The paper examines the accuracy of surface-adjusted estimations across a progression of spatial resolutions (10 m, 30 m, 100 m, and 1,000 m DEMs) and a suite of terrain types varying in latitude, altitude, slope, and roughness, validating off-centre estimates against direct elevation measurements on 3 m resolution lidar DEM. Results illustrate that the Bi-quadratic and Bi-cubic interpolation methods outperform Weighted Average, Linear, and Bi-linear methods at coarse resolutions and in rough or non-uniform terrain. In smooth or flat terrain and at finest resolutions, the interpolation method impacts estimation accuracy less or not at all. The type of contiguity configuration appears to play a role in estimation errors as well, with tighter neighbourhoods exhibiting higher accuracy. The analysis also examined regularized mathematical surfaces, adding autocorrelated randomly distributed noise to simulate terrain. The results of experiments based on regularized smooth mathematical surfaces do not translate directly to terrain modelling. The analysis also considers the balance between the increased computation times needed to measure surface-adjusted elevation against improvements in accuracy.

Extended local convergence for some inexact methods with applications

We present local convergence results for inexact iterative procedures of high convergence order in a normed space in order to approximate a locally unique solution. The hypotheses involve only Lipschitz conditions on the first Frechet-derivative of the operator involved. Earlier results involve Lipschitz-type hypotheses on higher than the first Frechet-derivative. The applicability of these methods is extended this way and under less computational cost. Special cases and applications are provided to show that these new results can apply to solve these equations.

Adaptive Regularization Algorithms with Inexact Evaluations for Nonconvex Optimization

A regularization algorithm using inexact function values and inexact derivatives is proposed and its evaluation complexity analyzed. This algorithm is applicable to unconstrained problems and to problems with inexpensive constraints (that is constraints whose evaluation and enforcement has negligible cost) under the assumption that the derivative of highest degree is $\beta$-H\"{o}lder continuous. It features a very flexible adaptive mechanism for determining the inexactness which is allowed, at each iteration, when computing objective function values and derivatives. The complexity analysis covers arbitrary optimality order and arbitrary degree of available approximate derivatives. It extends results of Cartis, Gould and Toint (2018) on the evaluation complexity to the inexact case: if a $q$th order minimizer is sought using approximations to the first $p$ derivatives, it is proved that a suitable approximate minimizer within $\epsilon$ is computed by the proposed algorithm in at most $O(\epsilon^{-\frac{p+\beta}{p-q+\beta}})$ iterations and at most $O(|\log(\epsilon)|\epsilon^{-\frac{p+\beta}{p-q+\beta}})$ approximate evaluations. An algorithmic variant, although more rigid in practice, can be proved to find such an approximate minimizer in $O(|\log(\epsilon)|+\epsilon^{-\frac{p+\beta}{p-q+\beta}})$ evaluations.While the proposed framework remains so far conceptual for high degrees and orders, it is shown to yield simple and computationally realistic inexact methods when specialized to the unconstrained and bound-constrained first- and second-order cases. The deterministic complexity results are finally extended to the stochastic context, yielding adaptive sample-size rules for subsampling methods typical of machine learning.

Inexact descent methods for elastic parameter optimization

Elastic parameter optimization has revealed its importance in 3D modeling, virtual reality, and additive manufacturing in recent years. Unfortunately, it is known to be computationally expensive, especially if there are many parameters and data samples. To address this challenge, we propose to introduce the inexactness into descent methods, by iteratively solving a forward simulation step and a parameter update step in an inexact manner. The development of such inexact descent methods is centered at two questions: 1) how accurate/inaccurate can the two steps be; and 2) what is the optimal way to implement an inexact descent method. The answers to these questions are in our convergence analysis, which proves the existence of relative error thresholds for the two inexact steps to ensure the convergence. This means we can simply solve each step by a fixed number of iterations, if the iterative solver is at least linearly convergent. While the use of the inexact idea speeds up many descent methods, we specifically favor a GPU-based one powered by state-of-the-art simulation techniques. Based on this method, we study a variety of implementation issues, including backtracking line search, initialization, regularization, and multiple data samples. We demonstrate the use of our inexact method in elasticity measurement and design applications. Our experiment shows the method is fast, reliable, memory-efficient, GPU-friendly, flexible with different elastic models, scalable to a large parameter space, and parallelizable for multiple data samples.

On uniform regularity and strong regularity

ABSTRACTWe investigate uniform versions of (metric) regularity and strong (metric) regularity on compact subsets of Banach spaces, in particular, along continuous paths. These two properties turn out to play a key role in analyzing path-following schemes for tracking a solution trajectory of a parametric generalized equation or, more generally, of a differential generalized equation (DGE). The latter model allows us to describe in a unified way several problems in control and optimization such as differential variational inequalities and control systems with state constraints. We study two inexact path-following methods for DGEs having the order of the grid error and , respectively. We provide numerical experiments, comparing the schemes derived, for simple problems arising in physics. Finally, we study metric regularity of mappings associated with a particular case of the DGE arising in control theory. We establish the relationship between the pointwise version of this property and its counterpart in function spaces.

The α-Representation Inexact T2 Fuzzy Sets Programming Model for Water Resources Management of the Southern Min River Basin under Uncertainty

In this study, a new α-representation of the inexact T2 fuzzy sets programming model (α-RITF2SP) is proposed for water resources planning under uncertainties. The developed model is a hybrid of inexact type-2 (T2) fuzzy sets (FSs) optimal programming based on α-cuts and α-planes (z-slices) algorithms, and both can be used in problems of water resource management without a loss of generality. Therefore, this novel method represents a complementary method to combine both of these algorithms to make inexact T2FSs methods easier and faster for users under uncertainties. This method provides not only a reduction in computations when computing fuzzy functions, but also a map of the primary variable of T2FSs in the secondary membership function. The α-RITF2SP is applied to a real case study of the operation of water resource systems in the southern Min River basin that involves three cities: Xiamen, Zhangzhou, and Quanzhou. The results of the water allocations and benefits have been analyzed. The symptoms of results indicate the higher profit with a higher wastewater treatment rate. It also represents that this model has enhanced abilities and it is relevant and practical as well as can serve as a guide for the development of this regional economy.

Optimal quotients for solving large eigenvalue problems

Quotients for eigenvalue problems (generalized or not) are considered. To have a quotient optimally approximating an eigenvalue, conditions are formulated to maximize the one-dimensional projection of the eigenvalue problem. Respective optimal quotient iterations are derived under the assumption that applying the inverse is affordable. Inexact methods are also considered if applying the inverse is not affordable. Then, to approximate an eigenvector, optimality conditions are formulated to minimize linear independency over a subspace. Equivalence transformations are performed for preconditioning iterations and steering the convergence. These ideas extend to subspaces in a natural way. For the standard eigenvalue problem, a new Arnoldi method arises as an alternative to the classical Arnoldi method.