Trust Region Technique Research Articles

Trust region based chaotic search for solving multi‐objective optimization problems

AbstractA numerical optimization technique used to address nonlinear programming problems is the trust region (TR) method. TR uses a quadratic model, which may represent the function adequately, to create a neighbourhood around the current best solution as a trust region in each step, rather than searching for the original function's objective solution. This allows the method to determine the next local optimum. The TR technique has been utilized by numerous researchers to tackle multi‐objective optimization problems (MOOPs). But there is not any publication that discusses the issue of applying a chaotic search (CS) with the TR algorithm for solving multi‐objective (MO) problems. From this motivation, the main contribution of this study is to introduce trust‐region (TR) technique based on chaotic search (CS) for solving MOOPs. First, the reference point interactive approach is used to convert MOOP to a single objective optimization problem (SOOP). The search space is then randomly initialized with a set of initial points. Second, in order to supply locations on the Pareto boundary, the TR method solves the SOOP. Finally, all points on the Pareto frontier are obtained using CS. A range of MO benchmark problems have demonstrated the efficiency of the proposed algorithm (TR based CS) in generating Pareto optimum sets for MOOPs. Furthermore, a demonstration of the suggested algorithm's ability to resolve real‐world applications is provided through a practical implementation of the algorithm to improve an abrasive water‐jet machining process (AWJM).

A trust-region scheme for constrained multi-objective optimization problems with superlinear convergence property

In this paper, a numerical approximation method is developed to find approximate solutions to a class of constrained multi-objective optimization problems. All the functions of the problem are not necessarily convex functions. At each iteration of the method, a particular type of subproblem is solved using the trust region technique, and the step is evaluated using the notions of actual reduction and predicted reduction. A non-differentiable l ∞ penalty function restricts the constraint violations. An adaptive BFGS update formula is introduced. Global convergence of the proposed algorithm is established under the Mangasarian-Fromovitz constraint qualification and some mild assumptions. Furthermore, it is justified that the proposed algorithm displays a super-linear convergence rate. Numerical results are provided to show the efficiency of the algorithm in the quality of the approximated Pareto front.

Overview of Trust-region Methods

The trust-region technique is an optimization algorithm for solving multidimensional nonlinear optimization complications. It is a class of derivative-based optimization approaches that relies on information about the gradient of the objective function and possibly Hessians. The main idea behind the Trust-region method is to remodel the objective function around the present iteration Solution of the inner subproblem. The confidence region is the region around the present iteration where the model is expected to be accurate. At each iteration, the trust region technique uses a quadratic model to estimate the objective function locally (Yuan, 2019). This model is based on the current iteration and the information about the gradient and Hessian. The confidence region limits the distance the algorithm can take from the current iteration to find the next one. The size of the trust region is adjusted dynamically based on the arrangement between the model and the actual performance values.

A New Adaptive Levenberg–Marquardt Method for Nonlinear Equations and Its Convergence Rate under the Hölderian Local Error Bound Condition

The Levenberg–Marquardt (LM) method is one of the most significant methods for solving nonlinear equations as well as symmetric and asymmetric linear equations. To improve the method, this paper proposes a new adaptive LM algorithm by modifying the LM parameter, combining the trust region technique and the non-monotone technique. It is interesting that the new algorithm is constantly optimized by adaptively choosing the LM parameter. To evaluate the effectiveness of the new algorithm, we conduct tests using various examples. To extend the convergence results, we prove the convergence of the new algorithm under the Hölderian local error bound condition rather than the commonly used local error bound condition. Theoretical analysis and numerical results show that the new algorithm is stable and effective.

A nonmonotone adaptive trust region technique with a forgetting factor

Nonmonotonicity has been considered to be essentially influential on the efficiency of the iterative procedures of nonlinear optimization. A review of the literature shows that the objective function values available from recent iterations provide worthier information in the nonmonotone schemes. So, with the aim of enhancing probability of applying more recent available function values, a nonmonotone trust region ratio is suggested using a forgetting factor. Meanwhile, modification of a recent adaptive formula for the trust region radius is devised by a nonmonotone reflection as well. Then, based on the two mentioned modifications, an adaptive nonmonotone trust region algorithm is given. In addition, convergence of the method is analysed under classic assumptions. To provide support for our theoretical arguments, computational merits of the given algorithm on a set of CUTEr test functions are depicted.

Nonconvex TVq model with l1 fidelity term in image restoration

Restoration of images based on a nonconvex �! variational model which contains the �" “norm,” q ∈ (0,1), is considered. The existence of a minimizer of the model is established in the discrete setting. To solve the minimization problem, Huber-type regularizations of the �" and the �!norms are performed on the model. A primal-dual semismooth Newton method with a trust region technique is adapted to solve the model and the convergence of the algorithm is presented. Applications are made on restoration of images corrupted with blur and impulse noise.

A Derivative-Free Optimization Algorithm Combining Line-Search and Trust-Region Techniques

A Derivative-Free Optimization Algorithm Combining Line-Search and Trust-Region Techniques

Quantifying strong point sources emissions of CO2 using spaceborne LiDAR: Method development and potential analysis

Accurate reporting of point source emissions of CO2 is fundamental to addressing climate change. Currently, bottom-up verification methods based on inventory statistics face significant challenges in this area. Satellite remote sensing has emerged as a promising approach for cost-effective global-scale verification of point source emissions, with spaceborne LiDAR offering high spatial resolution ideal for this purpose. However, the inversion of CO2 emissions from spaceborne LiDAR CO2 concentration observations requires urgent attention, as existing methods heavily rely on prior information in diffusion models and the accuracy of meteorological data. In this work, a novel emission inversion method based on genetic algorithms and trust-region techniques is proposed to estimate CO2 emissions from point sources using spaceborne LiDAR observations. A comparison between the CO2 emission rates calculated from actual airborne LiDAR data (as a prototype of spaceborne LiDAR) and emission inventories for the Suizhong power plant showed a deviation of less than 7.0%. Observing system simulation experiment (OSSE) demonstrated that using DQ-1 (spaceborne LiDAR) observation data as input, the relative error of emission rates would be less than 0.6% when the distance between the emission source and the observation footprint is less than 10 km. Furthermore, the developed model mitigates the impact of uncertainties in meteorological data and IPDA (Integrated-Path Differential Absorption) LiDAR measurements on the final emission quantification. The proposed approach is expected to enable DQ-1 to provide affordable and accurate carbon verification services for over 20.0% of the world's strong point source emissions.

Nonlinear solver based on trust region approximation for [formula omitted] utilization and storage in subsurface reservoir

Simulation of CO2 utilization and storage (CCUS) in subsurface reservoirs with complex heterogeneous structures requires a model that captures multiphase compositional flow and transport. Accurate simulation of these processes necessitates the use of stable numerical methods that are based on an implicit treatment of the flux term in the conservation equation. Due to the complicated thermodynamic phase behavior, including the appearance and disappearance of multiple phases, the discrete approximation of the governing equations is highly nonlinear. Consequently, robust and efficient techniques are needed to solve the resulting nonlinear system of algebraic equations. In this study, we present a powerful nonlinear solver based on a generalization of the trust-region technique for compositional multiphase flows. The approach is designed to embed a newly introduced Operator-Based Linearization technique and is grounded on the analysis of multi-dimensional tables related to parameterized convection operators. We split the parameter space of the nonlinear problem into a set of trust regions where the convection operators preserve the second-order behavior (i.e., they remain positive or negative definite). We approximate these trust regions in the solution process by detecting the boundary of convex regions via analysis of the directional derivative. This analysis is performed adaptively while tracking the nonlinear update trajectory in the parameter space. The proposed nonlinear solver locally constrains the update of the overall compositions across the boundaries of convex regions. We tested the performance of the proposed nonlinear solver for various scenarios. In many cases, our approach yields an improved behavior of the nonlinear solution in comparison to state-of-the-art solvers.

A New Adaptive Accelerated Levenberg–Marquardt Method for Solving Nonlinear Equations and Its Applications in Supply Chain Problems

In this paper, a new adaptive Levenberg–Marquardt method is proposed to solve the nonlinear equations including supply chain optimization problems. We present a new adaptive update rule which is a segmented function on the ratio between the actual and predicted reductions of the objective function to accept a large number of unsuccessful iterations and avoid jumping in local areas. The global convergence and quadratic convergence of the proposed method are proved by using the trust region technique and local error bound condition, respectively. In addition, we use the proposed algorithm to test on the symmetric and asymmetric linear equations. Numerical results show that the proposed method has good numerical performance and development prospects. Furthermore, we apply the algorithm to solve the fresh agricultural products supply chain optimization problems.

A Bundle Trust Region Algorithm for Minimizing Locally Lipschitz Functions

We propose a new algorithm for minimizing locally Lipschitz functions that combines both the bundle and trust region techniques. Based on the bundle methods the objective function is approximated by a piecewise linear working model which is updated by adding cutting planes at unsuccessful trial steps. The algorithm defines, at each iteration, a new trial point by solving a subproblem that employs the working model in the objective function subject to a region, which is called the trust region. The algorithm is studied from both theoretical and practical points of view. Under a assumption on the objective function, global convergence of it is verified to stationary points. In order to demonstrate the reliability and efficiency of the proposed algorithm, a MATLAB implementation of it is prepared and numerical experiments have been made using some academic nonsmooth test problems. Computational results show that the developed method is efficient for solving nonsmooth and nonconvex optimization problems.

Convergence rate of the modified Levenberg-Marquardt method under Hölderian local error bound

Abstract In this article, we analyze the convergence rate of the modified Levenberg-Marquardt (MLM) method under the Hölderian local error bound condition and the Hölderian continuity of the Jacobian, which are more general than the local error bound condition and the Lipschitz continuity of the Jacobian. Under special circumstances, the convergence rate of the MLM method coincides with the results presented by Fan. A globally convergent MLM algorithm by the trust region technique will also be given.

An active-set with barrier method and trust-region mechanism to solve a nonlinear Bilevel programming problem

<abstract><p>Nonlinear Bilevel programming (NBLP) problem is a hard problem and very difficult to be resolved by using the classical method. In this paper, Karush-Kuhn-Tucker (KKT) condition is used with Fischer-Burmeister function to convert NBLP problem to an equivalent smooth single objective nonlinear programming (SONP) problem. An active-set strategy is used with Barrier method and trust-region technique to solve the smooth SONP problem effectively and guarantee a convergence to optimal solution from any starting point. A global convergence theory for the active-set barrier trust-region (ACBTR) algorithm is studied under five standard assumptions. An applications to mathematical programs are introduced to clarify the effectiveness of ACBTR algorithm. The results show that ACBTR algorithm is stable and capable of generating approximal optimal solution to the NBLP problem.</p></abstract>

Trust-Region Solver of a Nonlinear Magnetometer Disturbance Estimation Problem

Earth geomagnetic field provides very important information for autonomous navigation. However, in practice, magnetometer measurements are easily distorted by outer disturbances, as shown in the right diagram. In a recent study, the real-time magnetometer disturbance estimation problem has been solved via a constrained nonlinear programming for better underwater navigation accuracy. However, the employed interior-point optimizer will require considerable computational resources and cannot always guarantee optimality during optimization updates. This paper further investigates this problem and refines the solution by introducing the trust-region method. The challenge of the designed approach mainly falls into finding out the feasible regions of the possible solutions. Geometric analysis and algebraic elimination methods are taken into account to give globally optimal solutions to the trust-region solver. Experimentations are conducted to verify the correctness of the proposed method. Moreover, it is also confirmed that the trust-region technique can continuously and accurately propagate so that high-frequency dynamic magnetic disturbances can be efficiently estimated.

Modification of high performance training algorithm for solving singular perturbation partial differential equations with cubic convergence

In this study, we used the update a neural network for solve singularly perturbed partial differential equations (SPPDEs) that have been singularly disturbed. We dealt with the modernization of artiftial networks by updating the most important types of high-performance training. For nonlinear equations, the modified Levenberg-Marquardt method (MLMM) is used, in which not only an LM step but also an approximation LM step is generated at each iteration. When applying the trust region technique, a new type of projected reduction for the merit function is created to assure global convergence of the new method. The modified LM method’s cubic convergence is demonstrated under the local error bound condition, which is weaker than nonsingularity. The numerical results demonstrate that the new method is quite efficient and might save numerous Jacobian calculations.

English

We propose a new and efficient nonmonotone adaptive trust region algorithm to solve unconstrained optimization problems. This algorithm incorporates two novelties: it benefits from a radius dependent shrinkage parameter for adjusting the trust region radius that avoids undesirable directions and exploits a new strategy to prevent sudden increments of objective function values in nonmonotone trust region techniques. Global convergence of this algorithm is investigated under some mild conditions. Numerical experiments demonstrate the efficiency and robustness of the proposed algorithm in solving a collection of unconstrained optimization problems from the CUTEst package.

An estimation–Based optimal scheduling model for settable renewable penetration level in energy hubs

Several recent studies suggest that power–to–gas (PtG) facilities can be operated to absorb and convert surplus renewable energy into hydrogen or synthetic natural gas, which is then supplied to the natural gas grid, forming an energy hub (EH). This paper argues and demonstrates that there is a trade off between maximizing the renewable penetration level (PRL) into the EH and minimizing the cost associated with it. The paper presents a new optimal scheduling model allowing variable and settable PRLs into the EH to minimize the system operating cost. The RPL, as determined by the grid operator, is taken in as a setting to the scheduling model. A new algorithm is presented by which the relationship between the RPL and scheduling setpoints of PtG and gas–fired generation (GfG) facilities are estimated. Accordingly, the optimal scheduling setpoints of PtG and GfG facilities are decided by the model, so that the required RPL can be met. The problem is formulated using the mixed integer nonlinear optimization method where the estimation algorithm is developed using the least error square technique. The developed problem is then solved using the combined interior point nonlinear programming and Newton trust region techniques. Numerical studies are conducted on a test system using historical operating data to validate the efficacy of the model. The system operating parameters are analyzed, and it is demonstrated that it is imperative to optimally manage the desired RPL taking into consideration the arbitrage revenue shortfall resulted from RPL enhancement in an EH.

Improved Sequential Convex Programming Algorithms for Entry Trajectory Optimization

Entry trajectory optimization for hypersonic vehicles has been formulated as constrained optimal control problems, which are difficult to solve because of the existing high nonlinearity and nonconvexity. In recent years, convex optimization has shown promise for real-time onboard applications with the state-of-the-art interior-point methods. In this paper, line-search and trust-region techniques are introduced to fundamentally improve the performance of the sequential convex programming method for entry trajectory optimization. Two improved algorithms are developed: line-search sequential convex programming and trust-region sequential convex programming. In addition, a new trajectory generation method is proposed by taking advantage of the predictor–corrector method to find an initial 3-D trajectory for the developed successive algorithms. As such, the convergence of the solution process is improved. To demonstrate the effectiveness and performance of the newly proposed algorithms, numerical simulations are presented for maximum-terminal-velocity and minimum-heat-load entry problems. Results show that these two problems are not well solved using the basic sequential convex programming algorithm. However, they can be efficiently solved by the two improved algorithms in a few seconds in MATLAB, showing a significant improvement over general-purpose solvers, such as GPOPS.

Shamanskii-Like Levenberg-Marquardt Method with a New Line Search for Systems of Nonlinear Equations

To save the calculations of Jacobian, a multi-step Levenberg-Marquardt method named Shamanskii-like LM method for systems of nonlinear equations was proposed by Fan (2013). Its convergence properties have been proved by using a trust region technique under the local error bound condition. However, the authors wonder whether the similar convergence properties are still true with standard line searches since the direction may not be a descent direction. For this purpose, the authors present a new nonmonotone m-th order Armijo type line search to guarantee the global convergence. Under the same condition as trust region case, the convergence rate also has been shown to be m + 1 by using this line search technique. Numerical experiments show the new algorithm can save much running time for the large scale problems, so it is efficient and promising.

A robust algorithm for solving nonlinear system of equations using trust-region and line-search techniques

Newton's method is an attractive method for solving nonlinear system of equations because of its fast convergence property. However, Newton's method may fail if the Jacobian matrices are singular. Newton's method with trust-region can be used to avoid such problem. In this work, a new trust-region technique for Newton's method was formulated to solve the nonlinear system of equations. The search direction in this method is computed by a sequence of factorisations of the Jacobian matrix with modified structure using a Lagrange multiplier associated with trust-region constraint such that the final modified Jacobian turns out to be well-conditioned (regularised). An optimal Lagrange multiplier was deduced using the same idea of unconstrained optimisation to satisfy the trust-region constraint. Furthermore, Armijo line-search technique is integrated with the method in order to improve the step length. Numerical tests were conducted to investigate the performance of Newton's method integrated with trust-region and line-search techniques.