A trust‐region funnel algorithm for gray‐box optimization

A globally convergent method to accelerate topology optimization using on-the-fly model reduction

Nonmonotone trust region method for solving optimization problems

Multi-scale optimization for process systems engineering

This monograph addresses the state of the art of reduced order methods for modeling and computational reduction of complex parametrized systems, governed by ordinary and/or partial differential equations, with a special emphasis on real time computing techniques and applications in computational mechanics, bioengineering and computer graphics. Several topics are covered, including: design, optimization, and control theory in real-time with applications in engineering; data assimilation, geometry registration, and parameter estimation with special attention to real-time computing in biomedical engineering and computational physics; real-time visualization of physics-based simulations in computer science; the treatment of high-dimensional problems in state space, physical space, or parameter space; the interactions between different model reduction and dimensionality reduction approaches; the development of general error estimation frameworks which take into account both model and discretization effects. This book is primarily addressed to computational scientists interested in computational reduction techniques for large scale differential problems.

An Algorithm for Least-Squares Estimation of Nonlinear Parameters

Heat Exchanger Network Optimization including Detailed Heat Exchanger Models using Trust Region Method

PurposeThe purpose of this paper is to employ stochastic techniques to increase efficiency of the classical algorithms for solving nonlinear optimization problems.Design/methodology/approachThe well-known simulated annealing strategy is employed to search successive neighborhoods of the classical trust region (TR) algorithm.FindingsAn adaptive formula for computing the TR radius is suggested based on an eigenvalue analysis conducted on the memoryless Broyden-Fletcher-Goldfarb-Shanno updating formula. Also, a (heuristic) randomized adaptive TR algorithm is developed for solving unconstrained optimization problems. Results of computational experiments on a set of CUTEr test problems show that the proposed randomization scheme can enhance efficiency of the TR methods.Practical implicationsThe algorithm can be effectively used for solving the optimization problems which appear in engineering, economics, management, industry and other areas.Originality/valueThe proposed randomization scheme improves computational costs of the classical TR algorithm. Especially, the suggested algorithm avoids resolving the TR subproblems for many times.

Purpose Nonlinear dynamical systems may, under certain conditions, be represented by a bilinear system. The paper is concerned with the construction of the controllability and observability gramians for the corresponding bilinear system. Such gramians form the core of model reduction schemes involving balancing. Design/methodology/approach The paper examines certain properties of the bilinear system and identifies parameters that capture important information relating to the behaviour of the system. Findings Novel approaches for the determination of approximate constant gramians for use in balancing-type model reduction techniques are presented. Numerical examples are given which indicate the efficacy of the proposed formulations. Research limitations/implications The systems under consideration are restricted to the so-called weakly nonlinear systems, i.e. those without strong nonlinearities where the essential type of behaviour of the system is determined by its linear part. Practical implications The suggested methods lead to an improvement in the accuracy of model reduction. Model reduction is a vital aspect of modern system simulation. Originality/value The proposed novel approaches for model reduction are particularly beneficial for the design of controllers for nonlinear systems and for the design of radio-frequency integrated circuits.

Annual Report: Carbon Capture Simulation Initiative (CCSI) (30 September 2013)

A theory of globally convergent trust-region methods for self-consistent field electronic structure calculations that use the density matrices as variables is developed. The optimization is performed by means of sequential global minimizations of a quadratic model of the true energy. The global minimization of this quadratic model, subject to the idempotency of the density matrix and the rank constraint, coincides with the fixed-point iteration. We prove that the global minimization of this quadratic model subject to the restrictions and smaller trust regions corresponds to the solution of level-shifted equations. The precise implementation of algorithms leading to global convergence is stated and a proof of global convergence is provided. Numerical experiments confirm theoretical predictions and practical convergence is obtained for difficult cases, even if their geometries are highly distorted. The reduction of the trust region is performed by a strategy that uses the structure of the energy function providing the algorithm with a nice practical behavior. This framework may be applied to any problem with idempotency constraints and for which the derivative of the objective function is a symmetric matrix. Therefore, application to calculations based both on Hartree–Fock or Kohn–Sham density functional theory are straightforward.

In this paper, a novel trust-region-based surrogate-assisted optimization method, called CBOILA (Constrained Black-box Optimization by Intrinsically Linear Approximation), has been proposed to reduce the number of black-box function evaluations and enhance the efficient performance for solving complex optimization problems. This developed optimization approach utilizes an assembly of intrinsically linear approximations to seek the optimum with incorporation of three strategies: (1) extended-box selection strategy (EBS), (2) global intelligence selection strategy (GIS) and (3) balanced trust-region strategy. EBS aims at reducing the number of function evaluations in current iteration by selecting points close to the given trust region boundary. Whilst, GIS is designed to improve the exploration performance by adaptively choosing points outside the trust region. The balanced trust-region strategy works with four indicators, which will be triggered by the quality of the approximation, the movement direction of the search, the location of the sub-optimum, and the condition of the termination, respectively. By modifying the move limit of each dimension accordingly, CBOILA is capable of attaining a balanced search between exploitation and exploration for the optimal solutions. To demonstrate the potentials of the proposed optimization method, four widely used benchmark problems have been examined and the results have also been compared with solutions by other metamodel-based algorithms in published works. Results show that the proposed method can efficiently and robustly solve constrained black-box optimization problems within an acceptable computational time.

This thesis deals with the development of numerical methods for solving nonconvex optimisation problems by means of decomposition and continuation techniques. We first introduce a novel decomposition algorithm based on alternating gradient projections and augmented Lagrangian relaxations. A proof of local convergence is given under standard assumptions. The effect of different stopping criteria on the convergence of the augmented Lagrangian loop is investigated. As a second step, a trust region algorithm for distributed nonlinear programs, named TRAP, is introduced. Its salient ingredient is an alternating gradient projection for computing a set of active constraints in a distributed manner, which is a novelty for trust region techniques. Global convergence as well as local almost superlinear convergence are proven. The numerical performance of the algorithm is assessed on nonconvex optimal power flow problems. The core of this thesis is the development and analysis of an augmented Lagrangian algorithm for tracking parameter-dependent optima. Despite their interesting features for large-scale and distributed optimisation, augmented Lagrangian methods have not been designed and fully analysed in a parametric setting. Therefore, we propose a novel optimality-tracking scheme that consists of fixed number of descent steps computed on an augmented Lagrangian subproblem and one dual update per parameter change. It is shown that the descent steps can be performed by means of first-order as well as trust region methods. Using the Kurdyka-Lojasiewicz property, an analysis of the local convergence rate of a class of trust region Newton methods is provided without relying on the finite detection of an optimal active set. This allows us to establish a contraction inequality for the parametric augmented Lagrangian algorithm. Hence, stability of the continuation scheme can be proven under mild assumptions. The effect of the number of primal iterations and the penalty is analysed by means of numerical examples. Finally, the efficacy of the augmented Lagrangian continuation scheme is successfully demonstrated on three examples in the field of optimal control. The first two examples consists of a real-time NMPC algorithm based on a multiple-shooting discretisation. In particular, it is shown that our C++ software package is competitive with the state-of-the-art codes on NMPC problems with long prediction horizons, and can address a more general class of real-time NMPC problems. The third case study is the distributed computation of solutions to multi-stage nonconvex optimal power flow problems in a real-time setting.

Multiobjective optimization has a significant number of real-life applications. For this reason, in this paper we consider the problem of finding Pareto critical points for unconstrained multiobjective problems and present a trust-region method to solve it. Under certain assumptions, which are derived in a very natural way from assumptions used to establish convergence results of the scalar trust-region method, we prove that our trust-region method generates a sequence which converges in the Pareto critical way. This means that our generalized marginal function, which generalizes the norm of the gradient for the multiobjective case, converges to zero. In the last section of this paper, we give an application to satisficing processes in Behavioral Sciences. Multiobjective trust-region methods appear to be remarkable specimens of much more abstract satisficing processes, based on “variational rationality” concepts. One of their important merits is to allow for efficient computations. This is a striking result in Behavioral Sciences.

We present a new algorithm for the solution of nonlinear least squares problems arising from parameterized imaging problems with diffuse optical tomographic data [D. Boas et al., IEEE Signal Process. Mag., 18 (2001), pp. 57-75]. The parameterization arises from the use of parametric level sets for regularization [M.E. Kilmer et al., Proc. SPIE, 5559 (2004), pp. 381-391], [A. Aghasi, M.E. Kilmer, and E.L. Miller, SIAM J. Imaging Sci., 4 (2011), pp. 618-650]. Such problems lead to Jacobians that have relatively few columns, a relatively modest number of rows, and are ill-conditioned. Moreover, such problems have function and Jacobian evaluations that are computationally expensive. Our optimization algorithm is appropriate for any inverse or imaging problem with those characteristics. In fact, we expect our algorithm to be effective for more general problems with ill-conditioned Jacobians. The algorithm aims to minimize the total number of function and Jacobian evaluations by analyzing which spectral components of the Gauss-Newton direction should be discarded or damped. The analysis considers for each component the reduction of the objective function and the contribution to the search direction, restricting the computed search direction to be within a trust region. The result is a truncated SVD-like approach to choosing the search direction. However, we do not necessarily discard components in order of decreasing singular value, and some components may be scaled to maintain fidelity to the trust region model. Our algorithm uses the Basic Trust Region Algorithm from [A.R. Conn, N.I.M. Gould, and Ph. L. Toint, Trust-Region Methods, SIAM, Philadelphia, 2000]. We prove that our algorithm is globally convergent to a critical point. Our numerical results show that the new algorithm generally outperforms competing methods applied to the DOT imaging problem with parametric level sets, and regularly does so by a significant factor.

We study a new trust region affine scaling method for general bound constrained optimization problems. At each iteration, we compute two trial steps. We compute one along some direction obtained by solving an appropriate quadratic model in an ellipsoidal region. This region is defined by an affine scaling technique. It depends on both the distances of current iterate to boundaries and the trust region radius. For convergence and avoiding iterations trapped around nonstationary points, an auxiliary step is defined along some newly defined approximate projected gradient. By choosing the one which achieves more reduction of the quadratic model from the two above steps as the trial step to generate next iterate, we prove that the iterates generated by the new algorithm are not bounded away from stationary points. And also assuming that the second-order sufficient condition holds at some nondegenerate stationary point, we prove the Q-linear convergence of the objective function values. Preliminary numerical experience for problems with bound constraints from the CUTEr collection is also reported.