Inexact Interior-Point Method for PDE-Constrained Nonlinear Optimization

In applied sciences PDEs model an extensive variety of phenomena. Typically the final goal of simulations is a system which is optimal in a certain sense. For instance optimal control problems identify a control to steer a system towards a desired state. Inverse problems seek PDE parameters which are most consistent with measurements. In these optimization problems PDEs appear as equality constraints. PDE-constrained optimization problems are large-scale and often nonconvex. Their numerical solution leads to large ill-conditioned linear systems. In many practical problems inequality constraints implement technical limitations or prior knowledge. In this thesis interior-point (IP) methods are considered to solve nonconvex large-scale PDE-constrained optimization problems with inequality constraints. To cope with enormous fill-in of direct linear solvers, inexact search directions are allowed in an inexact interior-point (IIP) method. This thesis builds upon the IIP method proposed in [Curtis, Schenk, Wachter, SIAM Journal on Scientific Computing, 2010]. SMART tests cope with the lack of inertia information to control Hessian modification and also specify termination tests for the iterative linear solver. The original IIP method needs to solve two sparse large-scale linear systems in each optimization step. This is improved to only a single linear system solution in most optimization steps. Within this improved IIP framework, two iterative linear solvers are evaluated: A general purpose algebraic multilevel incomplete L D L^T preconditioned SQMR method is applied to PDE-constrained optimization problems for optimal server room cooling in three space dimensions and to compute an ambient temperature for optimal cooling. The results show robustness and efficiency of the IIP method when compared with the exact IP method. These advantages are even more evident for a reduced-space preconditioned (RSP) GMRES solver which takes advantage of the linear system's structure. This RSP-IIP method is studied on the basis of distributed and boundary control problems originating from superconductivity and from two-dimensional and three-dimensional parameter estimation problems in groundwater modeling. The numerical results exhibit the improved efficiency especially for multiple PDE constraints. An inverse medium problem for the Helmholtz equation with pointwise box constraints is solved by IP methods. The ill-posedness of the problem is explored numerically and different regularization strategies are compared. The impact of box constraints and the importance of Hessian modification on the optimization algorithm is demonstrated. A real world seismic imaging problem is solved successfully by the RSP-IIP method.

Guaranteed satisfaction of inequality state constraints in PDE-constrained optimization

Previous chapter Next chapter Software, Environments, and Tools Parallel Processing for Scientific Computing16. Parallel Algorithms for PDE-Constrained OptimizationVolkan Akçelik, George Biros, Omar Ghattas, Judith Hill, David Keyes, and Bart van Bloemen WaandersVolkan Akçelik, George Biros, Omar Ghattas, Judith Hill, David Keyes, and Bart van Bloemen Waanderspp.291 - 322Chapter DOI:https://doi.org/10.1137/1.9780898718133.ch16PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutExcerpt PDE-constrained optimization refers to the optimization of systems governed by PDEs. The simulation problem is to solve the PDEs for the state variables (e.g., displacement, velocity, temperature, electric field, magnetic field, species concentration), given appropriate data (e.g., geometry, coefficients, boundary conditions, initial conditions, source functions). The optimization problem seeks to determine some of these data—the decision variables—given performance goals in the form of an objective function and possibly inequality or equality constraints on the behavior of the system. Since the behavior of the system is modeled by the PDEs, they appear as (usually equality) constraints in the optimization problem. We will refer to these PDE constraints as the state equations. Let u represent the state variables, d the decision variables, J the objective function, c the residual of the state equations, and h the residual of the inequality constraints. We can then state the general form of a PDE-constrained optimization problem as min u,d J (u,d) subject to c (u,d)=0, h (u,d)≥0 . 16.1 The PDE-constrained optimization problem (16.1) can represent an optimal design, optimal control, or inverse problem, depending on the nature of the objective function and decision variables. The decision variables correspondingly represent design, control, or inversion variables. Previous chapter Next chapter RelatedDetails Published:2006ISBN:978-0-89871-619-1eISBN:978-0-89871-813-3 https://doi.org/10.1137/1.9780898718133Book Series Name:Software, Environments, and ToolsBook Code:SE20Book Pages:xxiv + 383Key words:Parallel processing, scientific computing, parallel algorithms, high-performance computing, computational science and engineering

A scalable matrix-free spectral element approach for unsteady PDE constrained optimization using PETSc/TAO

This book fills a gap between theory-oriented investigations in PDE-constrained optimization and the practical demands made by numerical solutions of PDE optimization problems. The authors discuss computational techniques representing recent developments that result from a combination of modern techniques for the numerical solution of PDEs and for sophisticated optimization schemes. Computational Optimization of Systems Governed by Partial Differential Equations offers readers a combined treatment of PDE-constrained optimization and uncertainties and an extensive discussion of multigrid optimization. It provides a bridge between continuous optimization and PDE modeling and focuses on the numerical solution of the corresponding problems. Audience: This book is intended for graduate students working in PDE-constrained optimization and students taking a seminar on numerical PDE-constrained optimization. It is also suitable as an introduction for researchers in scientific computing with PDEs who want to work in the field of optimization and for those in optimization who want to consider methodologies from the field of numerical PDEs. It will help researchers in the natural sciences and engineering to formulate and solve optimization problems.

PDE-constrained optimization problems find many applications in medical image analysis, for example, neuroimaging, cardiovascular imaging, and oncological imaging. We review related literature and give examples on the formulation, discretization, and numerical solution of PDE-constrained optimization problems for medical imaging. We discuss three examples. The first one is image registration. The second one is data assimilation for brain tumor patients, and the third one data assimilation in cardiovascular imaging. The image registration problem is a classical task in medical image analysis and seeks to find pointwise correspondences between two or more images. The data assimilation problems use a PDE-constrained formulation to link a biophysical model to patient-specific data obtained from medical images. The associated optimality systems turn out to be sets of nonlinear, multicomponent PDEs that are challenging to solve in an efficient way. The ultimate goal of our work is the design of inversion methods that integrate complementary data, and rigorously follow mathematical and physical principles, in an attempt to support clinical decision making. This requires reliable, high-fidelity algorithms with a short time-to-solution. This task is complicated by model and data uncertainties, and by the fact that PDE-constrained optimization problems are ill-posed in nature, and in general yield high-dimensional, severely ill-conditioned systems after discretization. These features make regularization, effective preconditioners, and iterative solvers that, in many cases, have to be implemented on distributed-memory architectures to be practical, a prerequisite. We showcase state-of-the-art techniques in scientific computing to tackle these challenges.

This article summarizes several recent results on goal-oriented error estimation and mesh adaptation for the solution of elliptic PDE-constrained optimization problems with additional inequality constraints. The first part is devoted to the control constrained case. Then some emphasis is given to pointwise inequality constraints on the state variable and on its gradient. In the last part of the article regularization techniques for state constraints are considered and the question is addressed, how the regularization parameter can adaptively be linked to the discretization error.KeywordsGoal oriented error estimationPDE-constrained optimizationcontrol and state constraintsregularizationadaptivityadaptive finite elements

Optimal control problems with inequality path constraints (IPCs) are present in several engineering problems described by partial differential equations (PDE). We propose an algorithm to solve PDE-constrained dynamic optimization problems with guaranteed satisfaction of IPCs. The algorithm is based on a solution of a sequence of approximated dynamic optimization problems following the strategy of Fu et al. (Automatica, 2015). For the approximation, the path constraint is imposed only on a set of discrete points and is thus relaxed. At the same time, it is restricted by an adaptive back-off and by the error bound of the PDE solution. The approximation is iteratively improved: at iterations without violation, the restriction is reduced; at iterations with violation, the point of maximal violation is added to the discrete set. Under mild assumptions, we prove finite termination of the algorithm. We test the algorithm on an optimal grade change of a tubular reactor.

Optimal control problems with inequality path constraints (IPCs) are present in several engineering problems described by partial differential equations (PDE). We propose an algorithm to solve PDE-constrained dynamic optimization problems with guaranteed satisfaction of IPCs. The algorithm is based on a solution of a sequence of approximated dynamic optimization problems following the strategy of Fu et al. (Automatica, 2015). For the approximation, the path constraint is imposed only on a set of discrete points and is thus relaxed. At the same time, it is restricted by an adaptive back-off and by the error bound of the PDE solution. The approximation is iteratively improved: at iterations without violation, the restriction is reduced; at iterations with violation, the point of maximal violation is added to the discrete set. Under mild assumptions, we prove finite termination of the algorithm. We test the algorithm on an optimal grade change of a tubular reactor.

The solution of PDE-constrained optimal control problems is a computationally challenging task, and it involves the solution of structured algebraic linear systems whose blocks stem from the discretized first-order optimality conditions. In this paper we analyze the numerical solution of this large-scale system: we first perform a natural order reduction, and then we solve the reduced system iteratively by exploiting specifically designed preconditioning techniques. The analysis is accompanied by numerical experiments on two application problems.

We have developed an explicit inverse approach with a Hessian matrix for the least-squares (LS) implementation of prestack time migration (PSTM). A full Hessian matrix is divided into a series of computationally tractable small-sized matrices using a localized approach, thus significantly reducing the size of the inversion. The scheme is implemented by dividing the imaging volume into a series of subvolumes related to the blockwise Hessian matrices that govern the mapping relationship between the migrated result and corresponding reflectivity. The proposed blockwise LS-PSTM can be implemented in a target-oriented fashion. The localized approach that we use to modify the Hessian matrix can eliminate the boundary effects originating from the blockwise implementation. We derive the explicit formula of the offset-dependent Hessian matrix using the deconvolution imaging condition with an analytical Green’s function of PSTM. This avoids the challenging task of estimating the source wavelet. Moreover, migrated gathers can be generated with the proposed scheme. The smaller size of the blockwise Hessian matrix makes it possible to incorporate the total-variation regularization into the inversion, thus attenuating noises significantly. We revealed the proposed blockwise LS-PSTM with synthetic and field data sets. Higher quality common-reflection-point gathers of the field data are obtained.

Some first and second order algorithmic approaches for the solution of PDE‐constrained optimization problems are reviewed. An optimal control problem for the stationary Navier‐Stokes system with pointwise control constraints serves as an illustrative example. Some issues in treating inequality constraints for the state variable and alternative objective functions are also discussed (© 2010 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

In this paper we investigate the possibility of using a block triangular preconditioner for saddle point problems arising in PDE constrained optimization. In particular we focus on a conjugate gradient-type method introduced by Bramble and Pasciak which uses self adjointness of the preconditioned system in a non-standard inner product. We show when the Chebyshev semi-iteration is used as a preconditioner for the relevant matrix blocks involving the finite element mass matrix that the main drawback of the Bramble-Pasciak method – the appropriate scaling of the preconditioners – is easily overcome. We present an eigenvalue analysis for the block triangular preconditioners which gives convergence bounds in the non-standard inner product and illustrate their competitiveness on a number of computed examples.

Interior point methods provide an attractive class of approaches for solving linear, quadratic and nonlinear programming problems, due to their excellent efficiency and wide applicability. In this paper, we consider PDE-constrained optimization problems with bound constraints on the state and control variables, and their representation on the discrete level as quadratic programming problems. To tackle complex problems and achieve high accuracy in the solution, one is required to solve matrix systems of huge scale resulting from Newton iteration, and hence fast and robust methods for these systems are required. We present preconditioned iterative techniques for solving a number of these problems using Krylov subspace methods, considering in what circumstances one may predict rapid convergence of the solvers in theory, as well as the solutions observed from practical computations.

In this thesis we analyze and develop methods based on model order reduction (MOR) for the solution of optimization problems constrained by time-dependent partial differential equations (PDEs). The methods combine a direct solution approach with model reduction via proper orthogonal decomposition (POD) and the discrete empirical interpolation method (DEIM). The reduced-order models (ROMs) are used to approximate the high-dimensional dynamic systems originating from a spatial discretization of a PDE. However, when used in an optimization algorithm, conventional POD/DEIM ROMs often lack the ability to give adequate approximations of the gradient. We propose methods for a suitable enhancement of the ROMs for the optimization purpose which are based on the inclusion of derivative information in the POD and DEIM subspaces. We distinguish two types of error between quantities evaluated with the high-dimensional model and its ROM approximation in dependency on the optimization variable q: The reconstruction error which is evaluated with the same q0 which is used constructing the ROM and the prediction error which assesses approximations at q with a ROM constructed at q0 different to q. The novel reconstruction results we present include estimates for solutions of the adjoint equation and the sensitivity equations as well as for the gradient of the objective function. Based on the estimates we explain how the POD and DEIM bases should be extended with either adjoint or sensitivity information. The enhanced ROMs allow control of the reconstruction error for the objective and its gradient up to machine precision. Moreover, we propose a POD prediction estimate for the objective of the optimization problem in a neighborhood of q where the ROM is constructed. In case of sensitivity-extended POD and DEIM bases we give an analogous result for solutions of the states. The derivative-extended ROMs are then used to develop adaptive algorithms for the solution of optimal control and parameter estimation problems which results in great runtime improvements for the optimization while ensuring high approximation quality of the solution of the original problem. For the parameter estimation case a novel a posteriori error estimate is proposed which assesses the quality of suboptimal solutions obtained with the ROM. A further fundamental contribution is a discussion of discretize-then-optimize (DTO) vs. optimize-then-discretize (OTD) approaches in the context of MOR for optimization. We analyze advantages and disadvantages of both approaches and discuss to which extent our methods exhibit properties of either strategy. We also give examples of representative optimization problems in which standard POD/DEIM ROMs show an inacceptable behavior and can be successfully solved by derivative-extended ROMs. We have further implemented the developed methods emphasizing an efficient realization which is important for the investigation of the MOR potential. We showcase the practical performance of the proposed algorithms and the superiority of derivative-extended over conventional ROMs on two academic and one industry-relevant application which exhibit a variety of challenges for the model reduction approach in optimization.