PDE-constrained Optimization Research Articles

A Three-Block Inexact Heterogeneous Alternating Direction Method of Multipliers for Elliptic PDE-Constrained Optimization Problems with a Control Gradient Penalty Term

Optimization problems with PDE constraints are widely used in engineering and technical fields. In some practical applications, it is necessary to smooth the control variables and suppress their large fluctuations, especially at the boundary. Therefore, we propose an elliptic PDE-constrained optimization model with a control gradient penalty term. However, introducing this penalty term increases the complexity and difficulty of the problems. To solve the problems numerically, we adopt the strategy of “First discretize, then optimize”. First, the finite element method is employed to discretize the optimization problems. Then, a heterogeneous strategy is introduced to formulate the augmented Lagrangian function for the subproblems. Subsequently, we propose a three-block inexact heterogeneous alternating direction method of multipliers (three-block ihADMM). Theoretically, we provide a global convergence analysis of the three-block ihADMM algorithm and discuss the iteration complexity results. Numerical results are provided to demonstrate the efficiency of the proposed algorithm.

MultiShape: a spectral element method, with applications to Dynamic Density Functional Theory and PDE-constrained optimization

Abstract A new numerical framework is developed to solve general nonlinear and nonlocal PDEs on complicated two-dimensional domains. This enables the solution of a wide range of both steady and time-dependent problems on nonstandard geometries, as well as providing the ability to impose nonlinear and nonlocal boundary conditions (typical of those arising in the modelling of physical phenomena) in a flexible and automated way. This spectral element methodology, which we called MultiShape, is compatible with other state-of-the-art numerical methods, such as differential–algebraic equation solvers and optimization algorithms. MultiShape is an open-source Matlab library, in which the numerical implementation is designed to be user-friendly: the problem set-up and computations are done automatically through intuitive operator definitions and notation. Validation tests are presented, before we showcase the power and versatility of MultiShape with three motivating examples in Dynamic Density Functional Theory and PDE-constrained optimization.

Asymptotic properties of Monte Carlo methods in elliptic PDE-constrained optimization under uncertainty

Asymptotic properties of Monte Carlo methods in elliptic PDE-constrained optimization under uncertainty

A Multigrid Solver for PDE-Constrained Optimization with Uncertain Inputs

In this manuscript, we present a collective multigrid algorithm to solve efficiently the large saddle-point systems of equations that typically arise in PDE-constrained optimization under uncertainty, and develop a novel convergence analysis of collective smoothers and collective two-level methods. The multigrid algorithm is based on a collective smoother that at each iteration sweeps over the nodes of the computational mesh, and solves a reduced saddle-point system whose size is proportional to the number N of samples used to discretized the probability space. We show that this reduced system can be solved with optimal O(N) complexity. The multigrid method is tested both as a stationary method and as a preconditioner for GMRES on three problems: a linear-quadratic problem, possibly with a local or a boundary control, for which the multigrid method is used to solve directly the linear optimality system; a nonsmooth problem with box constraints and L1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^1$$\\end{document}-norm penalization on the control, in which the multigrid scheme is used as an inner solver within a semismooth Newton iteration; a risk-averse problem with the smoothed CVaR risk measure where the multigrid method is called within a preconditioned Newton iteration. In all cases, the multigrid algorithm exhibits excellent performances and robustness with respect to the parameters of interest.

Extended finite elements for 3D–1D coupled problems via a PDE-constrained optimization approach

In this work, we propose the application of the eXtended Finite Element Method (XFEM) in the context of the coupling between three-dimensional and one-dimensional elliptic problems. In particular, we consider the case in which the 3D–1D coupled problem arises from the geometrical model reduction of a fully three-dimensional problem, characterized by thin tubular inclusions embedded in a much wider domain. In the 3D–1D coupling framework, the use of non conforming meshes is widely adopted. However, since the inclusions typically behave as singular sinks or sources for the 3D problem, mesh adaptation near the embedded 1D domains may be necessary to enhance solution accuracy and recover optimal convergence rates. An alternative to mesh adaptation is represented by the XFEM, which we here propose to enhance the approximation capabilities of an optimization-based 3D–1D coupling approach. An effective quadrature strategy is devised to integrate the enrichment functions and numerical tests on single and multiple segments are proposed to demonstrate the effectiveness of the approach.

A five field formulation for flow simulations in porous media with fractures and barriers via an optimization based domain decomposition method

The present work deals with the numerical resolution of coupled 3D–2D problems arising from the simulation of fluid flow in fractured porous media modeled via the Discrete Fracture and Matrix (DFM) model. According to the DFM model, fractures are represented as planar interfaces immersed in a 3D porous matrix and can behave as preferential flow paths, in the case of conductive fractures, or can actually be a barrier for the flow, when, instead, the permeability in the normal-to-fracture direction is small compared to the permeability of the matrix. Consequently, the pressure solution in a DFM can be discontinuous across a barrier, as a result of the geometrical dimensional reduction operated on the fracture. The present work is aimed at developing a numerical scheme suitable for the simulation of the flow in a DFM with fractures and barriers, using a mesh for the 3D matrix non conforming to the fractures and that is ready for domain decomposition. This is achieved starting from a PDE-constrained optimization method, currently available in literature only for conductive fractures in a DFM. First, a novel formulation of the optimization problem is defined to account for non permeable fractures. These are described by a filtration-like coupling at the interface with the surrounding porous matrix. Also the extended finite element method with discontinuous enrichment functions is used to reproduce the pressure solution in the matrix around a barrier. The method is presented here in its simplest form, for clarity of exposition, i.e. considering the case of a single fracture in a 3D domain, also providing a proof of the well posedness of the resulting discrete problem. Four validation examples are proposed to show the viability and the effectiveness of the method.

PDE-Constrained Scale Optimization Selection for Feature Detection in Remote Sensing Image Matching

Feature detection and matching is the key technique for remote sensing image processing and related applications. In this paper, a PDE-constrained optimization model is proposed to determine the scale levels advantageous for feature detection. A variance estimation technique is introduced to treat the observation optical images polluted by additive zero-mean Gaussian noise and determine the parameter of a nonlinear scale space governed by the partial differential equation. Additive Operator Splitting is applied to efficiently solve the PDE constraint, and an iterative algorithm is proposed to approximate the optimal subset of the original scale level set. The selected levels are distributed more uniformly in the total variation sense and helpful for generating more accurate and robust feature points. The experimental results show that the proposed method can achieve about a 30% improvement in the number of correct matches with only a small increase in time cost.

One-Shot Learning of Surrogates in PDE-Constrained Optimization under Uncertainty

One-Shot Learning of Surrogates in PDE-Constrained Optimization under Uncertainty

A nonsmooth primal-dual method with interwoven PDE constraint solver

We introduce an efficient first-order primal-dual method for the solution of nonsmooth PDE-constrained optimization problems. We achieve this efficiency through not solving the PDE or its linearisation on each iteration of the optimization method. Instead, we run the method interwoven with a simple conventional linear system solver (Jacobi, Gauss–Seidel, conjugate gradients), always taking only one step of the linear system solver for each step of the optimization method. The control parameter is updated on each iteration as determined by the optimization method. We prove linear convergence under a second-order growth condition, and numerically demonstrate the performance on a variety of PDEs related to inverse problems involving boundary measurements.

PDE-constrained Optimization for Electroencephalographic Source Reconstruction

PDE-constrained Optimization for Electroencephalographic Source Reconstruction

AONN-2: An adjoint-oriented neural network method for PDE-constrained shape optimization

PDE-constrained shape optimization has been playing an important role in a large variety of engineering applications. Traditional mesh-dependent shape optimization methods often encounter challenges due to mesh deformation. To address this issue, we present a new adjoint-oriented neural network method, AONN-2, for PDE-constrained shape optimization problems. This method extends the capabilities of the original AONN method [1], which is developed for efficiently solving parametric optimal control problems. AONN-2 inherits from AONN the direct-adjoint looping (DAL) framework for computing the extremum of an objective functional and the involved neural network methods for solving complicated PDEs. Furthermore, AONN-2 expands the application scope to shape optimization by taking advantage of the shape derivatives to optimize the shape represented by discrete boundary points. AONN-2 is a fully mesh-free shape optimization approach, naturally sidestepping issues related to mesh deformation, with no needs for maintaining mesh quality and additional mesh corrections. A series of experimental results are presented, highlighting the flexibility, robustness, and accuracy of AONN-2.

Efficient mini-batch stochastic gradient descent with Centroidal Voronoi Tessellation for PDE-constrained optimization under uncertainty

The study of optimal control problems under uncertainty plays an important role in scientific numerical simulations. This class of optimization problems is frequently utilized in engineering, biology and finance. Although stochastic gradient descent is a well-known stochastic optimization technique for solving the approximate optimal control problem, it often exhibits a slow convergence rate due to the inherent variance in gradient approximation. To address this issue, we propose a more efficient stochastic optimization algorithm based on Centroidal Voronoi Tessellation sampling. This strategy reduces the error in gradient estimation compared to the conventional mini-batch stochastic gradient descent method. Our approach involves dividing the whole snapshot set into several Voronoi cells with low variance and extracting samples with good uniformity from each region to construct an unbiased estimation of the full gradient. Our method can economically and stably approximate numerical optimal control functions even with a fixed step size. Numerical results demonstrate that the proposed method is a reliable algorithm with great potential for applications in the field of optimization.

Iterative PDE-Constrained Optimization for Seismic Full-Waveform Inversion

Iterative PDE-Constrained Optimization for Seismic Full-Waveform Inversion

Differentiable solver for time-dependent deformation problems with contact

We introduce a general differentiable solver for time-dependent deformation problems with contact and friction. Our approach uses a finite element discretization with a high-order time integrator coupled with the recently proposed incremental potential contact method for handling contact and friction forces to solve ODE- and PDE-constrained optimization problems on scenes with complex geometry. It supports static and dynamic problems and differentiation with respect to all physical parameters involved in the physical problem description, which include shape, material parameters, friction parameters, and initial conditions. Our analytically derived adjoint formulation is efficient, with a small overhead (typically less than 10% for nonlinear problems) over the forward simulation, and shares many similarities with the forward problem, allowing the reuse of large parts of existing forward simulator code. We implement our approach on top of the open-source PolyFEM library and demonstrate the applicability of our solver to shape design, initial condition optimization, and material estimation on both simulated results and physical validations.

The Limit of Incomplete Information in Inverse Analysis for Hidden Corrosion Inference in Reinforced Concrete

One recurring problem in reinforced concrete structures is hidden corrosion. Detection of hidden corrosion is difficult due to incomplete information that can be measured. To overcome this problem, inverse analysis can be employed. Inverse analysis uses incomplete information from field measurement, and employs optimization to infer further information. Previous research has demonstrated the inference capability of inverse analysis for hidden corrosion detection in reinforced concrete. However, question remains on the limit of incompleteness of measurement data. The aim of this study is to simulate and examine the limit of incomplete information that can be allowed for an effective inverse analysis. The inverse analysis of hidden corrosion in reinforced concrete is formulated as a PDE-constrained optimization. The objective function of the optimization is the distance function between measurement data and FEM simulation. The search space is the PDE boundary conditions configuration. A FEM simulation is used to simulate measurement data by half-cell potential mapping on a concrete block of 2 by 1 meters. To simulate incomplete information, the number of measurement points is varied from 2 to 20 points on the surface of the concrete block. Results show that 20 measurement points are needed to successfully infer hidden corrosion profile. However, when the number of measurement points are below this, the effectiveness of inverse analysis varies and does not seem to correlate to the amount of information (i.e., number of measurement points). Further study is needed if this is due to a fluke or choice of measurement point’s location.

An iDCA with Sieving Strategy for PDE-Constrained Optimization Problems with $$L^{1-2}$$-Control Cost

An iDCA with Sieving Strategy for PDE-Constrained Optimization Problems with $$L^{1-2}$$-Control Cost

Inverse design of a grating metasurface for enhancing spontaneous emission through hyperbolic metamaterials

This work is concerned with inverse design of the grating metasurface over hyperbolic metamaterials (HMMs) in order to enhance spontaneous emission (SE). We formulate the design problem as a PDE-constrained optimization problem and employ the gradient descent method to solve the underlying optimization problem. The adjoint-state method is applied to compute the gradient of the objective function efficiently. Computational results show that the SE efficiency of the optical structure with the optimized metasurface increases by 600% in the near field compared to the bare HMM layer. In particular, an optimized double-slot metasurface obtained by this design method enhances the SE intensity by a factor of over 100 in the observation region.

A Discretize-then-Optimize Approach to PDE-Constrained Shape Optimization

We consider discretized two-dimensional PDE-constrained shape optimization problems, in which shapes are represented by triangular meshes. Given the connectivity, the space of admissible vertex positions was recently identified to be a smooth manifold, termed the manifold of planar triangular meshes. The latter can be endowed with a complete Riemannian metric, which allows large mesh deformations without jeopardizing mesh quality; see R. Herzog and E. Loayza-Romero, Math. Comput. 92 (2022) 1-50. Nonetheless, the discrete shape optimization problem of finding optimal vertex positions does not, in general, possess a globally optimal solution. To overcome this ill-possedness, we propose to add a mesh quality penalization term to the objective function. This allows us to simultaneously render the shape optimization problem solvable, and keep track of the mesh quality. We prove the existence of a globally optimal solution for the penalized problem and establish first-order necessary optimality conditions independently of the chosen Riemannian metric. Because of the independence of the existence results of the choice of the Riemannian metric, we can numerically study the impact of different Riemannian metrics on the steepest descent method. We compare the Euclidean, elasticity, and a novel complete metric, combined with Euclidean and geodesic retractions to perform the mesh deformation.

On fast convergence rates for generalized conditional gradient methods with backtracking stepsize

<p style='text-indent:20px;'>A generalized conditional gradient method for minimizing the sum of two convex functions, one of them differentiable, is presented. This iterative method relies on two main ingredients: First, the minimization of a partially linearized objective functional to compute a descent direction and, second, a stepsize choice based on an Armijo-like condition to ensure sufficient descent in every iteration. We provide several convergence results. Under mild assumptions, the method generates sequences of iterates which converge, on subsequences, towards minimizers. Moreover, a sublinear rate of convergence for the objective functional values is derived. Second, we show that the method enjoys improved rates of convergence if the partially linearized problem fulfills certain growth estimates. Most notably these results do not require strong convexity of the objective functional. Numerical tests on a variety of challenging PDE-constrained optimization problems confirm the practical efficiency of the proposed algorithm.</p>

Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks.

In recent years, advances in computing hardware and computational methods have prompted a wealth of activities for solving inverse problems in physics. These problems are often described by systems of partial differential equations (PDEs). The advent of machine learning has reinvigorated the interest in solving inverse problems using neural networks (NNs). In these efforts, the solution of the PDEs is expressed as NNs trained through the minimization of a loss function involving the PDE. Here, we show how to accelerate this approach by five orders of magnitude by deploying, instead of NNs, conventional PDE approximations. The framework of optimizing a discrete loss (ODIL) minimizes a cost function for discrete approximations of the PDEs using gradient-based and Newton's methods. The framework relies on grid-based discretizations of PDEs and inherits their accuracy, convergence, and conservation properties. The implementation of the method is facilitated by adopting machine-learning tools for automatic differentiation. We also propose a multigrid technique to accelerate the convergence of gradient-based optimizers. We present applications to PDE-constrained optimization, optical flow, system identification, and data assimilation. We compare ODIL with the popular method of physics-informed neural networks and show that it outperforms it by several orders of magnitude in computational speed while having better accuracy and convergence rates. We evaluate ODIL on inverse problems involving linear and nonlinear PDEs including the Navier-Stokes equations for flow reconstruction problems. ODIL bridges numerical methods and machine learning and presents a powerful tool for solving challenging, inverse problems across scientific domains.