Line Search Strategy Research Articles

Optimal control of PDEs using physics-informed neural networks

Physics-informed neural networks (PINNs) have recently become a popular method for solving forward and inverse problems governed by partial differential equations (PDEs). By incorporating the residual of the PDE into the loss function of a neural network-based surrogate model for the unknown state, PINNs can seamlessly blend measurement data with physical constraints. Here, we extend this framework to PDE-constrained optimal control problems, for which the governing PDE is fully known and the goal is to find a control variable that minimizes a desired cost objective. We provide a set of guidelines for obtaining a good optimal control solution; first by selecting an appropriate PINN architecture and training parameters based on a forward problem, second by choosing the best value for a critical scalar weight in the loss function using a simple but effective two-step line search strategy. We then validate the performance of the PINN framework by comparing it to adjoint-based nonlinear optimal control, which performs gradient descent on the discretized control variable while satisfying the discretized PDE. This comparison is carried out on several distributed control examples based on the Laplace, Burgers, Kuramoto-Sivashinsky, and Navier-Stokes equations. Finally, we discuss the advantages and caveats of using the PINN and adjoint-based approaches for solving optimal control problems constrained by nonlinear PDEs.

Integrating Conjugate Gradients Into Evolutionary Algorithms for Large-Scale Continuous Multi-Objective Optimization

Large-scale multi-objective optimization problems (LSMOPs) pose challenges to existing optimizers since a set of well-converged and diverse solutions should be found in huge search spaces. While evolutionary algorithms are good at solving small-scale multi-objective optimization problems, they are criticized for low efficiency in converging to the optimums of LSMOPs. By contrast, mathematical programming methods offer fast convergence speed on large-scale single-objective optimization problems, but they have difficulties in finding diverse solutions for LSMOPs. Currently, how to integrate evolutionary algorithms with mathematical programming methods to solve LSMOPs remains unexplored. In this paper, a hybrid algorithm is tailored for LSMOPs by coupling differential evolution and a conjugate gradient method. On the one hand, conjugate gradients and differential evolution are used to update different decision variables of a set of solutions, where the former drives the solutions to quickly converge towards the Pareto front and the latter promotes the diversity of the solutions to cover the whole Pareto front. On the other hand, objective decomposition strategy of evolutionary multi-objective optimization is used to differentiate the conjugate gradients of solutions, and the line search strategy of mathematical programming is used to ensure the higher quality of each offspring than its parent. In comparison with state-of-the-art evolutionary algorithms, mathematical programming methods, and hybrid algorithms, the proposed algorithm exhibits better convergence and diversity performance on a variety of benchmark and real-world LSMOPs.

A modified JFNK with line search method for solving k-eigenvalue neutronics problems with thermal-hydraulics feedback

The k-eigenvalue neutronics/thermal-hydraulics coupling calculation is a key issue for reactor design and analysis. Jacobian-free Newton-Krylov (JFNK) method, featured with super-linear convergence rate and high efficiency, has been attracting more and more attention to solve the multi-physics coupling problem. However, it may converge to the high-order eigenmode because of the multiple solutions nature of the k-eigenvalue form of multi-physics coupling issue. Based on our previous work, a modified JFNK with a line search method is proposed in this work, which can find the fundamental eigenmode together with thermal-hydraulics feedback in a wide range of initial values. In detail, the existing modified JFNK method is combined with the line search strategy, so that the intermediate iterative solution can avoid a sudden divergence and be adjusted into a convergence basin smoothly. Two simplified 2-D homogeneous reactor models, a PWR model, and an HTR model, are utilized to evaluate the performance of the newly proposed JFNK method. The results show that the performance of this proposed JFNK is more robust than the existing JFNK-based methods.

A new nonmonotone line search method for nonsmooth nonconvex optimization

In this paper, we develop a nonmonotone line search strategy for minimization of the locally Lipschitz functions. First, the descent direction (DD) is defined based on ∂ ϵ f ( ⋅ ) where ϵ > 0 . Next, we introduce a minimization algorithm to find a step length along the DD satisfying the nonsmooth nonmonotone Armijo condition. Choosing an adequate step length is the main purpose of the classic nonmonotone line search methods for a given DD, while in this paper both a search direction and step length are simultaneously computed. The global convergence of the minimization algorithm is proved by some assumptions on the DD. Finally, the proposed algorithm is implemented in the MATLAB environment and compared with another existing nonsmooth algorithm on some nonconvex nonsmooth optimization test problems. The efficiency of the proposed algorithm is shown by numerical results in solving some small-scale and large-scale nonsmooth optimization test problems.

A Scaled Dai–Yuan Projection-Based Conjugate Gradient Method for Solving Monotone Equations with Applications

In this paper, we propose two scaled Dai–Yuan (DY) directions for solving constrained monotone nonlinear systems. The proposed directions satisfy the sufficient descent condition independent of the line search strategy. We also reasonably proposed two different relations for computing the scaling parameter at every iteration. The first relation is proposed by approaching the quasi-Newton direction, and the second one is by taking the advantage of the popular Barzilai–Borwein strategy. Moreover, we propose a robust projection-based algorithm for solving constrained monotone nonlinear equations with applications in signal restoration problems and reconstructing the blurred images. The global convergence of this algorithm is also provided, using some mild assumptions. Finally, a comprehensive numerical comparison with the relevant algorithms shows that the proposed algorithm is efficient.

Robust and efficient fixed-point algorithm for the inverse elastostatic problem to identify myocardial passive material parameters and the unloaded reference configuration

Image-based computational models of the heart represent a powerful tool to shed new light on the mechanisms underlying physiological and pathological conditions in cardiac function and to improve diagnosis and therapy planning. However, in order to enable the clinical translation of such models, it is crucial to develop personalized models that are able to reproduce the physiological reality of a given patient. There have been numerous contributions in experimental and computational biomechanics to characterize the passive behavior of the myocardium. However, most of these studies suffer from severe limitations and are not applicable to high-resolution geometries. In this work, we present a novel methodology to perform an automated identification of in vivo properties of passive cardiac biomechanics. The highly-efficient algorithm fits material parameters against the shape of a patient-specific approximation of the end-diastolic pressure-volume relation (EDPVR). Simultaneously, an unloaded reference configuration is generated, where a novel line search strategy to improve convergence and robustness is implemented. Only clinical image data or previously generated meshes at one time point during diastole and one measured data point of the EDPVR are required as an input. The proposed method can be straightforwardly coupled to existing finite element (FE) software packages and is applicable to different constitutive laws and FE formulations. Sensitivity analysis demonstrates that the algorithm is robust with respect to initial input parameters.

Numerical solution to 3D bilinear Fokker-Planck control problem

A characterization and numerical scheme to control problem governed by three-dimensional (3D) time-dependent Fokker-Planck (FP) equation is presented. We formulate a control formulation that controls the drift of the stochastic (FP) process. In this way, the probability density function (PDF) attains a specific configuration. Moreover, a FP control strategy for collective motion is investigated and first-order optimality conditions are presented. On staggered grids, the Chang-Cooper (CC) discretization scheme that ensures the positivity, second-order accuracy, and conservativeness to the FP equation is employed, to the discretized state (resp. adjoint) system. Furthermore, a line search strategy is applied to update the control variable. Results of numerical experiments show the efficiency of the proposed numerical scheme to stochastic (FP) control problems.

A truncated Newton algorithm for nonconvex sparse recovery

Some second-order sufficient conditions are established for problems with ℓ1, SCAD and MCP penalties respectively and then an algorithm with active set strategy is proposed for solving ℓ1, SCAD and MCP penalties based on an approximation of the second order sufficient conditions. The active sets are estimated by an identification technique to identify accurately zero components in a neighborhood of a critical point of ℓ1, SCAD and MCP penalties. In the algorithm, a truncated Newton direction is used to update the free variables, while dk=−xk is used to update the active variables. To accelerate the convergence, a nonmonotone line search strategy is used to guarantee global convergence. Numerical results illustrate that the proposed algorithm is competitive with several well-known methods.

A three-term CGPM-based algorithm without Lipschitz continuity for constrained nonlinear monotone equations with applications

In this work, a new three-term conjugate gradient projection method (CGPM) with a new adaptive line search strategy is proposed for solving the large-scale nonlinear monotone equations with convex constraints. The proposed three-term CGPM-based algorithm has the following characteristics: (i) its search direction satisfies the sufficient descent and trust region properties which is independent of the line search; (ii) the nonlinear equations only satisfy continuous and monotone properties; (iii) its global convergence is analyzed and obtained without the Lipschitz continuity. Some preliminary numerical experiment results are reported and their corresponding performance profiles are illustrated, which show that the proposed algorithm is effective. Finally, the proposed algorithm is extended to solve sparse signal restoration problems.

Nonmonotone spectral gradient method based on memoryless symmetric rank-one update for large-scale unconstrained optimization

<p style='text-indent:20px;'>This paper proposes a nonmonotone spectral gradient method for solving large-scale unconstrained optimization problems. The spectral parameter is derived from the eigenvalues of an optimally sized memoryless symmetric rank-one matrix obtained under the measure defined as a ratio of the determinant of updating matrix over its largest eigenvalue. Coupled with a nonmonotone line search strategy where backtracking-type line search is applied selectively, the spectral parameter acts as a stepsize during iterations when no line search is performed and as a milder form of quasi-Newton update when backtracking line search is employed. Convergence properties of the proposed method are established for uniformly convex functions. Extensive numerical experiments are conducted and the results indicate that our proposed spectral gradient method outperforms some standard conjugate-gradient methods.</p>

Fast inertial relaxation engine in the CRYSTAL code

In the framework of ab initio simulations, the search for energy minimum atomic structures is the first step to perform in studying the properties of a system. One of the most used and efficient optimization algorithms is a quasi-Newton line-search scheme based on the Broyden–Fletcher–Goldfarb–Shanno (Bfgs) Hessian updating formula. However, recent studies [Bitzek et al., Phys. Rev. Lett. 97, 170201 (2006) and Guénolé et al., Comput. Mater. Sci. 175, 109584 (2020)] suggested that minimization methods based on molecular dynamics concepts, such as the Fast Inertial Relaxation Engine (Fire) algorithm, often exhibit better performance and accuracy in finding local minima than line-search based schemes. In the present work, the implementation of Fire, in the framework of Crystal ab initio quantum mechanical simulation package [Dovesi et al., Wiley Interdiscip. Rev.: Comput. Mol. Sci. 8, e1360 (2018)], has been described. Its efficiency and performance in comparison with Bfgs quasi-Newton scheme have been assessed using Hartree–Fock and density functional theory with Perdew–Burke–Ernzerhof and hybrid functionals to model the potential energy surface. Fire shows good convergence behavior for all the considered systems, well reproducing the minimum energy structures obtained by the Bfgs approach. As regards the computational cost, Fire requires more iterations to converge with respect to Bfgs, but each Fire iteration is faster than the Bfgs one. The overall efficiency of Fire improves as the size of the system increased so that this minimization method seems to be very promising for systems without symmetry (space group P1) with a large number of atoms.

Solving Nonlinear Systems of Equations Via Spectral Residual Methods: Stepsize Selection and Applications

Spectral residual methods are derivative-free and low-cost per iteration procedures for solving nonlinear systems of equations. They are generally coupled with a nonmonotone linesearch strategy and compare well with Newton-based methods for large nonlinear systems and sequences of nonlinear systems. The residual vector is used as the search direction and choosing the steplength has a crucial impact on the performance. In this work we address both theoretically and experimentally the steplength selection and provide results on a real application such as a rolling contact problem.

Global convergence of a descent PRP type conjugate gradient method for nonconvex optimization

Nonlinear conjugate gradient methods are often used to solve large-scale unconstrained optimization problems owing to their lower memory requirement and less computation cost. In this paper, we propose a new Polak-Ribiére-Polyak type conjugate gradient method, which satisfies the sufficient descent condition independent of any line search. A remarkable property about this method is its strong global convergence with the standard Wolfe line search conditions as well as the standard Armijo line search strategy without convexity assumption of the objective function. The numerical results demonstrating the efficiency of the proposed method are reported.

A comprehensive implicit substepping integration scheme for multisurface plasticity

AbstractA complex elastoplastic model requires a robust integration procedure of the evolution equations. The performance of the finite element solution is directly affected by the convergence characteristics of the state‐update procedure. Thereby, this study proposes a comprehensive numerical integration scheme to deal with generic multisurface plasticity models. This algorithm is based on the backward Euler method aiming at accuracy and stability, and on the Newton–Raphson method to solve the unconstrained optimization problem. In this scenario, a line search strategy is adopted to improve the convergence characteristics of the algorithm. The golden section method, an exact line search, is considered. Also, a substepping scheme is implemented to provide additional robustness to the state‐update procedure. Therefore, this work contributes to computational plasticity presenting an adaptive substep size scheme and a consistent tangent modulus according to the substepping technique. Finally, some numerical problems are evaluated using the proposed algorithm. Single‐surface and novel multisurface plasticity models are employed in these analyses. The results testify how the line search and substepping strategies can improve the robustness of the nonlinear analysis.

Newton-type methods near critical solutions of piecewise smooth nonlinear equations

It is well-recognized that in the presence of singular (and in particular nonisolated) solutions of unconstrained or constrained smooth nonlinear equations, the existence of critical solutions has a crucial impact on the behavior of various Newton-type methods. On the one hand, it has been demonstrated that such solutions turn out to be attractors for sequences generated by these methods, for wide domains of starting points, and with a linear convergence rate estimate. On the other hand, the pattern of convergence to such solutions is quite special, and allows for a sharp characterization which serves, in particular, as a basis for some known acceleration techniques, and for the proof of an asymptotic acceptance of the unit stepsize. The latter is an essential property for the success of these techniques when combined with a linesearch strategy for globalization of convergence. This paper aims at extensions of these results to piecewise smooth equations, with applications to corresponding reformulations of nonlinear complementarity problems.

A natural vector/matrix notation applied in an efficient and robust return-mapping algorithm for advanced yield functions

A fast and robust stress-integration algorithm is the key to full exploitation of advanced anisotropic yield functions in computational mechanics. Poor global convergence of a direct application of the Newton-Raphson scheme has been rectified by applying line search strategies during the Newton iterations. In this work the line-search approach is further improved by a better first guess. The new algorithm is implemented into a user-defined material subroutine (UMAT) in a finite-element (FE) software and tested. The implementation is made easier and more efficient by a new advantageous vector/matrix notation for symmetric second- and fourth-order tensors, which is the second result of this work. Benefits of this notation are discussed with respect to formulation of continuum-plasticity models as well as their implementations. FE simulations were run to demonstrate the performance of the new implementation, which is available as open-source software via GitLab repository (see Appendix). The new return-mapping algorithm implementation runs equally fast and robust as the simple von Mises and Hill standard implementations in the Abaqus/Standard software. This enables full exploitation of advanced yield functions as the new standard in industrial FE applications.

Alternative structured spectral gradient algorithms for solving nonlinear least-squares problems

The study of efficient iterative algorithms for addressing nonlinear least-squares (NLS) problems is of great importance. The NLS problems, which belong to a special class of unconstrained optimization problems, are of particular interest because of the special structure of their gradients and Hessians. In this paper, based on the spectral parameters of Barzillai and Borwein (1998), we propose three structured spectral gradient algorithms for solving NLS problems. Each spectral parameter in the respective algorithms incorporates the structured gradient and the information gained from the structured Hessian approximation. Moreover, we develop a safeguarding technique for the first two structured spectral parameters to avoid negative curvature directions. Moreso, using a nonmonotone line-search strategy, we show that the proposed algorithms are globally convergent under some standard conditions. The comparative computational results on some standard test problems show that the proposed algorithms are efficient.

Strong convergence inertial projection algorithm with self-adaptive step size rule for pseudomonotone variational inequalities in Hilbert spaces

Abstract In this paper, we introduce a new algorithm for solving pseudomonotone variational inequalities with a Lipschitz-type condition in a real Hilbert space. The algorithm is constructed around two algorithms: the subgradient extragradient algorithm and the inertial algorithm. The proposed algorithm uses a new step size rule based on local operator information rather than its Lipschitz constant or any other line search scheme and functions without any knowledge of the Lipschitz constant of an operator. The strong convergence of the algorithm is provided. To determine the computational performance of our algorithm, some numerical results are presented.

Multi-Material Decomposition for Single Energy CT Using Material Sparsity Constraint.

Multi-material decomposition (MMD) decomposes CT images into basis material images, and is a promising technique in clinical diagnostic CT to identify material compositions within the human body. MMD could be implemented on measurements obtained from spectral CT protocol, although spectral CT data acquisition is not readily available in most clinical environments. MMD methods using single energy CT (SECT), broadly applied in radiological departments of most hospitals, have been proposed in the literature while challenged by the inferior decomposition accuracy and the limited number of material bases due to the constrained material information in the SECT measurement. In this paper, we propose an image-domain SECT MMD method using material sparsity as an assistance under the condition that each voxel of the CT image contains at most two different elemental materials. L0 norm represents the material sparsity constraint (MSC) and is integrated into the decomposition objective function with a least-square data fidelity term, total variation term, and a sum-to-one constraint of material volume fractions. An accelerated primal-dual (APD) algorithm with line-search scheme is applied to solve the problem. The pixelwise direct inversion method with the two-material assumption (TMA) is applied to estimate the initials. We validate the proposed method on phantom and patient data. Compared with the TMA method, the proposed MSC method increases the volume fraction accuracy (VFA) from 92.0% to 98.5% in the phantom study. In the patient study, the calcification area can be clearly visualized in the virtual non-contrast image generated by the proposed method, and has a similar shape to that in the ground-truth contrast-free CT image. The high decomposition image quality from the proposed method substantially facilitates the SECT-based MMD clinical applications.

Multigrid Method for Optimal Control Problem Constrained by Stochastic Stokes Equations with Noise

Optimal control problems governed by stochastic partial differential equations have become an important field in applied mathematics. In this article, we investigate one such important optimization problem, that is, the stochastic Stokes control problem with forcing term perturbed by noise. A multigrid scheme with three-factor coarsening to solve the corresponding discretized control problem is presented. On staggered grids, a three-factor coarsening strategy helps in simplifying the inter-grid transfer operators and reduction in computation (CPU time). For smoothing, a distributive Gauss–Seidel scheme with a line search strategy is employed. To validate the proposed multigrid staggered grid framework, numerical results are presented with white noise at the end.