Augmented Lagrangian Approach Research Articles

Global Convergence in Learning Fully-Connected ReLU Networks Via Un-rectifying Based on the Augmented Lagrangian Approach

Global Convergence in Learning Fully-Connected ReLU Networks Via Un-rectifying Based on the Augmented Lagrangian Approach

Augmented Lagrangian Acceleration of Global-in-Time Pressure Schur Complement Solvers for Incompressible Oseen Equations

This work is focused on an accelerated global-in-time solution strategy for the Oseen equations, which highly exploits the augmented Lagrangian methodology to improve the convergence behavior of the Schur complement iteration. The main idea of the solution strategy is to block the individual linear systems of equations at each time step into a single all-at-once saddle point problem. By elimination of all velocity unknowns, the resulting implicitly defined equation can then be solved using a global-in-time pressure Schur complement (PSC) iteration. To accelerate the convergence behavior of this iterative scheme, the augmented Lagrangian approach is exploited by modifying the momentum equation for all time steps in a strongly consistent manner. While the introduced discrete grad-div stabilization does not modify the solution of the discretized Oseen equations, the quality of customized PSC preconditioners drastically improves and, hence, guarantees a rapid convergence. This strategy comes at the cost that the involved auxiliary problem for the velocity field becomes ill conditioned so that standard iterative solution strategies are no longer efficient. Therefore, a highly specialized multigrid solver based on modified intergrid transfer operators and an additive block preconditioner is extended to solution of the all-at-once problem. The potential of the proposed overall solution strategy is discussed in several numerical studies as they occur in commonly used linearization techniques for the incompressible Navier–Stokes equations.

Robust contact-constrained topology optimization considering uncertainty at the contact support

In this paper, the general framework for contact-constrained topology optimization of Strömberg and Klarbring (2010) is extended to robust topology optimization. In doing so, a linear elastic design domain is considered and the augmented Lagrangian approach is used to model the unilateral contact. For topology optimization, the design space is parametrized with the SIMP-approach and the Sigmund’s filter is applied. Additionally, the robust framework considers uncertainties at the contact support such as deviations of the geometry of the contact surface and the friction coefficient. Both uncertainties are described by the first-order second-moment method which leads to minimal additional costs. In fact, only two additional linear equations must be solved to obtain the robust objective and its gradient with respect to the design variables. Having both the objective and the gradient, the design update is computed with the method of moving asymptotes. The robust framework is applied to 2D and 3D examples to prove its scalability for real-world applications.

QAL-BP: an augmented Lagrangian quantum approach for bin packing

The bin packing is a well-known NP-Hard problem in the domain of artificial intelligence, posing significant challenges in finding efficient solutions. Conversely, recent advancements in quantum technologies have shown promising potential for achieving substantial computational speedup, particularly in certain problem classes, such as combinatorial optimization. In this study, we introduce QAL-BP, a novel Quadratic Unconstrained Binary Optimization (QUBO) formulation designed specifically for bin packing and suitable for quantum computation. QAL-BP utilizes the Augmented Lagrangian method to incorporate the bin packing constraints into the objective function while also facilitating an analytical estimation of heuristic, but empirically robust, penalty multipliers. This approach leads to a more versatile and generalizable model that eliminates the need for empirically calculating instance-dependent Lagrangian coefficients, a requirement commonly encountered in alternative QUBO formulations for similar problems. To assess the effectiveness of our proposed approach, we conduct experiments on a set of bin packing instances using a real Quantum Annealing device. Additionally, we compare the results with those obtained from two different classical solvers, namely simulated annealing and Gurobi. The experimental findings not only confirm the correctness of the proposed formulation, but also demonstrate the potential of quantum computation in effectively solving the bin packing problem, particularly as more reliable quantum technology becomes available.

A multi-tasking novel variational model for image decolorization and denoising

Image decolorization has a wide range of applications in digital printing, photo rendering and single-channel image processing. During the process of image decolorization, it is significant to retain more contrast information and avoid increasing the noise level of the original image. As we know, noise was hardly considered in previous decolorization methods. In this paper, we propose a new multi-tasking variational model with two tasks: decolorizing and denoising. The first task is image decolorization, that is, to transform color images to grayscale images. The other task is to control the noise level, and even eliminate the noise. Inspired by TV-Stokes model, we introduce the unit normal vector containing the basic geometry information of the color image in the model. The existence and the uniqueness of the solution to the variational problems are proved. For the variational model, two numerical methods are adopted. At first, we use gradient descent method to obtain evolution equations and construct finite difference schemes to solve the corresponding equations of proposed variational model. Furthermore, an efficient augmented Lagrangian approach is applied to solve the multi-tasking variational model directly. For noisy images and clean images, we employ two output patterns of grayscale images respectively. Especially, our model is stable for the noisy images. Numerical results and comparisons show that the proposed model can produce highly competitive results in terms of decolorization quality of noisy images, edge preservation and adjustment of parameters.

Optimum-pursuing method for constrained optimization and reliability-based design optimization problems using Kriging model

This paper proposes a new active learning method named as optimum-pursuing method (OPM) from the viewpoint of optimization theory, which aims to provide an effective tool for solving constrained optimization and reliability-based design optimization (RBDO) problems with low computation cost. It uses the cheap Kriging metamodel to replace the expensive physical response. The novelty of the proposed OPM primarily lies in two aspects. First, the OPM utilizes the advantage of the optimization theory rather than sampling technology. By using the augmented Lagrangian approach, it comprehensively considers the objective, constraints, and their relations, thereby automatic identification of important region in the vicinity of the optimum. Second, the accordingly optimum-pursuing function consists of three parts: Kriging mean, Kriging standard deviation, and merit function. Also, the target reliability surface is further considered to enhance the local accuracy of the reliability analysis. The performance of OPM is tested for both deterministic optimization and problems, in which two mathematical and three real-world engineering examples are selected to showcase the feasibility and validity. The results demonstrate that OPM is promising for solving both deterministic optimization and RBDO problems by comparing with the well-known active learning methods.

An augmented Lagrangian approach for cardinality constrained minimization applied to variable selection problems

To solve convex constrained minimization problems, that also include a cardinality constraint, we propose an augmented Lagrangian scheme combined with alternating projection ideas. Optimization problems that involve a cardinality constraint are NP-hard mathematical programs and typically very hard to solve approximately. Our approach takes advantage of a recently developed and analyzed continuous formulation that relaxes the cardinality constraint. Based on that formulation, we solve a sequence of smooth convex constrained minimization problems, for which we use projected gradient-type methods. In our setting, the convex constraint region can be written as the intersection of a finite collection of convex sets that are easy and inexpensive to project. We apply our approach to a variety of over and under determined constrained linear least-squares problems, with both synthetic and real data that arise in variable selection, and demonstrate its effectiveness.

Augmented Lagrangian approach for multi-objective topology optimization of energy storage flywheels with local stress constraints

Augmented Lagrangian approach for multi-objective topology optimization of energy storage flywheels with local stress constraints

Homogeneous multigrid for HDG applied to the Stokes equation

Abstract We propose a multigrid method to solve the linear system of equations arising from a hybrid discontinuous Galerkin (in particular, a single face hybridizable, a hybrid Raviart–Thomas, or a hybrid Brezzi–Douglas–Marini) discretization of a Stokes problem. Our analysis is centered around the augmented Lagrangian approach and we prove uniform convergence in this setting. Beyond this, we establish relations, which resemble those in Cockburn & Gopalakrishnan (2008, Error analysis of variable degree mixed methods for elliptic problems via hybridization. Math. Comput., 74, 1653–1677) for elliptic problems, between the approximates that are obtained by the single-face hybridizable, hybrid Raviart–Thomas and hybrid Brezzi–Douglas–Marini methods. Numerical experiments underline our analytical findings.

Efficient nonlinear model predictive motion controller for autonomous vehicles from standstill to extreme conditions based on split integration method

Autonomous driving motion control based on nonlinear model predictive control (NMPC) faces integration stability issues when driving at low speeds. However, this issue has not received sufficient attention or been adequately addressed. Ensuring integration stability requires the adoption of an implicit method, which incurs a high computational burden, making this phenomenon crucial to the performance of NMPC. In this paper, a split integration method is proposed to utilize the gradient-based augmented Lagrangian approach (GRAMPC) to solve the integration stability problem and significantly improve the efficiency of NMPC. The proposed split integration method integrates the explicit second-order Runge–Kutta (RK) method and the second-order L-stable Rosenbrock (ROS2) method. To enhance integration stability for the stiff part of the state equation, the ROS2 method is used. For the non-stiff part of the system, the RK method is used to improve its efficiency. This approach results in an efficient nonlinear NMPC system that can be adapted to full range speeds. Simulations and real-world experiments demonstrate that the low-speed integration stability problem of NMPC is entirely resolved without compromising its efficiency.

Lagrangian methods for optimal control problems governed by multivalued quasi-hemivariational inequalities

The primary objective of this paper is to develop a general augmented Lagrangian approach for optimal control problems governed by multivalued quasi-hemivariational inequalities. By imposing general coercivity and monotonicity-type conditions, we establish multiple existence results for the optimal control problems. Furthermore, we establish a comprehensive characterization of the zero duality gap property for the primal-dual problems, utilizing the concept of lower semicontinuity of new perturbation functions. Our results represent a significant advancement over recent literature on the subject.

Strength design of porous materials using B-spline based level set method

The inverse homogenization method can tailor some mechanical and physical effective properties by laying out materials in a periodic representative volume element. However, studies on strength design are yet to be developed because of the difficulties in numerically retrieving its value. Unlike traditional asymptotic homogenization, the fast Fourier transform-based homogenization method based on the augmented Lagrangian approach uses a Green operator in the frequency domain to replace time-consuming finite element analysis and inherently meet the periodic boundary conditions. Thus, it is developed in this work to retrieve material strength in terms of the von Mises yield criterion. The zero-level contour of a linear combination of cubic B-spline basis functions with repeated knots is used to represent the microstructure profile in the design of material strength. The effective strength or its squared difference with a prescribed target is minimized within the framework of the reaction diffusion-based B-spline level set method. The filtered, non-consistent discrete Green operator is adopted to avoid slow convergence of porous material. Examples of porous material demonstrate that the proposed method guarantees surface smoothness, optimization flexibility, and structural periodicity in strength design.

Squeeze cementing of micro-annuli: A visco-plastic invasion flow

Squeeze cementing is a process used to repair leaking oil and gas wells, in which a cement slurry is driven under pressure to fill an uneven leakage channel. This results in a Hele-Shaw type flow problem involving a yield stress fluid. We solve the flow problem using an augmented Lagrangian approach and advect forward the fluid concentrations until the flow stops. A planar invasion and a radial (perforation hole) invasion flow are studied. The characteristics of the flow penetration are linked to the channel thickness profile. The distribution of streamlines, flowing and non-flowing zones, evolves during the invasion flow. An interesting aspect of the results is the extreme variability in penetration metrics computed. These depend not only on the stochastic nature of the microannulus thickness, which has significant natural variation in both azimuthal and axial directions, but also on the “luck” of where the perforation hole is, relative to the larger/smaller microannulus gaps. This may explain the unreliability of the process.

An augmented Lagrangian approach with general constraints to solve nonlinear models of the large-scale reliable inventory systems

An augmented Lagrangian approach with general constraints to solve nonlinear models of the large-scale reliable inventory systems

A Resilient Distributed Control and Energy Management Approach for DERs and EVs with Application to EV Charging Stations

The rapid growth of Electric Vehicles (EVs) and their charging infrastructure has given rise to various challenges regarding the stable operation of EV charging stations (CSs). Vehicle-to-Vehicle (V2V) charging schemes decrease the dependency on CSs, but they also increase the reliance on a single source type. This maximizes the charging cost and minimizes the system continuity rate. In this work, energy routing-based resilient distributed control is presented by emphasizing the flexibility of Distributed Energy Resources (DERs). This provides an efficient energy management framework for EV CSs with optimal power sharing and cost-effective energy trading, by enabling DERs to EVs and V2V energy transactions. The dynamic performance of the proposed scheme is enhanced by using the Augmented Lagrangian approach. The resiliency of this distributed control approach is quantified by using a resiliency matrix to reflect its enhanced continuity under different system disturbances. The effectiveness of the proposed model is validated by considering different case study scenarios of EV-loading and system-level disturbances.

An efficient, floating-frame-of-reference-based recursive formulation to model planar flexible multibody applications

This paper describes a floating-frame-of-reference-based (FFRF-based) recursive formulation for flexible bodies. A new definition of the joint coordinate system is introduced that makes it simpler to formulate more concise velocity transformation matrices. A path matrix, which defines the connectivity of the mechanism for the open loop system, simplifies introducing the topology of the rigid and/or flexible spanning tree. The flexible multibody system is often subjected to large reference motion but small deformations. Both Euler–Bernoulli-based beam (EBB) and continuum-mechanics-based beam (CMB) elements are implemented to validate the proposed approach. The reference conditions mentioned here are used to eliminate the rigid body motion from the deformation field, i.e., removal of the coordinate redundancy, in the FFRF. Thus, different reference conditions, such as simply supported, single-cantilever, and double-cantilever are used. Finally, it is demonstrated that the proposed approach is not limited by the choice of reference conditions, which helps the modeler to apply the approach to different multibody applications.In the numerical analysis, the results of the total energy variation and constraint violations of the close loop systems are fulfilled with good accuracy by using both penalty formulation with augmented Lagrangian method and coordinate partitioning approach for constraint stabilization. Furthermore, the proposed semi-recursive method produces dynamic analysis results comparable to the ANCF with better computational efficiency. In the end, compared to fully recursive approach, the proposed semi-recursive approach shows better efficiency and easier implementation.

Fluid–structure interaction modeling with nonmatching interface discretizations for compressible flow problems: Computational framework and validation study

This work presents a strongly-coupled fluid–structure interaction (FSI) formulation for compressible flows that is developed based on an augmented Lagrangian approach. The method is suitable for handling problems that involve nonmatching fluid–structure interface discretizations. In this work, the fluid is modeled using a stabilized finite element method for the Navier–Stokes equations of compressible flows and is coupled to the structure, which is formulated using isogeometric Kirchhoff–Love shells. The strongly-coupled system is solved using a block-iterative approach. The proposed method is validated using two compressible flow benchmark problems to assess the accuracy of the developed formulation.

An augmented Lagrangian method for multiple nodal displacement-constrained topology optimization

This article proposes an augmented Lagrangian-based topology optimization approach to minimize the volume fraction subject to multiple nodal displacement constraints. The proposed method puts the multiple constraint equations in the objective function and transforms it into a sequence of unconstrained optimization problems. This study explains the theoretical aspects of the employed augmented Lagrangian approach in depth. Sensitivity expression in terms of design variables is identified by exercising the differentiate-then-discretize mechanism, and utilizes a moving asymptote algorithm to sort out a sequence of optimization subproblems. The optimized results are compared and analysed with those from conventional aggregation-based topology optimization. However, the predefined aggregation approach indicates a dependency on the supplied parameters. The numerical two- and three-dimensional examples reveal the viability and reliability of the suggested augmented Lagrangian scheme, which shows advantages in terms of robustness and efficiency.

A first order hyperbolic reformulation of the Navier-Stokes-Korteweg system based on the GPR model and an augmented Lagrangian approach

In this paper we present a novel first order hyperbolic reformulation of the barotropic Navier-Stokes-Korteweg system. The new formulation is based on a combination of the first order hyperbolic Godunov-Peshkov-Romenski (GPR) model of continuum mechanics with an augmented Lagrangian approach that allows to rewrite nonlinear dispersive systems in first order hyperbolic form at the aid of new evolution variables. The governing equations for the new evolution variables introduced by the rewriting of the dispersive part are endowed with curl involutions that need to be taken care of. In this paper, we account for the curl involutions at the aid of a thermodynamically compatible generalized Lagrangian multiplier (GLM) approach, similar to the GLM divergence cleaning introduced by Munz et al. for the divergence constraint of the magnetic field in the Maxwell and MHD equations. A key feature of the new mathematical model presented in this paper is its ability to restore hyperbolicity even for non-convex equations of state, such as the van der Waals equation of state, thanks to the use of an augmented Lagrangian approach and the resulting inclusion of surface tension terms into the hyperbolic flux.The governing PDE system proposed in this paper is solved at the aid of a high order ADER discontinuous Galerkin finite element scheme with a posteriori subcell finite volume limiter in order to deal with shock waves, discontinuities and steep gradients in the numerical solution. We propose an exact solution of our new mathematical model, show numerical convergence rates of up to sixth order of accuracy and show numerical results for several standard benchmark problems, including travelling wave solutions, stationary bubbles and Ostwald ripening in one and two space dimensions.

A Least-Squares Method for the Solution of the Non-smooth Prescribed Jacobian Equation

We consider a least-squares/relaxation finite element method for the numerical solution of the prescribed Jacobian equation. We look for its solution via a least-squares approach. We introduce a relaxation algorithm that decouples this least-squares problem into a sequence of local nonlinear problems and variational linear problems. We develop dedicated solvers for the algebraic problems based on Newton’s method and we solve the differential problems using mixed low-order finite elements. Various numerical experiments demonstrate the accuracy, efficiency and the robustness of the proposed method, compared for instance to augmented Lagrangian approaches.