Augmented Lagrangian Techniques Research Articles

Isogeometric analysis and Augmented Lagrangian Galerkin Least Squares Methods for residual minimization in dual norm

We explore how recent advances in Isogeometric analysis, Galerkin Least-Squares methods, and Augmented Lagrangian techniques can be applied to solve nonstandard problems, for which there is no classical stability theory, such as that provided by the Lax–Milgram lemma or the Banach-Necas-Babuska theorem. In particular, we consider continuation problems where a second-order partial differential equation with incomplete boundary data is solved given measurements of the solution on a subdomain of the computational domain. The use of higher regularity spline spaces leads to simplified formulations and potentially minimal multiplier space. We show that our formulation is inf-sup stable, and given appropriate a priori assumptions, we establish optimal order convergence.

Recent Scalability Improvements for Semidefinite Programming with Applications in Machine Learning, Control, and Robotics

Historically, scalability has been a major challenge for the successful application of semidefinite programming in fields such as machine learning, control, and robotics. In this article, we survey recent approaches to this challenge, including those that exploit structure (e.g., sparsity and symmetry) in a problem, those that produce low-rank approximate solutions to semidefinite programs, those that use more scalable algorithms that rely on augmented Lagrangian techniques and the alternating-direction method of multipliers, and those that trade off scalability with conservatism (e.g., by approximating semidefinite programs with linear and second-order cone programs). For each class of approaches, we provide a high-level exposition, an entry point to the corresponding literature, and examples drawn from machine learning, control, or robotics. We also present a list of software packages that implement many of the techniques discussed in the review. Our hope is that this article will serve as a gateway to the rich and exciting literature on scalable semidefinite programming for both theorists and practitioners.

A Unified Precoding Scheme for Generalized Spatial Modulation

Generalized spatial modulation (GSM) activates $n_{t}$ out of $N_{t}~(1 \leq n_{t} available transmit antennas, and information is conveyed through $n_{t}$ modulated symbols as well as the index of the $n_{t}$ activated antennas. GSM strikes an attractive tradeoff between spectrum efficiency and energy efficiency. Linear precoding that exploits channel state information at the transmitter enhances the system error performance. For GSM with $n_{t}=1$ (the traditional SM), the existing precoding methods suffer from high computational complexity. On the other hand, GSM precoding for $n_{t} \geq 2$ is not thoroughly investigated in the open literature. In this paper, we develop a unified precoding design for GSM systems, which universally works for all $n_{t}$ values. Based on the maximum minimum Euclidean distance criterion, we find that the precoding design can be formulated as a large-scale nonconvex quadratically constrained quadratic program problem. Then, we transform this challenging problem into a sequence of unconstrained subproblems by leveraging augmented Lagrangian and dual ascent techniques. These subproblems can be solved in an iterative manner efficiently. Numerical results show that the proposed method can substantially improve the system error performance relative to the GSM without precoding and features extremely fast convergence rate with a very low computational complexity.

Numerical Methods for Power-Law Diffusion Problems

In this paper, we consider numerical methods for nonlinear diffusion problems where the diffusion term follows a power law, e.g., $p$-Laplace-type problems. In the first part, we present continuous higher order finite element discretizations for the model problem and we derive error estimates. In the second part, we discuss Newton iterative methods based on residual-based line-search and error-oriented globalization, which are employed for the numerical solution of the produced nonlinear algebraic system. Third, we formulate the original problem as a saddle point problem in the frame of augmented Lagrangian techniques and present two iterative methods for its solution. We conduct a systematic investigation of all solution algorithms. These algorithms are compared with respect to computational cost and their efficiency. Numerical results demonstrating the theoretical error estimates are also presented in five examples.

Stability and Performance Limits of Adaptive Primal-Dual Networks

This paper studies distributed primal-dual strategies for adaptation and learning over networks from streaming data. Two first-order methods are considered based on the Arrow-Hurwicz (AH) and augmented Lagrangian (AL) techniques. Several revealing results are discovered in relation to the performance and stability of these strategies when employed over adaptive networks. The conclusions establish that the advantages that these methods exhibit for deterministic optimization problems do not necessarily carry over to stochastic optimization problems. It is found that they have narrower stability ranges and worse steady-state mean-square-error performance than primal methods of the consensus and diffusion type. It is also found that the AH technique can become unstable under a partial observation model, while the other techniques are able to recover the unknown under this scenario. A method to enhance the performance of AL strategies is proposed by tying the selection of the step-size to their regularization parameter. It is shown that this method allows the AL algorithm to approach the performance of consensus and diffusion strategies but that it remains less stable than these other strategies.

Performance of an Energy-Conserving Algorithm for Multi-Body Dynamics

This work presents the application to the dynamics of multibody systems of three methods based on the augmented Lagrangian techniques, compares them, and gives some criteria for their use in realistic problems.The methods are: an augmented Lagrangian formulation with trapezoidal rule plus projections of velocities and accelerations, an augmented Lagrangian energy-conserving formulation, and an augmented Lagrangian formulation with conserving integrator plus projections of velocities and accelerations.The simulation of two mechanical systems is carried out in this work: a spherical compound pendulum, which is an academic example that permits to see the properties of the formulations; andthe whole model of a car, which is a realistic and demanding example to test the efficiency and robustness of the formulations.

On the Time-Continuous Mass Transport Problem and Its Approximation by Augmented Lagrangian Techniques

In [J. D. Benamou and Y. Brenier}, Numer. Math., 84 (2000), pp. 375--393], a computational fluid dynamic approach was introduced for computing the optimal map occurring in the Monge--Kantorovich problem. Though the described augmented Lagrangian method involves a Hilbertian framework, the discussion was purely formal. Taking advantage of the recent progress in optimal transport theory [L. A. Caffarelli, Comm. Pure Appl. Math., 45 (1992), pp. 1141--1151], [L. A. Caffarelli, Ann. of Math. (2), 144 (1996), pp. 453--496], [D. Cordero-Erausquin, C. R. Acad. Sci. Paris Sér. I Math., 329 (1999), pp. 199--202], [R. J. McCann, Geom. Funct. Anal., 11 (2001), pp. 589--608] and despite the lack of coercivity of the Hilbertian problem, we establish an existence result. Then, under a reasonable assumption of positivity for the density, we prove the existence of saddle-points for both Lagrangians defined in Benamou and Brenier, and finally prove the convergence of the numerical method.

Log-Sigmoid Multipliers Method in Constrained Optimization

In this paper we introduced and analyzed the Log-Sigmoid (LS) multipliers method for con- strained optimization. The LS method is to the recently developed smoothing technique as augmented Lagrangian to the penalty method or modified barrier to classical barrier methods. At the same time the LS method has some specific properties, which make it substantially different from other nonquadratic augmented Lagrangian techniques. We established convergence of the LS type penalty method under very mild assumptions on the input data and estimated the rate of convergence of the LS multipliers method under the standard second order optimality condition for both exact and nonexact minimization. Some important properties of the dual function and the dual problem, which are based on the LS Lagrangian, were discovered and the primal-dual LS method was introduced.

On the first-order estimation of multipliers from Kuhn–Tucker systems

The minimization of a nonlinear function with linear and nonlinear constraints and simple bounds can be performed by minimizing an augmented Lagrangian function that includes only the nonlinear constraints subject to the linear constraints and simple bounds. It is then necessary to estimate the multipliers of the nonlinear constraints and variable reduction techniques can be used to carry out the successive minimizations. The viability of estimating the multipliers of the nonlinear constraints from the Kuhn–Tucker system is analyzed and an acceptability test on the residual of the estimation is put forward. The computational performance of the procedure is compared with that of the inexpensive Hestenes–Powell multiplier update. Scope and purpose It is possible to minimize a nonlinear function with linear and nonlinear constraints and simple bounds through the successive minimization of an augmented Lagrangian function including only the nonlinear constraints subject to the linear constraints and simple bounds. This method is particularly interesting when the linear constraints are flow conservation equations, as there are efficient techniques for solving nonlinear network problems. Regarding the successive estimation of the multipliers of the nonlinear constraints there is some doubt as to whether using the Kuhn–Tucker system could improve upon the inexpensive Hestenes–Powell update, especially considering that the Kuhn–Tucker system with partial augmented Lagrangians may not always lead to an acceptable multiplier estimation. Clarifying the computational efficiency of the multiplier update when there are linear or nonlinear side constraints is also a necessary previous step regarding the comparison between partial augmented Lagrangian techniques and either primal partitioning techniques for linear side constraints or projected Lagrangian methods in the case of nonlinear side constraints.

Relaxation labeling using augmented lagrange-hopfield method

A novel relaxation labeling (RL) method is presented based on Augmented Lagrangian multipliers and the graded Hopfield neural network (ALH). In this method, an RL problem is converted into a constrained optimization problem and solved by using the augmented Lagrangian and Hopfield techniques. The ALH method yields results comparable to the best of the existing RL algorithms in terms of the optimized objective values, yet it is more suitable for analog neural implementation. Experimental results are presented.

Least-squares methods for optimal control

Optimal control and optimal shape design problems for the Navier-Stokes equations arise in many important practical applications, such as design of optimal profiles [7], drag minimization [9], [11], and heating and cooling [12], among others. Typically, optimal control problems for the NavierStokes equations combine Lagrange multiplier techniques to enforce the constraints and to derive an optimality system (see, e.g., [11]-[12]), with mixed Galerkin discretization for the state equations. Resulting methods are well-studied theoretically, for example, an abstract framework that can be used for the analyses of such optimal control methods has been suggested in [13]. However, the use of Lagrange multipliers and mixed Galerkin discretizations is associated with some complications in the numerical computations which can reduce the overall efficiency and robustness of corresponding algorithms. For example, resulting discrete problems are in general indefinite. Similarly, it is now well-understood that stability of mixed discretizations does not allow one to choose independently the approximation spaces for the velocity and the pressure, and that these spaces are subject to a restrictive stability condition known as the inf-sup (or LBB) condition; see [8]. One possibility to remedy these difficulties is to consider optimal control methods in which the Navier-Stokes constraint is treated by augmented Lagrangian techniques; see [6]. Nevertheless, the use of mixed Galerkin discretization in the method of [6] still requires approximation by finite element spaces that are subject to the inf-sup condition.

Numerical comparison between two cost functions in contact shape optimization

A finite element approach to shape optimization in a 2D frictionless contact problem for two different cost functions is presented in this work. The goal is to find an appropriate shape for the contact boundary, performing an almost constant contact-stress distribution. The whole formulation, including the mathematical model for the unilateral problem, sensitivity analysis and geometry definition is treated in a continuous form, independently of the discretization in finite elements. Shape optimization is performed by a direct modification of the geometry throughB-spline curves and an automatic mesh generator is used at each new configuration to provide the finite element input data. Augmented-Lagrangian techniques (to solve the contact problem) and an interior-point mathematical-programming algorithm (for shape optimization) are used to obtain numerical results.

Domain decomposition with nonmatching grids: augmented Lagrangian approach

We propose and study a domain decomposition method which treats the constraint of displacement continuity at the interfaces by augmented Lagrangian techniques and solves the resulting problem by a parallel version of the Peaceman-Rachford algorithm. We prove that this algorithm is equivalent to the fictitious overlapping method introduced by P.L. Lions. We also prove its linear convergence independently of the discretization step h, even if the finite element grids do not match at the interfaces. A new preconditioner using fictitious overlapping and well adapted to three-dimensional elasticity problems is also introduced and is validated on several numerical examples.

Application of augmented Lagrangian techniques for non‐linear constitutive laws in contact interfaces

AbstractThe use of micromechanically based constitutive equations for contact interfaces leads to technically relevant parameters to ill‐conditioned finite‐element equations. In the paper an augmented Lagrangian technique is employed to overcome this difficulty and to provide a good converging algorithm.

An augmented lagrangian treatment of contact problems involving friction

A framework is presented within which the method of augmented Lagrangians is readily applied to problems involving contact with friction. This method, which has enjoyed considerable success in the treatment of constrained minimization problems, has been previously applied to problems involving incompressible flow, incompressible elasticity of solids and even frictionless contact. An additional challenge to the method is provided by frictional contact problems governed by a Coulomb law, due to the special form taken by the frictional constraint. This paper describes a new extension of the augmented Lagrangian technique to frictional problems which is well-suited to finite element implementation. The proposed treatment inherits the traditional advantages of augmented Lagrangian techniques over penalty methods; namely, decreased ill-conditioning of governing equations, and essentially exact satisfaction of constraints with finite penalties. A set of numerical examples is presented in which the utility of the method is demonstrated even in the presence of finite deformations and inelasticity.

Numerical Solution of Viscoplastic Flow Problems by Augmented Lagrangians

Abstract : This report describes an application of Augmented Lagrangian techniques to the numerical solution of quasistatic flow problems in incompressible viscoplasticity, focusing on cases where the internal viscoplastic dissipation potential is not a differentiable function of the material deformation rate. The stresses of elastic origin are neglected, and the variational formulation of these problems is approximated via mixed finite elements of order 1. Convergence results are proved or recalled, both for the finite element approximation and for the augmented lagrangian algorithm. A detailed study of the local minimization problems which occur in the augmented lagrangian decomposition of the above problems is also presented, together with several numerical results. These results were obtained using the MODULEF finite element code on a VAX 780 at the Mathematics Research Center and cover successively the case of Norton, of Bingham and of Tresca type materials. (Author)

Numerical Solution of Problems in Incompressible Finite Elasticity by Augmented Lagrangian Methods II. Three-Dimensional Problems

In a first paper [SIAM J. Appl. Math., 42 (1982), pp. 400–429] we introduced an augmented Lagrangian methodology for solving equilibrium problems in incompressible finite elasticity, and then applied it to the solution of several two-dimensional and axisymmetric test problems. In this second paper we extend our methods to the solution of genuine three-dimensional problems; actually using these augmented Lagrangian techniques we can reduce the solution of these complicated nonlinear, nonconvex problems, to the solution of a sequence of linear problems and also to the solution of a sequence of minimization problems for convex functionals on convex sets. The capabilities of our methods are illustrated by the results of numerical experiments.