Augmented Lagrangian Research Articles

An augmented Lagrangian approach for cardinality constrained minimization applied to variable selection problems

To solve convex constrained minimization problems, that also include a cardinality constraint, we propose an augmented Lagrangian scheme combined with alternating projection ideas. Optimization problems that involve a cardinality constraint are NP-hard mathematical programs and typically very hard to solve approximately. Our approach takes advantage of a recently developed and analyzed continuous formulation that relaxes the cardinality constraint. Based on that formulation, we solve a sequence of smooth convex constrained minimization problems, for which we use projected gradient-type methods. In our setting, the convex constraint region can be written as the intersection of a finite collection of convex sets that are easy and inexpensive to project. We apply our approach to a variety of over and under determined constrained linear least-squares problems, with both synthetic and real data that arise in variable selection, and demonstrate its effectiveness.

Electromagnetic and Mechanical Topology Optimization for SynRM Rotors Considering High Dimensional Constraints

The knowledge-based parametric design process of the rotor topology of the synchronous reluctance motor (SynRM) requires a tedious parameter definition, selection, and searching process until an optimal design can be obtained. This research proposes a topology optimization approach based on solid isotropic material with penalization method for the SynRM rotor topology design. The aim is to pursue a more intelligent and automatic design process. Specifically, the proposed method aims to generate the SynRM rotor design of high output torque profile. Meanwhile, the mechanical constraints including structural compliance and stress etc. are also considered for promising the structural reliability of the rotor. Considering the high dimensional constraints caused by the multi-physics performances, the augmented Lagrangian method based optimization framework is developed to solve the minimization problem in a compact scheme. To prove the effectiveness of the developed method, simulations and analysis on the topology optimization of SynRMs are performed in this research. Prototyping and experiment are also conducted for verifying the performance of the optimized structure.

Parameter-free preconditioning for nearly-incompressible linear elasticity

It is well known that via the augmented Lagrangian method, one can solve Stokes' system by solving the nearly incompressible linear elasticity equation. In this paper, we show that the converse holds, and approximate the inverse of the linear elasticity operator with a convex linear combination of parameter-free operators. In such a way, we construct a uniform preconditioner for linear elasticity for all values of the Lamé parameter λ∈[0,∞). Numerical results confirm that by using inf-sup stable finite-element spaces for the solution of Stokes' equations, the proposed preconditioner is robust in λ.

Parallel-in-time multiple shooting for optimal control problems governed by the Navier–Stokes equations

In the past decades, multiple shooting methods have proven to be a promising direction to speed up the optimization process, especially in the context of ODE-based optimization. Very recently, Fang et al. (Journal of Computational Physics, vol. 452, 110926, 2022) proposed a multiple shooting algorithm for large-scale PDE-constrained optimization. The current paper continues this line of work and explores the potential of multiple shooting methods for optimal control problems governed by the three-dimensional Navier–Stokes equations. Similar to Fang et al., the augmented Lagrangian (AL) method is used to solve the resulting equality-constrained optimization problem, and we employ the classical limited-memory BFGS method for the unconstrained subproblems inside the AL loop. In the current work, we exploit the multiple shooting paradigm in full by processing the shooting windows parallel-in-time, allowing for significant parallel speed-ups compared to single shooting. The proposed method is validated on a velocity tracking case, using up to 100 windows. Our analysis shows that the multiple shooting algorithm allows for considerable algorithmic and parallel speed-ups. While algorithmic speed-up depends on the exact tracking case and initialization of the shooting windows, the multiple shooting algorithm always outperforms single shooting in terms of computational time (if the number of windows is sufficiently high) due to the parallel-in-time implementation. For a given amount of resources, we also show that the proposed parallel-in-time strategy can outperform spatial parallelization alone, especially when the spatial parallelization is saturated.

An augmented Lagrangian method for nonconvex composite optimization problems with nonlinear constraints

An augmented Lagrangian method for nonconvex composite optimization problems with nonlinear constraints

Performance of matrix completion approaches for aquaponics data

Technological innovations in Internet of Things (IoT) have resulted in smart agricultural solutions such as a remotely monitored Aquaponics system and a wireless sensor network (WSN) of such systems (nodes). IoT enables continuous sensing of temperature and pH data at each node of the WSN, which is periodically transmitted to a remote fusion centre. In this regard, the data matrices acquired at the fusion centre often suffer from data vacancies and missing data problems, owing to typical wireless multipath fading environment, sensor malfunctions and node failures. This paper explores the applicability of different matrix completion approaches for missing data reconstruction. Specifically, the performance of baseline predictor, correlation based approaches such as baseline predictor with temporal model, k-nearest neighbors (kNN) and low rank based approaches such as Sparsity Regularized Singular Value Decomposition (SRSVD) and Augmented Lagrangian Sparsity Regularized Matrix Factorization (ALSRMF) have been explored. Reliable temperature and pH data for 19 independent acquisition hours with 60 samples per hour are acquired at the fusion centre via Ultra High Frequency (UHF) transmission at 470 MHz and suitable pre-processing. Simulating different data integrity scenarios, the reconstruction error plots from each of these matrix completion approaches is extracted. A hybrid of kNN and baseline predictor with temporal model rendered a Mean Absolute Percentage Error (MAPE) of 1.75% for temperature and 0.86% for pH, at 0.5 data integrity. Further, with ALSRMF, which exploits the low rank constraint, the error reduced to 1.25% for temperature and 0.7% for pH, thus substantiating a promising approach for Aquaponics system data reconstruction.

A numerical framework for three-dimensional optimization of cooling channels in thermoplastic printed molds

The rapid progress of additive manufacturing (AM) technology in the last few decades has unlocked new potentials for the injection molding industry by enabling the quick and cost-efficient manufacture of molds with complex geometries Despite the wide literature regarding experimental investigations on AM polymer-based soft tools specifically produced to cope with low-volume production of small-sized injected parts, their lifespan remains uncertain, and premature failures are often related to poor thermal performance. This paper is devoted to the numerical study and enhancement of the heat transfer within a thermoplastic 3D printed insert with cooling channels (CCs). Experimental thermal characterization is performed on printed composite samples made of polycarbonate reinforced with carbon fibers, considered as tool material. The simulations are performed on an industrial case, thereby facilitating a comprehensive validation of the proposed framework. Parametric studies show a marked cycle time sensitivity to insert thermal conductivity within the 0.1–1.0 W/(mK) range while demonstrating negligible influence on cycle times for polymer–polymer thermal contact resistance values below 10−3m2K/W. Furthermore, to find a suitable arrangement of the CCs’ layout, we propose here an accurate optimization methodology based on 3D overset meshes in the finite element method context coupled to the augmented Lagrangian particle swarm optimizer. The optimized bent CCs’ configuration enhances part surface temperature uniformity by 42% and reduces its temperature delta by over 6∘C, all while employing 67% of the reference cycle time. The thermal shortcomings of the thermoplastic AM mold, compared to its steel counterpart, are also addressed in this work.

Innovative and entrepreneurial characteristics of university students based on logistic regression model

Abstract This paper proposes a logistic regression model with structural sparsity to study the characteristics of innovation and entrepreneurship among college students. The article first analyzes the basic form of the logistic regression model, including the objective function and the selection method for the penalty function. Then, because the ADMM algorithm combines the advantages of augmented Lagrangian and pairwise decomposition, which can reduce the computational difficulty and complexity, based on this advantage, this paper designs the ADMM algorithm solution framework that is favorable for distributed computing. Finally, this paper analyzes the relationship between the development of innovation and entrepreneurship ability of students in R colleges and universities and their gender, grade, academic foundation, experience in clubs and discipline type. The results yielded that college students’ mean value of innovation and entrepreneurship competence in HEI R was 3.734. The mean value of the scores of each sub-competence ranged from 3.531 to 3.918, which puts the overall innovation and entrepreneurship competence of students in this university at an intermediate level. Therefore, this study plays an important role in understanding the innovation and entrepreneurial characteristics of students in higher education.

Enhancement and Segmentation of Low-Light Images Using Illumination Map Estimation based Level Set (IME-LS) Method

One of the most difficult aspects of image segmentation is that it cannot be successfully segmented if the image is dark or degraded. In this paper, proposes a method for the segmentation of low-quality and degraded images is put forth which is called Illumination Map Estimation based Level Set (IME-LS) Method. This proposed model has been classified into two parts: Firstly, we use an enhancement approach using illumination map approximation to enhance the input image. In this approach, Illumination map is constructed then refined. The refined illumination map undergoes an Augmented Lagrangian algorithm and then we use a sped-up solver for a considerably more efficient outcome. Secondly, Then the segmentation procedure begins once the image is enhanced. The enhanced image is segmented using level set bias method through Fuzzy clustering. In this method we employ the Fuzzy C-Means (FCM) algorithm to categorize data points into clusters. The fuzzy C-Means algorithm separates different entities in the image based on their varied intensities and sorts them into various clusters. The level set bias approach then tracks the variational boundaries of the image. We have designed the integrated algorithm in such a way that the image is classified or grouped into various clusters using a novel fuzzy Clustering algorithm and the variational boundaries of those clusters are tracked by employing the level set algorithm. In this paper, we further perform quantitative and comparative analysis of the suggested technique with respect to other segmentation techniques to illustrate the efficiency and flexibility of the suggested model.

Distributed online convex optimization with multiple coupled constraints: A double accelerated push–pull algorithm

This paper is concerned with the distributed online convex optimization problems with multiple coupled constraints over unbalanced digraphs, which are widely applied in diverse fields such as machine learning, smart grids, and resource allocation problems, etc. In such practical scenarios, multiple coupled constraints, where each coupled constraint might only contain a portion of all nodes, are more general and complex than global coupled constraints. Additionally, due to certain constraint conditions such as energy and communication limitations, each node can only exchange information with other nodes in an identical constraint. Besides, the network connectivity of practical constrained problems might not be only confined to one type of weight matrices. By adopting an augmented Lagrangian method, the multiple coupled constrained optimization problem is converted into a saddle-point problem. To tackle this problem, a novel double accelerated push–pull (DAPP) algorithm is proposed which concurrently employs row-stochastic (RS) and column-stochastic (CS) weight matrices. Specifically, each node pushes gradient information weighted by CS matrices to out-neighbors, while pulls dual variables fused by RS matrices from in-neighbors. Combined with CS matrices, the gradient tracking scheme is employed to promote the efficiency of tracking the averaged gradients. Furthermore, aiming at improving the convergence rate, the heavy-ball method is used to update both primal and dual variables. Then, rigorous theoretical results indicate that DAPP obtains the sublinear bounds of regret and constraint violation. Finally, the plug-in electric vehicles (PEVs) charging problem is utilized to demonstrate the validity of the proposed algorithm.

On the paper “Augmented Lagrangian algorithms for solving the continuous nonlinear resource allocation problem”

In the paper Torrealba et al. (2021) an augmented Lagrangian algorithm was proposed for resource allocation problems with the intriguing characteristic that instead of solving the box-constrained augmented Lagrangian subproblem, they propose projecting the solution of the unconstrained subproblem onto such box. A global convergence result for the quadratic case was provided, however, this is somewhat counterintuitive, as in usual augmented Lagrangian theory, this strategy can fail in solving the augmented Lagrangian subproblems. In this note we investigate further this algorithm and we show that the proposed method may indeed fail when the Hessian of the quadratic is not a multiple of the identity. In the paper, it is not clear enough that two different projections are being used: one for obtaining their convergence results and other in their implementation. However, despite the lack of theoretical convergence, their strategy works remarkably well in some classes of problems; thus, we propose a hybrid method which uses their idea as a starting point heuristics, switching to a standard augmented Lagrangian method under certain conditions. Our contribution consists in presenting an efficient way of determining when the heuristics is failing to improve the KKT residual of the problem, suggesting that the heuristic procedure should be abandoned. Numerical results are provided showing that this strategy is successful in accelerating the standard method.

Augmented Lagrangian approach for multi-objective topology optimization of energy storage flywheels with local stress constraints

Augmented Lagrangian approach for multi-objective topology optimization of energy storage flywheels with local stress constraints

A linear algebra perspective on the random multi-block ADMM: the QP case

Embedding randomization procedures in the Alternating Direction Method of Multipliers (ADMM) has recently attracted an increasing amount of interest as a remedy to the fact that the direct multi-block generalization of ADMM is not necessarily convergent. Even if, in practice, the introduction of such techniques could mitigate the diverging behaviour of the multi-block extension of ADMM, from the theoretical point of view, it can ensure just the convergence in expectation, which may not be a good indicator of its robustness and efficiency. In this work, analysing the strongly convex quadratic programming case from a linear algebra perspective, we interpret the block Gauss–Seidel sweep performed by the multi-block ADMM in the context of the inexact Augmented Lagrangian Method. Using the proposed analysis, we are able to outline an alternative technique to those present in the literature which, supported from stronger theoretical guarantees, is able to ensure the convergence of the multi-block generalization of the ADMM method.

A Filter-SQP strategy for training Support Vector Machine models

This paper introduces a filtering strategy for addressing optimization problems arising in binary Support Vector Machine classification. The training optimization problem aims to solve the dual formulation which involves a quadratic objective function subjected to a linear and box constraints. Our approach employs a Filter algorithm with Sequential Quadratic Programming iterations that minimize the quadratic Lagrangian approximations. Notably, we utilize the exact Hessian matrix in our numerical experiments to seek the desired classification function. Moreover, we present a Filter algorithm combined with the Augmented Lagrangian method aiming to accelerate the algorithm convergence. To substantiate our method's effectiveness, we conduct numerical experiments through MATLAB, comparing outcomes with alternative methodologies detailed in existing literature. Numerical experiments shows that the Filter--SQP combined with Augmented Lagrangian method is competitive and efficient method compared with an interior-point based solver and LIBSVM software in relation of classification metrics and CPU-time.

Homogeneous multigrid for HDG applied to the Stokes equation

Abstract We propose a multigrid method to solve the linear system of equations arising from a hybrid discontinuous Galerkin (in particular, a single face hybridizable, a hybrid Raviart–Thomas, or a hybrid Brezzi–Douglas–Marini) discretization of a Stokes problem. Our analysis is centered around the augmented Lagrangian approach and we prove uniform convergence in this setting. Beyond this, we establish relations, which resemble those in Cockburn & Gopalakrishnan (2008, Error analysis of variable degree mixed methods for elliptic problems via hybridization. Math. Comput., 74, 1653–1677) for elliptic problems, between the approximates that are obtained by the single-face hybridizable, hybrid Raviart–Thomas and hybrid Brezzi–Douglas–Marini methods. Numerical experiments underline our analytical findings.

Efficient nonlinear model predictive motion controller for autonomous vehicles from standstill to extreme conditions based on split integration method

Autonomous driving motion control based on nonlinear model predictive control (NMPC) faces integration stability issues when driving at low speeds. However, this issue has not received sufficient attention or been adequately addressed. Ensuring integration stability requires the adoption of an implicit method, which incurs a high computational burden, making this phenomenon crucial to the performance of NMPC. In this paper, a split integration method is proposed to utilize the gradient-based augmented Lagrangian approach (GRAMPC) to solve the integration stability problem and significantly improve the efficiency of NMPC. The proposed split integration method integrates the explicit second-order Runge–Kutta (RK) method and the second-order L-stable Rosenbrock (ROS2) method. To enhance integration stability for the stiff part of the state equation, the ROS2 method is used. For the non-stiff part of the system, the RK method is used to improve its efficiency. This approach results in an efficient nonlinear NMPC system that can be adapted to full range speeds. Simulations and real-world experiments demonstrate that the low-speed integration stability problem of NMPC is entirely resolved without compromising its efficiency.

Lagrangian methods for optimal control problems governed by multivalued quasi-hemivariational inequalities

The primary objective of this paper is to develop a general augmented Lagrangian approach for optimal control problems governed by multivalued quasi-hemivariational inequalities. By imposing general coercivity and monotonicity-type conditions, we establish multiple existence results for the optimal control problems. Furthermore, we establish a comprehensive characterization of the zero duality gap property for the primal-dual problems, utilizing the concept of lower semicontinuity of new perturbation functions. Our results represent a significant advancement over recent literature on the subject.

Strength design of porous materials using B-spline based level set method

The inverse homogenization method can tailor some mechanical and physical effective properties by laying out materials in a periodic representative volume element. However, studies on strength design are yet to be developed because of the difficulties in numerically retrieving its value. Unlike traditional asymptotic homogenization, the fast Fourier transform-based homogenization method based on the augmented Lagrangian approach uses a Green operator in the frequency domain to replace time-consuming finite element analysis and inherently meet the periodic boundary conditions. Thus, it is developed in this work to retrieve material strength in terms of the von Mises yield criterion. The zero-level contour of a linear combination of cubic B-spline basis functions with repeated knots is used to represent the microstructure profile in the design of material strength. The effective strength or its squared difference with a prescribed target is minimized within the framework of the reaction diffusion-based B-spline level set method. The filtered, non-consistent discrete Green operator is adopted to avoid slow convergence of porous material. Examples of porous material demonstrate that the proposed method guarantees surface smoothness, optimization flexibility, and structural periodicity in strength design.

A Derivative-free Trust-region Method for Optimization on the Ellipsoid

Optimization methods play a crucial role in various fields and applications. In some optimization problems, the derivative information of the objective function is unavailable. Such black-box optimization problems need to be solved by derivative-free optimization methods. At the same time, optimization problems with ellipsoidal constraints are important and have widespread applications in various fields as well. Following the development of the late professor M. J. D. Powell’s efficient derivative-free trust-region optimization methods, this paper considers solving derivative-free optimization problems on the ellipsoid. Our new optimization solver EC-NEWUOA for problems on the ellipsoid in ℜ n is designed based on Powell’s derivative-free software NEWUOA for unconstrained optimization problems. The proposed techniques for our new method mainly include using the Courant penalty function, the augmented Lagrangian method, and the projection technique. Details about the method and theoretical analysis are included in this paper. We also compare our new method with other algorithms by solving test problems and then show the numerical advantages of our new method.

Mission Profile Analysis of Point-to-Point Rocket Cargo Transportation System Using Trajectory Optimization

This paper proposes new concepts for the point-to-point rocket cargo transportation system (RCTS). The RCTS is a transportation system that can deliver cargo anywhere on Earth in an hour using reusable launch vehicle technologies. As a fundamental study of global transportation on Earth, we especially address the mission profiles for short-range transportation within hundreds of kilometers using small rocket engines, which are intended for logistics transportation within neighboring countries or domestic transportation in a country. Two mission profiles of the RCTS are introduced, and the flight phases of each concept are explained in detail. The trajectory optimization problem of the RCTS is formulated based on the mission profiles, incorporating the flight constraints, boundary conditions, and an objective function that aims to maximize the payload ratio. The explicit-guidance is employed during the landing burn phase to enhance the convergence of the optimization problem by expanding the feasible region. The coevolutionary augmented Lagrangian method, one of the evolutionary algorithms, is utilized to obtain the solution for the trajectory optimization problem. The optimal trajectory and state variables are presented through numerical simulations. Additionally, parametric studies are performed to determine the payload mass trend of the RCTS. The trajectory optimization results are summarized to provide an analysis and comparison of the proposed mission profiles.