Augmented Lagrangian Research Articles

Scalable log-ratio lasso regression for enhanced microbial feature selection with FLORAL.

Identifying predictive biomarkers of patient outcomes from high-throughput microbiome data is of high interest, while existing computational methods do not satisfactorily account for complex survival endpoints, longitudinal samples, and taxa-specific sequencing biases. We present FLORAL, an open-source tool to perform scalable log-ratio lasso regression and microbial feature selection for continuous, binary, time-to-event, and competing risk outcomes, with compatibility for longitudinal microbiome data as time-dependent covariates. The proposed method adapts the augmented Lagrangian algorithm for a zero-sum constraint optimization problem while enabling a two-stage screening process for enhanced false-positive control. In extensive simulation and real-data analyses, FLORAL achieved consistently better false-positive control compared to other lasso-based approaches and better sensitivity over popular differential abundance testing methods for datasets with smaller sample sizes. In a survival analysis of allogeneic hematopoietic cell transplant recipients, FLORAL demonstrated considerable improvement in microbial feature selection by utilizing longitudinal microbiome data over solely using baseline microbiome data.

A Three-Block Inexact Heterogeneous Alternating Direction Method of Multipliers for Elliptic PDE-Constrained Optimization Problems with a Control Gradient Penalty Term

Optimization problems with PDE constraints are widely used in engineering and technical fields. In some practical applications, it is necessary to smooth the control variables and suppress their large fluctuations, especially at the boundary. Therefore, we propose an elliptic PDE-constrained optimization model with a control gradient penalty term. However, introducing this penalty term increases the complexity and difficulty of the problems. To solve the problems numerically, we adopt the strategy of “First discretize, then optimize”. First, the finite element method is employed to discretize the optimization problems. Then, a heterogeneous strategy is introduced to formulate the augmented Lagrangian function for the subproblems. Subsequently, we propose a three-block inexact heterogeneous alternating direction method of multipliers (three-block ihADMM). Theoretically, we provide a global convergence analysis of the three-block ihADMM algorithm and discuss the iteration complexity results. Numerical results are provided to demonstrate the efficiency of the proposed algorithm.

Multi-task Learning for Gaussian Graphical Regressions with High Dimensional Covariates

Gaussian graphical regression is a powerful approach for regressing the precision matrix of a Gaussian graphical model on covariates, which permits the response variables and covariates to outnumber the sample size. However, traditional approaches of fitting the model via separate node-wise lasso regressions overlook the network-induced structure among these regressions, leading to high error rates, particularly when the number of nodes is large. To address this issue, we propose a multi-task learning estimator for fitting Gaussian graphical regression models, which incorporates a cross-task group sparsity penalty and a within-task element-wise sparsity penalty to govern the sparsity of active covariates and their effects on the graph, respectively. We also develop an efficient augmented Lagrangian algorithm for computation, which solves subproblems with a semi-smooth Newton method. We further prove that our multi-task learning estimator has considerably lower error rates than the separate node-wise regression estimates, as the cross-task penalty enables borrowing information across tasks. We examine the utility of our method through simulations and an application to a gene co-expression network study with brain cancer patients.

Enhanced PINNs with augmented Lagrangian method and transfer learning for hydrodynamic lubrication analysis

Purpose By seamlessly integrating physical laws, physics-informed neural networks (PINNs) have flexibly solved a wide variety of partial differential equations (PDEs). However, encoding PDEs and constraints as soft penalties in the loss function can cause gradient imbalances, leading to training and accuracy issues. This study aims to introduce the augmented Lagrangian method (ALM) and transfer learning to address these challenges and enhance the effectiveness of PINNs for hydrodynamic lubrication analysis. Design/methodology/approach The loss function was reformatted by ALM, adaptively adjusting the loss weights during training. Transfer learning was used to accelerate the convergence of PINNs under similar conditions. Additionally, the iterative process for load balancing was reframed as an inverse problem by extending film thickness as a trainable variable. Findings ALM-PINNs significantly reduced the maximum absolute boundary error by almost 80%. Transfer learning accelerated PINNs for solving the Reynolds equation, reducing training epochs by an order of magnitude. The iterative process for load balancing was effectively eliminated by extending the thickness as a trainable parameter, achieving a maximum percentage error of 2.31%. These outcomes demonstrated strong agreement with FDM results, analytical solutions and experimental data. Originality/value This study proposes a PINN-based approach for hydrodynamic lubrication analysis that significantly improves boundary accuracy and the training process. Additionally, it effectively replaces the load balancing procedure. This methodology demonstrates considerable potential for broader applications across various boundary value problems and iterative processes. Peer review The peer review history for this article is available at: https://publons.com/publon/10.1108/ILT-07-2024-0277/

Enhancing Ischemic Stroke Evaluation by a Model-Based Photoacoustic Tomography Algorithm.

Ischemic stroke (IS) is characterized by the sudden interruption of blood supply to the brain, resulting in neurological impairments and even mortality. Photoacoustic computed tomography (PACT) integrates the high contrast of optical imaging and the penetration of ultrasound imaging, enabling non-invasive IS evaluation. However, the image reconstruction quality significantly affects the oxyhemoglobin saturation (sO2) estimation. This study investigates a model-based with total variation minimized by augmented Lagrangian and alternating direction (MB-TVAL3) approach and compared it with the widely used back-projection (BP) and delay-and-sum (DAS) algorithms. Both simulations and invivo experiments are conducted to validate the performance of the MB-TVAL3 algorithm, showing a higher sO2 estimation accuracy and sensitivity in detecting infarct area compared to BP and DAS. The findings of this study emphasize the impact of acoustic inverse problem on the accuracy of sO2 estimation and the proposed approach offers valuable support for IS evaluation and cerebrovascular diagnosis.

Order dispatch problem of the inter-city or inter-district ridesplitting service

This study formulates the order dispatch problem of the inter-city/inter-district ridesplitting service as an integer linear programming, and a solution framework that combines Lagrangian relaxation (LR) and alternating direction methods of multipliers is constructed. Specifically, LR problem is constructed by relaxing coupling constraints; based on this, the augmented Lagrangian relaxation (ALR) problem is constructed by adding a quadratic penalty term. Then, the ALR problem can be decomposed into a series of interdependent vehicle route subproblems by using linearisation technique and block coordinate descent method. The upper bound of the model is generated by ALR and a heuristic, and the solution quality can be evaluated by the lower bound provided by LR. Finally, numerous experiments are conducted to verify the effectiveness of proposed algorithms. The results show that the proposed algorithm could achieve an average cost saving of 10% and an average detour reduction of over 20% than the existing algorithm.

Spur gear tooth root stress analysis by a 3D flexible multibody approach and a full-FE contact-based formulation

This paper proposes an original method to determine the gear tooth root stresses from a 3D finite element (FE) flexible multibody approach and a full-FE contact-based formulation. The contact problem is dealt with an augmented Lagrangian formulation whereas the analysis is performed by a preconditioned gradient solver (PCG). Tooth flank modifications are directly introduced within the 3D model. This one is thus able to take into account straightforwardly tooth bending and Hertzian-like deformations as well as the micro-geometry effect. Simulations are performed for several mesh periods, without making any assumptions about load distribution, tooth and gear blank flexibilities, and possible premature or delayed contacts between tooth pairs in quasi-static conditions. A precise distribution of tooth root stresses associated with instantaneous contacts conditions is then computed. For this study, a single stage spur gear with micro-geometry modifications corresponding to an arc-shaped profile crowning is modeled. Several output torques are considered. The obtained results are compared to those obtained using a 2D FE ISO-based model, where external forces are applied along the theoretical line of action.

Lattice Boltzmann simulations of unsteady Bingham fluid flows

Transient flows of viscoplastic fluids have very peculiar characteristics. The startup and cessation flows of viscoplastic materials have been subject to many theoretical and numerical investigations. The most challenging aspect of numerical solutions of viscoplastic fluids is the viscosity singularity during the transition from yielded to unyielded material. Hence, the proper representation of yield surfaces is the most critical aspect of numerical methods in viscoplastic fluid flow. In the present work, we use a lattice Boltzmann scheme to solve an ideal Bingham fluid’s startup and cessation flows. This numerical scheme advantage is that can represent infinite viscosity without noticeable numerical instabilities, producing yield surfaces with more accuracy and quality. Theoretical solutions for the startup flow are available in the literature. However, it is unclear which is more accurate and what their validity ranges are. Nonetheless, these solutions served as a reference for the present simulations. The overall aspect of the numerical solutions agreed with the theoretical models. The cessation flow of the Bingham fluid was also simulated. Unlike a Newtonian fluid, this type of flow is known to have a finite period until cessation. The simulations correctly reproduced this behavior. The transient yield surfaces matched very well with augmented Lagrangian solutions.

Nonlinear NMM analysis for large deformation and contact problems: Using full strain-rotation decomposition algorithm and augmented Lagrangian method enhanced open-closed iteration

Nonlinear analysis deals with problems that involve large displacements, strains, rotations, and contact problems. Accurate results can be obtained by choosing a reasonable method for decomposing strain and rotation to avoid errors and ensure the correct contact force. A novel full strain-rotation (S-R) decomposition and an augmented Lagrangian enhanced contact model are established within the numerical manifold method (NMM) framework. Limitations in simplified S-R NMM are overcome using our redesigned resolution procedure. A new method has been applied to analyse beams with large deflections, block columns under compression, block sliding, rock falling, and semi-ring contact with block problems. Promising results from the analysis indicate that proposed method is more accurate and effective in theory and numerically than prior approaches when resolving contact problems and large deformations.

Optimization Over Families of Quasi-Periodic Orbits

Quasi-periodic orbit families in astrodynamics are usually studied from a global standpoint without much attention to the specific orbits which are computed. Instead, we focus on the computation of particular quasi-periodic orbits and develop tools to do so. These tools leverage the parametric structure of families of quasi-periodic orbits to treat orbits only as a set of orbit frequencies instead of states in phase space. We develop a retraction on the family of quasi-periodic orbits to precisely navigate through frequency space, allowing us to compute orbits with specific frequencies. The retraction allows for movements in arbitrary directions. To combat the effects of resonances which slice through frequency space we develop resonance avoidance methods which detect resonances during continuation procedures and change the step size accordingly. We also develop an augmented Newton’s method for root-finding and an augmented gradient descent method for unconstrained optimization over a family of quasi-periodic orbits. Lastly, we implement an augmented Lagrangian method to solve constrained optimization problems. We note that many of the tools developed here are applicable to a wider range of solutions defined implicitly by a system of equations, but focus on quasi-periodic orbits.

Layer‐parallel training of residual networks with auxiliary variable networks

AbstractGradient‐based methods for training residual networks (ResNets) typically require a forward pass of input data, followed by back‐propagating the error gradient to update model parameters, which becomes time‐consuming as the network structure goes deeper. To break the algorithmic locking and exploit synchronous module parallelism in both forward and backward modes, auxiliary‐variable methods have emerged but suffer from communication overhead and a lack of data augmentation. By trading off the recomputation and storage of auxiliary variables, a joint learning framework is proposed in this work for training realistic ResNets across multiple compute devices. Specifically, the input data of each processor is generated from its low‐capacity auxiliary network (AuxNet), which permits the use of data augmentation and realizes forward unlocking. Backward passes are then executed in parallel, each with a local loss function derived from the penalty or augmented Lagrangian (AL) method. Finally, the AuxNet is adjusted to reproduce updated auxiliary variables through an end‐to‐end training process. We demonstrate the effectiveness of our method on ResNets and WideResNets across CIFAR‐10, CIFAR‐100, and ImageNet datasets, achieving speedup over the traditional layer‐serial training approach while maintaining comparable testing accuracy.

Scaled-PAKKT sequential optimality condition for multiobjective problems and its application to an Augmented Lagrangian method

Scaled-PAKKT sequential optimality condition for multiobjective problems and its application to an Augmented Lagrangian method

An alternating direction multiplier method with variable neighborhood search for electric vehicle routing problem with time windows and battery swapping stations

This paper studies a real-world electric vehicle routing problem (EVRP). Specifically, it is an EVRP with time windows and battery swapping stations (EVRP_TWBSS). The EVRP_TWBSS considers the routing of electric vehicles (EVs), the determination of each electric vehicle’s battery level, and the selection of battery swapping stations. The criterion of EVRP_TWBSS is to minimize the operating costs. To simplify the structure of model, a time-discrete and multi-commodity flow model based on extended state-space-time network (TMFM_ESSTN) is established. Meanwhile, an alternating direction multiplier method with variable neighborhood search (ADMM_VNS) is presented to address the TMFM_ESSTN. In ADMM_VNS, the augmented lagrangian relaxation (ALR) model constructed from the TMFM_ESSTN is decomposed and linearized to a series of least cost vehicle routing subproblems through the linear augmented lagrangian relaxation (LALR) decomposed technique. Then, each subproblem is iteratively solved by using the dynamic programming and two special designed VNS strategies in ADMM_VNS iterative framework. The solution’s quality can be controlled to a certain extent through monitoring the gap between the lower and upper bounds obtained after each iteration. Test results on instances with different scales and a real-world instance based on partial road network in Kunming City demonstrate that ADMM_VNS can achieve smaller gaps and better solutions than several state-of-the-art algorithms. In which, ADMM_VNS can reduce the optimal gap by up to 2.27 % compared to the other state-of-the-art algorithms in small-scale instances. The gap of ADMM_VNS is calculated based on the lower bound and the upper bound in the large-scale instances and the real-world instance are 10.36 % and 1.57 %, respectively.

Enhanced Safety in Autonomous Driving: Integrating a Latent State Diffusion Model for End-to-End Navigation.

Ensuring safety in autonomous driving is crucial for effective motion planning and navigation. However, most end-to-end planning methodologies lack sufficient safety measures. This study tackles this issue by formulating the control optimization problem in autonomous driving as Constrained Markov Decision Processes (CMDPs). We introduce an innovative, model-based approach for policy optimization, employing a conditional Value-at-Risk (VaR)-based soft actor-critic (SAC) to handle constraints in complex, high-dimensional state spaces. Our method features a worst-case actor to ensure strict compliance with safety requirements, even in unpredictable scenarios. The policy optimization leverages the augmented Lagrangian method and leverages latent diffusion models to forecast and simulate future trajectories. This dual strategy ensures safe navigation through environments and enhances policy performance by incorporating distribution modeling to address environmental uncertainties. Empirical evaluations conducted in both simulated and real environments demonstrate that our approach surpasses existing methods in terms of safety, efficiency, and decision-making capabilities.

Distributed optimization for multi-commodity urban traffic control

A distributed method for concurrent traffic signal and routing control of traffic networks is proposed. The method is based on the multi-commodity store-and-forward model, in which the destinations are the commodities. The system benefits from the communication between vehicles and infrastructure, providing optimal signal timings to intersections and routes to vehicles on a link-by-link basis. Using the augmented Lagrangian to model the constraints into the objective, the baseline centralized problem is decomposed into a set of objective-coupled subproblems, one for each intersection, enabling the solution to be computed by a distributed-gradient projection algorithm. The intersection agents only need to communicate and coordinate with neighboring intersections to ensure convergence to the optimal solution while tolerating suboptimal iterations that offer more flexibility, unlike other distributed approaches. Through microsimulation, we demonstrate the effectiveness of the proposed algorithm in traffic networks with time-varying demand. Computational analysis shows that the distributed problem is suitable for real-time applications. A robustness analysis show that the distributed formulation enables a graceful degradation of the system in case of failure.

Stochastic Augmented Lagrangian Method in Riemannian Shape Manifolds

In this paper, we present a stochastic augmented Lagrangian approach on (possibly infinite-dimensional) Riemannian manifolds to solve stochastic optimization problems with a finite number of deterministic constraints. We investigate the convergence of the method, which is based on a stochastic approximation approach with random stopping combined with an iterative procedure for updating Lagrange multipliers. The algorithm is applied to a multi-shape optimization problem with geometric constraints and demonstrated numerically.

Efficient method for symmetric nonnegative matrix factorization with an approximate augmented Lagrangian scheme

In this paper, we propose an efficient method for solving symmetric nonnegative matrix factorization following an approximate augmented Lagrangian scheme. The augmented Lagrangian subproblem was solved column by column under the block coordinate descent (BCD) framework. In particular, we extend the recursive formula for rank-k nonnegative least squares problems by Chu et al. (2021) to subproblems generated by BCD framework in our method. Thereafter we derive the closed-form solution for k=4. Experiments for clustering show that the proposed method is noticeably efficient and achieves competitive performance compared with state-of-the-art methods.

Distributed virtual formation control for railway trains with nonlinear dynamics and collision avoidance constraints

To improve the model accuracy and control efficiency for the movements of a virtual formation, this paper investigates distributed optimal control for the virtual formation control system in railways. Adopting the relative distance braking mode, a coupled optimal control problem with nonlinear train dynamics and constraints regarding collision avoidance and jerk is formulated for the virtual formation. To handle the non-convex constrained problem efficiently, a distributed augmented Lagrangian based alternating direction inexact newton (ALADIN) method under the model predictive control (MPC) framework is developed. For the execution of the distributed computational process, the copied variables are introduced to reformulate the original coupled problem in an objective separable form. By exploiting the problem separability, the ALADIN method decomposes the reformulation into a coordinated quadratic programming problem of small-scale and several local nonlinear programming problems that can be calculated in parallel, thereby facilitating real-time control and relieving communication burden. Numerical experiments on a metro line are carried out to verify the effectiveness of the proposed model and method. Experimental results demonstrate that high-performance tracking control for virtually coupled train units can be achieved in real time.

Global convergence of dislocation hyperbolic augmented Lagrangian algorithm for nonconvex optimization

In this work, we present a new augmented Lagrangian type algorithm based on the dislocation hyperbolic augmented Lagrangian function (DHALF). This algorithm is called dislocation hyperbolic augmented Lagrangian algorithm (DHALA). We ensure that DHALA converges to a global solution for the inequality constrained nonconvex optimization problem. We present DHALA considering the safeguards technique and ensure the convergence of this algorithm. Finally, we present a computational experiment to show the performance of our proposed algorithm.

Convergence analysis of the augmented Lagrangian method for $$\ell _{p}$$-norm cone optimization problems with $$p \ge 2$$

Convergence analysis of the augmented Lagrangian method for $$\ell _{p}$$-norm cone optimization problems with $$p \ge 2$$