Augmented Lagrangian Research Articles

Dislocation hyperbolic augmented Lagrangian algorithm in convex programming

The dislocation hyperbolic augmented Lagrangian algorithm (DHALA) is a new approach to the hyperbolic augmented Lagrangian algorithm (HALA). DHALA is designed to solve convex nonlinear programming problems. We guarantee that the sequence generated by DHALA converges towards a Karush-Kuhn-Tucker point. We are going to observe that DHALA has a slight computational advantage in solving the problems over HALA. Finally, we will computationally illustrate our theoretical results.

Augmented Lagrangian Acceleration of Global-in-Time Pressure Schur Complement Solvers for Incompressible Oseen Equations

This work is focused on an accelerated global-in-time solution strategy for the Oseen equations, which highly exploits the augmented Lagrangian methodology to improve the convergence behavior of the Schur complement iteration. The main idea of the solution strategy is to block the individual linear systems of equations at each time step into a single all-at-once saddle point problem. By elimination of all velocity unknowns, the resulting implicitly defined equation can then be solved using a global-in-time pressure Schur complement (PSC) iteration. To accelerate the convergence behavior of this iterative scheme, the augmented Lagrangian approach is exploited by modifying the momentum equation for all time steps in a strongly consistent manner. While the introduced discrete grad-div stabilization does not modify the solution of the discretized Oseen equations, the quality of customized PSC preconditioners drastically improves and, hence, guarantees a rapid convergence. This strategy comes at the cost that the involved auxiliary problem for the velocity field becomes ill conditioned so that standard iterative solution strategies are no longer efficient. Therefore, a highly specialized multigrid solver based on modified intergrid transfer operators and an additive block preconditioner is extended to solution of the all-at-once problem. The potential of the proposed overall solution strategy is discussed in several numerical studies as they occur in commonly used linearization techniques for the incompressible Navier–Stokes equations.

Generalized latent multi-view clustering with tensorized bipartite graph

Tensor-based multi-view spectral clustering algorithms use tensors to model the structure of multi-dimensional data to take advantage of the complementary information and high-order correlations embedded in the graph, thus achieving impressive clustering performance. However, these algorithms use linear models to obtain consensus, which prevents the learned consensus from adequately representing the nonlinear structure of complex data. In order to address this issue, we propose a method called Generalized Latent Multi-View Clustering with Tensorized Bipartite Graph (GLMC-TBG). Specifically, in this paper we introduce neural networks to learn highly nonlinear mappings that encode nonlinear structures in graphs into latent representations. In addition, multiple views share the same latent consensus through nonlinear interactions. In this way, a more comprehensive common representation from multiple views can be achieved. An Augmented Lagrangian Multiplier with Alternating Direction Minimization (ALM-ADM) framework is designed to optimize the model. Experiments on seven real-world data sets verify that the proposed algorithm is superior to state-of-the-art algorithms.

A new inexact gradient descent method with applications to nonsmooth convex optimization

The paper proposes and develops a novel inexact gradient method (IGD) for minimizing C 1 -smooth functions with Lipschitzian gradients, i.e. for problems of C 1 , 1 optimization. We show that the sequence of gradients generated by IGD converges to zero. The convergence of iterates to stationary points is guaranteed under the Kurdyka-Łojasiewicz (KL) property of the objective function with convergence rates depending on the KL exponent. The newly developed IGD is applied to designing two novel gradient-based methods of nonsmooth convex optimization such as the inexact proximal point methods (GIPPM) and the inexact augmented Lagrangian method (GIALM) for convex programs with linear equality constraints. These two methods inherit global convergence properties from IGD and are confirmed by numerical experiments to have practical advantages over some well-known algorithms of nonsmooth convex optimization.

An Effective Augmented Lagrangian Method for Fine-Grained Multi-View Optimization

The significance of multi-view learning in effectively mitigating the intricate intricacies entrenched within heterogeneous data has garnered substantial attention in recent years. Notwithstanding the favorable achievements showcased by recent strides in this area, a confluence of noteworthy challenges endures. To be specific, a majority of extant methodologies unceremoniously assign weights to data points view-wisely. This ineluctably disregards the intrinsic reality that disparate views confer diverse contributions to each individual sample, consequently neglecting the rich wellspring of sample-level structural insights harbored within the dataset. In this paper, we proposed an effective Augmented Lagrangian MethOd for fiNe-graineD (ALMOND) multi-view optimization. This innovative approach scrutinizes the interplay among multiple views at the granularity of individual samples, thereby fostering the enhanced preservation of local structural coherence. The Augmented Lagrangian Method (ALM) is elaborately incorporated into our framework, which enables us to achieve an optimal solution without involving an inexplicable intermediate variable as previous methods do. Empirical experiments on multi-view clustering tasks across heterogeneous datasets serve to incontrovertibly showcase the effectiveness of our proposed methodology, corroborating its preeminence over incumbent state-of-the-art alternatives.

Local regularization assisted split augmented Lagrangian shrinkage algorithm for feature selection in condition monitoring

Feature selection plays a vital role in improving the efficiency and accuracy of condition monitoring by constructing sparse but effective models. In this study, an advanced feature selection algorithm named the local regularization assisted split augmented Lagrangian shrinkage algorithm (LR-SALSA) is proposed. The feature selection is realized by solving a l1-norm optimization problem, which usually selects more sparse and representative features at less computational costs. The proposed algorithm operates in two stages, namely variable selection and coefficient estimation. In the stage of variable selection, the primal problem is converted into three subproblems which can be solved separately. Then individual penalty parameters are applied to every coefficient of the model when dealing with the first subproblem. Under the Bayesian evidence framework, an iterative algorithm is derived to optimize these hyperparameters. During the optimization process, redundant variables will be pruned to guarantee model sparsity and improve computational efficiency at the same time. In the second stage, the coefficients for the selected model terms are determined using the least squares technique. The superior performance and efficiency of the proposed LR-SALSA method are validated through two numerical examples and a real-world cutting tool wear prediction case study. Compared with the existing methods, the proposed method can generate a sparse model and ensure a good trade-off between estimation accuracy and computational efficiency.

An improved alternating direction method of multipliers for solving deterministic user equilibrium

An improved alternating direction method of multipliers (iADMM) algorithm is proposed for solving the deterministic user equilibrium (DUE) problem. The iADMM algorithm introduces a key improvement wherein the primal variables can be updated sequentially, possibly multiple times, before updating the dual variables. Subsequently, a sensitivity analysis method is applied to optimise the update frequency of primal variable, and two indices are introduced to evaluate the performance of various policies. Additionally, eliminating equality constraints through the augmented Lagrangian function makes it challenging to completely satisfy flow conservation conditions, resulting in the presence of primal residual. To address this issue, two schemes are proposed to handle the primal residual, involving modifications to the primal variables based on iterative messages. Numerical experiments highlight that the convergence of ADMM algorithm is significantly influenced by the update frequency of primal variable, and addressing the primal residual after each cycle enhances the convergence of ADMM algorithm.

Robust contact-constrained topology optimization considering uncertainty at the contact support

In this paper, the general framework for contact-constrained topology optimization of Strömberg and Klarbring (2010) is extended to robust topology optimization. In doing so, a linear elastic design domain is considered and the augmented Lagrangian approach is used to model the unilateral contact. For topology optimization, the design space is parametrized with the SIMP-approach and the Sigmund’s filter is applied. Additionally, the robust framework considers uncertainties at the contact support such as deviations of the geometry of the contact surface and the friction coefficient. Both uncertainties are described by the first-order second-moment method which leads to minimal additional costs. In fact, only two additional linear equations must be solved to obtain the robust objective and its gradient with respect to the design variables. Having both the objective and the gradient, the design update is computed with the method of moving asymptotes. The robust framework is applied to 2D and 3D examples to prove its scalability for real-world applications.

$$O(1/k^2)$$ convergence rates of (dual-primal) balanced augmented Lagrangian methods for linearly constrained convex programming

$$O(1/k^2)$$ convergence rates of (dual-primal) balanced augmented Lagrangian methods for linearly constrained convex programming

Solution of the Simultaneous Routing and Bandwidth Allocation Problem in Energy-Aware Networks Using Augmented Lagrangian-Based Algorithms and Decomposition

We discuss several algorithms for solving a network optimization problem of simultaneous routing and bandwidth allocation in green networks in a decomposed way, based on the augmented Lagrangian. The problem is difficult due to the nonconvexity caused by binary routing variables. The chosen algorithms, which are several versions of the Multiplier Method, including the Alternating Direction Method of Multipliers (ADMM), have been implemented in Python and tested on several networks’ data. We derive theoretical formulations for the inequality constraints of the Bertsekas, Tatjewski and SALA methods, formulated originally for problems with equality constraints. We also introduce some modifications to the Bertsekas and Tatjewski methods, without which they do not work for an MINLP problem. The final comparison of the performance of these algorithms shows a significant advantage of the augmented Lagrangian algorithms, using decomposition for big problems. In our particular case of the simultaneous routing and bandwidth allocation problem, these algorithms seem to be the best choice.

Variational sparse diffusion and its application in mesh processing

PurposeThe primary objective of this study is to tackle the enduring challenge of preserving feature integrity during the manipulation of geometric data in computer graphics. Our work aims to introduce and validate a variational sparse diffusion model that enhances the capability to maintain the definition of sharp features within meshes throughout complex processing tasks such as segmentation and repair.Design/methodology/approachWe developed a variational sparse diffusion model that integrates a high-order L1 regularization framework with Dirichlet boundary constraints, specifically designed to preserve edge definition. This model employs an innovative vertex updating strategy that optimizes the quality of mesh repairs. We leverage the augmented Lagrangian method to address the computational challenges inherent in this approach, enabling effective management of the trade-off between diffusion strength and feature preservation. Our methodology involves a detailed analysis of segmentation and repair processes, focusing on maintaining the acuity of features on triangulated surfaces.FindingsOur findings indicate that the proposed variational sparse diffusion model significantly outperforms traditional smooth diffusion methods in preserving sharp features during mesh processing. The model ensures the delineation of clear boundaries in mesh segmentation and achieves high-fidelity restoration of deteriorated meshes in repair tasks. The innovative vertex updating strategy within the model contributes to enhanced mesh quality post-repair. Empirical evaluations demonstrate that our approach maintains the integrity of original, sharp features more effectively, especially in complex geometries with intricate detail.Originality/valueThe originality of this research lies in the novel application of a high-order L1 regularization framework to the field of mesh processing, a method not conventionally applied in this context. The value of our work is in providing a robust solution to the problem of feature degradation during the mesh manipulation process. Our model’s unique vertex updating strategy and the use of the augmented Lagrangian method for optimization are distinctive contributions that enhance the state-of-the-art in geometry processing. The empirical success of our model in preserving features during mesh segmentation and repair presents an advancement in computer graphics, offering practical benefits to both academic research and industry applications.

Incorporating STATCOM into a Hydro-Thermal Optimal Power Flow Algorithm

This paper presents the results of the inclusion of Synchronous Static Compensator (STATCOM) power flow models into a hydro-thermal optimal power flow (HTOPF) algorithm. STATCOM basically introduces the voltage magnitude and phase angle of the source converter into the algorithm. For each incorporated STATCOM, an augmented Lagrangian function was formed. The first and second derivatives of these functions were added to the gradient vector and Hessian matrix of an existing algorithm. The modified algorithm was implemented using MATLAB R2018a and tested on 30 and 57 bus systems. Two and three STATCOMs were, respectively, tested on 30 and 57 bus systems. The results obtained showed that one of the STATCOMs used on 30-bus system injected reactive power that ranges from 8.99 MVAR to 28.22 MVAR while the other one injected reactive power in the range of 8.20 MVAR to 33.20 MVAR. For the STATCOMs placed on the 57-bus system, the range of reactive power absorption by the one placed at bus 5 is 7.83 MVAR to 22.86 MVAR while the one at bus 55 absorbed from 5.06 MVAR to 12.92 MVAR. The STATCOM’s reactive power injection at bus 31 ranges from 4.63 MVAR to 10.35 MVAR. All the lower and higher values were obtained at hours 3 and 7, respectively. While the STATCOMs significantly improved the systems’ voltage profile, the impacts of STATCOM on the total systems daily energy loss, daily energy generations (from both plants), daily fuel cost and hydro plant water worth are insignificant.

QAL-BP: an augmented Lagrangian quantum approach for bin packing

The bin packing is a well-known NP-Hard problem in the domain of artificial intelligence, posing significant challenges in finding efficient solutions. Conversely, recent advancements in quantum technologies have shown promising potential for achieving substantial computational speedup, particularly in certain problem classes, such as combinatorial optimization. In this study, we introduce QAL-BP, a novel Quadratic Unconstrained Binary Optimization (QUBO) formulation designed specifically for bin packing and suitable for quantum computation. QAL-BP utilizes the Augmented Lagrangian method to incorporate the bin packing constraints into the objective function while also facilitating an analytical estimation of heuristic, but empirically robust, penalty multipliers. This approach leads to a more versatile and generalizable model that eliminates the need for empirically calculating instance-dependent Lagrangian coefficients, a requirement commonly encountered in alternative QUBO formulations for similar problems. To assess the effectiveness of our proposed approach, we conduct experiments on a set of bin packing instances using a real Quantum Annealing device. Additionally, we compare the results with those obtained from two different classical solvers, namely simulated annealing and Gurobi. The experimental findings not only confirm the correctness of the proposed formulation, but also demonstrate the potential of quantum computation in effectively solving the bin packing problem, particularly as more reliable quantum technology becomes available.

How-to Augmented Lagrangian on Factor Graphs

How-to Augmented Lagrangian on Factor Graphs

Discriminative elastic-net broad learning systems for visual classification

The broad learning system (BLS) has garnered significant attention in the realm of visual classification due to its exceptional balance between accuracy and efficiency. However, the supervision mechanism in BLS typically relies on strict binary labels, limiting the approximation freedom and failing to represent the data distribution adequately. Furthermore, the inadequacy of the guidance mechanism for the output weights hinders the precise approximation of the target spaces by the input features. To tackle these issues, in this paper, we propose two discriminative elastic-net regularized BLS models with label enhancement, achieving the following significant objectives: Firstly, the proposed label enhancement technologies can substantially augment the margins between different classes while enhancing the diversity within label spaces. Secondly, the compactness and effectiveness of the output weights can be further improved via the guidance of elastic-net regularization of singular values. Thirdly, our proposed algorithms can be efficiently optimized with the augmented Lagrangian method, whose convergence and calculation complexity can be guaranteed well with solid theoretical analysis. Extensive experiments are intended on various popular databases to compare our proposed models with numerous other state-of-the-art recognition algorithms. The numerical results indicate that our proposed algorithms can achieve the best face recognition accuracy of 99.67%, and 97.54% for object recognition. Even in challenging recognition tasks, our methods can still yield an average improvement of 0.8 percentage points.

ADMM-TGV image restoration for scientific applications with unbiased parameter choice

AbstractImage restoration via alternating direction method of multipliers (ADMM) has gained large interest within the last decade. Solving standard problems of Gaussian and Poisson noise, the set of “Total Variation” (TV)-based regularizers proved to be efficient and versatile. In the last few years, the “Total Generalized Variation” (TGV) approach combined TV regularizers of different orders adaptively to better suit local regions in the image. This improved the technique significantly. The approach solved the staircase problem inherent of the first-order TV while keeping the beneficial edge preservation. The iterative minimization for the augmented Lagrangian of TGV problems requires four important parameters: two penalty parameters $${\rho }$$ ρ and $${\eta }$$ η and two regularization parameters $${\lambda _{0}}$$ λ 0 and $${\lambda _{1}}$$ λ 1 . The choice of penalty parameters decides on the convergence speed, and the regularization parameters decide on the impact of the respective regularizer and are determined by the noise level in the image. For scientific applications of such algorithms, an automated and thus objective method to determine these parameters is essential to receive unbiased results independent of the user. Obviously, both sets of parameters are to be well chosen to achieve optimal results, too. In this paper, a method is proposed to adaptively choose optimal $${\rho }$$ ρ and $${\eta }$$ η values for the iteration to converge faster, based on the primal and dual residuals arising from the optimality conditions of the augmented Lagrangian. Further, we show how to choose $${\lambda _{0}}$$ λ 0 and $${\lambda _{1}}$$ λ 1 based on the inherent noise in the image.

Inexact augmented Lagrangian method-based full-waveform inversion with randomized singular value decomposition

Abstract The main advantage of full-waveform inversion (FWI) is the ability to obtain useful subsurface structure information, such as velocity and density, from complex seismic data. We have developed a novel inversion algorithm to improve the capability of FWI to achieve high-resolution imaging, even under complex conditions caused by random noise contamination, initial model dependence, or the selection of parameters to be estimated. Our algorithm considers an effective image processing and dimension reduction tool, randomized singular value decomposition-weighted truncated nuclear norm regularization, for embedding FWI to achieve high-resolution imaging results. This algorithm obtains a truncated matrix approximating the original matrix by reducing the rank of the velocity increment matrix, thus achieving the truncation of noisy data, with the truncation range controlled by weighted truncated nuclear norm regularization. Subsequently, we employ an inexact augmented Lagrangian method algorithm in the optimization to compress the solution space range, thus relaxing the dependence of FWI and randomized singular value decomposition-weighted truncated nuclear norm regularization on the initial model and accelerating the convergence rate of the objective function. We tested on one set of synthetic data, and the results show that, compared with traditional FWI, our method can more effectively suppress the impact of random noise, thus obtaining higher resolution and more accurate subsurface model information. This work indicates that the combination of randomized singular value decomposition-weighted truncated nuclear norm regularization and FWI is an effective imaging strategy that can help to solve the challenges faced by traditional FWI.

Robust Ranking Kernel Support Vector Machine via Manifold Regularized Matrix Factorization for Multi-Label Classification

Multi-label classification has been extensively researched and utilized for several decades. However, the performance of these methods is highly susceptible to the presence of noisy data samples, resulting in a significant decrease in accuracy when noise levels are high. To address this issue, we propose a robust ranking support vector machine (Rank-SVM) method that incorporates manifold regularized matrix factorization. Unlike traditional Rank-SVM methods, our approach integrates feature selection and multi-label learning into a unified framework. Within this framework, we employ matrix factorization to learn a low-rank robust subspace within the input space, thereby enhancing the robustness of data representation in high-noise conditions. Additionally, we incorporate manifold structure regularization into the framework to preserve manifold relationships among low-rank samples, which further improves the robustness of the low-rank representation. Leveraging on this robust low-rank representation, we extract a resilient low-rank features and employ them to construct a more effective classifier. Finally, the proposed framework is extended to derive a kernelized ranking approach, for the creation of nonlinear multi-label classifiers. To effectively solve this non-convex kernelized method, we employ the augmented Lagrangian multiplier (ALM) and alternating direction method of multipliers (ADMM) techniques to obtain the optimal solution. Experimental evaluations conducted on various datasets demonstrate that our framework achieves superior classification results and significantly enhances performance in high-noise scenarios.

Inexact Exponential Penalty Function with the Augmented Lagrangian for Multiobjective Optimization Algorithms

This paper uses an augmented Lagrangian method based on an inexact exponential penalty function to solve constrained multiobjective optimization problems. Two algorithms have been proposed in this study. The first algorithm uses a projected gradient, while the second uses the steepest descent method. By these algorithms, we have been able to generate a set of nondominated points that approximate the Pareto optimal solutions of the initial problem. Some proofs of theoretical convergence are also proposed for two different criteria for the set of generated stationary Pareto points. In addition, we compared our method with the NSGA-II and augmented the Lagrangian cone method on some test problems from the literature. A numerical analysis of the obtained solutions indicates that our method is competitive with regard to the test problems used for the comparison.

A Customized Augmented Lagrangian Method for Block-Structured Integer Programming.

Integer programming with block structures has received considerable attention recently and is widely used in many practical applications such as train timetabling and vehicle routing problems. It is known to be NP-hard due to the presence of integer variables. We define a novel augmented Lagrangian function by directly penalizing the inequality constraints and establish the strong duality between the primal problem and the augmented Lagrangian dual problem. Then, a customized augmented Lagrangian method is proposed to address the block-structures. In particular, the minimization of the augmented Lagrangian function is decomposed into multiple subproblems by decoupling the linking constraints and these subproblems can be efficiently solved using the block coordinate descent method. We also establish the convergence property of the proposed method. To make the algorithm more practical, we further introduce several refinement techniques to identify high-quality feasible solutions. Numerical experiments on a few interesting scenarios show that our proposed algorithm often achieves a satisfactory solution and is quite effective.