Augmented Lagrange Multiplier Method Research Articles

${L_{1/2}}$ Norm and Spatial Continuity Regularized Low-Rank Approximation for Moving Object Detection in Dynamic Background

Low-rank modeling-based moving object detection approaches proposed so far use fixed ${{\boldsymbol{l}}_1}$ -norm penalty to capture the sparse nature of foreground in video, and thus, hardly adapt readily to the statistical variability of underlying foreground pixels in dynamic background. Additionally, they ignore the spatial continuity prior among the neighbor foreground pixels. Consequently, they cannot offer a satisfactory performance in practical dynamic background. In this letter, we present a unified regularization framework, namely ${{\boldsymbol{l}}_{1/2}}$ -norm and spatial continuity regularized low-rank approximation (SCLR- ${{\boldsymbol{l}}_{1/2}}$ ), to solve this problem. First, in order to promote accuracy, we introduce an ${{\boldsymbol{l}}_{1/2}}$ constraint into the framework. Second, to guarantee the continuity among the neighbor foreground pixels, we introduce a spatial continuity regularization term, motivated by total variation. Finally, we generalize our framework to the ${{\boldsymbol{l}}_{\boldsymbol{q}}}$ -norm penalized case (SCLR- ${{\boldsymbol{l}}_{\boldsymbol{q}}}$ ). By adjusting the shrinkage parameter ${\boldsymbol{q}}$ , the framework gets better flexibility to choose a reasonable sparse domain. To deal with the present constrained minimization problem, the augmented Lagrange multiplier method is employed and extended with the help of the alternating direction minimizing strategy. Experimental results show that the proposed method outperforms some state-of-the-art algorithms especially for the cases with dynamic backgrounds.

Unified Power Allocation for Receive Spatial Modulation Based on Approximate Optimization

In this paper, a novel unified power allocation (PA) framework is proposed for receive (pre-coding aided) spatial modulation (RSM). We find that the PA matrix design can be formulated as a non-convex quadratically constrained quadratic program (QCQP) problem, whose solution is generally intractable. To tackle this problem, we propose a pair of solvers having different trade-offs in terms of bit-error-rate and complexity. Specifically, we first propose a successive convex approximation (SCA) method, to convert the non-convex QCQP problem under consideration into a series of linear convex subproblems, where the latter can be easily solved by the classic polynomial-time-based optimization method, i.e., the interior point method. To further reduce the computational complexity, we propose an augmented Lagrangian multiplier (ALM) method, which transforms the challenging non-convex constrained PA optimization problem into its unconstrained counterpart, which can be efficiently solved by an iterative manner. Our simulation results show that both the proposed SCA and ALM methods are capable of substantially improving the system error performance compared with the conventional RSM system without PA as well as conventional PA-aided RSM schemes.

Small target detection based on reweighted infrared patch‐image model

To further improve the effect of infrared small target detection, a reweighted infrared patch-image model is proposed. First, the authors point out that the nuclear norm in the infrared patch-image model could easily leave some sparse background edges in the target patch-image, leading to an inaccurate background estimation. Then, to overcome this defect, the reweighted nuclear norm is adopted to constrain the background patch-image, which could preserve the background edges better. Considering that some non-target sparse points could not be suppressed by only using l1 norm, the authors introduce the reweighted l1 norm to further enhance the sparsity of target image. Finally, the proposed model is formulated as a reweighted robust principal component analysis problem and solved by the inexact augmented Lagrangian multiplier method. Extensive experiments show that the proposed model outperforms the other six competitive methods in suppressing background clutter and detecting target.

Truss Structure Optimization with Subset Simulation and Augmented Lagrangian Multiplier Method

This paper presents a global optimization method for structural design optimization, which integrates subset simulation optimization (SSO) and the dynamic augmented Lagrangian multiplier method (DALMM). The proposed method formulates the structural design optimization as a series of unconstrained optimization sub-problems using DALMM and makes use of SSO to find the global optimum. The combined strategy guarantees that the proposed method can automatically detect active constraints and provide global optimal solutions with finite penalty parameters. The accuracy and robustness of the proposed method are demonstrated by four classical truss sizing problems. The results are compared with those reported in the literature, and show a remarkable statistical performance based on 30 independent runs.

Outlier Suppression via Non-Convex Robust PCA for Efficient Localization in Wireless Sensor Networks

Due to the fact that outliers can degrade localization accuracy significantly in wireless sensor networks, we propose an outlier suppression approach via non-convex robust principal component analysis (Robust PCA). By introducing several non-convex penalty functions to approximate both the rank function and the sparse penalty function in the original Robust PCA problem, we establish a non-convex objective function and solve it efficiently by the augmented Lagrangian multiplier method and difference of convex programming technique. This framework creates the opportunity to safely perform localization with dimension reduction by standard PCA, which has well-known fragile characteristics in the presence of outliers. Thus, the localization algorithms obtained by sparse coding provide the benefits of computational efficiency in low-dimension. The device-free localization (DFL) experiment on a real-world data set shows the improvement in localization accuracy after standard PCA in the presence of outliers. A statistical distance, called the Wasserstein distance, is introduced to evaluate the processed result of outlier suppression, illustrating that the proposed approach can identify and eliminate outliers in DFL problems.

Topology Synthesis and Optimal Design of an Adaptive Compliant Gripper to Maximize Output Displacement

This paper presents the optimal design process of an innovative adaptive compliant gripper (ACG) for fast handling of objects with size and shape variations. An efficient soft-add topology optimization algorithm is developed to synthesize the optimal topology of the ACG. Unlike traditional hard-kill and soft-kill methods, the elements are equivalent to be numerically added into the analysis domain through the proposed soft-add scheme. A size optimization method incorporating Augmented Lagrange Multiplier (ALM) method, Simplex method, and nonlinear finite element analysis with the objective to maximize geometric advantage (which is defined as the ratio of output displacement to input displacement) of the analyzed compliant mechanism is carried out to optimize the design. The dynamic performance and contact behavior of the ACG is analyzed by using explicit dynamic finite element analysis. Three designs are prototyped using silicon rubber material. Experimental tests are performed, and the results agree well with the simulation models. The outcomes of this study provide numerical methods for design and analysis of compliant mechanisms with better computational efficiency, as well as to develop an innovative adaptive compliant gripper for fast grasping of unknown objects.

Data‐Driven Sparse Priors of 3D Shapes

AbstractWe present a sparse optimization framework for extracting sparse shape priors from a collection of 3D models. Shape priors are defined as point‐set neighborhoods sampled from shape surfaces which convey important information encompassing normals and local shape characterization. A 3D shape model can be considered to be formed with a set of 3D local shape priors, while most of them are likely to have similar geometry. Our key observation is that the local priors extracted from a family of 3D shapes lie in a very low‐dimensional manifold. Consequently, a compact and informative subset of priors can be learned to efficiently encode all shapes of the same family. A comprehensive library of local shape priors is first built with the given collection of 3D models of the same family. We then formulate a global, sparse optimization problem which enforces selecting representative priors while minimizing the reconstruction error. To solve the optimization problem, we design an efficient solver based on the Augmented Lagrangian Multipliers method (ALM). Extensive experiments exhibit the power of our data‐driven sparse priors in elegantly solving several high‐level shape analysis applications and geometry processing tasks, such as shape retrieval, style analysis and symmetry detection.

Consensus Regularized Multi-View Outlier Detection.

Identifying different types of data outliers with abnormal behaviors in multi-view data setting is challenging due to the complicated data distributions across different views. Conventional approaches achieve this by learning a new latent feature representation with the pairwise constraint on different view data. In this paper, we argue that the existing methods are expensive in generalizing their models from two-view data to three-view (or more) data, in terms of the number of introduced variables and detection performance. To address this, we propose a novel multi-view outlier detection method with consensus regularization on the latent representations. Specifically, we explicitly characterize each kind of outliers by the intrinsic cluster assignment labels and sample-specific errors. Moreover, we make a thorough discussion about the proposed consensus-regularization and the pairwise-regularization. Correspondingly, an optimization solution based on augmented Lagrangian multiplier method is proposed and derived in details. In the experiments, we evaluate our method on five well-known machine learning data sets with different outlier settings. Further, to show its effectiveness in real-world computer vision scenario, we tailor our proposed model to saliency detection and face reconstruction applications. The extensive results of both standard multi-view outlier detection task and the extended computer vision tasks demonstrate the effectiveness of our proposed method.

Denoising of Hyperspectral Images Using Nonconvex Low Rank Matrix Approximation

Hyperspectral image (HSI) denoising is challenging not only because of the difficulty in preserving both spectral and spatial structures simultaneously, but also due to the requirement of removing various noises, which are often mixed together. In this paper, we present a nonconvex low rank matrix approximation (NonLRMA) model and the corresponding HSI denoising method by reformulating the approximation problem using nonconvex regularizer instead of the traditional nuclear norm, resulting in a tighter approximation of the original sparsity-regularised rank function. NonLRMA aims to decompose the degraded HSI, represented in the form of a matrix, into a low rank component and a sparse term with a more robust and less biased formulation. In addition, we develop an iterative algorithm based on the augmented Lagrangian multipliers method and derive the closed-form solution of the resulting subproblems benefiting from the special property of the nonconvex surrogate function. We prove that our iterative optimization converges easily. Extensive experiments on both simulated and real HSIs indicate that our approach can not only suppress noise in both severely and slightly noised bands but also preserve large-scale image structures and small-scale details well. Comparisons against state-of-the-art LRMA-based HSI denoising approaches show our superior performance.

Two modified augmented Lagrange multiplier algorithms for Toeplitz matrix compressive recovery

In this paper, we propose two modified augmented Lagrange multiplier algorithms by mean-value for Toeplitz matrix compressive recovery. In the algorithms, the mean-value modification makes the iteration matrices keep the Toeplitz structure which contributes to reduce the SVD time and CPU time. Numerical experiments show that the proposed algorithms achieve better precision than the augmented Lagrange multiplier method, especially when the matrix E is less sparse. Convergence analysis of the proposed algorithms is also given in detail.

Compressed sensing reconstruction of hyperspectral images jointly using spatial smoothing feature and spectral correlation

A compressed sensing (CS) reconstruction algorithm of hyperspectral images jointly using spatial and spectral characteristics is considered. Specifically, in the sampling process, each band image is sampled by the block CS method independently. In the reconstruction process, how to utilize the spatial smoothing feature of each band image and spectral correlation between different band images to formulate the joint optimization problem is the focus of this paper. The total variation (TV) norm and multihypothesis prediction are introduced to express the spatial smoothing feature and the spectral correlation, respectively. Thus, the TV norm and the prediction residual are used as the regularization items in the reconstruction optimization problem. The resulting ill‐posed problem is solved by the augmented Lagrange multiplier method and alternating direction method in an iterative way, and the implementation process of the reconstruction algorithm is presented. Experimental results on four hyperspectral datasets reveal that the proposed algorithm significantly outperforms alternative strategies in terms of peak signal‐to‐noise ratio as well as visual quality. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

An orthogonal parallel symbiotic organism search algorithm embodied with augmented Lagrange multiplier for solving constrained optimization problems

Many practical engineering design problems need constrained optimization. The literature reports several meta-heuristic algorithms have been applied to solve constrained optimization problems. In many cases, the algorithms fail due to violation of constraints. Recently in 2014, a new meta-heuristic algorithm known as symbiotic organism search (SOS) is reported by Cheng and Prayogo. It is inspired by the natural phenomenon of interaction between organisms in an ecosystem which help them to survive and grow. In this paper, the SOS algorithm is combined with augmented Lagrange multiplier (ALM) method to solve the constrained optimization problems. The ALM is accurate and effective as the constraints in this case do not have the power to restrict the search space or search direction. The orthogonal array strategies have gained popularity among the meta-heuristic researchers due to its potentiality to enhance the exploitation process of the algorithms. Simultaneously, researchers are also looking at designing parallel version of the meta-heuristics to reduce the computational burden. In order to enhance the performance, an Orthogonal Parallel SOS (OPSOS) is developed. The OPSOS along with ALM method is a suitable combination which is used here to solve twelve benchmark nonlinear constrained problems and four engineering design problems. Simulation study reveals that the proposed approach has almost similar accuracy with lower run time than ALM with Orthogonal SOS. Comparative analysis also establish superior performance over ALM with orthogonal colliding bodies optimization, modified artificial bee colony, augmented Lagrangian-based particle swarm optimization and Penalty function-based genetic algorithm.

An improved artificial bee colony with modified augmented Lagrangian for constrained optimization

Artificial bee colony (ABC) algorithm has been successfully applied to solve constrained optimization problems (COPs). However, it is noteworthy that when using ABC to deal with COPs, the commonly used constraint-handling technique is the Deb’s feasibility-based rules. To our limited knowledge, the present ABC and its variants with augmented Lagrangian (AL) multiplier method have not been found applications to the COPs. In this paper, a novel constrained optimization method, named IABC-MAL, which integrates the benefit of the improved ABC (IABC) algorithm capability for obtaining the global optimum with the modified AL (MAL) method to handle constraints. This paper presents the first effort to integrate ABC algorithm with the AL method. To verify the performance of the proposed IABC-MAL, 24 well-known benchmark test problems at CEC2006, 18 benchmark test problems at CEC2010, and 5 engineering design problems are employed. Experiment results demonstrate that the proposed IABC-MAL algorithm shows better performance in comparison with other state-of-the-art algorithms from the literature.

An Improved Algorithm to Enhance Recovery Stability for Low-Rank Image

When recovering original image for low-rank image with noise, the effect usually has not been satisfactory through minimizing matrix nuclear norm to obtain low-rank resolution. Aiming at this problem, we introduced Frobenius norm of low-rank matrix, and combined with original low-rank matrix nuclear nor to form new regular item, and utilized an augmented lagrange multiplier method to resolve the problem after convex relaxation. Through adding 2 F A item on the base of original low-rank image recovery model, we can get better denoising result for low-rank image. The experimental results indicated that, by selecting proper parameters, the improved algorithm has superior recovery result compared to traditional LRMR model on wiping off impulse noise and gaussian noise.

Robust subspace clustering via joint weighted Schatten- p norm and Lq norm minimization

Low-rank representation (LRR) has been successfully applied to subspace clustering. However, the nuclear norm in the standard LRR is not optimal for approximating the rank function in many real-world applications. Meanwhile, the L21 norm in LRR also fails to characterize various noises properly. To address the above issues, we propose an improved LRR method, which achieves low rank property via the new formulation with weighted Schatten-p norm and Lq norm (WSPQ). Specifically, the nuclear norm is generalized to be the Schatten-p norm and different weights are assigned to the singular values, and thus it can approximate the rank function more accurately. In addition, Lq norm is further incorporated into WSPQ to model different noises and improve the robustness. An efficient algorithm based on the inexact augmented Lagrange multiplier method is designed for the formulated problem. Extensive experiments on face clustering and motion segmentation clearly demonstrate the superiority of the proposed WSPQ over several state-of-the-art methods.

CSOLNP: Numerical Optimization Engine for Solving Non-linearly Constrained Problems.

We introduce the optimizer CSOLNP, which is a C++ implementation of the R package RSOLNP (Ghalanos & Theussl, 2012, Rsolnp: General non-linear optimization using augmented Lagrange multiplier method. R package version, 1) alongside some improvements. CSOLNP solves non-linearly constrained optimization problems using a Sequential Quadratic Programming (SQP) algorithm. CSOLNP, NPSOL (a very popular implementation of SQP method in FORTRAN (Gill et al., 1986, User's guide for NPSOL (version 4.0): A Fortran package for nonlinear programming (No. SOL-86-2). Stanford, CA: Stanford University Systems Optimization Laboratory), and SLSQP (another SQP implementation available as part of the NLOPT collection (Johnson, 2014, The NLopt nonlinear-optimization package. Retrieved from http://ab-initio.mit.edu/nlopt)) are three optimizers available in OpenMx package. These optimizers are compared in terms of runtimes, final objective values, and memory consumption. A Monte Carlo analysis of the performance of the optimizers was performed on ordinal and continuous models with five variables and one or two factors. While the relative difference between the objective values is less than 0.5%, CSOLNP is in general faster than NPSOL and SLSQP for ordinal analysis. As for continuous data, none of the optimizers performs consistently faster than the others. In terms of memory usage, we used Valgrind's heap profiler tool, called Massif, on one-factor threshold models. CSOLNP and NPSOL consume the same amount of memory, while SLSQP uses 71 MB more memory than the other two optimizers.

Rigid-flexible coupling dynamic modeling and investigation of a redundantly actuated parallel manipulator with multiple actuation modes

A systematic dynamic modeling methodology is presented to develop the rigid-flexible coupling dynamic model (RFDM) of an emerging flexible parallel manipulator with multiple actuation modes. By virtue of assumed mode method, the general dynamic model of an arbitrary flexible body with any number of lumped parameters is derived in an explicit closed form, which possesses the modular characteristic. Then the completely dynamic model of system is formulated based on the flexible multi-body dynamics (FMD) theory and the augmented Lagrangian multipliers method. An approach of combining the Udwadia-Kalaba formulation with the hybrid TR-BDF2 numerical algorithm is proposed to address the nonlinear RFDM. Two simulation cases are performed to investigate the dynamic performance of the manipulator with different actuation modes. The results indicate that the redundant actuation modes can effectively attenuate vibration and guarantee higher dynamic performance compared to the traditional non-redundant actuation modes. Finally, a virtual prototype model is developed to demonstrate the validity of the presented RFDM. The systematic methodology proposed in this study can be conveniently extended for the dynamic modeling and controller design of other planar flexible parallel manipulators, especially the emerging ones with multiple actuation modes.

Sparse Low-Rank Matrix Approximation for Data Compression

Low-rank matrix approximation (LRMA) is a powerful technique for signal processing and pattern analysis. However, its potential for data compression has not yet been fully investigated. In this paper, we propose sparse LRMA (SLRMA), an effective computational tool for data compression. SLRMA extends conventional LRMA by exploring both the intra and inter coherence of data samples simultaneously. With the aid of prescribed orthogonal transforms (e.g., discrete cosine/wavelet transform and graph transform), SLRMA decomposes a matrix into a product of two smaller matrices, where one matrix is made up of extremely sparse and orthogonal column vectors and the other consists of the transform coefficients. Technically, we formulate SLRMA as a constrained optimization problem, i.e., minimizing the approximation error in the least-squares sense regularized by the $\ell _{0}$ -norm and orthogonality, and solve it using the inexact augmented Lagrangian multiplier method. Through extensive tests on real-world data, such as 2D image sets and 3D dynamic meshes, we observe that: 1) SLRMA empirically converges well; 2) SLRMA can produce approximation error comparable to LRMA but in a much sparse form; and 3) SLRMA-based compression schemes significantly outperform the state of the art in terms of rate-distortion performance.

Augmented Lagrangian alternating direction method for low-rank minimization via non-convex approximation

This paper concerns the low-rank minimization problems which consist of finding a matrix of minimum rank subject to linear constraints. Many existing approaches, which used the nuclear norm as a convex surrogate of the rank function, usually result in a suboptimal solution. To seek a tighter rank approximation, we develop a non-convex surrogate to approximate the rank function based on the Laplace function. An iterative algorithm based on the augmented Lagrangian multipliers method is developed. Empirical studies for practical applications including robust principal component analysis and low-rank representation demonstrate that our proposed algorithm outperforms many other state-of-the-art convex and non-convex methods developed recently in the literature.

PCA Based on Graph Laplacian Regularization and P-Norm for Gene Selection and Clustering

In modern molecular biology, the hotspots and difficulties of this field are identifying characteristic genes from gene expression data. Traditional reconstruction-error-minimization model principal component analysis (PCA) as a matrix decomposition method uses quadratic error function, which is known sensitive to outliers and noise. Hence, it is necessary to learn a good PCA method when outliers and noise exist. In this paper, we develop a novel PCA method enforcing P-norm on error function and graph-Laplacian regularization term for matrix decomposition problem, which is called as PgLPCA. The heart of the method designing for reducing outliers and noise is a new error function based on non-convex proximal P-norm. Besides, Laplacian regularization term is used to find the internal geometric structure in the data representation. To solve the minimization problem, we develop an efficient optimization algorithm based on the augmented Lagrange multiplier method. This method is used to select characteristic genes and cluster the samples from explosive biological data, which has higher accuracy than compared methods.