Augmented Lagrange Multiplier Research Articles

A Novel Regularized Model for Third-Order Tensor Completion

Inspired by the accuracy and efficiency of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\gamma$</tex-math></inline-formula> -norm of a matrix, which is closer to the original rank minimization problem than nuclear norm minimization (NNM), we generalize the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\gamma$</tex-math></inline-formula> -norm of a matrix to tensors and propose a new tensor completion approach within the tensor singular value decomposition (t-svd) framework. An efficient algorithm, which combines the augmented Lagrange multiplier and closed-resolution of a cubic equation, was developed to solve the associated nonconvex tensor multi-rank minimization problem. Experimental results show that the proposed approach has an advantage over current state of the art algorithms in recovery accuracy.

Modeling of jamming phenomenon in fixture design application: an analytical, numerical, and experimental study

Jamming may occur during the loading of the workpiece in the fixture and cause an improper locating process or damage to the workpiece, fixture, and loading system. Consideration of the jamming phenomenon is a necessary task in the planning of the workpiece loading strategy. In the present paper, the minimum norm principle is used to present an analytical model for predicting the jamming occurrence conditions through calculating reaction forces at the contact points. The implementation of this model results in an optimization problem that is solved using the augmented Lagrange multiplier method. The proposed model is applied to two well-known jamming problems, namely the peg-in-hole and block and palm configurations. Numerical analysis is also conducted in Adams software to verify the theoretical predictions. Finally, the experimental setups are fabricated for these configurations to validate the theoretical predictions and numerical results. The results show that the maximum errors between the theoretical predictions and experimental results are 14.4% and 3.8% for the jamming-out angle in the peg-in-hole study and jamming-in travel in the block and palm problem, respectively. Moreover, the worst-case error of 6.9% is obtained for the theoretical prediction of the jamming-in travel of block compared to the numerical result. Agreement between the theoretical, numerical, and experimental results proves the capabilities of the suggested model in the accurate prediction of the jamming occurrence conditions and its potentials for use in the three-dimensional jamming-prone fixturing systems.

Motion-Aware Structured Matrix Factorization for Foreground Detection in Complex Scenes

Foreground detection is one of the key steps in computer vision applications. Many foreground and background models have been proposed and achieved promising performance in static scenes. However, due to challenges such as dynamic background, irregular movement, and noise, most algorithms degrade sharply in complex scenes. To address the problem, we propose a motion-aware structured matrix factorization approach (MSMF), which integrates the structural and spatiotemporal motion information into a unified sparse-low-rank matrix factorization framework. Technologically, it has three main contributions: First, a variant of structured sparsity-inducing norm is proposed to constrain both structure and sparsity of foreground. The model is robust to the statistical variability of the underlying foreground pixels in complex scenes. Second, to capture the ambiguous pixels, a spatiotemporal cube-based motion trajectory is extracted for assisting matrix factorization. Finally, to solve the optimization problem of structured matrix factorization, we develop an augmented Lagrange multiplier method with the alternating direction strategy and Douglas-Rachford monotone operator splitting algorithm. Experiments demonstrate that the proposed approach achieves impressive performance in separating irregular moving foreground while suppressing the dynamic background and the noise, and outperforms some state-of-the-art algorithms.

A class of smooth exact penalty function methods for optimization problems with orthogonality constraints

ABSTRACT Updating the augmented Lagrangian multiplier by closed-form expression yields efficient first-order infeasible approach for optimization problems with orthogonality constraints. Hence, parallelization becomes tractable in solving this type of problems. Inspired by this closed-form updating scheme, we propose a novel penalty function with compact convex constraints (PenC). We show that PenC can act as an exact penalty model which shares the same global minimizers as the original problem with orthogonality constraints. Based on PenC, we first propose a first-order algorithm called PenCF and establish its global convergence and local linear convergence rate under some mild assumptions. For the case that the computation and storage of Hessian is achievable, and we pursue high precision solution and fast local convergence rate, a second-order approach called PenCS is proposed for solving PenC. To avoid expensive calculation or solving a hard subproblem in computing the Newton step, we propose a new strategy to do it approximately which still leads to quadratic convergence locally. Moreover, the main iterations of both PenCF and PenCS are orthonormalization-free and hence parallelizable. Numerical experiments illustrate that PenCF is comparable with the existing first-order methods. Furthermore, PenCS shows its stability and high efficiency in obtaining high precision solution comparing with the existing second-order methods.

Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory

A fast non-convex low-rank matrix decomposition method for potential field data separation is proposed. The singular value decomposition of the large size trajectory matrix, which is also a block Hankel matrix, is obtained using a fast randomized singular value decomposition algorithm in which fast block Hankel matrix-vector multiplications are implemented with minimal memory storage. This fast block Hankel matrix randomized singular value decomposition algorithm is integrated into the \texttt{Altproj} algorithm, which is a standard non-convex method for solving the robust principal component analysis optimization problem. The improved algorithm avoids the construction of the trajectory matrix. Hence, gravity and magnetic data matrices of large size can be computed. Moreover, it is more efficient than the traditional low-rank matrix decomposition method, which is based on the use of an inexact augmented Lagrange multiplier algorithm. The presented algorithm is also robust and, hence, algorithm-dependent parameters are easily determined. The improved and traditional algorithms are contrasted for the separation of synthetic gravity and magnetic data matrices of different sizes. The presented results demonstrate that the improved algorithm is not only computationally more efficient but it is also more accurate. Moreover, it is possible to solve far larger problems. As an example, for the adopted computational environment, matrices of sizes larger than $205 \times 205$ generate "out of memory" exceptions with the traditional method, but a matrix of size $2001\times 2001$ can be calculated in $1062.29$s with the new algorithm. Finally, the improved method is applied to separate real gravity and magnetic data in the Tongling area, Anhui province, China. Areas which may exhibit mineralizations are inferred based on the separated anomalies.

Partial multi-label learning with noisy side information

Partial multi-label learning (PML) aims to learn from the training data where each training example is annotated with a candidate label set, among which only a subset is relevant. Despite the success of existing PML approaches, a major drawback of them lies in lacking of robustness to noisy side information. To tackle this problem, we introduce a novel partial multi-label learning with noisy side information approach, which simultaneously removes noisy outliers from the training instances and trains robust partial multi-label classifier for unlabeled instances prediction. Specifically, we first represent the observed sample set as a feature matrix and then decompose it into an ideal feature matrix and an outlier feature matrix by using the low-rank and sparse decomposition scheme, where the former is constrained to be low rank by considering that the noise-free feature information always lies in a low-dimensional subspace and the latter is assumed to be sparse by considering that the outliers are usually sparse among the observed feature matrix. Secondly, we refine an ideal label confidence matrix from the observed label matrix and use the graph Laplacian regularization to constrain the confidence matrix to keep the intrinsic structure among feature vectors. Thirdly, we constrain the feature mapping matrix to be low rank by utilizing the label correlations. Finally, we obtain both the ideal features and ground-truth labels via minimizing the loss function, where the augmented Lagrange multiplier algorithm and quadratic programming are incorporated to solve the optimization problem. Extensive experiments conducted on ten different data sets demonstrate the effectiveness of our proposed method.

Feature-Enhanced Speckle Reduction via Low-Rank and Space-Angle Continuity for Circular SAR Target Recognition

With the development of synthetic aperture radar (SAR) system, automatic target recognition (ATR) has attracted wide attention in many decision-making tasks, in which an enhanced feature of SAR image is a powerful tool to improve the recognition accuracy. However, the presence of speckle noise and natural clutter inevitably contaminates SAR images and, thus, degrades image features. In this article, we explicitly address the speckle reduction problem for the circular SAR system, in which the motion of aircraft platform causes continuous angular variations so that different SAR images can be captured with the high interrelationship. By exploiting the underlying low-rank and continuous properties among different SAR images, a method called the $\ell _{p}$ -regularized low-rank and space-angle continuity extraction ( $\ell _{p}$ -LSCE) is proposed to suppress the noise and enhance the target feature. Taking into account the interrelationship between SAR images, we arrange the images in a 3-D tensor to investigate the space-angle continuity of the targets. Furthermore, we develop a robust $\ell _{p}$ -regularized scheme to incorporate the low-rank property of targets. Then, the joint optimization problem is solved via the framework of augmented Lagrange multiplier (ALM) with efficient computation of each ALM subproblem. The experimental results of circular SAR data sets of the moving and stationary target acquisition and recognition (MSTAR) and the VideoSAR demonstrate that the proposed method can efficiently despeckle SAR images with well-preserved target features, which is conducive to the improvement of ATR performance.

Robust Structured Graph Clustering.

Graph-based clustering methods have achieved remarkable performance by partitioning the data samples into disjoint groups with the similarity graph that characterizes the sample relations. Nevertheless, their learning scheme still suffers from two important problems: 1) the similarity graph directly constructed from the raw features may be unreliable as real-world data always involves adverse noises, outliers, and irrelevant information and 2) most graph-based clustering methods adopt two-step learning strategy that separates the similarity graph construction and clustering into two independent processes. Under such circumstance, the generated graph is unstructured and fixed. It may suffer from a low-quality clustering structure and thus lead to suboptimal clustering performance. To alleviate these limitations, in this article we propose a robust structured graph clustering (RSGC) model. We formulate a novel learning framework to simultaneously learn a robust structured similarity graph and perform clustering. Specifically, the structured graph with proper probabilistic neighborhood assignment is adaptively learned on a robust latent representation that resists the noises and outliers. Furthermore, an explicit rank constraint is imposed on the Laplacian matrix to structurize the graph such that the number of the connected components is exactly equal to the ground-truth cluster number. To solve the challenging objective formulation, we propose to first transform it into an equivalent one that can be tackled more easily. An iterative solution based on the augmented Lagrangian multiplier is then derived to solve the model. In RSGC, the discrete cluster labels can be directly obtained by partitioning the learned similarity graph without reliance on label discretization strategy as most graph-based clustering methods. Experiments on both synthetic and real data sets demonstrate the superiority of the proposed method compared with the state-of-the-art clustering techniques.

Twin Least Square Support Vector Regression Model Based on Gauss-Laplace Mixed Noise Feature with Its Application in Wind Speed Prediction.

In this article, it was observed that the noise in some real-world applications, such as wind power forecasting and direction of the arrival estimation problem, does not satisfy the single noise distribution, including Gaussian distribution and Laplace distribution, but the mixed distribution. Therefore, combining the twin hyperplanes with the fast speed of Least Squares Support Vector Regression (LS-SVR), and then introducing the Gauss–Laplace mixed noise feature, a new regressor, called Gauss-Laplace Twin Least Squares Support Vector Regression (GL-TLSSVR), for the complex noise. Subsequently, we apply the augmented Lagrangian multiplier method to solve the proposed model. Finally, we apply the short-term wind speed data-set to the proposed model. The results of this experiment confirm the effectiveness of our proposed model.

Micro‐Doppler effect removal for inverse synthetic aperture radar imaging based on augmented Lagrange multipliers

AbstractMicro‐Doppler (MD) effect may decrease the readability of inverse synthetic aperture radar (ISAR) imaging results, so it is more desirable to solve the problem of MD effect removal. To overcome this issue, a new algorithm based on Augmented Lagrange Multipliers (ALM) is proposed in this paper. In our study, the MD effect in ISAR image is proven to be of low‐rank. Therefore, the MD effect can be removed by the ALM algorithm. In addition to being capable of separating the MD effect from ISAR image, the proposal is robust to a low signal to noise ratio (SNR). Comprehensive experiments are provided to illustrate the effectiveness of the proposed method.

Small Target Detection for Infrared Image Based on Optimal Infrared Patch-Image Model by Solving Modified Adaptive RPCA Problem

Small target detection in infrared (IR) images has been widely applied for both military and civilian purposes. In this study, because IR images contain sparse and low-rank features in most scenarios, we propose an optimal IR patch-image (OIPI) model-based detection method to detect small targets in heavily cluttered IR images. First, the OIPI model was generated based on a conventional IR image model using a novel optimal patch size and sliding step adaptive selection algorithm. Secondly, the sparse and low-rank features of IR images were extracted and fused to generate an adaptive weighted parameter. Thirdly, the adaptive inexact augmented Lagrange multiplier (AIALM) algorithm was applied in the OIPI model to solve the robust principal component analysis (RPCA) optimization problem. Finally, an adaptive threshold method is proposed to segment and calibrate targets. Experimental results indicate that the proposed algorithm is capable of detecting small targets more stably and accurately, compared with state-of-the-art methods.

A local deviation constraint based non-rigid structure from motion approach

In many traditional non-rigid structure from motion &#x0028 NRSFM &#x0029 approaches, the estimation results of part feature points may significantly deviate from their true values because only the overall estimation error is considered in their models. Aimed at solving this issue, a local deviation-constrained-based column-space-fitting approach is proposed in this paper to alleviate estimation deviation. In our work, an effective model is first constructed with two terms: the overall estimation error, which is computed by a linear subspace representation, and a constraint term, which is based on the variance of the reconstruction error for each frame. Furthermore, an augmented Lagrange multipliers &#x0028 ALM &#x0029 iterative algorithm is presented to optimize the proposed model. Moreover, a convergence analysis is performed with three steps for the optimization process. As both the overall estimation error and the local deviation are utilized, the proposed method can achieve a good estimation performance and a relatively uniform estimation error distribution for different feature points. Experimental results on several widely used synthetic sequences and real sequences demonstrate the effectiveness and feasibility of the proposed algorithm.

Incentive mechanism design for edge‐cloud collaboration in mobile crowd sensing

AbstractIn this article, we propose a pricing‐based incentive mechanism for edge‐cloud collaboration (PIM‐EC) in mobile crowd sensing. In PIM‐EC, data can be exchanged among different regions with the support of edge‐cloud servers, which improves the data efficiency. For the utility conflicts between mobile sensing users and cloud servers in the traditional one‐stage game, we design a two‐stage game which includes the bargaining game between users and edge‐cloud servers, as well as a data trading game among different edge‐cloud servers deployed in different regions. For the first stage game, based on Stackelberg game model and Rubinstein bargaining model, we design a finite‐period dynamic bargaining algorithm. For the second stage game, based on the optimal auction mechanism and using the augmented Lagrange multiplier method, a quasi‐Newton iterative pricing algorithm is proposed. We investigate the performance of PIM‐EC through simulations. Compared with SWMA and IMC‐SS, the social welfare of PIM‐EC is increased by 41% and 29%, respectively.

A Synchrophasor Data-Driven Method for Forced Oscillation Localization Under Resonance Conditions

This paper proposes a data-driven algorithm for locating the source of forced oscillations and suggests a physical interpretation for the method. By leveraging the sparsity of forced oscillations along with the low-rank nature of synchrophasor data, the problem of source localization under resonance conditions is cast as computing the sparse and low-rank components using Robust Principal Component Analysis (RPCA), which can be efficiently solved by the exact Augmented Lagrange Multiplier method. Based on this problem formulation, an efficient and practically implementable algorithm is proposed to pinpoint the forced oscillation source during real-time operation. Furthermore, theoretical insights are provided for the efficacy of the proposed approach, by use of physical model-based analysis, specifically by highlighting the low-rank nature of the resonance component matrix. Without the availability of system topology information, the proposed method can achieve high localization accuracy in synthetic cases based on benchmark systems and real-world forced oscillations in the power grid of Texas.

Constrained Clustering With Dissimilarity Propagation-Guided Graph-Laplacian PCA.

In this article, we propose a novel model for constrained clustering, namely, the dissimilarity propagation-guided graph-Laplacian principal component analysis (DP-GLPCA). By fully utilizing a limited number of weakly supervisory information in the form of pairwise constraints, the proposed DP-GLPCA is capable of capturing both the local and global structures of input samples to exploit their characteristics for excellent clustering. More specifically, we first formulate a convex semisupervised low-dimensional embedding model by incorporating a new dissimilarity regularizer into GLPCA (i.e., an unsupervised dimensionality reduction model), in which both the similarity and dissimilarity between low-dimensional representations are enforced with the constraints to improve their discriminability. An efficient iterative algorithm based on the inexact augmented Lagrange multiplier is designed to solve it with the global convergence guaranteed. Furthermore, we innovatively propose to propagate the cannot-link constraints (i.e., dissimilarity) to refine the dissimilarity regularizer to be more informative. The resulting DP model is iteratively solved, and we also prove that it can converge to a Karush-Kuhn-Tucker point. Extensive experimental results over nine commonly used benchmark data sets show that the proposed DP-GLPCA can produce much higher clustering accuracy than state-of-the-art constrained clustering methods. Besides, the effectiveness and advantage of the proposed DP model are experimentally verified. To the best of our knowledge, it is the first time to investigate DP, which is contrast to existing pairwise constraint propagation that propagates similarity. The code is publicly available at https://github.com/jyh-learning/DP-GLPCA.

Urban and Rural Multi-objective Programming Based on Augmented Lagrange Multiplier Method for Nonlinear Mathematical Equations

Urban and Rural Multi-objective Programming Based on Augmented Lagrange Multiplier Method for Nonlinear Mathematical Equations

Multi-scale active patches fusion based on spatiotemporal LBP-TOP for micro-expression recognition

Micro-expressions are spontaneous emotions appearing on a face that is hard to conceal and thus making them different from normal facial expressions both in duration and subtlety. This paper investigates a challenging issue in micro-expression, where not all facial regions contribute equally to effective representation. Consequently, we proposed a multi-scale active patches fusion-based spatiotemporal LBP-TOP descriptor that considers the active contributions for different region area in faces. For the feature procedure, we exploit the average value of all patches under each scale to obtain the threshold that selectively fuses the local and global features. On the other hand, an improved weighted sparse representation based dual augmented Lagrange multiplier is adopted for the classification to remit the problem of sparse coefficients obtained by the traditional sparse representation algorithm. We conduct comprehensive experiments on CASME II and SAMM datasets and the accuracies respectively reach 77.30% and 58.82% using LOSO cross-validation.

Sound source localization of non-synchronous measurements beamforming with block Hermitian matrix completion

In this work, the block Hermitian matrix completion (BHMC) method, a fast non-iteration algorithm to complete the cross-spectral matrix, is proposed to realize the sound source localization of the non-synchronous measurements beamforming. Compared with the conventional beamforming (CBF) method of single measurement and the augmented Lagrange multiplier (ALM) method of non-synchronous measurements, the proposed method can obtain source maps with narrow mainlobes and no sidelobes. The simulation results show the matrix completion error and the sound source reconstruction error of BHMC are less than those of ALM. In terms of computational efficiency, the proposed method is more efficient than ALM. The computation time of BHMC increases slowly with the number of the non-synchronous measurements while that of ALM increases sharply. Finally, experiments were conducted to verify the feasibility of BHMC. The experimental results show that the proposed method is an accurate and fast non-synchronous measurements beamforming algorithm, which paves a way to the real-time sound source localization in industrial occasions.

Simultaneously learning feature-wise weights and local structures for multi-view subspace clustering

Multi-view clustering integrates multiple feature sets, which usually have a complementary relationship and can reveal distinct insights of data from different angles, to improve clustering performance. It remains challenging to productively utilize complementary information across multiple views since there is always noise in real data, and their features are highly redundant. Moreover, most existing multi-view clustering approaches only aimed at exploring the consistency of all views, but overlooked the local structure of each view. However, it is necessary to take the local structure of each view into consideration, because individual views generally present different geometric structures while admitting the same cluster structure. To ease the above issues, in this paper, a novel multi-view subspace clustering method is established by concurrently assigning weights for different features and capturing local information of data in view-specific self-representation feature spaces. In particular, a common clustering assignment regularization is adopted to explore the consistency among multiple views. An alternating iteration algorithm based on the augmented Lagrangian multiplier is also developed for optimizing the associated objective. Experiments conducted on diverse multi-view datasets manifest that the proposed method achieves state-of-the-art performance. We provide the Matlab code on https://github.com/Ekin102003/JFLMSC.

Unsupervised 3D Reconstruction and Grouping of Rigid and Non-Rigid Categories.

In this paper we present an approach to jointly recover camera pose, 3D shape, and object and deformation type grouping, from incomplete 2D annotations in a multi-instance collection of RGB images. Our approach is able to handle indistinctly both rigid and non-rigid categories. This advances existing work, which only addresses the problem for one single object or, they assume the groups to be known a priori when multiple instances are handled. In order to address this broader version of the problem, we encode object deformation by means of multiple unions of subspaces, that is able to span from small rigid motion to complex deformations. The model parameters are learned via Augmented Lagrange Multipliers, in a completely unsupervised manner that does not require any training data at all. Extensive experimental evaluation is provided in a wide variety of synthetic and real scenarios, including rigid and non-rigid categories with small and large deformations. We obtain state-of-the-art solutions in terms of 3D reconstruction accuracy, while also providing grouping results that allow splitting the input images into object instances and their associated type of deformation.