Lagrange Multiplier Algorithm Research Articles

A subspace-approximating algorithm for matrix completion

In this paper, we propose a new subspace-approximating algorithm for matrix completion based on the subspace-approximating of the singular value decomposition.Then we use quadratic programming to produce the closest and the best feasible matrix in the subspace. This algorithm can achieve the reduction of the rank of the subspace by gradually reducing the number of the singular value of the thresholding and get the optimal low-rank matrix. It is proved that the subspace-approximating algorithm is convergent under some conditions. Besides, compared with the augmented Lagrange multiplier algorithm and orthogonal rank-one matrix pursuit algorithm by random experiments,the proposed algorithm is more effective in the CPU time and the low-rank property.

Joint compressive representation for multi-feature tracking

Abstract Multi-feature tracking is an interesting but challenging task. Many previous works just consider the fusion of different features from identical spectral image or identical features from different spectral images alone, which makes them be quite distinct from each other and be difficult to be integrated naturally. To address this problem, we propose a unified multi-feature tracking framework based on joint compressive sensing in this paper. Our framework can accept features extracted from identical spectral or different spectral images, and provide the flexibility to arbitrary add or remove feature. We formulate the multi-feature tracking as a joint minimization problem of multiple l1-norms with inequality constraint of multiple l2-norms, and derive a customized augmented Lagrange multiplier algorithm to solve the minimization problem, which provides efficient object tracking with both low computational burden and high accuracy. Besides, a collaborative template update scheme induced by sparsity concentration index is developed to keep track of the most representative templates throughout the tracking procedure. Experimental results on challenging visible and infrared sequences clearly demonstrate the robustness and effectiveness of the proposed method.

An active three-way clustering method via low-rank matrices for multi-view data

In recent years, multi-view clustering algorithms have shown promising performance by combining multiple sources or views of datasets. A problem that has not been addressed satisfactorily is the uncertain relationship between an object and a cluster. Thus, this paper investigates an active three-way clustering method via low-rank matrices that can improve clustering accuracy as clustering proceeds for the multi-view data of high dimensionality. We adopt a three-way clustering representation to reflect the three types of relationships between an object and a cluster, namely, belong-to definitely, uncertain and not belong-to definitely. We construct the consensus low-rank matrix from each weighted low-rank matrix by taking account of the diversity of views, and give the method to solve the optimization problem of objective function based on the improved augmented Lagrangian multiplier algorithm. We suggest an active learning strategy to learn important informative pairwise constraints after measuring the uncertainty of an object based on the entropy concept. The experimental results conducted on real-world datasets have validated the effectiveness of the proposed method.

Joint optimisation of transmit waveform and receive filter for cognitive radar

In this study, the authors consider the problem of joint transmit waveform and receive filter design for cognitive radar. The problem is analysed in signal-dependent interference, as well as additive channel noise for an extended target with unknown target impulse response (TIR). To address this problem, an iterative algorithm is employed for target detection by maximising the average signal-to-interference-plus-noise ratio of the received echo on the premise of ensuring the TIR estimation precision. In this method, the transmit waveform and receive filter are optimally determined at each step based on the previous step. In particular, under the same constraint on waveform energy and bandwidth, the Lagrange multiplier method is also considered. Simulation results demonstrate that the proposed iterative algorithm waveform achieves a higher rate of performance significantly compared to Lagrange multiplier algorithm waveform and traditional linear frequency modulated waveform, in terms of estimation accuracy and detection performance.

Dimensional Optimization of Helical Gear Pair Through Finite Element Analysis Using Subproblem Approximation Method

Traditional helical gear stress analysis techniques such as AGMA and ISO standards provides very often over-dimensioned gear pairs with bulky design. Therefore, it is required to optimize the gear pair design using latest techniques like finite element analysis. Stress analysis technique chosen to investigate the feasibility of dimensional optimization of a gear was Finite Element Analysis platform ANSYS 12.1. ISO 6336 standards and Hertz contact theory neglect friction between contact teeth. This work reflects coefficient of friction between contact teeth using FEA predicts more accurate results for contact stress. In this work, case hardened steel helical gear is optimised for optimum tooth root stress level subsequently to reduce the face width of gears with other geometrical parameters unaffected. Efforts has been carried out to verify contact stress value using Lagrange multiplier algorithm of optimised helical gear pair. Subproblem approximation method was implemented for searching of optimal solution of design variable. Optimized gear pair was manufactured and experimentally investigated for actual stresses and strains, as per loading condition mention in the rules of TED. Percentage saving in the material for optimised gear pair also been included in this work.

Weighted sparse coding regularized nonconvex matrix regression for robust face recognition

Most existing regression based classification methods for robust face recognition usually characterize the representation error using L1-norm or Frobenius-norm for the pixel-level noise or nuclear norm for the image-level noise, and code the coefficients vector by l1-norm or l2-norm. To our best knowledge, nuclear norm can be used to describe the low-rank structural information but may lead to the suboptimal solution, while l1-norm or l2-norm can promote the sparsity or cooperativity but may neglect the prior information (e.g., the locality and similarity relationship) among data. To solve these drawbacks, we propose two weighted sparse coding regularized nonconvex matrix regression models including weighted sparse coding regularized matrix γ-norm based matrix regression (WSγMR) for the structural noise and weighted sparse coding regularized matrix γ-norm plus minimax concave plus (MCP) function based matrix regression (WSγM2R) for the mixed noise (e.g, structural noise plus sparse noise). The MCP induced nonconvex function can overcome the imbalanced penalization of different singular values and entries of the error image matrix, and the weighted sparse coding can consider the prior information by borrowing a novel distance metric. The variants of inexact augmented Lagrange multiplier (iALM) algorithm including nonconvex iALM (NCiALM) and majorization-minimization iALM (MMiALM) are developed to solve the proposed models, respectively. The matrix γ-norm based classifier is devised for classification. Finally, experiments on four popular face image databases can validate the superiority of our methods compared with the-state-of-the-art regression methods.

Self-Taught Low-Rank Coding for Visual Learning.

The lack of labeled data presents a common challenge in many computer vision and machine learning tasks. Semisupervised learning and transfer learning methods have been developed to tackle this challenge by utilizing auxiliary samples from the same domain or from a different domain, respectively. Self-taught learning, which is a special type of transfer learning, has fewer restrictions on the choice of auxiliary data. It has shown promising performance in visual learning. However, existing self-taught learning methods usually ignore the structure information in data. In this paper, we focus on building a self-taught coding framework, which can effectively utilize the rich low-level pattern information abstracted from the auxiliary domain, in order to characterize the high-level structural information in the target domain. By leveraging a high quality dictionary learned across auxiliary and target domains, the proposed approach learns expressive codings for the samples in the target domain. Since many types of visual data have been proven to contain subspace structures, a low-rank constraint is introduced into the coding objective to better characterize the structure of the given target set. The proposed representation learning framework is called self-taught low-rank (S-Low) coding, which can be formulated as a nonconvex rank-minimization and dictionary learning problem. We devise an efficient majorization-minimization augmented Lagrange multiplier algorithm to solve it. Based on the proposed S-Low coding mechanism, both unsupervised and supervised visual learning algorithms are derived. Extensive experiments on five benchmark data sets demonstrate the effectiveness of our approach.

Identificación de Parámetros Erróneos en Sistemas Eléctricos de Potencia y Corrección en base a Mediciones Sincrofasoriales

Los algoritmos convencionales de estimación de estado de sistemas de potencia se basan en la suposición de que los parámetros tales como la resistencia y reactancia de las líneas de transmisión, los taps de transformadores y el estado de los interruptores no tienen errores. En la realidad, estos parámetros no se conocen con certeza. Los errores en las impedancias de las ramas son menos visibles que los errores en los estados de los interruptores y pueden producir errores en los resultados del estimador que pueden pasar inadvertidos por largo tiempo. Los errores en los parámetros pueden producir un impacto sobre la estimación de estado similar o superior al de los errores no detectados en las mediciones.
 En este documento se desarrolla un programa de identificación de errores en los parámetros basado en un algoritmo que utiliza Multiplicadores de Lagrange. Este algoritmo se aplica a casos de estudio en diferentes condiciones de demanda obtenidos de los sistemas de tiempo real EMS y fuera de línea. Una vez identificado el equipo de red que presenta errores de parámetros, se plantean alternativas de cálculo para la corrección de parámetros, utilizando mediciones de equipos de medición fasorial PMUs.

Constrained Low-Rank Learning Using Least Squares-Based Regularization.

Low-rank learning has attracted much attention recently due to its efficacy in a rich variety of real-world tasks, e.g., subspace segmentation and image categorization. Most low-rank methods are incapable of capturing low-dimensional subspace for supervised learning tasks, e.g., classification and regression. This paper aims to learn both the discriminant low-rank representation (LRR) and the robust projecting subspace in a supervised manner. To achieve this goal, we cast the problem into a constrained rank minimization framework by adopting the least squares regularization. Naturally, the data label structure tends to resemble that of the corresponding low-dimensional representation, which is derived from the robust subspace projection of clean data by low-rank learning. Moreover, the low-dimensional representation of original data can be paired with some informative structure by imposing an appropriate constraint, e.g., Laplacian regularizer. Therefore, we propose a novel constrained LRR method. The objective function is formulated as a constrained nuclear norm minimization problem, which can be solved by the inexact augmented Lagrange multiplier algorithm. Extensive experiments on image classification, human pose estimation, and robust face recovery have confirmed the superiority of our method.

A novel multi-view clustering method via low-rank and matrix-induced regularization

Multi-view clustering algorithms have shown promising performance in various applications over the last few decades. Most of them, however, do not adequately take noises and correlation among multiple views into account, which may degrade the clustering performance. In this paper, we propose a novel multi-view clustering method to address these issues. In specific, we construct a low-rank consensus matrix and a sparse error matrix from each similarity matrix corresponding to each view. Furthermore, a matrix-induced regularization term is incorporated to reduce the redundancy and enhance the diversity among different views. The augmented Lagrangian multiplier algorithm is adopted to solve the resultant optimization problem. Comprehensive experiments are conducted to verify the effectiveness of the proposed algorithm. Results demonstrate that our algorithm outperforms several state-of-the-art ones on both synthetic and benchmark data sets.

A modified augmented lagrange multiplier algorithm for toeplitz matrix completion

In this paper, a modified scheme is proposed for iterative completion matrices generated by the augmented Lagrange multiplier (ALM) method based on the mean value. So that the iterative completion matrices generated by the new algorithm are of the Toeplitz structure, which decrease the computation of SVD and have better approximation to solution. Convergence is discussed. Finally, the numerical experiments and inpainted images show that the new algorithm is more effective than the accelerated proximal gradient (APG) algorithm, the singular value thresholding (SVT) algorithm and the ALM algorithm, in CPU time and accuracy.

Convergence analysis of the augmented Lagrange multiplier algorithm for a class of matrix compressive recovery

In this paper, we mainly discuss the convergence of the augmented Lagrange multiplier (ALM) algorithm for matrix compressive recovery presented in Wright et al. (2013). Because of the unknown ∂ ‖ P Ω ( A ) ‖ ∗ , it is hard to obtain the convergence. So we convert the model in Wright et al. (2013) to the equivalent model in Meng et al. (2014), and discuss the convergence of the ALM algorithm for the model in Meng et al. (2014). Finally, the numerical experiments show the convergence of the ALM algorithm for the matrix compressive recovery.

Complex and Quaternionic Principal Component Pursuit and Its Application to Audio Separation

Recently, the principal component pursuit has received increasing attention in signal processing research ranging from source separation to video surveillance. So far, all existing formulations are real-valued and lack the concept of phase, which is inherent in inputs such as complex spectrograms or color images. Thus, in this letter, we extend principal component pursuit to the complex and quaternionic cases to account for the missing phase information. Specifically, we present both complex and quaternionic proximity operators for the $\ell_1$- and trace-norm regularizers. These operators can be used in conjunction with proximal minimization methods such as the inexact augmented Lagrange multiplier algorithm. The new algorithms are then applied to the singing voice separation problem, which aims to separate the singing voice from the instrumental accompaniment. Results on the iKala and MSD100 datasets confirmed the usefulness of phase information in principal component pursuit.

Comparisons of several algorithms for Toeplitz matrix recovery

In this paper, we study algorithms for Toeplitz matrix recovery. Inspired by the singular value thresholding (SVT) algorithm for matrix completion and the alternating directions iterative method, we first propose a new mean value algorithm for Toeplitz matrix recovery. Then we apply our idea to the augmented Lagrange multiplier (ALM) algorithm for matrix recovery and put forward four modified ALM algorithms for Toeplitz matrix recovery. Convergence analysis of the new algorithms is discussed. All the iterative matrices generated by the five algorithms keep a Toeplitz structure that ensures the fast singular value decomposition (SVD) of Toeplitz matrices. Compared with the original algorithms, our algorithms are far superior in the time of SVD, as well as the CPU time.

Contact Stress Evaluation of Involute Gear Pairs, Including the Effects of Friction and Helix Angle

The aim of this technical brief is to provide a new viewpoint of friction factor for contact stress calculations of gears. The idea of friction factor has been coined, for the calculation of contact stresses along the tooth contact for different helical gear pairs. Friction factors were developed by evaluating contact stresses with and without friction for different gear pairs. In this paper, three-dimensional (3D) finite element method (FEM) and Lagrange multiplier algorithm have been used to evaluate the contact stresses. Initially, a spur gear finite element (FE) model was validated with the theoretical analysis under frictionless condition, which is based on Hertz's contact theory. Then, similar FE models were constructed for 5 deg, 15 deg, 25 deg, and 35 deg helical gear pairs. The contact stresses of these models were evaluated for different coefficients of friction. These results were employed for the development of friction factor.

Learning Balanced and Unbalanced Graphs via Low-Rank Coding

Graphs have been widely applied in modeling the relationships and structures in real-world applications. Graph construction is the most critical part in these models, while how to construct an effective graph is still an open problem. In this paper, we propose a novel approach to graph construction based on two observations. First, by virtue of recent advances in low-rank subspace recovery, the similarity between every two samples evaluated in the low-rank code space is more robust than that in the sample space. Second, a sparse and balanced graph can greatly increase the performance of learning tasks, such as label propagation in graph based semi-supervised learning. The <inline-formula> <tex-math notation="LaTeX">$k$</tex-math> </inline-formula> -NN sparsification can provide fast solutions to constructing unbalanced sparse graphs, and <inline-formula><tex-math notation="LaTeX">$b$</tex-math></inline-formula> -matching constraint is a necessary route for generating balanced graphs. These observations motivate us to jointly learn the low-rank codes and balanced (or unbalanced) graph simultaneously. In particular, two non-convex models are built by incorporating <inline-formula> <tex-math notation="LaTeX">$k$</tex-math> </inline-formula> -NN constraint and <inline-formula><tex-math notation="LaTeX">$b$</tex-math></inline-formula> -matching constraint into the low-rank representation model, respectively. We design a majorization-minimization augmented Lagrange multiplier (MM-ALM) algorithm to solve the proposed models. Extensive experimental results on four image databases demonstrate the superiority of our graphs over several state-of-the-art graphs in data clustering, transductive and inductive semi-supervised learning.

Advanced air-bearing modeling for operational shock analysis of a 2.5-inch HDD with ramp–disk contact

With the rapid adoption of mobile computing devices, it is important to analyze and predict the anti-shock performance of hard disk drive (HDD) systems accurately. This paper proposes an advanced air-bearing surface (ABS) model for analyzing the operational shock performance of a 2.5-inch HDD system, including the ramp---disk contact behavior. First, we developed a decoupled simulation method using four linear air-bearing springs using the finite element method and the Lagrange multiplier algorithm for modeling contact between the disk and ramp. With the finite element model, the effect of the linear air-bearing stiffness was investigated. We found that the air-bearing spring model affects the behavior of the slider in the HDD system with ramp---disk contact during the operational shock. Based on the numerical results, an advanced ABS model that reflects air-bearing characteristics and considers the natural frequencies and nodal lines of the ABS pitch and roll modes was proposed and evaluated.

Investigation of S-SPH for Hypervelocity Impact Calculations

In this paper, we investigate the Symmetric-SPH (S-SPH) method. S-SPH restores n-order consistency to the SPH formulation while remaining fully conservative by using a Taylor expansion of field variables to fit the kernel function. It has potential for HVI problems because it enables the ability to perform accurate stress and state calculations. We implement S-SPH in the Velodyne hydro-structural solver and evaluate its performance over a series of numerical examples including flyer impact, fragment penetration, and Taylor Rod impact. Idealized contact algorithms are employed to eliminate uncertainty in the flyer and Taylor impact problems while an advanced Lagrange multiplier algorithm is used for the fragment penetration test. We use the CTH hydrocode to provide a baseline response for each of the examples due to its ability to effectively handle penetration, hydrodynamic deformation, and shock propagation. Direct numerical comparisons are used to eliminate uncertainty from material characterizations, equation of state (EOS) models, and mesh resolution. We identify strengths and shortcomings of the S-SPH method and evaluate its utility for classes of HVI problems. We also compare the performance against SPH to compare relative accuracy, computational cost, and stability.

Folded-concave penalization approaches to tensor completion

The existing studies involving matrix or tensor completion problems are commonly under the nuclear norm penalization framework due to the computational efficiency of the resulting convex optimization problem. Folded-concave penalization methods have demonstrated surprising developments in sparse learning problems due to their nice practical and theoretical properties. To share the same light of folded-concave penalization methods, we propose a new tensor completion model via folded-concave penalty for estimating missing values in tensor data. Two typical folded-concave penalties, the minmax concave plus (MCP) penalty and the smoothly clipped absolute deviation (SCAD) penalty, are employed in the new model. To solve the resulting nonconvex optimization problem, we develop a local linear approximation augmented Lagrange multiplier (LLA-ALM) algorithm which combines a two-step LLA strategy to search a local optimum of the proposed model efficiently. Finally, we provide numerical experiments with phase transitions, synthetic data sets, real image and video data sets to exhibit the superiority of the proposed model over the nuclear norm penalization method in terms of the accuracy and robustness.

Defect inspection for TFT-LCD images based on the low-rank matrix reconstruction

Surface defect inspection of TFT-LCD panels is a critical task in LCD manufacturing. In this paper, an automatic defect inspection method based on the low-rank matrix reconstruction is proposed. The textured background of the LCD image is a low-rank matrix and the foreground image with defects can be treated as a sparse matrix. By utilizing the Inexact Augmented Lagrange Multipliers (IALM) algorithm, the segmentation of a LCD image can be converted into the reconstruction of a low-rank matrix with a fraction of its entries arbitrarily corrupted. This low-rank matrix reconstruction problem can be exactly solved via convex optimization that minimizes a combination of the nuclear norm and the l 1 -norm. Also, adaptive parameter selection strategy is proposed by conducting deep analysis on the IALM algorithm, which improves the generality of the IALM algorithm for different defect types. Experiment results show that our inspection algorithm is robust for the defect shapes and types under different illumination conditions. The shapes and edges of defect areas in the LCD images can be well preserved and segmented from textured background by our detection algorithm.