Inexact Augmented Lagrange Multiplier Research Articles

ScIALM: A method for sparse scRNA-seq expression matrix imputation using the Inexact Augmented Lagrange Multiplier with low error

Single-cell RNA sequencing (scRNA-seq) is a high-throughput sequencing technology that quantifies gene expression profiles of specific cell populations at the single-cell level, providing a foundation for studying cellular heterogeneity and patient pathological characteristics. It is effective for developmental, fertility, and disease studies. However, the cell-gene expression matrix of single-cell sequencing data is often sparse and contains numerous zero values. Some of the zero values derive from noise, where dropout noise has a large impact on downstream analysis. In this paper, we propose a method named scIALM for imputation recovery of sparse single-cell RNA data expression matrices, which employs the Inexact Augmented Lagrange Multiplier method to use sparse but clean (accurate) data to recover unknown entries in the matrix. We perform experimental analysis on four datasets, calling the expression matrix after Quality Control (QC) as the original matrix, and comparing the performance of scIALM with six other methods using mean squared error (MSE), mean absolute error (MAE), Pearson correlation coefficient (PCC), and cosine similarity (CS). Our results demonstrate that scIALM accurately recovers the original data of the matrix with an error of 10e-4, and the mean value of the four metrics reaches 4.5072 (MSE), 0.765 (MAE), 0.8701 (PCC), 0.8896 (CS). In addition, at 10%-50% random masking noise, scIALM is the least sensitive to the masking ratio. For downstream analysis, this study uses adjusted rand index (ARI) and normalized mutual information (NMI) to evaluate the clustering effect, and the results are improved on three datasets containing real cluster labels.

TVRPCA＋: Low-rank and sparse decomposition based on spectral norm and structural sparsity-inducing norm

Traditional low-rank sparse decomposition algorithms have trouble obtaining a clear and complete foreground representation in foreground–background separation due to the complex video environment and the noise. For this issue, We propose a more robust and higher-performance low-rank and sparse decomposition algorithm named TVRPCA＋ based on spectral norm, structured sparse norm and total variation (TV) regularization. The structured sparse norm and TV regularization are exploited to suppress noise and obtain much cleaner foregrounds. Spectral norm is used in our algorithm for the low-rank component to address the issue of over-punishment and restore more foreground information. Moreover, an efficient algorithm based on the inexact augmented Lagrange multiplier method is designed to solve the proposed optimization problem. Experimental results show that TVRPCA＋ obtained five top F-measures and three of the second-highest F-measures in eight noise-free test video sequences with complex backgrounds, while the highest average F-measure was also achieved in all ten experimental groups with noise.

A Tensor Method based on Enhanced Tensor Nuclear Norm and Hypergraph Laplacian Regularization for Pan-Cancer Omics Data Analysis.

As a powerful data representation technique, tensor robust principal component analysis (TRPCA) has been widely used for clustering and feature selection tasks. However, it ignores the significant difference in singular values of tensor data and the manifold information contained in different views, thereby causing serious degradation of conventional TRPCA performance. In this paper, a novel tensor method based on enhanced tensor nuclear norm and hypergraph Laplacian regularization (ETHLR) is developed to address the above problem. ETHLR can jointly learn the prior knowledge of singular values and high-order manifold structures in the unified tensor space and the view-specific feature spaces, respectively. Specifically, the enhanced tensor nuclear norm, namely, the weighted tensor Schatten p-norm, is used to shrink the singular values by fully considering the salient difference information of singular values and the complementary information embedded in the tensor space; the hypergraph Laplacian constraint helps encode high-order geometric structures among multiple samples in the nonlinear view-specific feature space. Furthermore, we employ inexact augmented Lagrange multipliers (ALM) to optimize the ETHLR method. Numerous experiments on pan-cancer omics data show that the superiority of ETHLR over several state-of-the-art competitors.

Discriminant Subspace Low-Rank Representation Algorithm for Electroencephalography-Based Alzheimer's Disease Recognition.

Alzheimer’s disease (AD) is a chronic progressive neurodegenerative disease that often occurs in the elderly. Electroencephalography (EEG) signals have a strong correlation with neuropsychological test results and brain structural changes. It has become an effective aid in the early diagnosis of AD by exploiting abnormal brain activity. Because the original EEG has the characteristics of weak amplitude, strong background noise and randomness, the research on intelligent AD recognition based on machine learning is still in the exploratory stage. This paper proposes the discriminant subspace low-rank representation (DSLRR) algorithm for EEG-based AD and mild cognitive impairment (MCI) recognition. The subspace learning and low-rank representation are flexibly integrated into a feature representation model. On the one hand, based on the low-rank representation, the graph discriminant embedding is introduced to constrain the representation coefficients, so that the robust representation coefficients can preserve the local manifold structure of the EEG data. On the other hand, the least squares regression, principle component analysis, and global graph embedding are introduced into the subspace learning, to make the model more discriminative. The objective function of DSLRR is solved by the inexact augmented Lagrange multiplier method. The experimental results show that the DSLRR algorithm has good classification performance, which is helpful for in-depth research on AD and MCI recognition.

Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory

A fast non-convex low-rank matrix decomposition method for potential field data separation is proposed. The singular value decomposition of the large size trajectory matrix, which is also a block Hankel matrix, is obtained using a fast randomized singular value decomposition algorithm in which fast block Hankel matrix-vector multiplications are implemented with minimal memory storage. This fast block Hankel matrix randomized singular value decomposition algorithm is integrated into the \texttt{Altproj} algorithm, which is a standard non-convex method for solving the robust principal component analysis optimization problem. The improved algorithm avoids the construction of the trajectory matrix. Hence, gravity and magnetic data matrices of large size can be computed. Moreover, it is more efficient than the traditional low-rank matrix decomposition method, which is based on the use of an inexact augmented Lagrange multiplier algorithm. The presented algorithm is also robust and, hence, algorithm-dependent parameters are easily determined. The improved and traditional algorithms are contrasted for the separation of synthetic gravity and magnetic data matrices of different sizes. The presented results demonstrate that the improved algorithm is not only computationally more efficient but it is also more accurate. Moreover, it is possible to solve far larger problems. As an example, for the adopted computational environment, matrices of sizes larger than $205 \times 205$ generate "out of memory" exceptions with the traditional method, but a matrix of size $2001\times 2001$ can be calculated in $1062.29$s with the new algorithm. Finally, the improved method is applied to separate real gravity and magnetic data in the Tongling area, Anhui province, China. Areas which may exhibit mineralizations are inferred based on the separated anomalies.

Small Target Detection for Infrared Image Based on Optimal Infrared Patch-Image Model by Solving Modified Adaptive RPCA Problem

Small target detection in infrared (IR) images has been widely applied for both military and civilian purposes. In this study, because IR images contain sparse and low-rank features in most scenarios, we propose an optimal IR patch-image (OIPI) model-based detection method to detect small targets in heavily cluttered IR images. First, the OIPI model was generated based on a conventional IR image model using a novel optimal patch size and sliding step adaptive selection algorithm. Secondly, the sparse and low-rank features of IR images were extracted and fused to generate an adaptive weighted parameter. Thirdly, the adaptive inexact augmented Lagrange multiplier (AIALM) algorithm was applied in the OIPI model to solve the robust principal component analysis (RPCA) optimization problem. Finally, an adaptive threshold method is proposed to segment and calibrate targets. Experimental results indicate that the proposed algorithm is capable of detecting small targets more stably and accurately, compared with state-of-the-art methods.

Constrained Clustering With Dissimilarity Propagation-Guided Graph-Laplacian PCA.

In this article, we propose a novel model for constrained clustering, namely, the dissimilarity propagation-guided graph-Laplacian principal component analysis (DP-GLPCA). By fully utilizing a limited number of weakly supervisory information in the form of pairwise constraints, the proposed DP-GLPCA is capable of capturing both the local and global structures of input samples to exploit their characteristics for excellent clustering. More specifically, we first formulate a convex semisupervised low-dimensional embedding model by incorporating a new dissimilarity regularizer into GLPCA (i.e., an unsupervised dimensionality reduction model), in which both the similarity and dissimilarity between low-dimensional representations are enforced with the constraints to improve their discriminability. An efficient iterative algorithm based on the inexact augmented Lagrange multiplier is designed to solve it with the global convergence guaranteed. Furthermore, we innovatively propose to propagate the cannot-link constraints (i.e., dissimilarity) to refine the dissimilarity regularizer to be more informative. The resulting DP model is iteratively solved, and we also prove that it can converge to a Karush-Kuhn-Tucker point. Extensive experimental results over nine commonly used benchmark data sets show that the proposed DP-GLPCA can produce much higher clustering accuracy than state-of-the-art constrained clustering methods. Besides, the effectiveness and advantage of the proposed DP model are experimentally verified. To the best of our knowledge, it is the first time to investigate DP, which is contrast to existing pairwise constraint propagation that propagates similarity. The code is publicly available at https://github.com/jyh-learning/DP-GLPCA.

Fast and Latent Low-Rank Subspace Clustering for Hyperspectral Band Selection

This article presents a fast and latent low-rank subspace clustering (FLLRSC) method to select hyperspectral bands. The FLLRSC assumes that all the bands are sampled from a union of latent low-rank independent subspaces and formulates the self-representation property of all bands into a latent low-rank representation (LLRR) model. The assumption ensures sufficient sampling bands in representing low-rank subspaces of all bands and improves robustness to noise. The FLLRSC first implements the Hadamard random projections to reduce spatial dimensionality and lower the computational cost. It then adopts the inexact augmented Lagrange multiplier algorithm to optimize the LLRR program and estimates sparse coefficients of all the projected bands. After that, it employs a correntropy metric to measure the similarity between pairwise bands and constructs an affinity matrix based on sparse representation. The correntropy metric could better describe the nonlinear characteristics of hyperspectral bands and enhance the block-diagonal structure of the similarity matrix for correctly clustering all subspaces. The FLLRSC conducts spectral clustering on the connected graph denoted by the affinity matrix. The bands that are closest to their separate cluster centroids form the final band subset. Experimental results on three widely used hyperspectral data sets show that the FLLRSC performs better than the classical low-rank representation methods with higher classification accuracy at a low computational cost.

Nonconvex dictionary learning based visual tracking method

Focusing on the heavy decrease of object tracking performance induced by complex circumstances, an object tracking method based on nonconvex discriminative dictionary learning (NDDL) is proposed. Firstly, the object and background samples are acquired according to the temporal and spatial local correlation of objects. Since object and background samples have some common features, an inconsistent constraint is imposed on dictionaries to improve their robustness and discriminability. In what follows, a nonconvex minimax concave plus (MCP) function can be used to penalize sparse encoding matrices to avoid over-punishment via some convex relaxation methods. Based on the sparse representation (SR) theory, a NDDL model can be constructed, which can be tackled by majorization-minimization inexact augmented Lagrange multiplier (MM-IALM) optimization method to achieve better convergence. After obtaining the optimal discriminative dictionary, the reconstruction errors of all candidates are calculated to construct the object observation model. Finally, the object tracking is implemented accurately based on the Bayesian inference framework. Compared to the existing state-of-the-art trackers, simulation results show that the proposed tracker can improve the precision and success rate of the object tracking significantly in complex circumstances.

Multi-Scale Infrared Small Target Detection Method via Precise Feature Matching and Scale Selection Strategy

Infrared small target detection is a crucial and challenging topic for various applications. In recent years, the spectrum scale space (SSS) algorithm has shown considerable potential in the field of target detection. However, the SSS algorithm is prone to high false alarm rates in infrared small target detection scenarios with complex background. This paper proposes an improved SSS (ISSS) algorithm via precise feature matching and scale selection strategy for efficient infrared small target detection, which includes background suppression, feature matching and optimal scale selection three stages. In the background suppression stage, a matrix decomposition method named inexact augmented Lagrange multiplier (IALM) algorithm is used to extract the sparse image matrix from the original image as the target foreground image. In the feature matching stage, the 16 elaborate Gaussian kernel functions convolve with the the amplitude spectrum of target foreground image to generate 16 scale saliency maps that precisely match the feature of small targets. In the optimal scale selection stage, a few proper candidate scale maps are screened out according to the difference between the pixel values of the target area and the background clutters, in which the target area was more highlighted, and the scale map corresponding to the maximum value of local information entropy of the candidate saliency map is the final detection result map. We mainly made three contributions: First, IALM algorithm is utilized as a preprocessing step, and we have verified it is indispensable in eliminating most backgrounds with self-correlation property. Second, an elaborate scale division strategy is proposed to obtain multi-scale saliency maps that match the feature of infrared small targets precisely. Third, the gray value difference and the maximum value of local information entropy are defined and used as the judgment criteria for optimal scale selection. Extensive experimental results demonstrate that the proposed method outperforms state-of-the-art techniques, especially on infrared images with thick clouds and high-brightness buildings.

A New Estimation Algorithm for Frequency and Amplitude in Harmonic Signal Processing

Low-rank matrix recovery is a large-scale data analysis and processing technology; its related theory has been widely used in image restoration, image denoising, video background modeling, signal recovery and other fields. This paper proposes an improved inexact augmented Lagrange multiplier (IALM) method to solve the harmonic recovery (HR) problem. After that, the performance of the original IALM and the improved IALM are compared when the sparse matrix is a diagonal sparse matrix satisfying certain conditions, and the results show that the improved IALM algorithm is more stable than the original algorithm. Then the improved strategy of the algorithm is extended to two kinds of occasions in which the positions of non-zero elements in sparse matrix are fixed or random, which provides a way to improve the algorithm in different application scenarios. Finally, the original IALM algorithm and the improved IALM algorithm are used to solve the HR problem, and the experimental results show that the improved IALM algorithm has better solution performance.

Low-rank matrix decomposition method for potential field data separation

Separation of potential field data forms the basis of inversion and interpretation. The low-rank matrix theory is used for the separation of potential field data. A theoretical analysis led to the approximate equation that demonstrates the relation between the amplitudes of the wavenumber components of potential field data and the singular values of the trajectory matrix embedded from the potential field data matrix. Therefore, the low-rank feature of the trajectory matrix of regional field data and the sparse feature of the trajectory matrix of residual field data can be obtained based on the features of the power spectrum of the potential field data. Based on this, we have developed a low-rank matrix decomposition model for the separation of the trajectory matrix of the potential field data. Minimizing the rank of the trajectory matrix of the regional field data and the [Formula: see text]-norm of the trajectory matrix of the residual field data is a double-objective optimization task, and this optimization task can be solved by the inexact augmented Lagrange multiplier algorithm. The weighting parameter is robust and easy to set. Numerical experiment results indicate that our method is robust, and the separation errors of the method are smaller compared to the matched filtering and wavelet analysis methods. In addition, our method can be easily implemented and has clear geophysical significance. Finally, our method is applied on real data sets in the Daye area, Hubei Province, China. The separated gravity and magnetic fields coincide well with target geologic sources.

Hyperspectral Image Denoising via Subspace Low-rank Representation and Spatial–spectral Total Variation

Abstract Hyperspectral images (HSIs) acquired actually often contain various types of noise, such as Gaussian noise, impulse noise, and dead lines. On the basis of land covers, the spectral vectors in HSI can be separated into different classifications, which means the spectral space can be regarded as a union of several low-rank (LR) subspaces rather than a single LR subspace. Recently, LR constraint has been widely applied for denoising HSI. However, those LR-based methods do not constrain the intrinsic structure of spectral space. And these methods cannot make better use of the spatial or spectral features in an HSI cube. In this article, a framework named subspace low-rank representation combined with spatial‐spectral total variation regularization (SLRR-SSTV) is proposed for HSI denoising, where the SLRR is introduced to more precisely satisfy the low-rank property of spectral space, and the SSTV regularization is involved for the spatial and spectral smoothness enhancement. An inexact augmented Lagrange multiplier method by alternative iteration is employed for the SLRR-SSTV model solution. Both simulated and real HSI experiment results demonstrate that the proposed method can achieve a state-of-the-art performance in HSI denoising.

SRSC: Selective, Robust, and Supervised Constrained Feature Representation for Image Classification.

Feature representation learning, an emerging topic in recent years, has achieved great progress. Powerful learned features can lead to excellent classification accuracy. In this article, a selective and robust feature representation framework with a supervised constraint (SRSC) is presented. SRSC seeks a selective, robust, and discriminative subspace by transforming the original feature space into the category space. Particularly, we add a selective constraint to the transformation matrix (or classifier parameter) that can select discriminative dimensions of the input samples. Moreover, a supervised regularization is tailored to further enhance the discriminability of the subspace. To relax the hard zero-one label matrix in the category space, an additional error term is also incorporated into the framework, which can lead to a more robust transformation matrix. SRSC is formulated as a constrained least square learning (feature transforming) problem. For the SRSC problem, an inexact augmented Lagrange multiplier method (ALM) is utilized to solve it. Extensive experiments on several benchmark data sets adequately demonstrate the effectiveness and superiority of the proposed method. The proposed SRSC approach has achieved better performances than the compared counterpart methods.

Similarity learning with joint transfer constraints for person re-identification

The inconsistency of data distributions among multiple views is one of the most crucial issues which hinder the accuracy of person re-identification. To solve the problem, this paper presents a novel similarity learning model by combining the optimization of feature representation via multi-view visual words reconstruction and the optimization of metric learning via joint discriminative transfer learning. The starting point of the proposed model is to capture multiple groups of multi-view visual words (MvVW) through an unsupervised clustering method (i.e. K-means) from human parts (e.g. head, torso, legs). Then, we construct a joint feature matrix by combining multi-group feature matrices with different body parts. To solve the inconsistent distributions under different views, we propose a method of joint transfer constraint to learn the similarity function by combining multiple common subspaces, each in charge of a sub-region. In the common subspaces, the original samples can be reconstructed based on MvVW under low-rank and sparse representation constraints, which can enhance the structure robustness and noise resistance. During the process of objective function optimization, based on confinement fusion of multi-view and multiple sub-regions, a solution strategy is proposed to solve the objective function using joint matrix transform. Taking all of these into account, the issue of person re-identification under inconsistent data distributions can be transformed into a consistent iterative convex optimization problem, and solved via the inexact augmented Lagrange multiplier (IALM) algorithm. Extensive experiments are conducted on three challenging person re-identification datasets (i.e., VIPeR, CUHK01 and PRID450S), which shows that our model outperforms several state-of-the-art methods.

KRMARO: Aerial Detection of Small-Size Ground Moving Objects Using Kinematic Regularization and Matrix Rank Optimization

Detecting moving objects has been well studied in the past due to its importance in computer vision applications. Nevertheless, in aerial imagery, the small sizes of moving objects and the camera motion present challenges to existing well-known detection methods. Most moving object detection methods have reported either high true detection rates associated with high false-detection rates, or low false-detection rates at the expense of lowering true detection rates. This paper proposes a novel method, Kinematic Regularization and Matrix Rank Optimization (KRMARO), to achieve high true-detection rates and reduce false-detection rates significantly. KRMARO introduces a formulation of the moving objects detection problem that integrates a novel kinematic regularization into the principal component pursuit. This formulation models moving objects as sparse, which is located in regions exhibiting unique kinematic properties, while the background is modeled as a low-rank matrix that is corrupted by this sparse. To solve the former formulation accurately, KRMARO proposes a solution based on the inexact Newton method and the inexact augmented Lagrange multiplier with backtracking behavior. The robustness of KRMARO is verified through testing on DARPA VIVID, UCF aerial action, and VIRAT aerial data sets and then comparing the results with relevant state-of-the-art methods.

Lateral-Slice Sparse Tensor Robust Principal Component Analysis for Hyperspectral Image Classification

This letter proposes a lateral-slice sparse tensor robust principal component analysis (LSSTRPCA) method to remove gross errors or outliers from hyperspectral images so as to promote the performance of subsequent classification. The LSSTRPCA assumes that a three-order hyperspectral tensor has a low-rank structure, and gross errors or outliers are sparsely scattered in a 2-D space (i.e., lateral-slice) of the tensor. It formulates a low-rank and sparse tensor decomposition problem into a convex problem and then implements the inexact augmented Lagrange multiplier method to solve it. The experiments on two hyperspectral data sets show that the LSSTRPCA can successfully remove outliers or gross errors and achieve higher accuracies than both the original robust principal component analysis (RPCA) and tensor robust principal component analysis (TRPCA).

An inexact augmented Lagrangian multiplier method for solving quadratic complementary problems: An adapted algorithmic framework combining specific resolution techniques

In this paper, an inexact augmented Lagrangian multiplier method (ALM) is designed for solving the quadratic complementarity problem (QCP). The primary goal is proposing an algorithm exploring the structures of QCPs and incorporating them into the implementation. In the implementation, an inexact proximal alternating minimization method is employed to solve the augmented Lagrangian subproblems. As a result, all the subproblems are either strongly convex quadratic programs or linear equations with positive semidefinite symmetric coefficient matrices, which in turn can either be solved with explicitly formula solutions, or by an accelerated proximal gradient method, or a conjugate gradient method. Convergence rates of these subproblems are given, and global convergence of the ALM algorithm is established under mild assumptions. Preliminary numerical experiments show that the algorithm is feasible and efficient.

Robust subspace clustering via symmetry constrained latent low rank representation with converted nuclear norm

Subspace clustering, which is devoted to classifying data samples derived from a union of linear subspaces, has been widely applied to various fields such as pattern recognition, artificial intelligence and computer vision. In this paper, we propose a symmetry constrained latent low rank representation with converted nuclear norm (SLLRRC) algorithm for robust subspace clustering, which extends the original latent low rank representation (LLRR) algorithm by introducing a kind of converted nuclear norm and integrating strategy of the symmetric constraint. SLLRRC both enhances the sparsity of the coefficient matrix and guarantees weight consistency for each pair of data samples when seeking the low rank representation. The symmetric coefficient matrix that will no longer be overly dense is acquired by the inexact augmented Lagrange multipliers (IALM) method. With further exploiting the angular information of the principal directions of that, the sparse affinity matrix for spectral clustering is identified. Extensive experimental results on face clustering and motion segmentation datasets confirm the availability and robustness of the proposed algorithm, and it is highly competitive compared to the state-of-the-art subspace clustering algorithms.

A matrix completion-based multiview learning method for imputing missing values in buoy monitoring data

Buoy systems have been deployed to monitor coastal waters, and these systems generate massive amounts of data with high temporal resolution. In practice, however, many of these records are often lost, which adversely impacts further data analysis. Current imputation methods place more emphasis on spatial and temporal correlations but ignore the crucial interactions between different parameters. To fully employ these interactions and further improve the imputation performance, a matrix completion-based multiview learning (MC-MVL) method is proposed in this paper to fill in the missing values in buoy monitoring data. MC-MVL considers three hybrid views, i.e., the temporal-parameter view, the spatial-parameter view and the spatiotemporal view, and each view is formulated as a matrix. The fixed-point continuation with approximate singular value decomposition (FPCA), the non-convex (Non-Convex) and the inexact augmented Lagrange multipliers (IALM) algorithms are separately utilized to reconstruct these three matrices. Then, a ridge regression-based multiview learning algorithm is used to aggregate the estimates of the three views into the final results. We used a buoy monitoring dataset from the Zhejiang coastal area to verify the imputation ability of the proposed model. The results confirm that MC-MVL achieves better performance than do 8 baseline approaches.