Augmented Lagrange Multiplier Method Research Articles

ScIALM: A method for sparse scRNA-seq expression matrix imputation using the Inexact Augmented Lagrange Multiplier with low error

Single-cell RNA sequencing (scRNA-seq) is a high-throughput sequencing technology that quantifies gene expression profiles of specific cell populations at the single-cell level, providing a foundation for studying cellular heterogeneity and patient pathological characteristics. It is effective for developmental, fertility, and disease studies. However, the cell-gene expression matrix of single-cell sequencing data is often sparse and contains numerous zero values. Some of the zero values derive from noise, where dropout noise has a large impact on downstream analysis. In this paper, we propose a method named scIALM for imputation recovery of sparse single-cell RNA data expression matrices, which employs the Inexact Augmented Lagrange Multiplier method to use sparse but clean (accurate) data to recover unknown entries in the matrix. We perform experimental analysis on four datasets, calling the expression matrix after Quality Control (QC) as the original matrix, and comparing the performance of scIALM with six other methods using mean squared error (MSE), mean absolute error (MAE), Pearson correlation coefficient (PCC), and cosine similarity (CS). Our results demonstrate that scIALM accurately recovers the original data of the matrix with an error of 10e-4, and the mean value of the four metrics reaches 4.5072 (MSE), 0.765 (MAE), 0.8701 (PCC), 0.8896 (CS). In addition, at 10%-50% random masking noise, scIALM is the least sensitive to the masking ratio. For downstream analysis, this study uses adjusted rand index (ARI) and normalized mutual information (NMI) to evaluate the clustering effect, and the results are improved on three datasets containing real cluster labels.

TVRPCA＋: Low-rank and sparse decomposition based on spectral norm and structural sparsity-inducing norm

Traditional low-rank sparse decomposition algorithms have trouble obtaining a clear and complete foreground representation in foreground–background separation due to the complex video environment and the noise. For this issue, We propose a more robust and higher-performance low-rank and sparse decomposition algorithm named TVRPCA＋ based on spectral norm, structured sparse norm and total variation (TV) regularization. The structured sparse norm and TV regularization are exploited to suppress noise and obtain much cleaner foregrounds. Spectral norm is used in our algorithm for the low-rank component to address the issue of over-punishment and restore more foreground information. Moreover, an efficient algorithm based on the inexact augmented Lagrange multiplier method is designed to solve the proposed optimization problem. Experimental results show that TVRPCA＋ obtained five top F-measures and three of the second-highest F-measures in eight noise-free test video sequences with complex backgrounds, while the highest average F-measure was also achieved in all ten experimental groups with noise.

Twin extreme learning machine based on heteroskedastic Gaussian noise model and its application in short-term wind-speed forecasting

Extreme learning machine (ELM) has received increasingly more attention because of its high efficiency and ease of implementation. However, the existing ELM algorithms generally suffer from the drawbacks of noise sensitivity and poor robustness. Therefore, we combine the advantages of twin hyperplanes with the fast speed of ELM, and then introduce the characteristics of heteroscedastic Gaussian noise. In this paper, a new regressor is proposed, which is called twin extreme learning machine based on heteroskedastic Gaussian noise (TELM-HGN). In addition, the augmented Lagrange multiplier method is introduced to optimize and solve the presented model. Finally, a significant number of experiments were conducted on different data-sets including real wind-speed data, Boston housing price dataset and stock dataset. Experimental results show that the proposed algorithms not only inherits most of the merits of the original ELM, but also has more stable and reliable generalization performance and more accurate prediction results. These applications demonstrate the correctness and effectiveness of the proposed model.

Trajectory Optimization for Target Localization Using Time Delays and Doppler Shifts in Bistatic Sonar-Based Internet of Underwater Things

Efficient target localization is critical to many marine applications in the Internet of Underwater Things (IoUT). Doppler effect becomes more predominant in the underwater environment when the relative speed of the moving object, i.e., the observer, to the signal propagation speed in water is much larger than that in the air. This along with the observer-target geometry will bring in a notable impact on the localization performance. In this paper, we derive the Cram´er–Rao lower bound (CRLB) and formulate the A-optimality criterion-based observer trajectory optimization problem to improve the localization performance based on time delay and Doppler shift measurements. We show that in the worst-case scenario, there will be a conflict between the non-accessible zone constraint and the observer dynamics constraint, which will lead to an erroneous result. To address this problem, we propose a warning zone-based augmented Lagrange multiplier method (ALMM) where the non-accessible zone constraint is relaxed to resolve the conflict and ensure the non-accessible requirement of the targeted zone is maintained. Performance evaluations are conducted through extensive simulations for different scenarios, and the results are compared to other methods with or without trajectory optimization. We demonstrate that trajectory optimization using time delays and Doppler shifts can greatly improve the target localization accuracy in underwater networks.

Identifying potential small molecule-miRNA associations via Robust PCA based on γ-norm regularization.

Dysregulation of microRNAs (miRNAs) is closely associated with refractory human diseases, and the identification of potential associations between small molecule (SM) drugs and miRNAs can provide valuable insights for clinical treatment. Existing computational techniques for inferring potential associations suffer from limitations in terms of accuracy and efficiency. To address these challenges, we devise a novel predictive model called RPCA$\Gamma $NR, in which we propose a new Robust principal component analysis (PCA) framework based on $\gamma $-norm and $l_{2,1}$-norm regularization and design an Augmented Lagrange Multiplier method to optimize it, thereby deriving the association scores. The Gaussian Interaction Profile Kernel Similarity is calculated to capture the similarity information of SMs and miRNAs in known associations. Through extensive evaluation, including Cross Validation Experiments, Independent Validation Experiment, Efficiency Analysis, Ablation Experiment, Matrix Sparsity Analysis, and Case Studies, RPCA$\Gamma $NR outperforms state-of-the-art models concerning accuracy, efficiency and robustness. In conclusion, RPCA$\Gamma $NR can significantly streamline the process of determining SM-miRNA associations, thus contributing to advancements in drug development and disease treatment.

Multi-view clustering with Laplacian rank constraint based on symmetric and nonnegative low-rank representation

Multi-view spectral clustering has attracted more and more attention due to its promising advantages in data clustering, and many related methods have been proposed, among which the method based on low-rank representation have become increasingly popular. However, the similarity matrices learned by the existing methods inevitably contain some negative entries and are generally nonsymmetric. In order to guarantee weight consistency for each pair of data points and highly correlated data points to be together, we propose a new multi-view spectral clustering model based on the symmetric and nonnegative low-rank representation. This model can learn the view-specific consistent similarity matrix and also capture the local structure and discriminative information among data by imposing symmetric and nonnegative constraints on the representation matrix. By employing the compatible and complementary information from multi-view data points, it can also learn the consensus similarity matrix for all views and perform the spectral clustering simultaneously. By the augmented Lagrangian multiplier method, an alternating iterative optimization algorithm is designed for solving this model and its computational complexity and convergence is analyzed. Comprehensive experiments on multiple multi-view data sets also verify the effectiveness and superiority of the proposed approach.

Day-ahead robust optimal dispatching method for urban power grids containing high proportion of renewable energy

Urban energy supply includes more renewable energy, but renewable energy, such as wind and photovoltaic, is intermittent and connected to the grid and impacts the safe operation of the power system. For urban multi-type energy dispatching, this paper proposed a day-ahead multi-energy robust optimization dispatching method for an urban power grid with a high proportion of renewable power. Firstly, the power supply and load operation scenarios of the urban energy supply system are constructed. Multiple typical scenarios are formed based on fully considering the uncertainties caused by renewable energy and load prediction. A robust optimal scheduling model with system constraints was constructed based on the tuning capacity of adjustable generator sets and energy storage devices in the power grid, with multi-system functional cost optimization as the objective function. Extending the augmented Lagrange multiplier Method, the Alternating Direction Method of Multipliers (ADMM) is proposed to solve the constructed model. The effectiveness of the proposed method is verified by some examples based on the data of an actual power grid in China.

Robust subspace clustering via multi-affinity matrices fusion

Subspace clustering (SC) methods have attracted widespread attention from interested researchers, and a variety of SC methods have been proposed recently. In general, SC methods obtain an affinity matrix, on which spectral clustering is used. Obviously, affinity matrices obtained by different algorithms are not the same, and their clustering effects are also different. A question then arises of whether the affinity matrices obtained by different algorithms can be merged to improve the clustering performance. The answer may be yes I n this paper, we design a new SC method, i.e., robust subspace clustering via multi-affinity matrices fusion (RSC/MAMF). Specifically, several classical SC algorithms, i.e., sparse subspace clustering (SSC), low-rank representation (LRR) and least squares regression (LSR), are first chosen to derive their own affinity matrices. To make full use of the information of different affinity matrices, and to mine the subspace structure thoroughly, we further fuse the derived affinity matrices, i.e., splice them into a 3-order tensor, and impose a weighted tensor nuclear norm (WTNN) to it, which not only mines and fuses the information of the different affinity matrices but also removes their noise. Additionally, to further explore the consistency of the different affinity matrices, spectral embedding is also unified into the final objective function. We propose an optimization algorithm to address the optimization problem of RSC/MAMF, which utilizes the Augmented Lagrange Multiplier (ALM) method. Experiments show that the multi-affinity matrices fusion idea is feasible, and RSC/MAMF outperforms the state-of-the-art SC methods.

A numerical framework for modelling tire mechanics accounting for composite materials, large strains and frictional contact

We present a general framework for the analysis and modelling of frictional contact involving composite materials. The study has focused on composite materials formed by a matrix of rubber and synthetic or metallic fibres, which is the case of standard tires. We detail the numerical treatment of incompressibility at large deformations that rubber can experience, as well as the stiffening effect that properly oriented fibres will induce within the rubber. To solve the frictional contact between solids, a Dual Augmented Lagrangian Multiplier Method is used together with the Mortar method. This ensures a variationally consistent estimation of the contact forces. A modified Serial-Parallel Rule of Mixtures is employed to model the behaviour of composite materials. This is a simple and novel methodology that allows the blending of constitutive behaviours as diverse as rubber (very low stiffness and incompressible behaviour) and steel (high stiffness and compressible behaviour) taking into account the orientation of the fibres within the material. The locking due to the incompressibility constraint in the rubber material has been overcome by using Total Lagrangian mixed displacement-pressure elements. A collection of numerical examples is provided to show the accuracy and consistency of the methodology presented when solving frictional contact, incompressibility and composite materials under finite strains.

Remaining useful life prediction of lithium-ion batteries based on KS agglomeration function integrating multi-expert knowledge

In recent years, lithium-ion batteries have been widely used in the field of new energy, and the prediction of the remaining useful life (RUL) of lithium-ion batteries has become the focus in this field. There are few literatures about the fusion of multi-expert knowledge, integrated into the training process of the BP Neural Network, in RUL prediction based on small sample data, but it can improve the performance of the BP Neural Network and require less training sample data in non-linear regression model, which is the innovation of this paper. When there is a lot of expert knowledge, there is no general framework that can simultaneously integrate multiple pieces of expert knowledge. Therefore, this paper proposes a multi-expert knowledge processing method based on the Kreisselmeier-Steinhauser (KS) agglomeration function and applies it to the prediction of the remaining life of lithium-ion batteries. First, indirect health factors are extracted based on the NASA lithium-ion battery aging dataset. Secondly, the initial weights and thresholds of the Back-Propagation (BP) Neural Network are optimized by a genetic algorithm. The KS agglomeration function is used again to condense multiple pieces of expert knowledge into one. Finally, the expert knowledge is integrated into the training process of the BP Neural Network by the Augmented Lagrange multiplier method. The simulation results show that, compared with the mainstream algorithms, the improved algorithm proposed in this paper reduces the relative error (RE) and mean absolute error (MAE) obviously and has better prediction results. It means that the improved algorithm can the effectively improve the RUL prediction accuracy of lithium-ion batteries when there are many expert knowledges.

Robust and optimal neighborhood graph learning for multi-view clustering

In recent years, researchers have proposed many graph-based multi-view clustering (GMC) algorithms to solve the multi-view clustering (MVC) problem. However, there are still some limitations in the existing GMC algorithm. In these algorithms, a graph is usually constructed to represent the relationship between the samples in a view; however, it cannot represent the relationship very well since it is often preconstructed. In addition, these algorithms ignore the robustness problem of each graph and high-level information between different graphs. Then, in the paper, we propose a novel MVC method, i.e., robust and optimal neighborhood graph learning for MVC (RONGL/MVC). Specifically, we first build an initial graph for each view. However, these initial graphs cannot represent the relationship between the samples in each view well, so we look for the optimal graph with k connected components in the neighborhood of each initial graph, where k is the number of clusters. Then, to improve the robustness of RONGL/MVC, we reconstruct the optimal graph with the self-representation matrix. Furthermore, we stack all the self-representation matrices into a tensor and impose the tensor low-rank constraint, which can maximize consistent features and explore the high-order relationship between optimal graphs. In addition, we provide an optimization strategy by utilizing the Augmented Lagrange Multiplier (ALM) method. Experimental results on several datasets indicate that RONGL/MVC outperforms state-of-the-art methods.

Matrix Completion via Successive Low-rank Matrix Approximation

In this paper, a successive low-rank matrix approximation algorithm is presented for the matrix completion (MC) based on hard thresholding method, which approximate the optimal low-rank matrix from rank-one matrix step by step. The algorithm enables the distance between the matrix with the observed elements and the projection on low-rank manifold to be minimum. The optimal low-rank matrix with observed elements is obtained when the distance is zero. In theory, convergence and convergent error of the new algorithm are analyzed in detail. Furthermore, some numerical experiments show that the algorithm is more effective in CPU time and precision than the orthogonal rank-one matrix pursuit(OR1MP) algorithm and the augmented Lagrange multiplier (ALM) method when the sampling rate is low.

Scalability Improvement of IEEE 802.11ah IoT Networks

In this paper, we propose a non-orthogonal multiple access (NOMA) based grouping method for IEEE 802.11ah, a promising platform for the internet of things (IoT). The grouping method improves the scalability of IoT networks, by reducing collisions in the access points (APs). The proposed method puts those IoT devices (IoT-Ds) whose channel gains are far enough from each other, i.e., who satisfy NOMA constraints, in the same group. Therefore, using successive interference cancellation (SIC), the AP is able to decode the simultaneous signal transmissions from IoT-Ds in a group. To assign IoT-Ds into groups and determine their transmission power, we formulate a total throughput maximization problem as a joint optimal grouping and power allocation problem, which is a non-convex mixed-integer programming problem. We convert it to a convex problem using quadratic fractional programming, and then we solve it using augmented Lagrange multiplier (ALM) method. Moreover, to reduce the complexity of the solution, we propose a fast grouping method to allocate power to each group in parallel. Simulation results show that the proposed methods have outstanding performance compared to conventional association identifier (AID)-based grouping method; besides, scalability of the network in terms of throughput, power consumption and channel utilization improves dramatically because of the collision reduction of IoT-Ds, which is achieved by deploying NOMA and SIC. Furthermore, the fast grouping method decreases the computational complexity greatly at the expense of a small reduction in network performance.

Robust multi-view subspace enhanced representation based on collaborative constraints and HSIC induction

The design of effective multi-view subspace clustering (MSC) algorithms has recently garnered significant research attention. Herein, to effectively improve the recognition performance and anti-noise interference ability of an MSC model, we propose a novel MSC algorithm, termed as robust multi-view subspace enhancement representation, based on collaborative constraints and a Hilbert–Schmidt independence criterion (HSIC) induction method. To mine the complementary information between different views, we apply the HSIC as a diversity regularization term. Specifically, to enhance the diagonal block structure of a subspace representation, a new sparse constraint is introduced on the product of itself and the transpose of the subspace representation matrix in a multi-view subspace learning model. Furthermore, hypergraph regularization and a low-rank idea are considered to capture the local geometric structure and clean data. In addition, to optimize our model, we adopt an augmented Lagrangian multiplier method and discuss the convergence of the model. Extensive experiments on six challenging datasets reveal that the proposed method achieves a highly competent objective performance with and without noisy views, as compared with several state-of-the-art multi-view clustering methods.

One-step incomplete multiview clustering with low-rank tensor graph learning

Incomplete multiview clustering (IMVC) has attracted extensive attention in the field of machine learning due to its excellent performance in handling incomplete multiview data (IMVD). However, existing IMVC methods often use spectral clustering or k-means on the obtained affinity matrices to obtain the final clustering results, that is, the affinity matrices learning process and clustering process are separated. In this paper, we propose a new ingenious IMVC method, one-step incomplete multiview clustering with low-rank tensor graph learning (OIMVC/LTGL), to solve the problem. This method combines graph learning, low-rank tensor constraint, common representation learning and clustering into a unified optimization framework. First, we utilize the adjacency relationship between views to complete the similarity graph matrices. Second, we stack all similarity graph matrices into a third-order tensor. To mine higher-order correlations among different views, we impose a low-rank tensor constraint, the tensor nuclear norm (TNN), into the constructed tensor. Third, to explore consistent information between views, we introduce a common representation learning term to learn the optimal consensus representation. Last, we introduce spectral rotation to the consensus representation matrices to directly obtain the clustering labels. These steps promote each other for better clustering performance. We solve the optimization problem by using the augmented Lagrange multiplier (ALM) method. Experiments based on five well-known datasets show the superiority of our method.

Random noise attenuation with weak feature preservation via total variation regularization

Dictionary learning (DL) methods have been successfully applied to seismic data denoising. However, when the noise is strong, weak signals cannot be well preserved. Considering the characteristics of seismic data in the spatial-time domain, we propose a dictionary learning method with total variation regularization (DLTV). The DLTV method consists of two steps. The first step is to learn the dictionary; it uses the sparse feature of the data in the dictionary domain and the learned dictionary contains data features. The second step is to use the augmented Lagrange multiplier method to restore the data, and total variation regularization to preserve the weak signal by sampling data information around it. An example with seismic data shows that the proposed method retains the weak features better than the traditional F-X deconvolution (FX-Decon) method and the data-driven tight frame method (DDTF).

High-order manifold regularized multi-view subspace clustering with robust affinity matrices and weighted TNN

Multi-view subspace clustering achieves impressive performance for high-dimensional data. However, many of these models do not sufficiently mine the intrinsic information among samples and consider the robustness problem of the affinity matrices, resulting in the degradation of clustering performance. To address these problems, we propose a novel high-order manifold regularized multi-view subspace clustering with robust affinity matrices and a weighted tensor nuclear norm (TNN) model (termed HMRMSC) to characterize real-world data. Specifically, all the similarity matrices of different views are first stacked into a third-order tensor. However, the constructed tensor may contain an additional inter-class representation since the data are usually noisy. Then, we use a technique similar to tensor principal component analysis (TPCA) to obtain a more robust similarity tensor, which is constrained by the so-called weighted TNN since the original TNN treats each singular value equally and usually considers no prior information of singular values. In addition, a high-order manifold regularized term is also added to utilize the manifold information of data. Finally, all the steps are unified into a framework, which is resolved by the augmented Lagrange multiplier (ALM) method. Experimental results on six representative datasets show that our model outperforms several state-of-the-art counterparts.

Total Variation Weighted Low-Rank Constraint for Infrared Dim Small Target Detection

Infrared dim small target detection is the critical technology in the situational awareness field currently. The detection algorithm of the infrared patch image (IPI) model combined with the total variation term is a recent research hotspot in this field, but there is an obvious staircase effect in target detection, which reduces the detection accuracy to some extent. This paper further investigates the problem of accurate detection of infrared dim small targets and a novel method based on total variation weighted low-rank constraint (TVWLR) is proposed. According to the overlapping edge information of image background structure characteristics, the weights of constraint low-rank items are adaptively determined to effectively suppress the staircase effect and enhance the details. Moreover, an optimization algorithm combined with the augmented Lagrange multiplier method is proposed to solve the established TVWLR model. Finally, the experimental results of multiple sequence images indicate that the proposed algorithm has obvious improvements in detection accuracy, including receiver operating characteristic (ROC) curve, background suppression factor (BSF) and signal-to-clutter ratio gain (SCRG). Furthermore, the proposed method has stronger robustness under complex background conditions such as buildings and trees.

Subspace clustering via joint [formula omitted] and [formula omitted] norms

Some of the data collected from practical applications are usually heavily corrupted. In subspace clustering, the common method is to use the specific regularization strategy to correct these corrupted data by virtue of the prior knowledge, which could result in a suboptimal clustering solution. To alleviate the problem, we develop a novel formulation named subspace clustering via joint ℓ1,2 and ℓ2,1 (L12-21) norms (SCJL12-21). Specifically, we identify and exclude the heavily corrupted data points (unimportant data points) from participating in the linear representation of other points by imposing the ℓ1,2 norm on the representation matrix, and improve the robustness to outliers by enforcing the ℓ2,1 constraint on the error matrix. The joint ℓ1,2 and ℓ2,1 minimization leads to a good representation matrix which enhances the clustering performance. Related ℓ1,2 and ℓ2,1 norm constrained optimization problem is solved by utilizing the augmented Lagrange multiplier method. The effectiveness of the proposed method is demonstrated through experiments on the constructed datasets as well as the two practical problems of motion segmentation and face clustering.

Analysis of Wide-Frequency Dense Signals Based on Fast Minimization Algorithm

To improve the detection speed for wide-frequency dense signals (WFDSs), a fast minimization algorithm (FMA) was proposed in this study. Firstly, this study modeled the WFDSs and performed a Taylor-series expansion of the sampled model. Secondly, we simplified the sampling model based on the augmented Lagrange multiplier (ALM) method and then calculated the augmented Lagrange function of the sampling model. Finally, according to the alternating minimization strategy, the Lagrange multiplier vector and the sparse block phasor in the function were iterated individually to realize the measurement of the original signal components. The results show that the algorithm improved the analysis accuracy of the WFDS by 35% to 46% on the IEEE C37.118.1a-2014 standard for the wide-frequency noise test, harmonic modulation test, and step-change test, providing a theoretical basis for the development of the P-class phasor measurement unit (PMU).