Nonconvex Relaxation Research Articles

A Novel Nonconvex Relaxation Approach to Low-Rank Matrix Completion of Inexact Observed Data

A Novel Nonconvex Relaxation Approach to Low-Rank Matrix Completion of Inexact Observed Data

A log-based non-convex relaxation regularized regression for robust face recognition

Regression methods are widely employed in face recognition applications. The vector-based norm regression model only represents the pixel by pixel correlation and exhibits poor performance with contiguous occlusion. In contrast, methods based on the nuclear norm describe the low-rank structural information, but it is difficult to achieve satisfactory performance when dealing with complex noise. To address this, a log-based non-convex relaxation regularized regression (log-NCRR) model for robust face recognition is proposed in this paper. We adopt the log-based matrix loss without additional parameters to characterize the low-rank part of error image. Considering sparsity, we design a log-based vector loss to describe the sparse part of the error matrix, which achieves a balance between the l0-norm and l1-norm. Additionally, a weighted non-convex relaxation of l2,1-norm is proposed to ensure the group sparsity of the regression coefficient. The optimization problem is optimized by ADMM, with the corresponding sub-problems easily solved by the generalized singular value shrinkage operator. Finally, experimental results on four public databases for various types of noise validate the effectiveness and robustness of log-NCRR in face recognition.

A Novel Tensor Ring Sparsity Measurement for Image Completion.

As a promising data analysis technique, sparse modeling has gained widespread traction in the field of image processing, particularly for image recovery. The matrix rank, served as a measure of data sparsity, quantifies the sparsity within the Kronecker basis representation of a given piece of data in the matrix format. Nevertheless, in practical scenarios, much of the data are intrinsically multi-dimensional, and thus, using a matrix format for data representation will inevitably yield sub-optimal outcomes. Tensor decomposition (TD), as a high-order generalization of matrix decomposition, has been widely used to analyze multi-dimensional data. In a direct generalization to the matrix rank, low-rank tensor modeling has been developed for multi-dimensional data analysis and achieved great success. Despite its efficacy, the connection between TD rank and the sparsity of the tensor data is not direct. In this work, we introduce a novel tensor ring sparsity measurement (TRSM) for measuring the sparsity of the tensor. This metric relies on the tensor ring (TR) Kronecker basis representation of the tensor, providing a unified interpretation akin to matrix sparsity measurements, wherein the Kronecker basis serves as the foundational representation component. Moreover, TRSM can be efficiently computed by the product of the ranks of the mode-2 unfolded TR-cores. To enhance the practical performance of TRSM, the folded-concave penalty of the minimax concave penalty is introduced as a nonconvex relaxation. Lastly, we extend the TRSM to the tensor completion problem and use the alternating direction method of the multipliers scheme to solve it. Experiments on image and video data completion demonstrate the effectiveness of the proposed method.

Tensor recovery based on rank adaptive and non-convex methods

The growing popularity of tensor singular value decomposition (T-SVD) in tensor recovery problems and the N-tubal rank can be applied to higher order tensors. However, this method faces new challenges as it does not fully use the rank prior information of tensors and lacks non-convex relaxation methods. Thus, this paper applies a new non-convex function, called Minimax Logarithmic Concave Penalty (MLCP), based on the N-tubal rank method. Two MLCP models based on N-tubal rank are introduced for solving low-rank tensor completion (LRTC) and tensor robust principal component analysis (TRPCA) problems, along with corresponding solving algorithms. The effectiveness and superiority of these models are demonstrated through experiments on real datasets.

NF-3DLogTNN: An effective hyperspectral and multispectral image fusion method based on nonlocal low-fibered-rank regularization

In this paper, we develop a framelet representation of the three-directional log-based tensor nuclear norm (F-3DLogTNN) for hyperspectral and multispectral image fusion (HSI-MSI fusion). The three-directional log-based tensor nuclear norm, as the nonconvex relaxation of tensor fibered rank, is computed by mode-n tensor singular value decomposition (t-SVD) based on the mode-n tensor-tensor product (t-prod). The mode-n t-prod is based on the mode-n discrete Fourier transform. Hereafter, we suggest using the mode-n framelet transform to define the mode-n t-prod and subsequently the mode-n t-SVD. Due to the redundancy of the framelet basis, each fiber along mode-n of the tensor can be sparsely represented. When the tensor’s slices (including frontal, lateral, and horizontal slices) are correlated, respectively, the corresponding sum of the rank of framelet transformed slices should be small. Thereby, there has lower tensor fibered rank performance. With that, we propose a novel nonlocal low-fibered-rank regularization to depict the local spatial-spectral correlation and nonlocal self-similarity of high-spatial-resolution hyperspectral image. Since minimizing the fibered rank directly is an NP-hard problem, we suggest F-3DLogTNN as its nonconvex relaxation. Subsequently, nonlocal based F-3DLogTNN (NF-3DLogTNN) method is developed for HSI-MSI fusion. To deal with the proposed model, we design an algorithm based on the alternating direction multipliers method. Experimental results on three datasets prove the proposed method’s superiority over the related state-of-the-art HSI-MSI fusion methods.

Matrix Completion via Non-Convex Relaxation and Adaptive Correlation Learning.

The existing matrix completion methods focus on optimizing the relaxation of rank function such as nuclear norm, Schatten- p norm, etc. They usually need many iterations to converge. Moreover, only the low-rank property of matrices is utilized in most existing models and several methods that incorporate other knowledge are quite time-consuming in practice. To address these issues, we propose a novel non-convex surrogate that can be optimized by closed-form solutions, such that it empirically converges within dozens of iterations. Besides, the optimization is parameter-free and the convergence is proved. Compared with the relaxation of rank, the surrogate is motivated by optimizing an upper-bound of rank. We theoretically validate that it is equivalent to the existing matrix completion models. Besides the low-rank assumption, we intend to exploit the column-wise correlation for matrix completion, and thus an adaptive correlation learning, which is scaling-invariant, is developed. More importantly, after incorporating the correlation learning, the model can be still solved by closed-form solutions such that it still converges fast. Experiments show the effectiveness of the non-convex surrogate and adaptive correlation learning.

Multimodal Core Tensor Factorization and its Applications to Low-Rank Tensor Completion

Low-rank tensor completion has been widely used in computer vision and machine learning. This paper develops a novel multimodal core tensor factorization (MCTF) method combined with a tensor low-rankness measure and a better nonconvex relaxation form of this measure (NC-MCTF). The proposed models encode low-rank insights for general tensors provided by Tucker and T-SVD and thus are expected to simultaneously model spectral low-rankness in multiple orientations and accurately restore the data of intrinsic low-rank structure based on few observed entries. Furthermore, we study the MCTF and NC-MCTF regularization minimization problem and design an effective block successive upper-bound minimization (BSUM) algorithm to solve them. Theoretically, we prove that the iterates generated by the proposed models converge to the set of coordinatewise minimizers. This efficient solver can extend MCTF to various tasks such as tensor completion. A series of experiments including hyperspectral image (HSI), video and MRI completion confirm the superior performance of the proposed method.

Tensor Recovery Based on a Novel Non-Convex Function Minimax Logarithmic Concave Penalty Function.

Non-convex relaxation methods have been widely used in tensor recovery problems, compared with convex relaxation methods, and can achieve better recovery results. In this paper, a new non-convex function, Minimax Logarithmic Concave Penalty (MLCP) function, is proposed, and some of its intrinsic properties are analyzed, among which it is interesting to find that the Logarithmic function is an upper bound of the MLCP function. The proposed function is generalized to tensor cases, yielding tensor MLCP and weighted tensor Lγ -norm. Consider that its explicit solution cannot be obtained when applying it directly to the tensor recovery problem. Therefore, the corresponding equivalence theorems to solve the such problem are given, namely, tensor equivalent MLCP theorem and equivalent weighted tensor Lγ -norm theorem. In addition, we propose two EMLCP-based models for classic tensor recovery problems, namely low-rank tensor completion (LRTC) and tensor robust principal component analysis (TRPCA), and design proximal alternate linearization minimization (PALM) algorithms to solve them individually. Furthermore, based on the Kurdyka-Åasiwicz property, it is proved that the solution sequence of the proposed algorithm has a finite length and converges to the critical point globally. Finally, extensive experiments show that the proposed algorithm achieves good results, and it is confirmed that the MLCP function is indeed better than the Logarithmic function in the minimization problem, which is consistent with the analysis of theoretical properties.

Nonconvex Tensor Completion for 5-D Seismic Data Reconstruction

Multidimensional pre-stack seismic data reconstruction can be viewed as a low-rank tensor completion problem. Recently, the nuclear norm has been widely used as a convex surrogate of the tensor rank function for low-rank tensor recovery and has been successfully applied to 5D seismic data reconstruction. However, solving the nuclear norm-based relaxed convex problem typically leads to a suboptimal solution of the original rank minimization problem, often degrading the reconstruction performance. In this study, to seek solutions to the aforementioned problems, we established a non-convex logDet function as a smooth approximation for the tensor rank instead of the convex tensor nuclear norm and applied it to solve the 5D seismic data reconstruction problem. Thereafter, we propose solving the obtained non-convex relaxation problem using an alternating direction method of multipliers (ADMM) algorithm. Numerical experiments of our approach on synthetic 5D seismic data demonstrated remarkable reconstruction performance compared with the performances of HOSVD, nulear norm, and PMF methods in terms of visual examination and numerical test. We further illustrate the performance of the proposed method using a land data survey.

A matrix nonconvex relaxation approach to unconstrained binary polynomial programs

A matrix nonconvex relaxation approach to unconstrained binary polynomial programs

A new nonconvex relaxation approach for low tubal rank tensor recovery

In this paper, we consider the tensor recovery problem within the low tubal rank framework. A new nonconvex surrogate is proposed to approximate the tensor tubal rank. The main advantage is that it uses a class of nonconvex functions to penalize the singular tubes directly instead of penalizing the singular values as in existing methods. Our proposed surrogate can continuously approximate the tubal rank without breaking its composition structure and keep the intrinsic structural information of the tensor better than existing methods. We then develop an efficient iteratively reweighted tube thresholding algorithm to solve the tensor recovery model equipped with the new tubal rank surrogate and provide the theoretical guarantee for convergence. Simulation results on synthetical and practical data demonstrate the superior performance of the proposed method over several widely used methods in the literature.

Low-rank matrix factorization with nonconvex regularization and bilinear decomposition

Low-rank matrix factorization (LRMF) is a powerful approach that can recover useful information with low-rank features from corrupted data and has been broadly applied in various scenarios such as computer vision and statistics. Existing convex relaxation-based LRMF models' performance and solutions are often limited by unsatisfied recovery accuracy and prolonged execution time due to excessive relaxation bias and heavy computational burden. In this paper, we first explore one class of parameterized non-convex penalty function and generalize it as a non-convex relaxation substitution term for the original rank function in the LRMF, which can effectively enhance the recovery capability for low-rank data. In order to reduce the computational burden of the nuclear-norm-based optimization scheme and make LRMF applicable to large-scale data. We propose to perform bilinear decomposition on the low-rank matrix and derive the bilinear factorization formula under the parameterized non-convex alternative. With the above considering, a low-rank matrix recovery model via bilinear decomposition and non-convex regularization, denoted as NCBF, is formulated. Further, the optimal solution algorithm for the NCBF model is developed in the framework of the block coordinate descent, combining simultaneously three strategies of decoupling, proximal gradient descent, and block prox-linear. Also, the convergence condition for the proposed algorithm has been given. We perform extensive experiments using synthetic data and real-world image datasets. The synthetic data test results demonstrate the dual superiority of the bilinear decomposition scheme in terms of operation time and estimation accuracy. Application of denoising to both generic and CT image datasets, experimental results show that NCBF outperforms several existing methods under both subjective and objective metrics evaluations.

On generalized surrogate duality in mixed-integer nonlinear programming

The most important ingredient for solving mixed-integer nonlinear programs (MINLPs) to global epsilon -optimality with spatial branch and bound is a tight, computationally tractable relaxation. Due to both theoretical and practical considerations, relaxations of MINLPs are usually required to be convex. Nonetheless, current optimization solvers can often successfully handle a moderate presence of nonconvexities, which opens the door for the use of potentially tighter nonconvex relaxations. In this work, we exploit this fact and make use of a nonconvex relaxation obtained via aggregation of constraints: a surrogate relaxation. These relaxations were actively studied for linear integer programs in the 70s and 80s, but they have been scarcely considered since. We revisit these relaxations in an MINLP setting and show the computational benefits and challenges they can have. Additionally, we study a generalization of such relaxation that allows for multiple aggregations simultaneously and present the first algorithm that is capable of computing the best set of aggregations. We propose a multitude of computational enhancements for improving its practical performance and evaluate the algorithm’s ability to generate strong dual bounds through extensive computational experiments.

Packing under convex quadratic constraints

We consider a general class of binary packing problems with a convex quadratic knapsack constraint. We prove that these problems are mathsf {APX}-hard to approximate and present constant-factor approximation algorithms based upon two different algorithmic techniques: a rounding technique tailored to a convex relaxation in conjunction with a non-convex relaxation, and a greedy strategy. We further show that a combination of these techniques can be used to yield a monotone algorithm leading to a strategyproof mechanism for a game-theoretic variant of the problem. Finally, we present a computational study of the empirical approximation of these algorithms for problem instances arising in the context of real-world gas transport networks.

Low-rank factorization for rank minimization with nonconvex regularizers

Rank minimization is of interest in machine learning applications such as recommender systems and robust principal component analysis. Minimizing the convex relaxation to the rank minimization problem, the nuclear norm, is an effective technique to solve the problem with strong performance guarantees. However, nonconvex relaxations have less estimation bias than the nuclear norm and can more accurately reduce the effect of noise on the measurements. We develop efficient algorithms based on iteratively reweighted nuclear norm schemes, while also utilizing the low rank factorization for semidefinite programs put forth by Burer and Monteiro. We prove convergence and computationally show the advantages over convex relaxations and alternating minimization methods. Additionally, the computational complexity of each iteration of our algorithm is on par with other state of the art algorithms, allowing us to quickly find solutions to the rank minimization problem for large matrices.

Nonconvex Low-Rank Kernel Sparse Subspace Learning for Keyframe Extraction and Motion Segmentation.

By exploiting the kernel trick, the sparse subspace model is extended to the nonlinear version with one or a combination of predefined kernels, but the high-dimensional space induced by predefined kernels is not guaranteed to be able to capture the features of the nonlinear data in theory. In this article, we propose a nonconvex low-rank learning framework in an unsupervised way to learn a kernel to replace the predefined kernel in the sparse subspace model. The learned kernel by a nonconvex relaxation of rank can better exploiting the low-rank property of nonlinear data to induce a high-dimensional Hilbert space that more closely approaches the true feature space. Furthermore, we give a global closed-form optimal solution of the nonconvex rank minimization and prove it. Considering the low-rank and sparseness characteristics of motion capture data in its feature space, we use them to verify the better representation of nonlinear data with the learned kernel via two tasks: keyframe extraction and motion segmentation. The performances on both tasks demonstrate the advantage of our model over the sparse subspace model with predefined kernels and some other related state-of-art methods.

A Continuous Relaxation of the Constrained $$\ell _2-\ell _0$$ Problem

We focus on the minimization of the least square loss function under a k-sparse constraint encoded by a \(\ell _0\) pseudo-norm. This is a non-convex, non-continuous and NP-hard problem. Recently, for the penalized form (sum of the least square loss function and a \(\ell _0\) penalty term), a relaxation has been introduced which has strong results in terms of minimizers. This relaxation is continuous and does not change the global minimizers, among other favorable properties. The question that has driven this paper is the following: can a continuous relaxation of the k-sparse constraint problem be developed following the same idea and same steps as for the penalized \(\ell _2-\ell _0\) problem? We calculate the convex envelope of the constrained problem when the observation matrix is orthogonal and propose a continuous non-smooth, non-convex relaxation of the k-sparse constraint functional. We give some equivalence of minimizers between the original and the relaxed problems. The subgradient is calculated as well as the proximal operator of the new regularization term, and we propose an algorithm that ensures convergence to a critical point of the k-sparse constraint problem. We apply the algorithm to the problem of single-molecule localization microscopy and compare the results with well-known sparse minimization schemes. The results of the proposed algorithm are as good as the state-of-the-art results for the penalized form, while fixing the constraint constant is usually more intuitive than fixing the penalty parameter.

Global Convergence Guarantees of (A)GIST for a Family of Nonconvex Sparse Learning Problems.

In recent years, most of the studies have shown that the generalized iterated shrinkage thresholdings (GISTs) have become the commonly used first-order optimization algorithms in sparse learning problems. The nonconvex relaxations of the l0 -norm usually achieve better performance than the convex case (e.g., l1 -norm) since the former can achieve a nearly unbiased solver. To increase the calculation efficiency, this work further provides an accelerated GIST version, that is, AGIST, through the extrapolation-based acceleration technique, which can contribute to reduce the number of iterations when solving a family of nonconvex sparse learning problems. Besides, we present the algorithmic analysis, including both local and global convergence guarantees, as well as other intermediate results for the GIST and AGIST, denoted as (A)GIST, by virtue of the Kurdyka-Łojasiewica (KŁ) property and some milder assumptions. Numerical experiments on both synthetic data and real-world databases can demonstrate that the convergence results of objective function accord to the theoretical properties and nonconvex sparse learning methods can achieve superior performance over some convex ones.

Power Allocation for Reducing PAPR of Artificial-Noise-Aided Secure Communication System

The research of improving the secrecy capacity (SC) of wireless communication system using artificial noise (AN) is one of the classic models in the field of physical layer security communication. In this paper, we consider the peak-to-average power ratio (PAPR) problem in this AN-aided model. A power allocation algorithm for AN subspaces is proposed to solve the nonconvex optimization problem of PAPR. This algorithm utilizes a series of convex optimization problems to relax the nonconvex optimization problem in a convex way based on fractional programming, difference of convex (DC) functions programming, and nonconvex quadratic equality constraint relaxation. Furthermore, we also derive the SC of the proposed signal under the condition of the AN-aided model with a finite alphabet and the nonlinear high-power amplifiers (HPAs). Simulation results show that the proposed algorithm reduces the PAPR value of transmit signal to improve the efficiency of HPA compared with benchmark AN-aided secure communication signals in the multiple-input single-output (MISO) model.

Hyperspectral Image Recovery Using Non-Convex Low-Rank Tensor Approximation

Low-rank tensors have received more attention in hyperspectral image (HSI) recovery. Minimizing the tensor nuclear norm, as a low-rank approximation method, often leads to modeling bias. To achieve an unbiased approximation and improve the robustness, this paper develops a non-convex relaxation approach for low-rank tensor approximation. Firstly, a non-convex approximation of tensor nuclear norm (NCTNN) is introduced to the low-rank tensor completion. Secondly, a non-convex tensor robust principal component analysis (NCTRPCA) method is proposed, which aims at exactly recovering a low-rank tensor corrupted by mixed-noise. The two proposed models are solved efficiently by the alternating direction method of multipliers (ADMM). Three HSI datasets are employed to exhibit the superiority of the proposed model over the low rank penalization method in terms of accuracy and robustness.