Proximal Alternating Minimization Method Research Articles

Proximal alternating minimization method for Poisson noise removal

Proximal alternating minimization method for Poisson noise removal

Linear Convergence of a Proximal Alternating Minimization Method with Extrapolation for \(\boldsymbol{\ell_1}\) -Norm Principal Component Analysis

A popular robust alternative of the classic principal component analysis (PCA) is the -norm PCA (L1-PCA), which aims to find a subspace that captures the most variation in a dataset as measured by the -norm. L1-PCA has shown great promise in alleviating the effect of outliers in data analytic applications. However, it gives rise to a challenging nonsmooth, nonconvex optimization problem, for which existing algorithms are either not scalable or lack strong theoretical guarantees on their convergence behavior. In this paper, we propose a proximal alternating minimization method with extrapolation (PAMe) for solving a two-block reformulation of the L1-PCA problem. We then show that for both the L1-PCA problem and its two-block reformulation, the Kurdyka–Łojasiewicz exponent at any of the limiting critical points is . This allows us to establish the linear convergence of the sequence of iterates generated by PAMe and to determine the criticality of the limit of the sequence with respect to both the L1-PCA problem and its two-block reformulation. To complement our theoretical development, we show via numerical experiments on both synthetic and real-world datasets that PAMe is competitive with a host of existing methods. Our results not only significantly advance the convergence theory of iterative methods for L1-PCA but also demonstrate the potential of our proposed method in applications.

An Iterative Regularization Method Based on Tensor Subspace Representation for Hyperspectral Image Super-Resolution

Hyperspectral image super-resolution (HSI-SR) can be achieved by fusing a paired multispectral image (MSI) and hyperspectral image (HSI), which is a prevalent strategy. But, how to precisely reconstruct the high spatial resolution hyperspectral image (HR-HSI) by fusion technology is a challenging issue. In this paper, we propose an iterative regularization method based on tensor subspace representation (IR-TenSR) for MSI-HSI fusion, thus HSI-SR. First, we propose a tensor subspace representation (TenSR)-based regularization model that integrates the global spectral-spatial low-rank and the nonlocal self-similarity priors of HR-HSI. These two priors have been proven effective, but previous HSI-SR works cannot simultaneously exploit them. Subsequently, we design an iterative regularization procedure to utilize the residual information of acquired low-resolution images, which are ignored in other works that produce suboptimal results. Finally, we develop an effective algorithm based on the proximal alternating minimization method to solve the TenSR-regularization model. With that, we obtain the iterative regularization algorithm. Experiments implemented on the simulated and real datasets illustrate the advantages of the proposed IR-TenSR compared with state-of-the-art fusion approaches. The code is available at https://github.com/liangjiandeng/IR-TenSR.

Proximal alternating minimization method for adaptive TGV-based image restoration

This article presents an adaptive total generalized variation regularized strategy for image reconstruction. Unlike the traditional fixed weights schemes, our weights can be adaptively updated according to the latest computations. This helps to obtain more accurate numerical solutions and avoid the troublesome parameters selection. Subsequently, by artfully employing the Moreau decomposition and proximal algorithm, we develop a highly efficient proximal alternating minimization method to optimize the objective function in detail. The introduced technique has the capability of automatically estimating the regularization parameter and restoring the deteriorated image. Finally, several numerical simulations concertedly illustrate the competitive superiority of our proposed scheme for image restoration, especially in terms of suppressing staircase artifacts and maintaining edge details.

Semi-blind image deblurring by a proximal alternating minimization method with convergence guarantees

The semi-blind image deblurring problem aims to simultaneously estimate the clean image and the point spread function (PSF), which results in a (jointly) nonconvex optimization problem. In this paper, we develop an efficient algorithm to tackle the corresponding minimization problem based on the framework of the proximal alternating minimization (PAM). We also establish the convergence of the proposed algorithm under a mild assumption. Numerical experiments demonstrate our approach could obtain a more robust performance than the related state-of-the-art semi-blind image deblurring method.

A Discriminative Projection and Representation-Based Classification Framework for Face Recognition

The sparse representation-based classifier (SRC) has been developed and verified as having great potential for real-world face recognition. In this paper, we propose a discriminative projection and representation-based classification (DPRC) method to enhance the discriminant ability of the SRC. The proposed method first obtains a discriminative projection matrix not only maximizing the ratio of the distance within interclass over the distance within intraclass, but also minimizing the linear approximation error within intraclass. Then it maps the original data onto the discriminative space, and adopts an SRC method to obtain the final solution. An inexact augmented Lagrangian method of multiplier is proposed for finding the optimal representation vector in our framework, and a proximal alternating minimization method is adopted to the iteration subproblems of the proposed method. The proposed method is proven to have the subsequence convergence property. Experimental results on Yale, ORL, and AR face image databases demonstrate that, compared with some existing feature extraction methods based on the SRC, the proposed DPRC method is more efficient.

Majorized Proximal Alternating Imputation for regularized rank constrained matrix completion

Low rank approximation in the presence of missing data is ubiquitous in many areas of science and engineering. As an NP-hard problem, spectral regularization (e.g., nuclear norm, Schatten p-norm) is widely used. However, solving this problem directly can be computationally expensive due to the unknown rank of variables and full rank Singular Value Decompositions (SVD). Bilinear factorization is efficient and effectively benefits from the fast numerical optimization. However, most of existing methods require explicit knowledge of the rank and often tend to be flatlining or over fitting. To balance and take advantage of both, we formulate the matrix completion problem under the regularized rank constrained conditions. We relax this problem to the surrogates with Majorization Minimization (MM). By employing a proximal alternating minimization method to optimize the corresponding surrogates, we propose an efficient and accurate algorithm named Majorized Proximal Alternating Imputation (MPA-IMPUTE) algorithm. The proposed method iteratively replaces the missing data at each step with the most recent estimate, and only involves simple matrix computation and small scale SVD. Theoretically, the proposed method is guaranteed to converge to the stationary point under mild conditions. Moreover, the proposed method can be generalized to a family of non-convex regularization problems. Extensive experiments over synthetic data and real-world dataset demonstrate the superior performance of our method.

A sparse rank-1 approximation algorithm for high-order tensors

For the best sparse rank-1 approximation to higher-order tensors, we propose a proximal alternating minimization method in which each iteration can be easily computed. Its global convergence is established without any assumption and the numerical experiments are provided to show the efficiency of the proposed method.

An inexact augmented Lagrangian method for computing strongly orthogonal decompositions of tensors

A strongly orthogonal decomposition of a tensor is a rank one tensor decomposition with the two component vectors in each mode of any two rank one tensors are either colinear or orthogonal. A strongly orthogonal decomposition with few number of rank one tensors is favorable in applications, which can be represented by a matrix-tensor multiplication with orthogonal factor matrices and a sparse tensor; and such a decomposition with the minimum number of rank one tensors is a strongly orthogonal rank decomposition. Any tensor has a strongly orthogonal rank decomposition. In this article, computing a strongly orthogonal rank decomposition is equivalently reformulated as solving an optimization problem. Different from the ill-posedness of the usual optimization reformulation for the tensor rank decomposition problem, the optimization reformulation of the strongly orthogonal rank decomposition of a tensor is well-posed. Each feasible solution of the optimization problem gives a strongly orthogonal decomposition of the tensor; and a global optimizer gives a strongly orthogonal rank decomposition, which is however difficult to compute. An inexact augmented Lagrangian method is proposed to solve the optimization problem. The augmented Lagrangian subproblem is solved by a proximal alternating minimization method, with the advantage that each subproblem has a closed formula solution and the factor matrices are kept orthogonal during the iteration. Thus, the algorithm always can return a feasible solution and thus a strongly orthogonal decomposition for any given tensor. Global convergence of this algorithm to a critical point is established without any further assumption. Extensive numerical experiments are conducted, and show that the proposed algorithm is quite promising in both efficiency and accuracy.

An inexact augmented Lagrangian multiplier method for solving quadratic complementary problems: An adapted algorithmic framework combining specific resolution techniques

In this paper, an inexact augmented Lagrangian multiplier method (ALM) is designed for solving the quadratic complementarity problem (QCP). The primary goal is proposing an algorithm exploring the structures of QCPs and incorporating them into the implementation. In the implementation, an inexact proximal alternating minimization method is employed to solve the augmented Lagrangian subproblems. As a result, all the subproblems are either strongly convex quadratic programs or linear equations with positive semidefinite symmetric coefficient matrices, which in turn can either be solved with explicitly formula solutions, or by an accelerated proximal gradient method, or a conjugate gradient method. Convergence rates of these subproblems are given, and global convergence of the ALM algorithm is established under mild assumptions. Preliminary numerical experiments show that the algorithm is feasible and efficient.

Overcomplete Transform Learning With the <inline-formula> <tex-math notation="LaTeX">$log$ </tex-math> </inline-formula> Regularizer

Transform learning has been proposed as a new and effective formulation for analysis dictionary learning, where the $\ell _{0}$ norm or the $\ell _{1}$ norm are generally used as sparsity constraint. The sparse solutions can be obtained by the hard thresholding or the soft thresholding. The hard thresholding is actually a greedy algorithm, which only obtains the approximate solutions; while the soft thresholding has a certain bias for the large elements. In this paper, we propose to employ the $log$ regularizer instead of the $\ell _{0}$ norm and the $\ell _{1}$ norm in the overcomplete transform learning problem. Our minimization problem is nonconvex due to the $log$ regularizer. We propose to employ a simple proximal alternating minimization method, where a closed-form solution of the $log$ function could be obtained based on the proximal operator. Hence, an efficient and fast overcomplete transform learning algorithm is developed, which iterates based on the analysis coding stage and the transform update stage. The proposed algorithm can obtain sparser solutions and more accurate results from the theoretical analysis. Numerical experiments verify that the proposed algorithm outperforms existing transform learning approaches with the $\ell _{0}$ norm or the $\ell _{1}$ norm. Furthermore, the proposed algorithm is on par with the state-of-the-art image denoising algorithms.

Random Coordinate Descent Methods for <inline-formula> <tex-math notation="TeX">$\ell_{0}$</tex-math></inline-formula> Regularized Convex Optimization

In this paper, we study the minimization of ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> regularized optimization problems, where the objective function is composed of a smooth convex function and the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> regularization. We analyze optimality conditions for this nonconvex problem which lead to the separation of local minima into two restricted classes that are nested and around the set of global minima. Based on these restricted classes of local minima, we devise two new random coordinate descent type methods for solving these problems. In particular, we analyze the convergence properties of an iterative hard thresholding based random coordinate descent algorithm for which we prove that any limit point is a local minimum from the first restricted class of local minimizers. Then, we analyze the convergence of a random proximal alternating minimization method and show that any limit point of this algorithm is a local minima from the second restricted class of local minimizers. We also provide numerical experiments which show the superior behavior of our methods in comparison with the usual iterative hard thresholding algorithm.