Alternating Minimization Method Research Articles

Resolution Enhancement in Terahertz Digital In-line Holography by Sparsity-Based Extrapolation

We introduce a generalized extrapolation framework as a constrained optimization problem to enhance the resolution in terahertz digital in-line holography. The alternating minimization method is employed to iteratively extrapolate the hologram beyond the actually detecting area and to reconstruct the complex amplitude distribution of the object wavefront. Within this framework, we propose a sparsity-based extrapolation model based on L1-norm, where the object mask is not required and the generalized positive absorption constraint can be utilized. This work can achieve super-resolution reconstruction completely by numerical postprocessing without any modification to the imaging system. Both the simulation and experiments on the terahertz band demonstrate the feasibility of the proposed algorithms, and the limit of resolution can be extended by a factor of 1.67.

Low-rank factorization for rank minimization with nonconvex regularizers

Rank minimization is of interest in machine learning applications such as recommender systems and robust principal component analysis. Minimizing the convex relaxation to the rank minimization problem, the nuclear norm, is an effective technique to solve the problem with strong performance guarantees. However, nonconvex relaxations have less estimation bias than the nuclear norm and can more accurately reduce the effect of noise on the measurements. We develop efficient algorithms based on iteratively reweighted nuclear norm schemes, while also utilizing the low rank factorization for semidefinite programs put forth by Burer and Monteiro. We prove convergence and computationally show the advantages over convex relaxations and alternating minimization methods. Additionally, the computational complexity of each iteration of our algorithm is on par with other state of the art algorithms, allowing us to quickly find solutions to the rank minimization problem for large matrices.

Alternating minimization methods for strongly convex optimization

Abstract We consider alternating minimization procedures for convex and non-convex optimization problems with the vector of variables divided into several blocks, each block being amenable for minimization with respect to its variables while maintaining other variables blocks constant. In the case of two blocks, we prove a linear convergence rate for an alternating minimization procedure under the Polyak–Łojasiewicz (PL) condition, which can be seen as a relaxation of the strong convexity assumption. Under the strong convexity assumption in the many-blocks setting, we provide an accelerated alternating minimization procedure with linear convergence rate depending on the square root of the condition number as opposed to just the condition number for the non-accelerated method. We also consider the problem of finding an approximate non-negative solution to a linear system of equations A ⁢ x = y {Ax=y} with alternating minimization of Kullback–Leibler (KL) divergence between Ax and y.

A coupling model and ADMM algorithm based on TGV and shearlet regularization term for MRI reconstruction

Motivated by the ideas from two step model and its deformation, we propose a coupling model for MR image reconstruction, based on the advantages of TGV and shearlet regularization terms. By using variable splitting technique, splitting Bregman iteration scheme and alternating minimization method, the proposed model can be decomposed into several subproblems to avoid solving high-order PDEs. The u subproblem can be solved by Cramer’s rule and the diagonalization technique of the Fourier transform. The other subproblems can be solved simply by some shrinkage formulas. We also use the Barzilai-Borwein step selection scheme to accelerate these subproblem’s solutions. Finally, an ADMM algorithm is proposed to solve the coupling model. The numerical results show that the proposed coupling model and algorithm are feasible and effective.

A partially proximal linearized alternating minimization method for finding Dantzig selectors

The Dantzig selector (DS) is an efficient estimator designed for high-dimensional linear regression problems, especially for the case where the number of samples n is much less than the dimension of features (or variables) p. In this paper, we first reformulate the underlying DS model as an unconstrained minimization problem of the sum of two nonsmooth convex functions and a smooth coupled function. Then by exploiting the structure of the resulting model, we propose a partially proximal linearized alternating minimization method (P-PLAM), whose two subproblems are easy enough with closed-form solutions. Another remarkable advantage is that P-PLAM requires only one starting point, which is potentially helpful for saving computing time. A series of computational experiments on synthetic and real-world data sets demonstrate that the proposed P-PLAM has promising numerical performance in the sense that P-PLAM can obtain higher quality solutions by taking less computing time than some existing state-of-the-art first-order solvers.

An inexact PAM method for computing Wasserstein barycenter with unknown supports

Wasserstein barycenter is the centroid of a collection of discrete probability distributions which minimizes the average of the $$\ell _2$$ -Wasserstein distance. This paper focuses on the computation of Wasserstein barycenters under the case where the support points are free, which is known to be a severe bottleneck in the D2-clustering due to the large scale and nonconvexity. We develop an inexact proximal alternating minimization (iPAM) method for computing an approximate Wasserstein barycenter, and provide its global convergence analysis. This method can achieve a good accuracy with a reduced computational cost when the unknown support points of the barycenter have low cardinality. Numerical comparisons with the 3-block B-ADMM in Ye et al. (IEEE Trans Signal Process 65:2317–2332, 2017) and an alternating minimization method involving the LP subproblems on synthetic and real data show that the proposed iPAM can yield comparable even a little better objective values in less CPU time, and hence, the computed barycenter will render a better role in the D2-clustering.

Inhomogeneous image segmentation based on local constant and global smoothness priors

In this article, we propose a new variational model for segmenting images with intensity inhomogeneity. The proposed model applies simultaneously the local constant and global smoothness priors to describe the bias part such that our model can obtain more precise segmentation results. This is different from the existing models in which either of such two priors is considered. Also, our method is developed to segment the multiphase inhomogeneous image with noise. Theoretically, we show the existence of minimizers of the proposed variational model. Moreover, by using the alternating minimization method, we design an effective algorithm to numerically solve the solution of the model. Numerical experimental results are provided to verify the better performance of our method than other test methods.

Incremental CP Tensor Decomposition by Alternating Minimization Method

In practical applications, incremental tensors are very common: only a portion of tensor data is available, and new data are arriving in the next time step or continuously over time. To handle this type of tensors time-saving algorithms are required for online computation. In this paper, we consider incremental CP (CANDECOMP/PARAFAC) decomposition, which requires one to update the CP decomposition after new tensor data, together with the exiting tensor, are ready for analysis. There exist several incremental CP decomposition algorithms, but almost all of these algorithms assume that the number of CP decomposition components remains fixed in the incremental process. The main contribution of this paper is the study of how to add components in the incremental CP decomposition. We derive the coordinate representation of the incremental CP decomposition with respect to a special basis and show related properties of tensor rank and the uniqueness of such incremental CP decomposition. Under the framework of this representation, the proposed method can be solved by using an alternating minimization algorithm. Numerical examples are presented to show the good performance of the proposed algorithms in terms of computational time and data fitting compared with existing methods.

Video SAR Imaging Based on Low-Rank Tensor Recovery

Due to its ability of forming continuous images for a ground scene of interest, the video synthetic aperture radar (SAR) has been studied in recent years. However, as video SAR needs to reconstruct many frames, the data are of enormous amount and the imaging process is of large computational cost, which limits its applications. In this article, we exploit the redundancy property of multiframe video SAR data, which can be modeled as low-rank tensor, and formulate the video SAR imaging process as a low-rank tensor recovery problem, which is solved by an efficient alternating minimization method. We empirically compare the proposed method with several state-of-the-art video SAR imaging algorithms, including the fast back-projection (FBP) method and the compressed sensing (CS)-based method. Experiments on both simulated and real data show that the proposed low-rank tensor-based method requires significantly less amount of data samples while achieving similar or better imaging performance.

Non-Convex High Order Total Variation With Overlapping Group Sparsity Denoising Model Under Cauchy Noise

It is widely known that the total variation regularization model preserves the edges well in the restored images but has some staircase effects. We consider using non-convex high-order total variation and overlapping group sparsity as a hybrid regularization to present a new denoising model. The proposed model can well preserve edges and reduce the staircase effect in the smooth region of the restored images. In order to solve the proposed hybrid model, we develop an efficient alternating minimization method. Compared with other models for removing Cauchy noise, numerical experimental results demonstrate that the superiority of the proposed model and algorithm, both in terms of visual and quantitative measures.

Proximal alternating minimization method for adaptive TGV-based image restoration

This article presents an adaptive total generalized variation regularized strategy for image reconstruction. Unlike the traditional fixed weights schemes, our weights can be adaptively updated according to the latest computations. This helps to obtain more accurate numerical solutions and avoid the troublesome parameters selection. Subsequently, by artfully employing the Moreau decomposition and proximal algorithm, we develop a highly efficient proximal alternating minimization method to optimize the objective function in detail. The introduced technique has the capability of automatically estimating the regularization parameter and restoring the deteriorated image. Finally, several numerical simulations concertedly illustrate the competitive superiority of our proposed scheme for image restoration, especially in terms of suppressing staircase artifacts and maintaining edge details.

Multiple Kernel k-Means Clustering by Selecting Representative Kernels.

To cluster data that are not linearly separable in the original feature space, k -means clustering was extended to the kernel version. However, the performance of kernel k -means clustering largely depends on the choice of the kernel function. To mitigate this problem, multiple kernel learning has been introduced into the k -means clustering to obtain an optimal kernel combination for clustering. Despite the success of multiple kernel k -means clustering in various scenarios, few of the existing work update the combination coefficients based on the diversity of kernels, which leads to the result that the selected kernels contain high redundancy and would degrade the clustering performance and efficiency. We resolve this problem from the perspective of subset selection in this article. In particular, we first propose an effective strategy to select a diverse subset from the prespecified kernels as the representative kernels, and then incorporate the subset selection process into the framework of multiple k -means clustering. The representative kernels can be indicated as a significant combination weights. Due to the nonconvexity of the obtained objective function, we develop an alternating minimization method to optimize the combination coefficients of the selected kernels and the cluster membership alternatively. In particular, an efficient optimization method is developed to reduce the time complexity of optimizing the kernel combination weights. Finally, extensive experiments on benchmark and real-world data sets demonstrate the effectiveness and superiority of our approach in comparison with existing methods.

Image Compressive Sensing via Hybrid Nonlocal Sparsity Regularization.

This paper focuses on image compressive sensing (CS). As the intrinsic properties of natural images, nonlocal self-similarity and sparse representation have been widely used in various image processing tasks. Most existing image CS methods apply either self-adaptive dictionary (e.g., principle component analysis (PCA) dictionary and singular value decomposition (SVD) dictionary) or fixed dictionary (e.g., discrete cosine transform (DCT), discrete wavelet transform (DWT), and Curvelet) as the sparse basis, while single dictionary could not fully explore the sparsity of images. In this paper, a Hybrid NonLocal Sparsity Regularization (HNLSR) is developed and applied to image compressive sensing. The proposed HNLSR measures nonlocal sparsity in 2D and 3D transform domain simultaneously, and both self-adaptive singular value decomposition (SVD) dictionary and fixed 3D transform are utilized. We use an efficient alternating minimization method to solve the optimization problem. Experimental results demonstrate that the proposed method outperforms existing methods in both objective evaluation and visual quality.

Multi-energy CT reconstruction using tensor nonlocal similarity and spatial sparsity regularization.

Multi-energy computed tomography (MECT) based on a photon-counting detector is an emerging imaging modality that collects projections at several energy bins with a single scan. However, the limited number of photons collected into the divided, narrow energy bins results in high quantum noise levels in reconstructed images. This study aims to improve MECT image quality by minimizing noise levels while retaining image details. A novel MECT reconstruction method was proposed by exploiting the nonlocal tensor similarity among interchannel images and spatial sparsity in single-channel images. Similar patches were initially extracted from the interchannel images in spectral and spatial domains, then stacked into a new three-order tensor. Intrinsic tensor sparsity regularization that combined the Tuker and canonical polyadic (CP) low-rank decomposition techniques were applied to exploit the nonlocal similarity of the formulated tensor. Spatial sparsity in single-channel images was modeled by total variation (TV) regularization that utilizes the compressibility of gradient image. A new MECT reconstruction model was established by simultaneously incorporating the intrinsic tensor sparsity and TV regularizations. The iterative alternating minimization method was utilized to solve the reconstruction model based on a flexible framework. The proposed method was applied to the digital phantom and real mouse data to assess its feasibility and reliability. The reconstruction and decomposition results in the mouse data were encouraging and demonstrated the ability of the proposed method in noise suppression while preserving image details, not observed with other methods. Imaging data from the digital phantom illustrated this method as achieving the best intuitive reconstruction and decomposition results among all compared methods. They reduced the root mean square error (RMSE) by 89.75%, 50.75%, and 36.54% on the reconstructed images compared with analytic, TV-based, and tensor-based methods, respectively. This phenomenon was also observed with decomposition results, where the RMSE was also reduced by 97.96%, 67.74%, 72.05%, respectively. In this study, we proposed a reconstruction method for photon counting detector-based MECT, using the intrinsic tensor sparsity and TV regularizations. Improvements in noise suppression and detail preservation in the digital phantom and real mouse data were validated by the qualitative and quantitative evaluations on the reconstruction and decomposition results, verifying the potential of the proposed method in MECT reconstruction.

Anomaly-Aware Network Traffic Estimation via Outlier-Robust Tensor Completion

Accurately estimating network traffic from the partial measurements plays a crucial role in network management. However, the potential anomaly existing in real networks usually makes this goal difficult to achieve. Existing network traffic estimation methods generally impute network traffic independent of anomaly detection, which incurs significant performance degradation with network anomaly. To address this issue in the realistic network scenario, we propose a novel anomaly-aware network traffic estimation method to recover network traffic data concurrently with network anomaly detection. Specifically, by exploiting the inherent spatio-temporal characteristics, we first formulate the network traffic estimation as a low-rank tensor completion problem. Then, an outlier-robust tensor completion (OrTC) model is constructed by introducing both L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm regularization and L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</sub> -norm regularization, which can not only well fit the intrinsic low-rank property of real traffic data, but also is robust against both the dense noise and the sparse anomaly. Furthermore, an effective optimization algorithm OrTC-AM is designed to solve the non-convex and non-smooth OrTC model based on the popular alternating minimization method. Finally, the extensive experiments performed on the public dataset demonstrate that our proposed OrTC-AM method outperforms the previously widely used network traffic estimation methods.

Spatially Adaptive Regularization in Image Segmentation

We present a total-variation-regularized image segmentation model that uses local regularization parameters to take into account spatial image information. We propose some techniques for defining those parameters, based on the cartoon-texture decomposition of the given image, on the mean and median filters, and on a thresholding technique, with the aim of preventing excessive regularization in piecewise-constant or smooth regions and preserving spatial features in nonsmooth regions. Our model is obtained by modifying a well-known image segmentation model that was developed by T. Chan, S. Esedoḡlu, and M. Nikolova. We solve the modified model by an alternating minimization method using split Bregman iterations. Numerical experiments show the effectiveness of our approach.

Hierarchical Codebook-Based Multiuser Beam Training for Millimeter Wave Massive MIMO

In this article, multiuser beam training based on hierarchical codebook for millimeter wave massive multi-input multi-output is investigated, where the base station (BS) simultaneously performs beam training with multiple user equipments (UEs). For the UEs, an alternative minimization method with a closed-form expression (AMCF) is proposed to design the hierarchical codebook under the constant modulus constraint. To speed up the convergence of the AMCF, an initialization method based on Zadoff-Chu sequence is proposed. For the BS, a simultaneous multiuser beam training scheme based on an adaptively designed hierarchical codebook is proposed, where the codewords in the current layer of the codebook are designed according to the beam training results of the previous layer. The codewords at the BS are designed with multiple mainlobes, each covering a spatial region for one or more UEs. Simulation results verify the effectiveness of the proposed hierarchical codebook design schemes and show that the proposed multiuser beam training scheme can approach the performance of the beam sweeping but with significantly reduced beam training overhead.

The Analysis of Alternating Minimization Method for Double Sparsity Constrained Optimization Problem

This work analyzes the alternating minimization (AM) method for solving double sparsity constrained minimization problem, where the decision variable vector is split into two blocks. The objective function is a separable smooth function in terms of the two blocks. We analyze the convergence of the method for the non-convex objective function and prove a rate of convergence of the norms of the partial gradient mappings. Then, we establish a non-asymptotic sub-linear rate of convergence under the assumption of convexity and the Lipschitz continuity of the gradient of the objective function. To solve the sub-problems of the AM method, we adopt the so-called iterative thresholding method and study their analytical properties. Finally, some future works are discussed.

Sparse probabilistic K-means

Abstract The goal of clustering is to partition a set of data points into groups of similar data points, called clusters. Clustering algorithms can be classified into two categories: hard and soft clustering. Hard clustering assigns each data point to one cluster exclusively. On the other hand, soft clustering allows probabilistic assignments to clusters. In this paper, we propose a new model which combines the benefits of these two models: clarity of hard clustering and probabilistic assignments of soft clustering. Since the majority of data usually have a clear association, only a few points may require a probabilistic interpretation. Thus, we apply the l1 norm constraint to impose sparsity on probabilistic assignments. Moreover, we also incorporate outlier detection in our clustering model to simultaneously detect outliers which can cause serious problems in statistical analyses. To optimize the model, we introduce an alternating minimization method and prove its convergence. Numerical experiments and comparisons with existing models show the soundness and effectiveness of the proposed model.

A General Framework for Deep Supervised Discrete Hashing

With the rapid growth of image and video data on the web, hashing has been extensively studied for image or video search in recent years. Benefiting from recent advances in deep learning, deep hashing methods have shown superior performance over the traditional hashing methods. However, there are some limitations of previous deep hashing methods (e.g., the semantic information is not fully exploited). In this paper, we develop a general deep supervised discrete hashing framework based on the assumption that the learned binary codes should be ideal for classification. Both the similarity information and the classification information are used to learn the hash codes within one stream framework. We constrain the outputs of the last layer to be binary codes directly, which is rarely investigated in deep hashing algorithms. Besides, both the pairwise similarity information and the triplet ranking information are exploited in this paper. In addition, two different loss functions are presented: $${l_2}$$ loss and hinge loss, which are carefully designed for the classification term under the one stream framework. Because of the discrete nature of hash codes, an alternating minimization method is used to optimize the objective function. Experimental results have shown that our approach outperforms current state-of-the-art methods on benchmark datasets.