Norm Minimization Method Research Articles

An Improved Weighted Nuclear Norm Minimization Method for Image Denoising

Patch-based low rank matrix approximation has shown great potential in image denoising. Among state-of-the-art methods in this topic, the weighted nuclear norm minimization (WNNM) has been attracting significant attention due to its competitive denoising performance. For each local patch in an image, the WNNM method groups nonlocal similar patches by block matching to formulate a low-rank matrix. However, the WNNM often chooses irrelevant patches such that it may lose fine details of the image, resulting in undesirable artifacts in the final reconstruction. In this regards, this paper aims to provide a denoising algorithm which further improves the performance of the WNNM method. For this purpose, we develop a new nonlocal similarity measure by exploiting both pixel intensities and gradients and present a filter that enhances edge information in a patch to improve the performance of low rank approximation. The experimental results on widely used test images demonstrate that the proposed denoising algorithm performs better than other state-of-the-art denoising algorithms in terms of PSNR and SSIM indices as well as visual quality.

A Regularized Weighted Smoothed L₀ Norm Minimization Method for Underdetermined Blind Source Separation.

Compressed sensing (CS) theory has attracted widespread attention in recent years and has been widely used in signal and image processing, such as underdetermined blind source separation (UBSS), magnetic resonance imaging (MRI), etc. As the main link of CS, the goal of sparse signal reconstruction is how to recover accurately and effectively the original signal from an underdetermined linear system of equations (ULSE). For this problem, we propose a new algorithm called the weighted regularized smoothed -norm minimization algorithm (WReSL0). Under the framework of this algorithm, we have done three things: (1) proposed a new smoothed function called the compound inverse proportional function (CIPF); (2) proposed a new weighted function; and (3) a new regularization form is derived and constructed. In this algorithm, the weighted function and the new smoothed function are combined as the sparsity-promoting object, and a new regularization form is derived and constructed to enhance de-noising performance. Performance simulation experiments on both the real signal and real images show that the proposed WReSL0 algorithm outperforms other popular approaches, such as SL0, BPDN, NSL0, and L-RLSand achieves better performances when it is used for UBSS.

Vandermonde Factorization of Hankel Matrix for Complex Exponential Signal Recovery—Application in Fast NMR Spectroscopy

Many signals are modeled as a superposition of exponential functions in spectroscopy of chemistry, biology and medical imaging. This paper studies the problem of recovering exponential signals from a random subset of samples. We exploit the Vandermonde structure of the Hankel matrix formed by the exponential signal and formulate signal recovery as Hankel matrix completion with Vandermonde factorization (HVaF). A numerical algorithm is developed to solve the proposed model and its sequence convergence is analyzed theoretically. Experiments on synthetic data demonstrate that HVaF succeeds over a wider regime than the state-of-the-art nuclear-normminimization-based Hankel matrix completion method, while has a less restriction on frequency separation than the state-of-the-art atomic norm minimization and fast iterative hard thresholding methods. The effectiveness of HVaF is further validated on biological magnetic resonance spectroscopy data.

Fuzzy Double Trace Norm Minimization for Recommendation Systems

Recovering low-rank matrices from incomplete observations is a fundamental problem with many applications, especially in recommender systems. In theory, under certain conditions, this problem can be solved by convex or nonconvex relaxation. However, most existing provable algorithms suffer from superlinear per-iteration cost, which severely limits their applicability to large-scale problems. In this paper, we propose a novel fuzzy double trace norm minimization (DTNM) method for recommender systems. We first present a tractable DTNM model, in which we can integrate both the user social relationship and the user reputation information using a fuzzy weighting way and coupling fuzzy matrix factorization. In essence, our model is a Schatten- ${1/2}$ quasi-norm minimization problem. Moreover, we develop two efficient augmented Lagrangian algorithms to solve the proposed problems, and prove the convergence of our algorithms. Finally, we investigate the empirical recoverability properties of our model and its advantage over classical trace norm. Extensive experimental results on both synthetic and real-world data sets verified both the efficiency and effectiveness of our method compared with the state-of-the-art algorithms.

De-noising research on terahertz holographic reconstructed image based on weighted nuclear norm minimization method

Terahertz imaging is one of the forefront topics of imaging technology today. Denoising process is the key for improving the resolution of the terahertz holographic reconstructed image. Based on the fact that the weighted nuclear norm minimization (WNNM) method preserves the details of the reconstructed image well and the nonlocal mean (NLM) algorithm performs better in the removal of background noise, this paper proposes a new method in which the NLM algorithm is used to improve the WNNM method. The experimental observation and quantitative analysis of the denoising results prove that the new method has better denoising effect for the terahertz holographic reconstructed image.

A Fast and Accurate Matrix Completion Method Based on QR Decomposition and L2,1 -Norm Minimization.

Low-rank matrix completion aims to recover matrices with missing entries and has attracted considerable attention from machine learning researchers. Most of the existing methods, such as weighted nuclear-norm-minimization-based methods and Qatar Riyal (QR)-decomposition-based methods, cannot provide both convergence accuracy and convergence speed. To investigate a fast and accurate completion method, an iterative QR-decomposition-based method is proposed for computing an approximate singular value decomposition. This method can compute the largest singular values of a matrix by iterative QR decomposition. Then, under the framework of matrix trifactorization, a method for computing an approximate SVD based on QR decomposition (CSVD-QR)-based L2,1 -norm minimization method (LNM-QR) is proposed for fast matrix completion. Theoretical analysis shows that this QR-decomposition-based method can obtain the same optimal solution as a nuclear norm minimization method, i.e., the L2,1 -norm of a submatrix can converge to its nuclear norm. Consequently, an LNM-QR-based iteratively reweighted L2,1 -norm minimization method (IRLNM-QR) is proposed to improve the accuracy of LNM-QR. Theoretical analysis shows that IRLNM-QR is as accurate as an iteratively reweighted nuclear norm minimization method, which is much more accurate than the traditional QR-decomposition-based matrix completion methods. Experimental results obtained on both synthetic and real-world visual data sets show that our methods are much faster and more accurate than the state-of-the-art methods.

How to compute the optimal parameters of the parameterized PMHSS preconditioner?

Based on the preconditioned modified Hermitian and skew-Hermitian splitting (PMHSS) iteration method and its alternating-directional variant, a parameterized PMHSS (P2MHSS) preconditioner is proposed for a class of block two-by-two systems of linear equations. To optimize the preconditioning effect of the P2MHSS preconditioner, practical formulas for computing the optimal parameters of the P2MHSS preconditioner are derived by employing the scaled norm minimization method. Finally, we test the feasibility and the effectiveness of the P2MHSS preconditioner.

Scaled norm minimization method for computing the parameters of the HSS and the two‐parameter HSS preconditioners

SummaryThe performance of the Hermitian and skew‐Hermitian splitting (HSS) preconditioner for the non‐Hermitian positive definite system of linear equations is largely dependent on the choice of its parameter value. In this work, an efficient scaled norm minimization (SNM) method is proposed to compute the parameter value of the HSS preconditioner. In addition, by choosing different parameters for the Hermitian and the skew‐Hermitian matrices in the HSS preconditioner, a two‐parameter HSS preconditioner is proposed. Moreover, an efficient and practical formula for computing the parameter values of this new preconditioner is also derived by using the SNM method. Numerical examples are illustrated to verify the performances of the HSS and the two‐parameter HSS preconditioners when their parameters are computed by the SNM method.

Atomic Norm Denoising-Based Joint Channel Estimation and Faulty Antenna Detection for Massive MIMO

We consider joint channel estimation and faulty antenna detection for massive multiple-input multiple-output (MIMO) systems operating in time-division duplexing (TDD) mode. For systems with faulty antennas, we show that the impact of faulty antennas on uplink (UL) data transmission does not vanish even with unlimited number of antennas. However, the signal detection performance can be improved with a priori knowledge on the indices of faulty antennas. This motivates us to propose the approach for simultaneous channel estimation and faulty antenna detection. By exploiting the fact that the degrees of freedom of the physical channel matrix are smaller than the number of free parameters, the channel estimation and faulty antenna detection can be formulated as an extended atomic norm denoising problem and solved efficiently via the alternating direction method of multipliers (ADMM). Furthermore, we improve the computational efficiency by proposing a fast algorithm and show that it is a good approximation of the corresponding extended atomic norm minimization method. Numerical simulations are provided to compare the performances of the proposed algorithms with several existing approaches and demonstrate the performance gains of detecting the indices of faulty antennas.

Nuclear Norm Subspace Identification Of Continuous Time State-Space Models

Subspace identification techniques derive approximate models rather than models that are optimal with respect to a goodness of fit criterion. To obtain low rank models, a nuclear norm minimization method for estimating the system matrices of linear time invariant continuous time state-space models in the presence of measurement noise is proposed. In the proposed approach, Generalized Poisson Moment Functional (GPMF) method is used to circumvent the time-derivative problem which is inherent in continuous time models. To make the proposed algorithm consistent, instrumental variables (Hankel matrix of past inputs) are considered. The accuracy of the proposed method is demonstrated with the help of numerical simulations on a variety of systems.

Truncated Nuclear Norm Minimization Based Group Sparse Representation for Image Restoration

Group sparse representation has shown great potential in image restoration, which can be considered as a low-rank matrix approximation problem. The nuclear norm minimization method, as a convex rel...

Incoherent Tensor Norms and Their Applications in Higher Order Tensor Completion

In this paper, we investigate the sample size requirement for a general class of nuclear norm minimization methods for higher order tensor completion. We introduce a class of tensor norms by allowing for different levels of coherence, which allows us to leverage the incoherence of a tensor. In particular, we show that a $k$ th-order tensor of multilinear rank $r$ and dimension $d \times \cdots \times d$ can be recovered perfectly from as few as $O((r^{(k-1)/2}d^{3/2}+r^{k-1}d)(\log (d))^{2})$ uniformly sampled entries through an appropriate incoherent nuclear norm minimization. Our results demonstrate some key differences between completing a matrix and a higher order tensor: they not only point to potential room for improvement over the usual nuclear norm minimization but also highlight the importance of explicitly accounting for incoherence, when dealing with higher order tensors. Although our focus is primarily on the theoretical guarantees for nuclear norm minimization, such insights may prove useful for understanding performance of other related methods and developing improved practical algorithms.

Seismic noise suppression using weighted nuclear norm minimization method

The weighted nuclear norm minimization method as an extension of nuclear-norm minimization was applied to image denoising originally. It is a kind of low rank matrix approximation method that can estimate the noiseless matrix from its noise version. The effective structures of image have a certain degree of repeatability and the weighted nuclear norm minimization method just utilizes this property to construct an approximate low rank matrix. Taking into account the spatial characteristics of seismic data and the redundancies of valid information, we propose to adopt the weighted nuclear norm minimization method to suppress seismic random noise. In this method the block matching algorithm is helpful for the recovery of seismic events because the texture blocks sharing the same reflection events are the most similar. Even when the signal to noise ratio is −10dB, this novel method still be able to clearly recover signals. Experiments on both synthetic and real seismic data show that the weighted nuclear norm minimization method can not only suppress the random noise but also better preserves the valid information of seismic signal when compared to the common seismic denoising methods such as the Wavelet and Time Frequency Peak Filter.

Fast iterative WSVT algorithm in WNN minimization problem for multiuser massive MIMO channel estimation

SummaryIn this paper, channel estimation for multiuser massive MIMO system is addressed for the scenario, where the number of scatterers is small compared to the base station antennas and single antenna users in the cell. If the number of scatterers is limited, then the corresponding angle of arrivals are finite. Moreover, if all the users share the same angle of arrivals, then the correlation among the channel vector increases. Thus, the high‐dimensional channel is approximated to the low‐rank matrix. This rank minimization problem is formulated as the weighted nuclear norm problem and is estimated using iterative weighted singular value thresholding (IWSVT) algorithm. To increase the convergence rate, fast IWSVT algorithm is proposed and the performance is measured based on mean square error and uplink and downlink achievable sum‐rate. The simulation study shows that the proposed weighted nuclear norm minimization method with fast IWSVT algorithm performs better than the conventional least square and the nuclear norm minimization method for various finite scatterers in different signal‐to‐noise ratio levels.

System Identification Algorithm for Non-Uniformly Sampled Data

Considerable effort has been devoted to the development of algorithms for identification of parsimonious discrete time models from noisy input/output data sets since this facilitates controller design. Several methods, such as nuclear norm minimization, have been used to provide approximate solutions to this non-convex problem. However, even though the field of continuous time system identification is now mature, results on parsimonious model identification of continuous time systems are still very limited. In this paper, an atomic norm minimization method is proposed for this purpose that can handle non-uniformly sampled data without preprocessing. The proposed approach provides an efficient way to use noisy, non-uniformly sampled data to determine a reliable, low-order continuous time model. Numerical performance is illustrated using academic examples and simulated behavioral data from a smoking cessation study.

Sparse EEG/MEG source estimation via a group lasso.

Non-invasive recordings of human brain activity through electroencephalography (EEG) or magnetoencelphalography (MEG) are of value for both basic science and clinical applications in sensory, cognitive, and affective neuroscience. Here we introduce a new approach to estimating the intra-cranial sources of EEG/MEG activity measured from extra-cranial sensors. The approach is based on the group lasso, a sparse-prior inverse that has been adapted to take advantage of functionally-defined regions of interest for the definition of physiologically meaningful groups within a functionally-based common space. Detailed simulations using realistic source-geometries and data from a human Visual Evoked Potential experiment demonstrate that the group-lasso method has improved performance over traditional ℓ2 minimum-norm methods. In addition, we show that pooling source estimates across subjects over functionally defined regions of interest results in improvements in the accuracy of source estimates for both the group-lasso and minimum-norm approaches.

Joint Precoding and RRH Selection for User-Centric Green MIMO C-RAN

This paper jointly optimizes the precoding matrices and the set of active remote radio heads (RRHs) to minimize the network power consumption for a user-centric cloud radio access network, where both the RRHs and users have multiple antennas and each user is served by its nearby RRHs. Both users' rate requirements and per-RRH power constraints are considered. Due to these conflicting constraints, this optimization problem may be infeasible. In this paper, we propose to solve this problem in two stages. In Stage I, a low-complexity user selection algorithm is proposed to find the largest subset of feasible users. In Stage II, a low-complexity algorithm is proposed to solve the optimization problem with the users selected from Stage I. Specifically, the re-weighted l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm minimization method is used to transform the original problem with non-smooth objective function into a series of weighted power minimization (WPM) problems, each of which can be solved by the weighted minimum mean square error (WMMSE) method. The solution obtained by the WMMSE method is proved to satisfy the Karush-Kuhn-Tucker conditions of the WPM problem. Moreover, a low-complexity algorithm based on Newton's method and the gradient descent method is developed to update the precoder matrices in each iteration of the WMMSE method. Simulation results demonstrate the rapid convergence of the proposed algorithms and the benefits of equipping multiple antennas at the user side. Moreover, the proposed algorithm is shown to achieve near-optimal performance in terms of NPC.

Channel Estimation for Finite Scatterers Massive Multi-User MIMO System

In finite scattering propagation environment, when the number of scatterers is very small compared to the number of base station antennas and users in the cell and if the same scatterers are shared by all users, then the correlation among the users increases. Hence, the high-dimensional multi-user MIMO system is likely to have low-rank channel. To estimate the channel matrix, weighted nuclear norm minimization method is proposed in this paper. Iterative weighted singular value thresholding algorithm is used to solve the optimization problem. To recover the low-rank channel, a partial random Fourier matrix (PRFM) satisfying the restricted isometric property is adapted as the training matrix. The PRFM forces the iterative algorithm to converge in one iteration which reduces the huge computational complexity of the proposed channel estimation method. The mean square error and achievable sum rate are the criteria used to measure the performance of the proposed method. The results show that the proposed method outperforms the least square estimation method and the nuclear norm minimization method for various finite scatterers in different SNR levels.

Moving Target's HRRP Synthesis With Sparse Frequency-Stepped Chirp Signal via Atomic Norm Minimization

Compressive sensing (CS) has been introduced into inverse synthetic aperture radar (ISAR) imaging with partial measurements. However, in the case of transmitting sparse frequency-stepped chirp signal (FSCS), the CS-based method will produce an irregular range cell migration (IRCM) problem in the recovered high-resolution range profiles (HRRPs). The IRCM is induced by the basis mismatch problem in CS, and it will degrade the ISAR image. To obviate the IRCM, an atomic norm minimization (ANM) method is proposed in this letter. By reformulating the ANM as a semidefinite program (SDP), the echo with full FSCS can be recovered using off-the-shelf SDP solvers. Thus, HRRPs without IRCM can be achieved via the conventional inverse fast Fourier transform. As a result, an improved ISAR image will be obtained. Real data results demonstrate the advantages of the proposed method over the CS-based and matrix completion-based methods.

L 1 norm minimization in partial errors-in-variables model

The weighted total least-squares (WTLS) estimate is sensitive to outliers and will be strongly disturbed if there are outliers in the observations and coefficient matrix of the partial errors-in-variables (EIV) model. The L 1 norm minimization method is a robust technique to resist the bad effect of outliers. Therefore, the computational formula of the L 1 norm minimization for the partial EIV model is developed by employing the linear programming theory. However, the closed-form solution cannot be directly obtained since there are some unknown parameters in constrained condition equation of the presented optimization problem. The iterated procedure is recommended and the proper condition for stopping iteration is suggested. At the same time, by treating the partial EIV model as the special case of the non-linear Gauss–Helmert (G–H) model, another iterated method for the L 1 norm minimization problem is also developed. At last, two simulated examples and a real data of 2D affine transformation are conducted. It is illustrated that the results derived by the proposed L 1 norm minimization methods are more accurate than those by the WTLS method while the observations and elements of the coefficient matrix are contaminated with outliers. And the two methods for the L 1 norm minimization problem are identical in the sense of robustness. By comparing with the data-snooping method, the L 1 norm minimization method may be more reliable for detecting multiple outliers due to masking. But it leads to great computation burden.