L1-norm Minimization Problem Research Articles

Sparse recovery of missing image samples using a convex similarity index

Abstract This paper investigates the problem of recovering missing samples using methods based on sparse representation adapted for visually enhanced quality of reconstruction of image signals. Although, the popular Mean Square Error (MSE) criterion is convex and simple, it may not be entirely consistent with Human Visual System (HVS). Thus, instead of l2-norm or MSE, a new perceptual quality measure is used as the similarity criterion between the original and the reconstructed images. The proposed criterion called Convex SIMilarity (CSIM) index is a modified version of the Structural SIMilarity (SSIM) index, which despite its predecessor, is convex and uni-modal. We derive mathematical properties for the proposed index and show how to optimally choose the parameters of the proposed criterion, investigating the Restricted Isometry (RIP) and error-sensitivity properties. We also propose an iterative sparse recovery method based on a constrained l1-norm minimization problem, incorporating CSIM as the fidelity criterion. The resulting convex optimization problem is solved via an algorithm based on Alternating Direction Method of Multipliers (ADMM). Taking advantage of the convexity of the CSIM index, we also prove the convergence of the algorithm to the globally optimal solution of the proposed optimization problem, starting from any arbitrary point. Simulation results confirm the performance of the new similarity index as well as the proposed algorithm for missing sample recovery of image patch signals.

Geometric artifacts reduction for cone-beam CT via L0-norm minimization without dedicated phantoms.

For cone-beam computed tomography (CBCT), transversal shifts of the rotation center exist inevitably, which will result in geometric artifacts in CT images. In this work, we propose a novel geometric calibration method for CBCT, which can also be used in micro-CT. The symmetry property of the sinogram is used for the first calibration, and then L0-norm of the gradient image from the reconstructed image is used as the cost function to be minimized for the second calibration. An iterative search method is adopted to pursue the local minimum of the L0-norm minimization problem. The transversal shift value is updated with affirmatory step size within a search range determined by the first calibration. In addition, graphic processing unit (GPU)-based FDK algorithm and acceleration techniques are designed to accelerate the calibration process of the presented new method. In simulation experiments, the mean absolute difference (MAD) and the standard deviation (SD) of the transversal shift value were less than 0.2 pixels between the noise-free and noisy projection images, which indicated highly accurate calibration applying the new calibration method. In real data experiments, the smaller entropies of the corrected images also indicated that higher resolution image was acquired using the corrected projection data and the textures were well protected. Study results also support the feasibility of applying the proposed method to other imaging modalities.

Accelerated 3D blind separation of convolved mixtures based on the fast iterative shrinkage thresholding algorithm for adaptive multiple subtraction

After multiple prediction, adaptive multiple subtraction is essential for the success of multiple removal. The 3D blind separation of convolved mixtures (3D BSCM) method, which is effective in conducting adaptive multiple subtraction, needs to solve an optimization problem containing L1-norm minimization constraints on primaries by the iterative reweighted least-squares (IRLS) algorithm. The 3D BSCM method can better separate primaries and multiples than the 1D/2D BSCM method and the method with energy minimization constraints on primaries. However, the 3D BSCM method has high computational cost because the IRLS algorithm achieves nonquadratic optimization with an LS optimization problem solved in each iteration. In general, it is good to have a faster 3D BSCM method. To improve the adaptability of field data processing, the fast iterative shrinkage thresholding algorithm (FISTA) is introduced into the 3D BSCM method. The proximity operator of FISTA can solve the L1-norm minimization problem efficiently. We demonstrate that our FISTA-based 3D BSCM method achieves similar accuracy of estimating primaries as that of the reference IRLS-based 3D BSCM method. Furthermore, our FISTA-based 3D BSCM method reduces computation time by approximately 60% compared with the reference IRLS-based 3D BSCM method in the synthetic and field data examples.

Sparse decomposition of seismic data and migration using Gaussian beams with nonzero initial curvature

We study problems associated with seismic data decomposition and migration imaging. We first represent the seismic data utilizing Gaussian beam basis functions, which have nonzero curvature, and then consider the sparse decomposition technique. The sparse decomposition problem is an l0-norm constrained minimization problem. In solving the l0-norm minimization, a polynomial Radon transform is performed to achieve sparsity, and a fast gradient descent method is used to calculate the waveform functions. The waveform functions can subsequently be used for sparse Gaussian beam migration. Compared with traditional sparse Gaussian beam methods, the seismic data can be properly reconstructed employing fewer Gaussian beams with nonzero initial curvature. The migration approach described in this paper is more efficient than the traditional sparse Gaussian beam migration.

Direction-of-arrival estimation of multipath signals using independent component analysis and compressive sensing

Multipath signal is often considered an interference that must be removed. The coherence between multipath and direct component makes it difficult to use conventional direction-of-arrival (DOA) estimation methods in a smart antenna system. This study demonstrates a new multipath signal DOA estimation technique. Unlike the common methods, without decoherence preprocessing, the proposed algorithm first apply a complex fast independent component analysis (cFastICA) algorithm to obtain the steering vectors with multipath information that corresponds to each source signal. Then, according to the special structure of the obtained steering vectors and spatial sparsity of the multipath signal components, the algorithm uses the solution of the sparse signal reconstruction problem in the compressive sensing (CS) theory, and the DOA estimation of the multipath signal is translated into an l1 norm minimization problem. Finally, we search the space spectrums to acquire the DOAs for each direct component and multipath component. Comparative simulation tests and analysis prove the effectiveness of the proposed algorithm in estimation accuracy in underdetermined conditions.

Structural learning in artificial neural networks using sparse optimization

In this paper, the problem of simultaneously estimating the structure and parameters of artificial neural networks with multiple hidden layers is considered. A method based on sparse optimization is proposed. The problem is formulated as an ℓ0-norm minimization problem, so that redundant weights are eliminated from the neural network. Such problems are in general combinatorial, and are often considered intractable. Hence, an iterative reweighting heuristic for relaxing the ℓ0-norm is presented. Experiments have been carried out on simple benchmark problems, both for classification and regression, and on a case study for estimation of waste heat recovery in ships. All experiments demonstrate the effectiveness of the algorithm.

Iterative sparsity score for feature selection and its extension for multimodal data

As a key dimensionality reduction technique in pattern recognition, feature selection has been widely used in information retrieval, text classification and genetic data analysis. In recent years, structural information contained in samples for guiding feature selection has become a new hot spot in machine learning field. Although tremendous feature selection methods have been developed, less important features are still used to construct the structure in those conventional structure based feature selection approaches. In this paper, we propose a new filter-type feature selection method called iterative sparsity score, which is independent of any learning algorithm. The proposed method can preserve the structural information by sparse representation, which can be efficiently solved by a ℓ1-norm minimization problem. To exclude data noise, at one time we discard last m features and iteratively optimize the ℓ1-norm minimization problem. We perform clustering and classification experiments on numerous bench mark datasets. Furthermore, its extension for multimodal data is also developed. We adopt the multi-modality alzheimer’s disease data for classification to evaluate the extended method. The experimental results show the effectiveness of our proposed methods compared with several popular feature selection approaches.

L 1 norm minimization in partial errors-in-variables model

The weighted total least-squares (WTLS) estimate is sensitive to outliers and will be strongly disturbed if there are outliers in the observations and coefficient matrix of the partial errors-in-variables (EIV) model. The L 1 norm minimization method is a robust technique to resist the bad effect of outliers. Therefore, the computational formula of the L 1 norm minimization for the partial EIV model is developed by employing the linear programming theory. However, the closed-form solution cannot be directly obtained since there are some unknown parameters in constrained condition equation of the presented optimization problem. The iterated procedure is recommended and the proper condition for stopping iteration is suggested. At the same time, by treating the partial EIV model as the special case of the non-linear Gauss–Helmert (G–H) model, another iterated method for the L 1 norm minimization problem is also developed. At last, two simulated examples and a real data of 2D affine transformation are conducted. It is illustrated that the results derived by the proposed L 1 norm minimization methods are more accurate than those by the WTLS method while the observations and elements of the coefficient matrix are contaminated with outliers. And the two methods for the L 1 norm minimization problem are identical in the sense of robustness. By comparing with the data-snooping method, the L 1 norm minimization method may be more reliable for detecting multiple outliers due to masking. But it leads to great computation burden.

Multi-stage convex relaxation method for low-rank and sparse matrix separation problem

The low-rank and sparse matrix separation problem, recovering a low-rank matrix and a sparse matrix from a small number of linear observations, has arisen in a broad set of applications. The desire for efficiency motivated the research on it, however, most of them focus on the nuclear norm plus ℓ1-norm minimization problem, which is a single convex relaxation of the low-rank and sparse matrix separation problem. This paper is devoted to investigate a multi-stage convex relaxation method to recover the low-rank and sparse components via solving a sequence of convex relaxations, while the first stage of the method includes the nuclear norm plus ℓ1-norm minimization problem. Numerical studies indicate that the multi-stage convex relaxation outperforms the one-stage convex relaxation, especially when observations are incomplete and noisy. We have also verified the efficiency of the proposed method by applying it to the static background extraction from surveillance video.

Robust Generalized Low Rank Approximations of Matrices.

In recent years, the intrinsic low rank structure of some datasets has been extensively exploited to reduce dimensionality, remove noise and complete the missing entries. As a well-known technique for dimensionality reduction and data compression, Generalized Low Rank Approximations of Matrices (GLRAM) claims its superiority on computation time and compression ratio over the SVD. However, GLRAM is very sensitive to sparse large noise or outliers and its robust version does not have been explored or solved yet. To address this problem, this paper proposes a robust method for GLRAM, named Robust GLRAM (RGLRAM). We first formulate RGLRAM as an l 1-norm optimization problem which minimizes the l 1-norm of the approximation errors. Secondly, we apply the technique of Augmented Lagrange Multipliers (ALM) to solve this l 1-norm minimization problem and derive a corresponding iterative scheme. Then the weak convergence of the proposed algorithm is discussed under mild conditions. Next, we investigate a special case of RGLRAM and extend RGLRAM to a general tensor case. Finally, the extensive experiments on synthetic data show that it is possible for RGLRAM to exactly recover both the low rank and the sparse components while it may be difficult for previous state-of-the-art algorithms. We also discuss three issues on RGLRAM: the sensitivity to initialization, the generalization ability and the relationship between the running time and the size/number of matrices. Moreover, the experimental results on images of faces with large corruptions illustrate that RGLRAM obtains the best denoising and compression performance than other methods.

A fast and accurate algorithm for ℓ 1 minimization problems in compressive sampling

Abstract An accurate and efficient algorithm for solving the constrained ℓ 1-norm minimization problem is highly needed and is crucial for the success of sparse signal recovery in compressive sampling. We tackle the constrained ℓ 1-norm minimization problem by reformulating it via an indicator function which describes the constraints. The resulting model is solved efficiently and accurately by using an elegant proximity operator-based algorithm. Numerical experiments show that the proposed algorithm performs well for sparse signals with magnitudes over a high dynamic range. Furthermore, it performs significantly better than the well-known algorithm NESTA (a shorthand for Nesterov’s algorithm) and DADM (dual alternating direction method) in terms of the quality of restored signals and the computational complexity measured in the CPU-time consumed.

Video segmentation with L0 gradient minimization

Video segmentation is an important preprocessing step for many computer vision and graphics tasks. Its main goal is to group the voxels in the video volume with similar appearance and motion into spatio-temporally consistent supervoxels. In this paper, we formulate video segmentation as an L0 gradient minimization problem, so that the spatio-temporal coherence can be effectively enforced through a gradient sparsity pursuit way. In our method, the appearance and motion descriptor space is first built for over-segmented image patches of each video frame. Then the L0 gradient minimization is performed in the descriptor space, for both spatial and temporal dimensions. To solve the non-convex L0 norm minimization problem, we extend the fused coordinate descent algorithm from 2D image grids to 3D video volume. We conduct quantitative evaluation of our method in a public video segmentation benchmark LIBSVX. The experimental results demonstrate our superior performance to state-of-the-arts in segmentation accuracy and undersegmentation error, and comparable performance in boundary recall and explained variation.

Improved sparse reconstruction for fluorescence molecular tomography with L1/2 regularization.

Fluorescence molecular tomography (FMT) is a promising imaging technique that allows in vivo visualization of molecular-level events associated with disease progression and treatment response. Accurate and efficient 3D reconstruction algorithms will facilitate the wide-use of FMT in preclinical research. Here, we utilize L1/2-norm regularization for improving FMT reconstruction. To efficiently solve the nonconvex L1/2-norm penalized problem, we transform it into a weighted L1-norm minimization problem and employ a homotopy-based iterative reweighting algorithm to recover small fluorescent targets. Both simulations on heterogeneous mouse model and in vivo experiments demonstrated that the proposed L1/2-norm method outperformed the comparative L1-norm reconstruction methods in terms of location accuracy, spatial resolution and quantitation of fluorescent yield. Furthermore, simulation analysis showed the robustness of the proposed method, under different levels of measurement noise and number of excitation sources.

Exploiting spatiospectral correlation for impulse denoising in hyperspectral images

This paper proposes a technique for reducing impulse noise from corrupted hyperspectral images. We exploit the spatiospectral correlation present in hyperspectral images to sparsify the datacube. Since impulse noise is sparse, denoising is framed as an l1-norm regularized l1-norm data fidelity minimization problem. We derive an efficient split Bregman-based algorithm to solve the same. Experiments on real datasets show that our proposed technique, when compared with state-of-the-art denoising algorithms, yields better results.

L0-norm sparse representation based on modified genetic algorithm for face recognition

The typical sparse representation for classification (SRC) exploits the training samples to represent the test samples, and classifies the test samples based on the representation results. SRC is essentially an L0-norm minimization problem which can theoretically yield the sparsest representation and lead to the promising classification performance. We know that it is difficult to directly resolve L0-norm minimization problem by applying usual optimization method. To effectively address this problem, we propose the L0-norm based SRC by exploiting a modified genetic algorithm (GA), termed GASRC, in this paper. The basic idea of GASRC is that it modifies the traditional genetic algorithm and then uses the modified GA (MGA) to select a part of the training samples to represent a test sample. Compared with the conventional SRC based on L1-norm optimization, GASRC can achieve better classification performance. Experiments on several popular real-world databases show the good classification effectiveness of our approach.

MR image super-resolution reconstruction using sparse representation, nonlocal similarity and sparse derivative prior

In magnetic resonance (MR) imaging, image spatial resolution is determined by various instrumental limitations and physical considerations. This paper presents a new algorithm for producing a high-resolution version of a low-resolution MR image. The proposed method consists of two consecutive steps: (1) reconstructs a high-resolution MR image from a given low-resolution observation via solving a joint sparse representation and nonlocal similarity L1-norm minimization problem; and (2) applies a sparse derivative prior based post-processing to suppress blurring effects. Extensive experiments on simulated brain MR images and two real clinical MR image datasets validate that the proposed method achieves much better results than many state-of-the-art algorithms in terms of both quantitative measures and visual perception.

Multi-task l0 gradient minimization for visual tracking

In most object tracking algorithms based on sparse representation, the optimization problem is often formulated as an l1 or l2 minimization problem, because its primal l0-norm minimization problem is NP-hard. In this paper, a visual tracking method is proposed based upon l0-norm minimization which directly seeks solution to the primal l0 problem. To avoid solving a large number of l0 minimization problems, we introduce to encode all samples simultaneously in a multi-task manner, which means that the number of minimization problem to be solved is only one, and an algorithm is presented to solve the minimization problem. Our tracking algorithm is then implemented under the framework of particle filter. Experiments on different challenging video sequences demonstrate that our method can achieve robust tracking results.

Locality-sensitive kernel sparse representation classification for face recognition

In this paper a new classification method called locality-sensitive kernel sparse representation classification (LS-KSRC) is proposed for face recognition. LS-KSRC integrates both sparsity and data locality in the kernel feature space rather than in the original feature space. LS-KSRC can learn more discriminating sparse representation coefficients for face recognition. The closed form solution of the l1-norm minimization problem for LS-KSRC is also presented. LS-KSRC is compared with kernel sparse representation classification (KSRC), sparse representation classification (SRC), locality-constrained linear coding (LLC), support vector machines (SVM), the nearest neighbor (NN), and the nearest subspace (NS). Experimental results on three benchmarking face databases, i.e., the ORL database, the Extended Yale B database, and the CMU PIE database, demonstrate the promising performance of the proposed method for face recognition, outperforming the other used methods.

Sparse Recovery of Streaming Signals Using <formula formulatype="inline"><tex Notation="TeX">$\ell_1$</tex></formula>-Homotopy

Most of the existing sparse-recovery methods assume a static system: the signal is a finite-length vector for which a fixed set of measurements and sparse representation are available and an l1 problem is solved for the reconstruction. However, the same representation and reconstruction framework is not readily applicable in a streaming system: the signal changes over time, and it is measured and reconstructed sequentially over small intervals. This is particularly desired when dividing signals into disjoint blocks and processing each block separately is infeasible or inefficient. In this paper, we discuss two streaming systems and a new homotopy algorithm for quickly solving the associated l1 problems: 1) recovery of smooth, time-varying signals for which, instead of using block transforms, we use lapped orthogonal transforms for sparse representation and 2) recovery of sparse, time-varying signals that follows a linear dynamic model. For both systems, we iteratively process measurements over a sliding interval and solve a weighted l1-norm minimization problem for estimating sparse coefficients. Since we estimate overlapping portions of the signal while adding and removing measurements, instead of solving a new l1 program as the system changes, we use available signal estimates as starting point in a homotopy formulation and update the solution in a few simple steps. We demonstrate with numerical experiments that our proposed streaming recovery framework provides better reconstruction compared to the methods that represent and reconstruct signals as independent, disjoint blocks, and that our proposed homotopy algorithm updates the solution faster than the current state-of-the-art solvers.

Linearized alternating directions method for [formula omitted]-norm inequality constrained [formula omitted]-norm minimization

The ℓ1-regularization is popular in compressive sensing due to its ability to promote sparsity property. In the past few years, intensive research activities have been attracted to the algorithms for ℓ1-regularized least squares or its multifarious variations. In this study, we consider the ℓ1-norm minimization problems simultaneously with ℓ1-norm inequality constraints. The formulation of this problem is preferable when the measurement of a large and sparse signal is corrupted by an impulsive noise, in the mean time the noise level is given. This study proposes and investigates an inexact alternating direction method. At each iteration, as the closed-form solution of the resulting subproblem is not clear, we apply a linearized technique such that the closed-form solutions of the linearized subproblem can be easily derived. Global convergence of the proposed method is established under some appropriate assumptions. Numerical results, including comparisons with another algorithm are reported which demonstrate the superiority of the proposed algorithm. Finally, we extend the algorithm to solve ℓ2-norm constrained ℓ1-norm minimization problem, and show that the linearized technique can be avoided.