Sparsity-promoting Regularization Research Articles

Spatio-temporal wavelet regularization for parallel MRI reconstruction: application to functional MRI

Parallel magnetic resonance imaging (MRI) is a fast imaging technique that helps acquiring highly resolved images in space/time. Its performance depends on the reconstruction algorithm, which can proceed either in the k-space or in the image domain. To improve the performance of the widely used SENSE algorithm, 2D regularization in the wavelet domain has been investigated. In this paper, we first extend this approach to 3D-wavelet representations and the 3D sparsity-promoting regularization term, in order to address reconstruction artifacts that propagate across adjacent slices. The resulting optimality criterion is convex but nonsmooth, and we resort to the parallel proximal algorithm to minimize it. Second, to account for temporal correlation between successive scans in functional MRI (fMRI), we extend our first contribution to 3D + t acquisition schemes by incorporating a prior along the time axis into the objective function. Our first method (3D-UWR-SENSE) is validated on T1-MRI anatomical data for gray/white matter segmentation. The second method (4D-UWR-SENSE) is validated for detecting evoked activity during a fast event-related functional MRI protocol. We show that our algorithm outperforms the SENSE reconstruction at the subject and group levels (15 subjects) for different contrasts of interest (motor or computation tasks) and two parallel acceleration factors (R = 2 and R = 4) on 2 × 2 × 3 MM(3) echo planar imaging (EPI) images.

Load Curve Data Cleansing and Imputation Via Sparsity and Low Rank

The smart grid vision is to build an intelligent power network with an unprecedented level of situational awareness and controllability over its services and infrastructure. This paper advocates statistical inference methods to robustify power monitoring tasks against the outlier effects owing to faulty readings and malicious attacks, as well as against missing data due to privacy concerns and communication errors. In this context, a novel load cleansing and imputation scheme is developed leveraging the low intrinsic-dimensionality of spatiotemporal load profiles and the sparse nature of "bad data.'' A robust estimator based on principal components pursuit (PCP) is adopted, which effects a twofold sparsity-promoting regularization through an $\ell_1$-norm of the outliers, and the nuclear norm of the nominal load profiles. Upon recasting the non-separable nuclear norm into a form amenable to decentralized optimization, a distributed (D-) PCP algorithm is developed to carry out the imputation and cleansing tasks using networked devices comprising the so-termed advanced metering infrastructure. If D-PCP converges and a qualification inequality is satisfied, the novel distributed estimator provably attains the performance of its centralized PCP counterpart, which has access to all networkwide data. Computer simulations and tests with real load curve data corroborate the convergence and effectiveness of the novel D-PCP algorithm.

A local-order regularization for geophysical inverse problems

Abstract Different types of regularization have been developed to obtain stable solutions to linear inverse problems. Among these, total variation (TV) is known as an edge preserver method, which leads to piecewise constant solutions and has received much attention for solving inverse problems arising in geophysical studies. However, the method shows staircase effects and is not suitable for the models including smooth regions. To overcome the staircase effect, we present a method, which employs a local-order difference operator in the regularization term. This method is performed in two steps: First, we apply a pre-processing step to find the edge locations in the regularized solution using a properly defined minmod limiter, where the edges are determined by a comparison of the solutions obtained using different order regularizations of the TV types. Then, we construct a local-order difference operator based on the information obtained from the pre-processing step about the edge locations, which is subsequently used as a regularization operator in the final sparsity-promoting regularization. Experimental results from the synthetic and real seismic traveltime tomography show that the proposed inversion method is able to retain the smooth regions of the regularized solution, while preserving sharp transitions presented in it.

Smoothly Clipped Absolute Deviation (SCAD) regularization for compressed sensing MRI Using an augmented Lagrangian scheme

PurposeCompressed sensing (CS) provides a promising framework for MR image reconstruction from highly undersampled data, thus reducing data acquisition time. In this context, sparsity-promoting regularization techniques exploit the prior knowledge that MR images are sparse or compressible in a given transform domain. In this work, a new regularization technique was introduced by iterative linearization of the non-convex smoothly clipped absolute deviation (SCAD) norm with the aim of reducing the sampling rate even lower than it is required by the conventional l1 norm while approaching an l0 norm. Materials and MethodsThe CS-MR image reconstruction was formulated as an equality-constrained optimization problem using a variable splitting technique and solved using an augmented Lagrangian (AL) method developed to accelerate the optimization of constrained problems. The performance of the resulting SCAD-based algorithm was evaluated for discrete gradients and wavelet sparsifying transforms and compared with its l1-based counterpart using phantom and clinical studies. The k-spaces of the datasets were retrospectively undersampled using different sampling trajectories. In the AL framework, the CS-MRI problem was decomposed into two simpler sub-problems, wherein the linearization of the SCAD norm resulted in an adaptively weighted soft thresholding rule with a sparsity enhancing effect. ResultsIt was demonstrated that the proposed regularization technique adaptively assigns lower weights on the thresholding of gradient fields and wavelet coefficients, and as such, is more efficient in reducing aliasing artifacts arising from k-space undersampling, when compared to its l1-based counterpart. ConclusionThe SCAD regularization improves the performance of l1-based regularization technique, especially at reduced sampling rates, and thus might be a good candidate for some applications in CS-MRI.

Sparse Stochastic Processes and Discretization of Linear Inverse Problems

We present a novel statistically-based discretization paradigm and derive a class of maximum a posteriori (MAP) estimators for solving ill-conditioned linear inverse problems. We are guided by the theory of sparse stochastic processes, which specifies continuous-domain signals as solutions of linear stochastic differential equations. Accordingly, we show that the class of admissible priors for the discretized version of the signal is confined to the family of infinitely divisible distributions. Our estimators not only cover the well-studied methods of Tikhonov and l1-type regularizations as particular cases, but also open the door to a broader class of sparsity-promoting regularization schemes that are typically nonconvex. We provide an algorithm that handles the corresponding nonconvex problems and illustrate the use of our formalism by applying it to deconvolution, magnetic resonance imaging, and X-ray tomographic reconstruction problems. Finally, we compare the performance of estimators associated with models of increasing sparsity.

Non-negative least squares for high-dimensional linear models: Consistency and sparse recovery without regularization

Least squares fitting is in general not useful for high-dimensional linear models, in which the number of predictors is of the same or even larger order of magnitude than the number of samples. Theory developed in recent years has coined a paradigm according to which sparsity-promoting regularization is regarded as a necessity in such setting. Deviating from this paradigm, we show that non-negativity constraints on the regression coefficients may be similarly effective as explicit regularization if the design matrix has additional properties, which are met in several applications of non-negative least squares (NNLS). We show that for these designs, the performance of NNLS with regard to prediction and estimation is comparable to that of the lasso. We argue further that in specific cases, NNLS may have a better $\ell_{\infty}$-rate in estimation and hence also advantages with respect to support recovery when combined with thresholding. From a practical point of view, NNLS does not depend on a regularization parameter and is hence easier to use.

Inverse Eigenvalue Problems for Exploring the Dynamics of Systems Biology Models

This paper describes inverse eigenvalue problems that arise in studying qualitative dynamics in systems biology models. An algorithm based on lift-and- project iterations is proposed, where the lifting step entails solving a constrained matrix inverse eigenvalue problem. In particular, prior to carrying out the iterative steps, a-priori bounds on the entries of the Jacobian matrix are computed by relying on the reaction network structure as well as the form of the rate law expressions for the model under consideration. Numerical results on a number of models show that the proposed algorithm can be used to computationally explore the possible dynamical scenarios while identifying the important mechanisms via the use of sparsity-promoting regularization.