Total Variation Penalty Research Articles

Prediction of activation patterns preceding hallucinations in patients with schizophrenia using machine learning with structured sparsity.

Despite significant progress in the field, the detection of fMRI signal changes during hallucinatory events remains difficult and time-consuming. This article first proposes a machine-learning algorithm to automatically identify resting-state fMRI periods that precede hallucinations versus periods that do not. When applied to whole-brain fMRI data, state-of-the-art classification methods, such as support vector machines (SVM), yield dense solutions that are difficult to interpret. We proposed to extend the existing sparse classification methods by taking the spatial structure of brain images into account with structured sparsity using the total variation penalty. Based on this approach, we obtained reliable classifying performances associated with interpretable predictive patterns, composed of two clearly identifiable clusters in speech-related brain regions. The variation in transition-to-hallucination functional patterns not only from one patient to another but also from one occurrence to the next (e.g., also depending on the sensory modalities involved) appeared to be the major difficulty when developing effective classifiers. Consequently, second, this article aimed to characterize the variability within the prehallucination patterns using an extension of principal component analysis with spatial constraints. The principal components (PCs) and the associated basis patterns shed light on the intrinsic structures of the variability present in the dataset. Such results are promising in the scope of innovative fMRI-guided therapy for drug-resistant hallucinations, such as fMRI-based neurofeedback.

On the total variation regularized estimator over a class of tree graphs

We generalize to tree graphs obtained by connecting path graphs an oracle result obtained for the Fused Lasso over the path graph. Moreover we show that it is possible to substitute in the oracle inequality the minimum of the distances between jumps by their harmonic mean. In doing so we prove a lower bound on the compatibility constant for the total variation penalty. Our analysis leverages insights obtained for the path graph with one branch to understand the case of more general tree graphs. As a side result, we get insights into the irrepresentable condition for such tree graphs.

Low-Dose CBCT Reconstruction Using Hessian Schatten Penalties.

Cone-beam computed tomography (CBCT) has been widely used in radiation therapy. For accurate patient setup and treatment target localization, it is important to obtain high-quality reconstruction images. The total variation (TV) penalty has shown the state-of-the-art performance in suppressing noise and preserving edges for statistical iterative image reconstruction, but it sometimes leads to the so-called staircase effect. In this paper, we proposed to use a new family of penalties-the Hessian Schatten (HS) penalties-for the CBCT reconstruction. Consisting of the second-order derivatives, the HS penalties are able to reflect the smooth intensity transitions of the underlying image without introducing the staircase effect. We discussed and compared the behaviors of several convex HS penalties with orders 1, 2, and for CBCT reconstruction. We used the majorization-minimization approach with a primal-dual formulation for the corresponding optimization problem. Experiments on two digital phantoms and two physical phantoms demonstrated the proposed penalty family's outstanding performance over TV in suppressing the staircase effect, and the HS penalty with order 1 had the best performance among the HS penalties tested.

Quantitative Analysis of Intravoxel Incoherent Motion (IVIM) Diffusion MRI using Total Variation and Huber Penalty Function.

Quantitative analysis in intravoxel incoherent motion (IVIM) imaging commonly uses voxel-wise estimation of the bi-exponential model, which might not be reliable for clinical interpretation. Improving model fitting performance and qualitative and quantitative parametric estimation, two novel methodologies are proposed here. Five IVIM analyses methodologies: (a) Bi-exponential (BE) model, (b) Segmented BE method with two-parameter fitting (BEseg-2), (c) Segmented BE method with one-parameter fitting (BEseg-1), (d) BE with adaptive Total Variation penalty function (BE+TV) and (e) BE with adaptive Huber penalty function (BE+HPF) were evaluated. Relative root-mean-square error (RRMSE), relative bias (RB) and relative parameters (Drel,Drel∗,&frel) were calculated to estimate the accuracy of methods in simulations. Empirical datasets from 14 patients with bone tumor were analyzed using these methodologies. Coefficient of variation (CV) were estimated for each IVIM parameter in tumor volume to measure the precision of the estimation methods invivo. Both BE+TV and BE+HPF showed consistently lower RRMSE (~10-42%) and lower RB (-4 to 8%) at all noise levels, compared to BE, BEseg-2 and BEseg-1 (RRMSE: ~15-120% and RB: -20 to 62%). Estimated Drel,Drel∗&frel for both BE+TV and BE+HPF methods were ~1 (0.96-1.08), whereas BE, BEseg-2 and BEseg-1 showed sub-optimal parameter estimation (0.80-1.62). For clinical data BE+TV and BE+HPF showed 30-50% improved CV in estimating D, D*, and f than BE and improved CV in estimating D* (7-23%) and f (26-30%) than BEseg-2 and BEseg-1. Bi-exponential model with penalty function showed quantitatively and qualitatively improved IVIM parameter estimation for both simulated and clinical dataset of bone tumors, thus potentially making this approach suitable for clinical applications in future.

Diffuse Optical Tomography Based on Radiative Transfer Equation with L∞ Data Fidelity and Total Variation Penalty Regularization

Diffuse Optical Tomography Based on Radiative Transfer Equation with <i>L</i>^∞ Data Fidelity and Total Variation Penalty Regularization

Model-based iterative reconstruction using higher-order regularization of dynamic synchrotron data

We present a novel iterative reconstruction method applied to in situ x-ray synchrotron tomographic data of dendrite formation during the solidification of magnesium alloy. Frequently, fast dynamic imaging projection data are undersampled, noisy, of poor contrast and can contain various acquisition artifacts. Direct reconstruction methods are not suitable and iterative reconstruction techniques must be adapted to the existing data features. Normally, an accurate modelling of the objective function can guarantee a better reconstruction. In this work, we design a special cost function where the data fidelity term is based on the Group-Huber functional to minimize ring artifacts and the regularization term is a higher-order variational penalty. We show that the total variation penalty is unsuitable for some cases and higher-order regularization functionals can ensure a better fit to the expected properties of the data. Additionally, we highlight the importance of 3D regularization over 2D for the problematic data. The proposed method shows a promising performance dealing with angular undersampled noisy dynamic data with ring artifacts.

Significant Anatomy Detection Through Sparse Classification: A Comparative Study.

We present a comparative study for discriminative anatomy detection in high dimensional neuroimaging data. While most studies solve this problem using mass univariate approaches, recent works show better accuracy and variable selection using a sparse classification model. Two types of image-based regularization methods have been proposed in the literature based on either a Graph Net (GN) model or a total variation (TV) model. These studies showed increased classification accuracy and interpretability of results when using image-based regularization, but did not look at the accuracy and quality of the recovered significant regions. In this paper, we theoretically prove bounds on the recovered sparse coefficients and the corresponding selected image regions in four models (two based on GN penalty and two based on TV penalty). Practically, we confirm the theoretical findings by measuring the accuracy of selected regions compared with ground truth on simulated data. We also evaluate the stability of recovered regions over cross-validation folds using real MRI data. Our findings show that the TV penalty is superior to the GN model. In addition, we showed that adding an l2 penalty improves the accuracy of estimated coefficients and selected significant regions for the both types of models.

Superiorized algorithm for reconstruction of CT images from sparse-view and limited-angle polyenergetic data

Recent work in CT image reconstruction has seen increasing interest in the use of total variation (TV) and related penalties to regularize problems involving reconstruction from undersampled or incomplete data. Superiorization is a recently proposed heuristic which provides an automatic procedure to ‘superiorize’ an iterative image reconstruction algorithm with respect to a chosen objective function, such as TV. Under certain conditions, the superiorized algorithm is guaranteed to find a solution that is as satisfactory as any found by the original algorithm with respect to satisfying the constraints of the problem; this solution is also expected to be superior with respect to the chosen objective.Most work on superiorization has used reconstruction algorithms which assume a linear measurement model, which in the case of CT corresponds to data generated from a monoenergetic x-ray beam. Many CT systems generate x-rays from a polyenergetic spectrum, however, in which the measured data represent an integral of object attenuation over all energies in the spectrum. This inconsistency with the linear model produces the well-known beam hardening artifacts, which impair analysis of CT images.In this work we superiorize an iterative algorithm for reconstruction from polyenergetic data, using both TV and an anisotropic TV (ATV) penalty. We apply the superiorized algorithm in numerical phantom experiments modeling both sparse-view and limited-angle scenarios. In our experiments, the superiorized algorithm successfully finds solutions which are as constraints-compatible as those found by the original algorithm, with significantly reduced TV and ATV values. The superiorized algorithm thus produces images with greatly reduced sparse-view and limited angle artifacts, which are also largely free of the beam hardening artifacts that would be present if a superiorized version of a monoenergetic algorithm were used.

A Robust Method for Inferring Network Structures

Inferring the network structure from limited observable data is significant in molecular biology, communication and many other areas. It is challenging, primarily because the observable data are sparse, finite and noisy. The development of machine learning and network structure study provides a great chance to solve the problem. In this paper, we propose an iterative smoothing algorithm with structure sparsity (ISSS) method. The elastic penalty in the model is introduced for the sparse solution, identifying group features and avoiding over-fitting, and the total variation (TV) penalty in the model can effectively utilize the structure information to identify the neighborhood of the vertices. Due to the non-smoothness of the elastic and structural TV penalties, an efficient algorithm with the Nesterov’s smoothing optimization technique is proposed to solve the non-smooth problem. The experimental results on both synthetic and real-world networks show that the proposed model is robust against insufficient data and high noise. In addition, we investigate many factors that play important roles in identifying the performance of ISSS.

Assessment of vectorial total variation penalties on realistic dual-energy CT data

Vectorial extensions of total variation have recently been developed for regularizing the reconstruction and denoising of multi-channel images, such as those arising in spectral computed tomography. Early studies have focused mainly on simulated, piecewise-constant images whose structure may favor total-variation penalties. In the current manuscript, we apply vectorial total variation to real dual-energy CT data of a whole turkey in order to determine if the same benefits can be observed in more complex images with anatomically realistic textures. We consider the total nuclear variation () as well as another vectorial total variation based on the Frobenius norm () and standard channel-by-channel total variation (). We performed a series of 3D TV denoising experiments comparing the three TV variants across a wide range of smoothness parameter settings, optimizing each regularizer according to a very-high-dose ‘ground truth’ image. Consistent with the simulation studies, we find that both vectorial TV variants achieve a lower error than the channel-by-channel TV and are better able to suppress noise while preserving actual image features. In this real data study, the advantages are subtler than in the previous simulation study, although the penalty is found to have clear advantages over either or when comparing material images formed from linear combinations of the denoised energy images.

On the prediction performance of the Lasso

Although the Lasso has been extensively studied, the relationship between its prediction performance and the correlations of the covariates is not fully understood. In this paper, we give new insights into this relationship in the context of multiple linear regression. We show, in particular, that the incorporation of a simple correlation measure into the tuning parameter can lead to a nearly optimal prediction performance of the Lasso even for highly correlated covariates. However, we also reveal that for moderately correlated covariates, the prediction performance of the Lasso can be mediocre irrespective of the choice of the tuning parameter. We finally show that our results also lead to near-optimal rates for the least-squares estimator with total variation penalty.

Photon-limited depth and reflectivity imaging with sparsity regularization

We demonstrate a depth and reflectivity imaging system at low light level based on sparsity regularization method. Depth and reflectivity imaging from the time-correlated single photon counting (TCSPC) measurement in limit of few photon counts are reconstructed through exploiting transform-domain sparsity. Two different sparsity-based penalty function: total variation (TV) penalty and l1 norm penalty measuring sparsity in the discrete cosine transform(DCT) basis, are applied to the experimental data. The results show that compared with traditional image denoising method, sparsity regularization approach achieves better accuracy with fewer photon measurements. Further more, the performance of TV regularization is proved better than l1-DCT regularization method for photon-limited imaging at first time, especially in the case of depth imaging. Our system is a photon-limited imaging device for a variety of applications, such as target detection, space surveillance, and distance measurement.

Attributing hacks with survival trend filtering

In this paper we describe an algorithm for estimating the provenance of hacks on websites. That is, given properties of sites and the temporal occurrence of attacks, we are able to attribute individual attacks to joint causes and vulnerabilities, as well as estimate the evolution of these vulnerabilities over time. Specifically, we use hazard regression with a time-varying additive hazard function parameterized in a generalized linear form. The activation coefficients on each feature are continuous-time functions over time. We formulate the problem of learning these functions as a constrained variational maximum likelihood estimation problem with total variation penalty and show that the optimal solution is a $0$th order spline (a piecewise constant function) with a finite number of adaptively chosen knots. This allows the inference problem to be solved efficiently and at scale by solving a finite dimensional optimization problem. Extensive experiments on real data sets show that our method significantly outperforms Cox’s proportional hazard model. We also conduct case studies and verify that the fitted functions of the features respond to real-life campaigns.

Penalized B-spline estimator for regression functions using total variation penalty

We carry out a study on a penalized regression spline estimator with total variation penalty. In order to provide a spatially adaptive method, we consider total variation penalty for the estimating regression function. This paper adopts B-splines for both numerical implementation and asymptotic analysis because they have small supports, so the information matrices are sparse and banded. Once we express the estimator with a linear combination of B-splines, the coefficients are estimated by minimizing a penalized residual sum of squares. A new coordinate descent algorithm is introduced to handle total variation penalty determined by the B-spline coefficients. For large-sample inference, a nonasymptotic oracle inequality for penalized B-spline estimators is obtained. The oracle inequality is then used to show that the estimator is an optimal adaptive for the estimation of the regression function up to a logarithm factor.

Hyperspectral Feature Extraction Using Total Variation Component Analysis

In this paper, a novel feature extraction method, called orthogonal total variation component analysis (OTVCA), is proposed for remotely sensed hyperspectral data. The features are extracted by minimizing a total variation (TV) penalized optimization problem. The TV penalty promotes piecewise smoothness of the extracted features which is useful for classification. A cyclic descent algorithm called OTVCA-CD is proposed for solving the minimization problem. In the experiments, OTVCA is applied on a rural hyperspectral image having low spatial resolution and an urban hyperspectral image having high spatial resolution. The features extracted by OTVCA show considerable improvements in terms of classification accuracy compared with features extracted by other state-of-the-art methods.

Profile-encoding reconstruction for multiple-acquisition balanced steady-state free precession imaging.

The scan-efficiency in multiple-acquisition balanced steady-state free precession imaging can be maintained by accelerating and reconstructing each phase-cycled acquisition individually, but this strategy ignores correlated structural information among acquisitions. Here, an improved acceleration framework is proposed that jointly processes undersampled data across N phase cycles. Phase-cycled imaging is cast as a profile-encoding problem, modeling each image as an artifact-free image multiplied with a distinct balanced steady-state free precession profile. A profile-encoding reconstruction (PE-SSFP) is employed to recover missing data by enforcing joint sparsity and total-variation penalties across phase cycles. PE-SSFP is compared with individual compressed-sensing and parallel-imaging (ESPIRiT) reconstructions. In the brain and the knee, PE-SSFP yields improved image quality compared to individual compressed-sensing and other tested methods particularly for higher N values. On average, PE-SSFP improves peak SNR by 3.8 ± 3.0 dB (mean ± s.e. across N = 2-8) and structural similarity by 1.4 ± 1.2% over individual compressed-sensing, and peak SNR by 5.6 ± 0.7 dB and structural similarity by 7.1 ± 0.5% over ESPIRiT. PE-SSFP attains improved image quality and preservation of high-spatial-frequency information at high acceleration factors, compared to conventional reconstructions. PE-SSFP is a promising technique for scan-efficient balanced steady-state free precession imaging with improved reliability against field inhomogeneity. Magn Reson Med 78:1316-1329, 2017. © 2016 International Society for Magnetic Resonance in Medicine.

Structure-adaptive CBCT reconstruction using weighted total variation and Hessian penalties.

The exposure of normal tissues to high radiation during cone-beam CT (CBCT) imaging increases the risk of cancer and genetic defects. Statistical iterative algorithms with the total variation (TV) penalty have been widely used for low dose CBCT reconstruction, with state-of-the-art performance in suppressing noise and preserving edges. However, TV is a first-order penalty and sometimes leads to the so-called staircase effect, particularly over regions with smooth intensity transition in the reconstruction images. A second-order penalty known as the Hessian penalty was recently used to replace TV to suppress the staircase effect in CBCT reconstruction at the cost of slightly blurring object edges. In this study, we proposed a new penalty, the TV-H, which combines TV and Hessian penalties for CBCT reconstruction in a structure-adaptive way. The TV-H penalty automatically differentiates the edges, gradual transition and uniform local regions within an image using the voxel gradient, and adaptively weights TV and Hessian according to the local image structures in the reconstruction process. Our proposed penalty retains the benefits of TV, including noise suppression and edge preservation. It also maintains the structures in regions with gradual intensity transition more successfully. A majorization-minimization (MM) approach was designed to optimize the objective energy function constructed with the TV-H penalty. The MM approach employed a quadratic upper bound of the original objective function, and the original optimization problem was changed to a series of quadratic optimization problems, which could be efficiently solved using the Gauss-Seidel update strategy. We tested the reconstruction algorithm on two simulated digital phantoms and two physical phantoms. Our experiments indicated that the TV-H penalty visually and quantitatively outperformed both TV and Hessian penalties.

MO-FG-CAMPUS-IeP2-04: Multiple Penalties with Different Orders for Structure Adaptive CBCT Reconstruction

Purpose: To combine total variation (TV) and Hessian penalty in a structure adaptive way for cone-beam CT (CBCT) reconstruction. Methods: TV is a widely used first order penalty with good ability in suppressing noise and preserving edges but leads to the staircase effect in regions with smooth intensity transition. The second order Hessian penalty can effectively suppress the staircase effect with extra cost of blurring object edges. To take the best of both penalties we proposed a novel method to combine both for CBCT reconstruction in a structure adaptive way. The proposed method adaptively determined the weight of each penalty according to the geometry of local regions. An specially-designed exponent term with image gradient involved was used to characterize the local geometry such that the weights for Hessian and TV were 1 and 0 respectively at uniform local regions and 0 and 1 at edge regions. For other local regions the weights varied from 0 to 1. The objective functional was minimized using the majorzationminimization approach. We evaluated the proposed method on a modified 3D shepp-logan and a CatPhan 600 phantom. The full-width-at-halfmaximum (FWHM) and contrast-to-noise (CNR) were calculated. Results: For 3D shepp-logan the reconstructed images using TV had an obvious staircase effect while those using the proposed method and Hessian preserved the smooth transition regions well. FWHMs of the proposed method TV and Hessian penalty were 1.75 1.61 and 3.16 respectively, indicating that both TV and the proposed method is able to preserve edges. For CatPhan 600 CNR values of the proposed method were similar to those of TV and Hessian. Conclusion: The proposed method retains favorable properties of TV like preserving edges and also has the ability in better preserving gradual transition structure as Hessian does. All methods performs similarly in suppressing noise. This work was supported in part by National Natural Science Foundation of China (NNSFC) under Grant Nos.60971112 and 61375018 grants from the Cancer Prevention and Research Institute of Texas (RP130109 and RP110562-P2) National Institute of Biomedical Imaging and Bioengineering (R01 EB020366) and a grant from the American Cancer Society (RSG-13-326-01-CCE).

An adaptive penalty multi-pitch estimator with self-regularization

This work treats multi-pitch estimation, and in particular the common misclassification issue wherein the pitch at half the true fundamental frequency, the sub-octave, is chosen instead of the true pitch. Extending on current group LASSO-based methods for pitch estimation, this work introduces an adaptive total variation penalty, which enforces both group- and block sparsity, as well as deals with errors due to sub-octaves. Also presented is a scheme for signal adaptive dictionary construction and automatic selection of the regularization parameters. Used together with this scheme, the proposed method is shown to yield accurate pitch estimates when evaluated on synthetic speech data. The method is shown to perform as good as, or better than, current state-of-the-art sparse methods while requiring fewer tuning parameters than these, as well as several conventional pitch estimation methods, even when these are given oracle model orders. When evaluated on a set of ten musical pieces, the method shows promising results for separating multi-pitch signals.

A hybrid regularization method combining Tikhonov with total variation for electrical resistance tomography

Electrical resistance tomography (ERT) is a promising measurement technique in industrial process imaging. However, image reconstruction in ERT is an ill-posed inverse problem. Regularization methods have been developed to solve the ill-posed inverse problem. Since the penalty term is a form of L2-norm, Tikhonov regularization method guarantees the stability of the solution, but it always makes the image edge oversmoothed. Total variation (TV) regularization method has good ability of preserving image edges. A hybrid regularization method, which combines Tikhonov with TV regularization method, is proposed to get better reconstructed images. The choice of the adaptive weighted parameter between TV and Tikhonov penalty term has been discussed in detail. In the proposed hybrid regularization method, the function of conductivity gradients is used as the adaptive weighted parameter to control automatically the weighting between the penalty terms from TV and Tikhonov regularization. For the model with sharp edges, the proportion of the penalty term from TV regularization is increased to preserve the edges, while for the model with smooth edges, the proportion of penalty term from Tikhonov regularization is increased to make the solution stable and robust to noise. Both simulation and experimental results of Tikhonov, TV and hybrid regularization method are shown respectively, which indicates that the hybrid regularization method can improve the reconstruction quality with sharp edges and is more robust to noise, and it is applicable for models with different edge characteristic.