Total Generalized Variation Research Articles

High-resolution depth imaging with a small-scale SPAD array based on the temporal-spatial filter and intensity image guidance.

Currently single-photon avalanche diode (SPAD) arrays suffer from a small-scale pixel count, which makes it difficult to achieve high-resolution 3D imaging directly through themselves. We established a CCD camera-assisted SPAD array depth imaging system. Based on illumination laser lattice generated by a diffractive optical element (DOE), the registration of the low-resolution depth image gathered by SPAD and the high-resolution intensity image gathered by CCD is realized. The intensity information is used to guide the reconstruction of a resolution-enhanced depth image through a proposed method consisting of total generalized variation (TGV) regularization and temporal-spatial (T-S) filtering algorithm. Experimental results show that an increasement of 4 × 4 times for native depth image resolution is achieved and the depth imaging quality is also improved by applying the proposed method.

A framework from point clouds to workpieces

Combining computer-aided design and computer numerical control (CNC) with global technical connections have become interesting topics in the manufacturing industry. A framework was implemented that includes point clouds to workpieces and consists of a mesh generation from geometric data, optimal surface segmentation for CNC, and tool path planning with a certified scallop height. The latest methods were introduced into the mesh generation with implicit geometric regularization and total generalized variation. Once the mesh model was obtained, a fast and robust optimal surface segmentation method is provided by establishing a weighted graph and searching for the minimum spanning tree of the graph for extraordinary points. This method is easy to implement, and the number of segmented patches can be controlled while preserving the sharp features of the workpiece. Finally, a contour parallel tool-path with a confined scallop height is generated on each patch based on B-spline fitting. Experimental results show that the proposed framework is effective and robust.

Dualization and Automatic Distributed Parameter Selection of Total Generalized Variation via Bilevel Optimization

Total Generalized Variation (TGV) regularization in image reconstruction relies on an infimal convolution type combination of generalized first- and second-order derivatives. This helps to avoid the staircasing effect of Total Variation (TV) regularization, while still preserving sharp contrasts in images. The associated regularization effect crucially hinges on two parameters whose proper adjustment represents a challenging task. In this work, a bilevel optimization framework with a suitable statistics-based upper level objective is proposed in order to automatically select these parameters. The framework allows for spatially varying parameters, thus enabling better recovery in high-detail image areas. A rigorous dualization framework is established, and for the numerical solution, a Newton type method for the solution of the lower level problem, i.e. the image reconstruction problem, and a bilevel TGV algorithm are introduced. Denoising tests confirm that automatically selected distributed regularization parameters lead in general to improved reconstructions when compared to results for scalar parameters.

HFP-QSMGAN: QSM from homodyne-filtered phase images.

PurposeHomodyne filtering is a standard preprocessing step in the estimation of SWI. Unfortunately, SWI is not quantitative, and QSM cannot be accurately estimated from filtered phase images. Compared with gradient‐echo sequences suitable for computing QSM, SWI is more readily available and is often the only susceptibility‐sensitive sequence acquired in the clinical setting. In this project, we aimed to quantify susceptibility from the homodyne‐filtered phase (HFP), acquired for computing susceptibility‐weighted images, using convolutional neural networks to solve the compounded problem of (1) computing the solution to the inverse dipole problem, and (2) compensating for the effects of the homodyne filtering.MethodsTwo convolutional neural networks, the U‐Net and a modified QSMGAN architecture (HFP‐QSMGAN), were trained to predict QSM maps at different TEs from HFP images. The QSM maps were quantified from a gradient‐echo sequence acquired in the same individuals using total generalized variation (TGV)‐QSM. The QSM maps estimated directly from the HFP were also included for comparison. Voxel‐wise predictions and, importantly, regional predictions of susceptibility with adjustment to a reference region, were compared.ResultsOur results indicate that the U‐Net model provides more accurate voxel‐wise predictions of susceptibility compared with HFP‐QSMGAN and HFP‐QSM. However, regional estimates of susceptibility predicted by HFP‐QSMGAN are more strongly correlated with the values from TGV–QSM compared with those of U‐Net and HFP‐QSM.ConclusionAccurate prediction of susceptibility can be achieved from filtered SWI phase using convolutional neural networks.

Variational image decomposition model TGV-Hilbert-BM3D for phase retrieval in FPP and its application for an on-site wall surface bulge.

Mass loss from wall surface bulge deformation can be used to estimate the strength loss of reinforcement, bond reduction, and ductility degradation, so it is very important to accurately measure the three-dimensional (3D) shape of on-site wall surface bulge. In this paper, we try to solve the problem by use of fringe projection profilometry. In the fringe projection patterns of wall surface bulge, the contrast of the fringes is very weak, and there are sometimes cracks in patterns. We first present a preprocessing method to inpaint fringes if there are damaged fringes caused by cracks. Then we propose a new, to the best of our knowledge, image decomposition model, total generalized variation (TGV)-Hilbert-block-matching (BM)3D, to effectively extract the fringe component. Finally, we use Fourier transform, phase unwrapping, and carrier-removal methods to obtain the unwrapped phase. We test the proposed method on a simulated fringe projection pattern and two real fringe projection patterns of wall surface bulge. We compare our method with the advanced total variation space-generalized functions space-BM3D, TV-Hilbert-L2, and Beppo-Levi-space-Hilbert-BM3D methods. In addition, we perform ablation experiments to prove that our preprocessing method is necessary. The experimental results demonstrate that our method can effectively measure the 3D shape of wall surface bulge from a single fringe projection pattern for the first time, to our knowledge.

Multi-directional rain streak removal based on infimal convolution of oscillation TGV

Image deraining is essential in both image processing and computer vision. However it is still challenging to remove rain streaks, due to the fact that they are complex and usually have multiple directions. In this paper, we propose a novel infimal convolution type functional based model to remove multi-directional rain streaks. Specifically, we employ total generalized variation (TGV) to represent the rain-free background and m-fold infimal convolution of oscillation TGV (ICTGVmosci) to capture the rain streaks with m different directions. The existence of solutions to the proposed model is proved. We also design an algorithm using the classical first-order Primal–Dual method for solving the proposed model. Numerical experiments on both synthetic and real images demonstrate that the proposed model outperforms the state-of-the-art methods quantitatively and qualitatively.

Quantitative evaluation of simultaneous spatial and temporal regularization in dynamic contrast-enhanced MRI of the liver using Gd-EOB-DTPA

The purpose of this study was to quantitatively evaluate the usefulness of simultaneous spatial and temporal regularization using total variation (TV), total generalized variation (TGV), a combination of low-rank decomposition (LRD) and TV (LRD+TV), a combination of LRD and TGV (LRD+TGV), and nuclear norm (NN) when applied to dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) in rats with concanavalin A (ConA)-induced acute hepatic injury.The rats were divided into three groups: normal control (NC) (n = 10), ConA10 (n = 8), and ConA20 (n = 7). Rats in the ConA10 and ConA20 groups were intravenously injected with 10 and 20 mg/kg of ConA, respectively; those in the NC group were intravenously injected with the same volume of saline. DCE-MRI studies were performed using gadolinium ethoxybenzyl diethylenetriamine pentaacetic acid (Gd-EOB-DTPA; 0.025 mmol Gd/kg) as a contrast agent (CA), 24 h after the ConA or saline injection. After the DCE-MRI study, we generated zero-filled and undersampled k-space data from the original images using a pseudoradial sampling scheme with 4 to 64 spokes. We subsequently reconstructed images from these data using the above regularizers and calculated the signal-to-error ratio (SERimg) and structural similarity index measure (SSIM) using the original and reconstructed images. We also calculated the area under the curve (AUC), rate of CA washout (λw), maximum relative enhancement (REmax), and time to REmax (Tmax) from time-intensity curves using an empirical mathematical model (EMM) and the signal-to-error ratio for curve fitting (SERfit) from the original and fit curves. We also compared the parameters obtained using the pseudoradial and Cartesian sampling schemes in the NC group.When using LRD+TV and LRD+TGV, both SERimg and SSIM were greater than those for the other regularizers at all spoke numbers studied; the SERfit for TGV was the greatest. When using TGV and LRD+TGV, in the majority of cases the AUCs did not significantly differ from those obtained from the original images, whereas those for LRD+TV and NN were significantly less at several spoke numbers. The λw for NN was significantly greater at numerous spoke numbers in the NC group; the REmax values for LRD+TV and NN were significantly less at several spoke numbers in all groups. The Tmax values for TV, TGV, and LRD+TGV were significantly greater at numerous spoke numbers in the NC group. Although there were significant differences in SERimg and SSIM between the pseudoradial and Cartesian sampling schemes, the kinetic parameters obtained by the EMM did not significantly differ between the two sampling schemes, with certain exceptions.In conclusion, our results suggest that simultaneous spatial and temporal regularization using TGV or LRD+TGV is useful for accelerating DCE-MRI without significant reduction in the accuracy of the kinetic parameter estimation, even at extremely low sampling factors.

An efficient augmented Lagrangian method with semismooth Newton solver for total generalized variation

<p style='text-indent:20px;'>Total generalization variation (TGV) is a very powerful and important regularization for various inverse problems and computer vision tasks. In this paper, we propose a semismooth Newton based augmented Lagrangian method for solving this problem. The augmented Lagrangian method (also called as method of multipliers) is widely used for lots of smooth or nonsmooth variational problems. However, its efficiency heavily depends on solving the corresponding coupled and nonlinear system together and simultaneously. With efficient primal-dual semismooth Newton methods for the challenging and highly coupled nonlinear subproblems involving total generalized variation, we develop a highly efficient and competitive augmented Lagrangian method compared with some fast first-order method. With the analysis of the metric subregularities of the corresponding functions, we give both the global convergence and local linear convergence rate for the proposed augmented Lagrangian methods.</p>

Generalized Hessian-Schatten Norm Regularization for Image Reconstruction

Regularization plays a crucial role in reliably utilizing imaging systems for scientific and medical investigations. It helps to stabilize the process of computationally undoing any degradation caused by physical limitations of the imaging process. In the past decades, total variation regularization played a dominant role in the literature. Two forms of total variation regularizations, namely the first-order and the second-order total variation (TV-1 and TV-2) have been widely used. TV-1 has a disadvantage: it reconstructs images in the form of piece-wise constants when the noise and/or under-sampling is severe, while TV-2 reconstructs natural-looking images under such scenarios. On the other hand, TV-1 can recover sharp jumps better than TV-2. Two forms of generalizations, namely Hessian-Schatten norm (HSN) regularization, and total generalized variation (TGV) regularization, have been proposed and have become significant developments in the area of regularization for imaging inverse problems owing to their performance. While the strength of TGV is that it can combine the advantages of TV-1 and TV-2, HSN has better structure-preserving property. Here, we develop a novel regularization for image recovery that combines the strengths of TGV and HSN. We achieve this by restricting the maximization space in the dual form of HSN in the same way that TGV is obtained from TV-2. We call the new regularization the generalized Hessian-Schatten norm regularization (GHSN). We develop a novel computational method for image reconstruction using the new form of regularization based on the well-known framework called the alternating direction method of multipliers (ADMM). We demonstrate the strength of the GHSN using some reconstruction examples.

A wavelet frame constrained total generalized variation model for imaging conductivity distribution

<p style='text-indent:20px;'>Electrical impedance tomography (EIT) is a sensing technique with which conductivity distribution can be reconstructed. It should be mentioned that the reconstruction is a highly ill-posed inverse problem. Currently, the regularization method has been an effective approach to deal with this problem. Especially, total variation regularization method is advantageous over Tikhonov method as the edge information can be well preserved. Nevertheless, the reconstructed image shows severe staircase effect. In this work, to enhance the quality of reconstruction, a novel hybrid regularization model which combines a total generalized variation method with a wavelet frame approach (TGV-WF) is proposed. An efficient mean doubly augmented Lagrangian algorithm has been developed to solve the TGV-WF model. To demonstrate the effectiveness of the proposed method, numerical simulation and experimental validation are conducted for imaging conductivity distribution. Furthermore, some comparisons are made with typical regularization methods. From the results, it can be found that the proposed method shows better performance in the reconstruction since the edge of the inclusion can be well preserved and the staircase effect is effectively relieved.</p>

Mediastinal Lymph Node Detection and Segmentation Using Deep Learning

Automatic lymph node (LN) segmentation and detection for cancer staging are critical. In clinical practice, computed tomography (CT) and positron emission tomography (PET) imaging detect abnormal LNs. Despite its low contrast and variety in nodal size and form, LN segmentation remains a challenging task. Deep convolutional neural networks frequently segment items in medical photographs. Most state-of-the-art techniques destroy image's resolution through pooling and convolution. As a result, the models provide unsatisfactory results. Keeping the issues in mind, a well-established deep learning technique UNet was modified using bilinear interpolation and total generalized variation (TGV) based upsampling strategy to segment and detect mediastinal lymph nodes. The modified UNet maintains texture discontinuities, selects noisy areas, searches appropriate balance points through backpropagation, and recreates image resolution. Collecting CT image data from TCIA, 5-patients, and ELCAP public dataset, a dataset was prepared with the help of experienced medical experts. The UNet was trained using those datasets, and three different data combinations were utilized for testing. Utilizing the proposed approach, the model achieved 94.8% accuracy, 91.9% Jaccard, 94.1% recall, and 93.1% precision on COMBO_3. The performance was measured on different datasets and compared with state-of-the-art approaches. The UNet++ model with hybridized strategy performed better than others.

An [formula omitted]-overlapping group sparse total variation for impulse noise image restoration

Total variation (TV) based methods are effective models in image restoration. For eliminating impulse noise, an effective way is to use the ℓ1-norm total variation model. However, the TV image restoration always yields staircase artifacts, especially in high-density noise levels. Additionally, the ℓ1-norm tends to over penalize solutions and is not robust to outlier characteristics of impulse noise. In this paper, we propose a new total variation model to effectively remove the staircase effects and eliminate impulse noise. The proposed model uses the ℓ0-norm data fidelity to effectively remove the impulse noise while the overlapping group sparse total variation (OGSTV) acts as a regularizer to eliminate the staircase artifacts. Since the proposed method requires solving an ℓ0-norm and an OGSTV optimization problem, a formulation using the mathematical program with equilibrium constraints (MPEC) and the majorization–minimization (MM) method are respectively used together with the alternating direction method of multipliers (ADMM). Experiments demonstrate that our proposed model performs better than several state-of-the-art algorithms such as the ℓ1 total generalized variation, ℓ0 total variation, and the ℓ1 overlapping group sparse total variation in terms of the peak signal-to-noise ratio (PSNR) and the structural similarity index measure (SSIM).

Fast Hyperparameter Calibration of Sparsity Enforcing Penalties in Total Generalised Variation Penalised Reconstruction Methods for XCT Using a Planted Virtual Reference Image

The reconstruction problem in X-ray computed tomography (XCT) is notoriously difficult in the case where only a small number of measurements are made. Based on the recently discovered Compressed Sensing paradigm, many methods have been proposed in order to address the reconstruction problem by leveraging inherent sparsity of the object’s decompositions in various appropriate bases or dictionaries. In practice, reconstruction is usually achieved by incorporating weighted sparsity enforcing penalisation functionals into the least-squares objective of the associated optimisation problem. One such penalisation functional is the Total Variation (TV) norm, which has been successfully employed since the early days of Compressed Sensing. Total Generalised Variation (TGV) is a recent improvement of this approach. One of the main advantages of such penalisation based approaches is that the resulting optimisation problem is convex and as such, cannot be affected by the possible existence of spurious solutions. Using the TGV penalisation nevertheless comes with the drawback of having to tune the two hyperparameters governing the TGV semi-norms. In this short note, we provide a simple and efficient recipe for fast hyperparameters tuning, based on the simple idea of virtually planting a mock image into the model. The proposed trick potentially applies to all linear inverse problems under the assumption that relevant prior information is available about the sought for solution, whilst being very different from the Bayesian method.

A retinex based non-local total generalized variation framework for OCT image restoration

A retinex driven non-local total generalized variational (TGV) model is proposed in this paper to restore and enhance speckled images. The combined first and second-order TGV controlled by a balancing parameter are used to improve the enhancement and restoration process. The distribution of the speckle is estimated from input images using detailed statistical analysis. The model is designed to handle speckle-noise following a Gamma distribution, as analyzed later in this paper. The non-local TGV model is shown to restore images without causing any visual artefacts, unlike the normal total variation (TV) model. Moreover, a retinex framework shows a remarkable improvement to the contrast features of the data without distorting the natural image characteristics as quantified visually and statistically in the experimental section of this work. A fast numerical approximation based on the Split-Bregman scheme is employed to improve the efficiency of the model in terms of computation. The proposed model is verified to have despeckled and enhanced the Optical Coherence Tomography (OCT) data to a greater extent compared to the state-of-the-art models as observable from the results shown in this paper.

Multimodal image fusion and denoising in NSCT domain using CNN and FOTGV

Multimodal medical image fusion is the most popular tool to integrate important information of multimodal medical images into a single complete informative image. Fusion provides an effective way for medical image diagnosis and treatment. However the acquired medical images may be corrupted with noise due to patient movement or faulty transmission, which misguides the image analysis and requires denoising. Therefore, in the suggested scheme non-subsampled contourlet transform (NSCT) is first used to extract features from the noisy source images. Then, a Siamese convolutional neural network (sCNN) is utilized for weighted fusion of important features from the two multimodal images. Simultaneously, a fractional order total generalized variation (FOTGV) is implemented for noise removal with improved degree of freedom. The image processing results demonstrate that the NSCT + sCNN + FOTGV scheme performs effectively for clean as well as noisy images in comparison to the other state-of-the-art conventional techniques on the basis of visual and quantitative analysis.

Total generalized variation and wavelet transform for impulsive image restoration

Total generalized variation and wavelet transform for impulsive image restoration

Second-order total generalized variation based model for restoring images with mixed Poisson — Gaussian noise

Introduction: A common problem in image restoration is image denoising. Among many noise models, the mixed Poisson-Gaussian model has recently aroused considerable interest. Purpose: Development of a model for denoising images corrupted by mixed Poisson-Gaussian noise, along with an algorithm for solving the resulting minimization problem. Results: We proposed a new total variation model for restoring an image with mixed Poisson-Gaussian noise, based on second-order total generalized variation. In order to solve this problem, an efficient alternating minimization algorithm is used. To illustrate its comparison with related methods, experimental results are presented, demonstrating the high efficiency of the proposed approach. Practical relevance: The proposed model allows you to remove mixed Poisson-Gaussian noise in digital images, preserving the edges. The presented numerical results demonstrate the competitive features of the proposed model.

Sparsity reconstruction using nonconvex TGpV-shearlet regularization and constrained projection

In many sparsity-based image processing problems, compared with the convex ℓ1 norm approximation of the nonconvex ℓ0 quasi-norm, one can often preserve the structures better by taking full advantage of the nonconvex ℓp quasi-norm (0≤p<1). In this paper, we propose a nonconvex ℓp quasi-norm approximation in the total generalized variation (TGV)-shearlet regularization for image reconstruction. By introducing some auxiliary variables, the nonconvex nonsmooth objective function can be solved by an efficient alternating direction method of multipliers with convergence analysis. Especially, we use a generalized iterated shrinkage operator to deal with the ℓp quasi-norm subproblem, which is easy to implement. Extensive experimental results show clearly that the proposed nonconvex sparsity approximation outperforms some state-of-the-art algorithms in both the visual and quantitative measures for different sampling ratios and noise levels.

Enhanced total generalized variation method based on moreau envelope

Image restoration with total generalized variation (TGV) regularization has been proved to be a creditable method and used broadly, but this method is imperfect in retaining image details. This paper proposes a non-convex and non-separable regularization term based on TGV enhancement, which can maintain the strict convexity of the total cost function and avoid the underestimation of the TGV method. The new regularization is obtained by subtracting the Moreau envelope of TGV from the TGV term, in which the former is defined by infimal convolution. To minimize the cost function, the proximal forward-backward splitting (FBS) algorithm is applied, and the alternating direction method of multipliers with adaptive parameter estimation (APE-ADMM), which avoids the drawbacks of FBS algorithm, is proposed by applying the variable splitting and the augment Lagrangian method (ALM). This proposed algorithm can update the regularization parameter adaptively. Experiments varify that the proposed method is more accurate in solution estimation than the other promoted ones, and the new ADMM algorithm with new regularization shows improved effects of image restoration.

Combined higher order non-convex total variation with overlapping group sparsity for impulse noise removal

A typical approach to eliminate impulse noise is to use the l1-norm for both the data fidelity term and the regularization terms. However, the l1-norm tends to over penalize signal entries which is one of its underpinnings. Hence, we propose a variational model that uses the non-convex lp-norm, 0 < p < 1 for both the data fidelity and a second-order total variation regularization term combined with an overlapping group sparse regularizer. Specifically, to robustly eliminate impulse noise, the proposed method uses a non-convex data fidelity term. The hybrid combination of a second-order non-convex total variation and an overlapping group sparse regularization term is used to eliminate the remaining staircase artifacts while maintaining a sharp restored image. A mathematical formulation is derived and to implement it, the iterative re-weighted l1 (IRL1) based alternating direction method of multipliers (ADMM) is used to solve the constraints and the subproblems. Experimental results for image denoising and deblurring on several widely used standard images demonstrate that the proposed method performed better when compared to the l1-norm total variation (TV), total generalized variation (TGV) model, and the non-convex lp-norm TV-based data fidelity model in terms of peak signal-to-noise ratio (PSNR) and structure similarity index measure (SSIM).