Staircase Artifacts Research Articles

CNN-based superresolution reconstruction of 3D MR images using thick-slice scans

Due to inherent physical and hardware limitations, 3D MR images are often acquired in the form of orthogonal thick slices, resulting in highly anisotropic voxels. This causes the partial volume effect, which introduces blurring of image details, appearance of staircase artifacts and significantly decreases the diagnostic value of images. To restore high resolution isotropic volumes, we propose to use a convolutional neural network (CNN) driven by patches taken from three orthogonal thick-slice images. To assess the validity and efficiency of this postprocessing approach, we used 1x1x1 mm3-voxel brain images of different modalities, available via the well known BrainWeb database. They served as a high resolution reference and were numerically preprocessed to create input images of different slice thickness and anatomical orientation, for CNN training, validation and testing. The visual quality of reconstructed images was indeed superior, compared to images obtained by fusion of interpolated thick-slice images, or to images reconstructed with the CNN using a single input MR scan. The significant increase of objectively computed figures of merit, e.g. the Structural Similarity Image Metric, was also noticed. Keeping in mind that any single value of such quality metrics represents a number of psychophysical effects, we applied the CNN trained on brain images for superresolution reconstruction of synthetic and acquired blood vessel tree images. We then used the restored superresolution volumes for estimation of vessel radii. It was demonstrated that vessel radius values derived from superresolution images of simulated vessel trees are significantly more accurate than those obtained from a standard fusion of interpolated thick-slice orthogonal scans. Superiority of the CNN-based superresolution images was also observed for scanner-acquired MR scans according to the evaluated parameters. These three experiments show the efficiency of CNN-based image reconstruction for qualitative and quantitative improvement of its diagnostic quality, as well as illustrates the practical usefulness of transfer learning - networks trained on example images of one kind can be used to restore superresolution images of physically different objects.

Infrared Image Super-Resolution Reconstruction Based on Quaternion and High-Order Overlapping Group Sparse Total Variation.

Owing to the limitations of imaging principles and system imaging characteristics, infrared images generally have some shortcomings, such as low resolution, insufficient details, and blurred edges. Therefore, it is of practical significance to improve the quality of infrared images. To make full use of the information on adjacent points, preserve the image structure, and avoid staircase artifacts, this paper proposes a super-resolution reconstruction method for infrared images based on quaternion total variation and high-order overlapping group sparse. The method uses a quaternion total variation method to utilize the correlation between adjacent points to improve image anti-noise ability and reconstruction effect. It uses the sparsity of a higher-order gradient to reconstruct a clear image structure and restore smooth changes. In addition, we performed regularization by using the denoising method, alternating direction method of multipliers, and fast Fourier transform theory to improve the efficiency and robustness of our method. Our experimental results show that this method has excellent performance in objective evaluation and subjective visual effects.

Image Deblurring under Impulse Noise via Total Generalized Variation and Non-Convex Shrinkage

Image deblurring under the background of impulse noise is a typically ill-posed inverse problem which attracted great attention in the fields of image processing and computer vision. The fast total variation deconvolution (FTVd) algorithm proved to be an effective way to solve this problem. However, it only considers sparsity of the first-order total variation, resulting in staircase artefacts. The L1 norm is adopted in the FTVd model to depict the sparsity of the impulse noise, while the L1 norm has limited capacity of depicting it. To overcome this limitation, we present a new algorithm based on the Lp-pseudo-norm and total generalized variation (TGV) regularization. The TGV regularization puts sparse constraints on both the first-order and second-order gradients of the image, effectively preserving the image edge while relieving undesirable artefacts. The Lp-pseudo-norm constraint is employed to replace the L1 norm constraint to depict the sparsity of the impulse noise more precisely. The alternating direction method of multipliers is adopted to solve the proposed model. In the numerical experiments, the proposed algorithm is compared with some state-of-the-art algorithms in terms of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), signal-to-noise ratio (SNR), operation time, and visual effects to verify its superiority.

A Novel Total Generalized Variation Model for Image Dehazing

In this paper, we propose a new variational model for removing haze from a single input image. The proposed model combines two total generalized variation (TGV) regularizations, which are related to the image intensity and the transmission map, respectively, to build an optimization problem. Actually, TGV functionals are more appropriate for describing a natural color image and its transmission map with slanted plane. By minimizing the energy functional with double-TGV regularizations, we obtain the final haze-free image and the refined transmission map simultaneously instead of the general two-step framework. The existence and uniqueness of solutions to the proposed variational model are further obtained. Moreover, the variational model can be solved in a unified way by realizing a primal–dual method for associated saddle-point problems. A number of experimental results on natural hazy images are presented to demonstrate our superior performance, in comparison with some state-of-the-art methods in terms of the subjective and objective visual quality assessments. Compared with the total variation-based models, the proposed model can generate a haze-free image with less staircasing artifacts in the slanted plane and more details in the remote scene of an input image.

NLTV-Gabor-based models for image decomposition and denoising

By using the nonlocal total variation (NLTV) as the regularization and Gabor functions as the fidelity, this paper proposes two novel models for image decomposition and denoising. The presented models closely incorporate the advantages of the NLTV and Gabor wavelets-based methods. These improvements are aimed at overcoming the drawbacks of staircase artifacts and loss of edge details caused by the traditional variational frameworks. Furthermore, on the basis of Chambolle’s projection algorithm, we introduce two extremely efficient numerical methods to solve the resulting optimization problems. Finally, compared with several popular and powerful numerical methods, this article confirms the superiorities of the developed strategies for image decomposition and denoising in terms of visual quality and quantitative assessments.

Total variation with overlapping group sparsity and Lp quasinorm for infrared image deblurring under salt-and-pepper noise

Because of the limitations of the infrared imaging principle and the properties of infrared imaging systems, infrared images have some drawbacks, including a lack of details, indistinct edges, and a large amount of salt-andpepper noise. To improve the sparse characteristics of the image while maintaining the image edges and weakening staircase artifacts, this paper proposes a method that uses the Lp quasinorm instead of the L1 norm and for infrared image deblurring with an overlapping group sparse total variation method. The Lp quasinorm introduces another degree of freedom, better describes image sparsity characteristics, and improves image restoration. Furthermore, we adopt the accelerated alternating direction method of multipliers and fast Fourier transform theory in the proposed method to improve the efficiency and robustness of our algorithm. Experiments show that under different conditions for blur and salt-and-pepper noise, the proposed method leads to excellent performance in terms of objective evaluation and subjective visual results.

Adaptive Surface Mesh Reconstruction for Computed Tomography Images

In medical applications, it is important to reconstruct surface meshes from Computed Tomography (CT) images. Surface mesh reconstruction of biological tissues actually suffers from staircase artifacts, due to anisotropic CT data. To solve this problem, this paper proposes an adaptive surface mesh reconstruction method. We convert the contour pixels of medical image to contour points and exploit the adaptive spherical cover to produce an approximating surface based on the contour points. Due to the reconstruction quality depending on the accurate normal estimation, computing the normal vectors from the negative gradient based on 3D binary volume data instead of classical principal component analysis (PCA), and then covering contour points by adaptive spheres, linking the auxiliary points in the spheres for reconstructing adaptive triangular meshes. The presented method has been used in CT images of the first cervical vertebrae (C1), scapula, as well as the third lumbar vertebrae (L3) and the results are analyzed regarding their smoothness, accuracy and mesh quality. The results show that our method can reconstruct smooth, accurate and high-quality adaptive surface meshes.

Hybrid non-convex second-order total variation with applications to non-blind image deblurring

Total variation is a very popular method for image restoration, yet it produces undesirable staircase artifacts. In this paper, a combined second-order non-convex total variation with overlapping group sparse regularizer for staircase artifact removal is proposed. The non-convex higher-order TV regularizer is introduced to model the observation that the staircase effect must be appropriately smoothen out in the restored image while preserving its edges. However, using the non-convex higher-order TV alone tends to smoothen the image while amplifying speckle artifacts. To deal with this, the overlapping group sparse regularizer is added to balance the effects produced by the non-convex higher-order TV regularizer. An efficient re-weighted $$\ell _1$$ alternating direction method is formulated to solve the corresponding iterative scheme. Comparative analysis with three algorithms, namely overlapping group sparse total variation, total generalized variation and the fast non-smooth non-convex method, with two different blur kernels is carried out. The results from peak signal-to-noise ratio and structural similarity index measure show the effectiveness of our proposed method when compared to the mentioned algorithms.

Seismic fault attribute estimation using a local fault model

Faults in the subsurface can be an avenue of, or a barrier to, hydrocarbon flow and pressure communication. Manual interpretation of discontinuities on 3D seismic amplitude volume is the most common way to define faults within a reservoir. Unfortunately, 3D seismic fault interpretation can be a time-consuming and tedious task. Seismic attributes such as coherence help define faults, but suffer from “staircase” artifacts and nonfault-related stratigraphic discontinuities. We assume that each sample of the seismic data is located at a potential fault plane. The hypothesized fault divides the seismic data centered at the analysis sample into two subwindows. We then compute the coherence for the two subwindows and full analysis window. We repeat the process by rotating the hypothesized fault plane along a set of user-defined discrete fault dip and azimuth. We obtain almost the same coherence values for the subwindows and the full window if the analysis point is not located at a fault plane. The “best” fault plane results in maximum coherence for the subwindows and minimum coherence for the full window if the analysis point is located at a fault plane. To improve the continuity of the fault attributes, we finally smooth the fault probability attribute along the estimated fault plane. We illustrate the effectiveness of our workflow by applying it to a synthetic and two real seismic data. The results indicate that our workflow successfully produces a continuous fault attribute without staircase artifacts and stratigraphic discontinuities.

A generalized hybrid nonconvex variational regularization model for staircase reduction in image restoration

Total variation (TV) regularization model has the excellent performance in noise-removing and edge-preserving. However, it often yields staircase artifacts in the smooth region of the restorations. To attack this problem, we propose a generalized hybrid nonconvex variational regularization model in this paper, which utilizes two nonconvex regularizers to impose the priors to the two different components of the images, respectively. One is nonconvex TV regularizer that measures the piecewise constant component of the input image, and the other is nonconvex Laplacian regularizer that models the piecewise smooth component. New model inherits the advantages of the nonconvex regularization and the first- and second-order hybrid variational regularization, which can well remove the noises while preserving edges and reducing staircase artifacts. To solve this nonconvex minimization problem efficiently, we propose a first-order algorithm based on alternating direction method of multipliers (ADMM) combining with majorization–minimization (MM) scheme. In addition, a sufficient condition for the convergence of the proposed algorithm is provided. In the experiments, we compare the proposed model with several state-of-the-art image denoising models, numerical results illustrate the effectiveness of the proposed model and algorithm for both synthetic and real images in terms of peak signal to noise ratio (PSNR) and structural similarity (SSIM) indexes.

Image Inpainting Based on Fractional-Order Nonlinear Diffusion for Image Reconstruction

Image inpainting, image deblurring, and noise removal are influential concepts in the field of digital image processing. Second-order diffusion-based image restoration models suffer from staircase effects and connectivity principle, while fourth-order models suffer from speckle artifacts. In this article, a robust image inpainting model using fractional-order nonlinear diffusion driven by difference curvature is proposed and fractional-order variational model is utilized to remove the noise and blur. Fractional-order derivatives can deal well with edges and attain good trade-off between edges preservation and elimination of staircase and speckle artifacts of an image. Difference curvature is a feature descriptor which can effectively characterize the intensity variations in the image. In this work, difference curvature-based a bi-weight, adaptive conductance coefficient is proposed to restore the image and fractional-order derivative is implemented by using discrete Fourier transform. Simulation results validate that the proposed model can adequately complete the damaged regions, solve the connectivity principle, and also avoid the staircase and speckle artifacts.

A new adaptive boosting total generalized variation (TGV) technique for image denoising and inpainting

In this paper we present a new adaptive boosting technique for total generalized variation (TGV) based image denoising and inpainting. Instead of the strengthening and substracting steps in existing boosting techniques, the proposed technique is iteratively operated by two steps: the first step is to take average of restored image with observed image, and updated parameter; the second step is to operate the TGV restoration algorithm with the average and dynamic parameter. For each iteration, as the input contains more correct information, the restoration algorithm can produce signals with more details. We have solved our boosting TGV model by primal-dual method, and applied the boosting TGV technique for gray/color image denoising and inpainting. Our algorithms have been discussed about influence of parameters, computational cost and compared with several typical existing methods. Plenty of experimental results show that our method can produce images with more structures and prevent staircase artifacts effectively.

An Efficient Method for Euler’s Elastica Based Image Deconvolution

Variational models involving Euler's elastica energy have a wide range of applications in digital image processing. Recently, fast methods, such as the proximal-augmented Lagrangian method (PALM), have been successfully used to solve nonlinear higher order models for image restoration. In this paper, we extend fast method PALM to Euler's elastica deconvolution models with quadratic and nonquadratic fidelity terms. The proposed variational model can eliminate blur and noise and preserve edges while reducing the blocky and staircase artifacts during image restoration. We present an efficient and effective solution to the proposed minimization problems by a proximal-based numerical scheme. Our numerical experiments demonstrate several results on image deblurring and denoising, which shows a clear improvement of the proposed model over standard variational models such as total variation and Hessian-based model.

Image Denoising via Nonlocal Low Rank Approximation With Local Structure Preserving

The nuclear norm minimization method emerged from a patch-based low-rank model leads to an excellent image denoising performance, where the non-local self-similarity over image patches is exploited. However, natural images are normally with complex and irregular image patches, which cannot be well represented using only a low-rank model, and thus most of them suffer from the over-penalty problem especially for images with lots of local irregular structures (e.g., fine details or sharp edges), and then results in over-smoothing problem after denoising. On the other hand, in order to represent the irregular components, edges defined over pixel level are often exploited. While the total variation (TV) is a well-known prior to remove noises and preserve edges, it might yield undesired staircase artifacts. The total generalized variation (TGV), a generalization of TV, can largely alleviate such staircase artifacts. Consequently, in order to deal with the over-smoothing problem aroused by a low-rank model, we propose a re-weighted TGV regularized nuclear norm minimization model for local structure preserving image denoising. Thanks to the split Bregman method, our proposed model can be effectively solved. A re-weighted strategy is developed to adaptively update the weight parameters of TGV regularization. The encouraging experimental results on noisy images demonstrate the effectiveness of our proposed method.

Nonconvex Low Rank Approximation With Phase Congruency Regularization for Mixed Noise Removal

Mixed noise removal from a natural image is a challenging task since the complex noise distribution usually is inestimable. Many noise removal methods based on the low rank approximation have an excellent image denoising performance and are effective for recovering the images corrupted by Gaussian noise. These methods based on the additive white Gaussian noise(AWGN) model are sensitive to the outliers and non-Gaussian noise, such as the salt-and-pepper impulse noise (SPIN) and random valued impulse noise (RVIN). Such methods for mixed noise removal, however, are less effective in preserving image structures and tend to undesired staircase artifacts. This paper presents a novel Nonconvex Low Rank Model with Phase congruency and overlapping Group sparsity regularization (NLRM-PG) for mixed noise removal. Moreover, an efficient optimization algorithm under the alternating direction method of multipliers and majorization minimization framework is proposed to solve the NLRM-PG model. We demonstrate that the proposed method is effective for preserving local irregular structures and it reduces staircase artifacts with the two types of mixture noise, namely, AWGN+SPIN and AWGN+RVIN. Both qualitative and quantitative experiment results on synthetic noisy images and real noisy images illustrate that the proposed method can remove mixture noise in images more efficiently than the existing methods can do. And the results also outperform those obtained by using the competing state-of-the-art methods, particularly for the removal of high-density impulse noise.

Retraction notice.

Reducing radiation dose while maintaining the quality of the reconstructed images is a major challenge in the computed tomography (CT) community. In light of the non-stationary Gaussian noise distribution, we developed a model that incorporates a noise-level weighted total variation (NWTV) regularization term for denoising the projection data. Contrary to the well-known edge-weighted total variation method, which aims for better edge preserving, the proposed NWTV tries to adapt the regularization with the spatially varying noise levels. Experiments on simulated data as well as the real imaging data suggest that the proposed NWTV regularization could achieve quite competitive results. For sinograms with sharp edges, the NWTV could do a better job at balancing noise reduction and edge preserving, such that noise is removed in a more uniform manner. Another conclusion from our experiments is that the well-recognized stair-casing artifacts of TV regularization play little role in the reconstructed images when the NWTV method is applied to low-dose CT imaging data.

Triangulated Surface Denoising using High Order Regularization with Dynamic Weights

Recovering high quality surfaces from noisy triangulated surfaces is a fundamental important problem in geometry processing. Sharp features including edges and corners can not be well preserved in most existing denoising methods except the recent total variation (TV) and $\ell_0$ regularization methods. However, these two methods have suffered producing staircase artifacts in smooth regions. In this paper, we first introduce a second order regularization method for restoring a surface normal vector field, and then propose a new vertex updating scheme to recover the desired surface according to the restored surface normal field. The proposed model can preserve sharp features and simultaneously suppress the staircase effects in smooth regions which overcomes the drawback of the first order models. In addition, the new vertex updating scheme can prevent ambiguities introduced in existing vertex updating methods. Numerically, the proposed high order model is solved by the augmented Lagrangian method with a dynamic weighting strategy. Intensive numerical experiments on a variety of surfaces demonstrate the superiority of our method by visually and quantitatively.

Intensity Guided Depth Upsampling Using Edge Sparsity and Super-Weighted $L_0$ Gradient Minimization

Although depth cameras acquire depth in dynamic scenes, the captured depth images are often noisy and of low resolution. Depth images have the physical nature of being represented by smooth regions and edges in between them, i.e. depth images have high edge sparsity in the gradient domain. In this paper, we propose intensity guided depth upsampling using edge sparsity and super-weighted L 0 gradient minimization. First, we get mutual structure between intensity and depth using joint mutual structure filtering. Second, we generate an initial depth image using recursive interpolation. Next, we generate weights for L 0 gradient minimization based on gradient and entropy of the intensity image, and upsample the depth image using super-weighted L 0 gradient minimization. In super-weighted L 0 gradient minimization, we combine two main terms: Hybrid data fidelity and weighted L 0 gradient regularization. The hybrid data fidelity term combines both zero-order and first-order data differences to suppress staircase artifacts, while the weighted L 0 gradient regularization term preserves depth structures and removes noise. Finally, we further refine the depth image using adaptive fast weighted median filtering. Experiments on Middlebury and realworld scene datasets verify that the proposed method produces edge preserving depth upsampling results and outperforms state-of-the-arts in terms of both visual quality and quantitative measurements.

Infimal convolution-based regularization for SPECT reconstruction.

Total variation (TV) regularization is efficient in suppressing noise, but is known to suffer from staircase artifacts. The goal of this work was to develop a regularization method using the infimal convolution of the first- and the second-order derivatives to reduce or even prevent staircase artifacts in the reconstructed images, and to investigate if the advantage in noise suppression by this TV-type regularization can be translated into dose reduction. In the present work, we introduce the infimal convolution of the first- and the second-order total variation (ICTV) as the regularization term in penalized maximum likelihood reconstruction. The preconditioned alternating projection algorithm (PAPA), previously developed by the authors of this article, was employed to produce the reconstruction. Using Monte Carlo-simulated data, we evaluate noise properties and lesion detectability in the reconstructed images and compare the results with conventional total variation (TV) and clinical EM-based methods with Gaussian post filter (GPF-EM). We also evaluate the quality of ICTV regularized images obtained for lower photon number data, compared with clinically used photon number, to verify the feasibility of radiation-dose reduction to patients by use of the ICTV reconstruction method. By comparison with GPF-EM reconstructed images, we have found that the ICTV-PAPA method can achieve a lower background variability level while maintaining the same level of contrast. Images reconstructed by the ICTV-PAPA method with 80,000 counts per view exhibit even higher channelized Hotelling observer (CHO) signal-to-noise ratio (SNR), as compared to images reconstructed by the GPF-EM method with 120,000 counts per view. In contrast to the TV-PAPA method, the ICTV-PAPA reconstruction method avoids substantial staircase artifacts, while producing reconstructed images with higher CHO SNR and comparable local spatial resolution. Simulation studies indicate that a 33% dose reduction is feasible by switching to the ICTV-PAPA method, compared with the GPF-EM clinical standard.

Infrared Image Super-Resolution Reconstruction Based on Quaternion Fractional Order Total Variation with Lp Quasinorm

Owing to the limitations of the imaging principle as well as the properties of imaging systems, infrared images often have some drawbacks, including low resolution, a lack of detail, and indistinct edges. Therefore, it is essential to improve infrared image quality. Considering the information of neighbors, a description of sparse edges, and by avoiding staircase artifacts, a new super-resolution reconstruction (SRR) method is proposed for infrared images, which is based on fractional order total variation (FTV) with quaternion total variation and the L p quasinorm. Our proposed method improves the sparsity exploitation of FTV, and efficiently preserves image structures. Furthermore, we adopt the plug-and-play alternating direction method of multipliers (ADMM) and the fast Fourier transform (FFT) theory for the proposed method to improve the efficiency and robustness of our algorithm; in addition, an accelerated step is adopted. Our experimental results show that the proposed method leads to excellent performances in terms of an objective evaluation and the subjective visual effect.