A joint deep learning model to recover information and reduce artifacts in missing-wedge sinograms for electron tomography and beyond

Guanglei Ding,Huolin L Xin,Rui Zhang,Yitong Liu

doi:10.1038/s41598-019-49267-x

Abstract

We present a joint model based on deep learning that is designed to inpaint the missing-wedge sinogram of electron tomography and reduce the residual artifacts in the reconstructed tomograms. Traditional methods, such as weighted back projection (WBP) and simultaneous algebraic reconstruction technique (SART), lack the ability to recover the unacquired project information as a result of the limited tilt range; consequently, the tomograms reconstructed using these methods are distorted and contaminated with the elongation, streaking, and ghost tail artifacts. To tackle this problem, we first design a sinogram filling model based on the use of Residual-in-Residual Dense Blocks in a Generative Adversarial Network (GAN). Then, we use a U-net structured Generative Adversarial Network to reduce the residual artifacts. We build a two-step model to perform information recovery and artifacts removal in their respective suitable domain. Compared with the traditional methods, our method offers superior Peak Signal to Noise Ratio (PSNR) and the Structural Similarity Index (SSIM) to WBP and SART; even with a missing wedge of 45°, our method offers reconstructed images that closely resemble the ground truth with nearly no artifacts. In addition, our model has the advantage of not needing inputs from human operators or setting hyperparameters such as iteration steps and relaxation coefficient used in TV-based methods, which highly relies on human experience and parameter fine turning.

Highlights

The reconstruction of tomography images or tomograms has great significances for physical, materials, medical sciences because it offers capabilities to investigate the internal structures of a non-transparent object without having to dissect or disrupt it
The real data is the sinogram without any missing wedges, denoted as the complete sinogram from here on, and the fake data is the sinograms with the missing wedge inpainted; the discriminative model will distinguish whether a sinogram is a complete sinogram or an inpainted sinogram generated by the generative model
To systematically evaluate the performance of our method compared with other reconstruction approaches, we investigate the Peak Signal to Noise Ratio (PSNR), the Structural Similarity Index (SSIM), and Perceptual Index (PI)[28] of the tomograms reconstructed by our joint model, weighted back projection (WBP), simultaneous algebraic reconstruction technique (SART), and Total Variation Minimization (TVM) from the complete and the inpainted sinograms, respectively

Summary

Method

We present the construction of the two-step joint model that can efficiently recover the missing-wedge of information without introducing visible artifacts in the reconstructed sinogram. For the training of the de-artifacts model, we collect reconstructed tomograms of the missing-wedge sinograms, the ground-truth sinograms, and the inpainted sinograms. We use them as the input of de-artifacts model and the original cross-sectional images as the ground truth to compute the loss. Sinograms with and without the missing wedge were created by Radon transforming the library of cross-sectional images (see Supplementary Materials for detail). The images generated via inpainting model is the core of the training data set. A small number of missing-wedge sinograms are directly transformed by WBP and SART It will generate slightly different artifact patterns to improve the robustness on the de-artifact model. As for discriminator, it has a similar structure with the discriminator in inpainting GAN, as shown in Supplementary Fig. S2

Result

Conclusion