Medical Image Reconstruction Research Articles

Overview of image-to-image translation by use of deep neural networks: denoising, super-resolution, modality conversion, and reconstruction in medical imaging.

Since the advent of deep convolutional neural networks (DNNs), computer vision has seen an extremely rapid progress that has led to huge advances in medical imaging. Every year, many new methods are reported at conferences such as the International Conference on Medical Image Computing and Computer-Assisted Intervention and Machine Learning for Medical Image Reconstruction, or published online at the preprint server arXiv. There is a plethora of surveys on applications of neural networks in medical imaging (see [1] for a relatively recent comprehensive survey). This article does not aim to cover all aspects of the field, but focuses on a particular topic, image-to-image translation. Although the topic may not sound familiar, it turns out that many seemingly irrelevant applications can be understood as instances of image-to-image translation. Such applications include (1) noise reduction, (2) super-resolution, (3) image synthesis, and (4) reconstruction. The same underlying principles and algorithms work for various tasks. Our aim is to introduce some of the key ideas on this topic from a uniform viewpoint. We introduce core ideas and jargon that are specific to image processing by use of DNNs. Having an intuitive grasp of the core ideas of applications of neural networks in medical imagingand a knowledge of technical terms would be of great help to the reader for understanding the existing and future applications. Most of the recent applications which build on image-to-image translation are based on one of two fundamental architectures, called pix2pix and CycleGAN, depending on whether the available training data are paired or unpaired (see Sect.1.3). We provide codes ([2, 3]) which implement these two architectures with various enhancements. Our codes are available online with use of the very permissive MIT license. We provide a hands-on tutorial for training a model for denoising based on our codes (see Sect.6). We hope that this article, together with the codes, will provide both an overview and the details of the key algorithms and that it will serve as a basis for the development of new applications.

A Roadmap for Foundational Research on Artificial Intelligence in Medical Imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop.

Imaging research laboratories are rapidly creating machine learning systems that achieve expert human performance using open-source methods and tools. These artificial intelligence systems are being developed to improve medical image reconstruction, noise reduction, quality assurance, triage, segmentation, computer-aided detection, computer-aided classification, and radiogenomics. In August 2018, a meeting was held in Bethesda, Maryland, at the National Institutes of Health to discuss the current state of the art and knowledge gaps and to develop a roadmap for future research initiatives. Key research priorities include: 1, new image reconstruction methods that efficiently produce images suitable for human interpretation from source data; 2, automated image labeling and annotation methods, including information extraction from the imaging report, electronic phenotyping, and prospective structured image reporting; 3, new machine learning methods for clinical imaging data, such as tailored, pretrained model architectures, and federated machine learning methods; 4, machine learning methods that can explain the advice they provide to human users (so-called explainable artificial intelligence); and 5, validated methods for image de-identification and data sharing to facilitate wide availability of clinical imaging data sets. This research roadmap is intended to identify and prioritize these needs for academic research laboratories, funding agencies, professional societies, and industry.

3d Medical Image Reconstruction of Modified Closed Reduction and Three Hollow Nail for The Treatment of Gardon Iv Femoral Neck Fracture

3d Medical Image Reconstruction of Modified Closed Reduction and Three Hollow Nail for The Treatment of Gardon Iv Femoral Neck Fracture

Improving Sparse Compressed Sensing Medical CT Image Reconstruction

For the incomplete scanning data, the traditional algorithms cannot guarantee that the medical CT reconstruction image meets the diagnostic requirements. In this case, a medical CT image reconstruction method with the limited angle projection is proposed. According to the theory of compressed sensing, medical CT images with a sparse representation can be reconstructed from the incomplete scanning data and provide reliable information for the diagnosis. The sparse representation of CT images is performed by the sparse patch-ordering wavelet-tree transform, and the digital features of sparse coefficients are used as regularization terms to ensure the validity of the solution. Meantime, the weighting term is added into the fidelity item to reduce the influence of noise on the reconstruction results. The extended Lagrange method is used to solve the constrained objective function iteratively so as to realize the reconstruction of low dose medical CT images. Simulation results demonstrate that the reconstructed image can not only satisfy the completeness condition of projection data, but also can reconstruct the high quality image and effectively improve the mean square error and the structural similarity index.

Simulation Study for Feature Identification of Dynamic Medical Image Reconstruction Technique Based on Singular Value Decomposition

Simulation Study for Feature Identification of Dynamic Medical Image Reconstruction Technique Based on Singular Value Decomposition

ASIC modelling of SENSE for parallel MRI

Magnetic Resonance Imaging (MRI) is widely used in medical diagnostics and image reconstruction is a vital part of MRI systems. In Parallel MRI (pMRI), imaging process is accelerated by acquiring less data (undersampled) using multiple receiver coils and offline reconstruction algorithms are applied to reconstruct the fully sampled image. In this research, an Application Specific Integrated Circuits (ASIC) model of SENSE (a pMRI algorithm) is presented which reconstructs the image from the undersampled data right on the data acquisition module of the scanner. The proposed ASIC HDL architecture is compared with SENSE reconstruction model implemented on FPGAs, Multi-core CPU and Graphics Processing Units. The proposed architecture is validated using simulated brain data with 8-channel receiver coils and a human cardiac dataset with 20-channel receiver coils. The quality of the reconstructed images is analyzed using Artifact Power (0.0098), Peak Signal-to-Noise Ratio (53.4) and Structured Similarity Index (0.871) which validate the quality of the reconstructed images using the proposed design. The results show that the proposed ASIC HDL SENSE reconstruction model is ∼8000 times faster as compared to the multi-core CPU reconstruction, ∼700 times faster than the GPU implementation and ∼16 times faster as compared to the FPGA reconstruction model. The proposed architecture is suitable for image reconstruction right on the data acquisition system of the scanner and will open new ways for faster image reconstruction on portable MRI scanners.

CT image reconstruction with SuiteSparseQR factorization package

SuiteSparseQR is a factorization package for sparse matrices oriented to parallelism in multicore architectures. It employs BLAS and LAPACK as well as Intel's Threading Building Blocks to achieve high performance. Through the SPQR method implemented in this package we can use the QR decomposition to reconstruct CT images efficiently. In this paper, we analyze the behavior of the package applied to the reconstruction of medical CT images, studying the quality of the obtained image. To this purpose, we use the image dataset DeepLesion, which provides various CT studies of different lesions in different organs or tissues. We also compare it to our previous iterative reconstruction method called LSQR. This new method is promising since the computations are simplified if we compare it to the iterative options and the reconstructions are high-quality, as the results show.

Descriptive anatomy of the largest known specimen of Protoichthyosaurus prostaxalis (Reptilia: Ichthyosauria) including computed tomography and digital reconstruction of a three-dimensional skull.

Ichthyosaur fossils are abundant in Lower Jurassic sediments with nine genera found in the UK. In this paper, we describe the partial skeleton of a large ichthyosaur from the Lower Jurassic (lower Sinemurian) of Warwickshire, England, which was conserved and rearticulated to form the centrepiece of a new permanent gallery at the Thinktank, Birmingham Science Museum in 2015. The unusual three-dimensional preservation of the specimen permitted computed tomography (CT) scanning of individual braincase elements as well as the entire reassembled skull. This represents one of the first times that medical imaging and three-dimensional reconstruction methods have been applied to a large skull of a marine reptile. Data from these scans provide new anatomical information, such as the presence of branching vascular canals within the premaxilla and dentary, and an undescribed dorsal (quadrate) wing of the pterygoid hidden within matrix. Scanning also revealed areas of the skull that had been modelled in wood, clay and other materials after the specimen’s initial discovery, highlighting the utility of applying advanced imaging techniques to historical specimens. Additionally, the CT data served as the basis for a new three-dimensional reconstruction of the skull, in which minor damage was repaired and the preserved bones digitally rearticulated. Thus, for the first time a digital reconstruction of the skull and mandible of a large marine reptile skull is available. Museum records show the specimen was originally identified as an example of Ichthyosaurus communis but we identify this specimen as Protoichthyosaurus prostaxalis. The specimen features a skull nearly twice as long as any previously described specimen of P. prostaxalis, representing an individual with an estimated total body length between 3.2 and 4 m.

2D and 3D Medical Image Analysis by Discrete Orthogonal Moments

The 2D and 3D medical imaging play a very important role in medical diagnosis to detect and isolate various pathologies. The treatment, exploitation and the interpretation of the obtained images are valuable tools to help establish a medical diagnosis. In this context, we will present in this paper an effective method for the analysis of different types of 2D and 3D medical image based on the discrete orthogonal moments of Tchebichef, Krawtchouk, Hahn, Charlier and Meixner. By this method, the 2D and 3D medical images are described by moment descriptors. The use of these moment descriptors has allowed us to significantly reduce the storage space of 2D and 3D medical images and speed up the image transfer process. The results of simulations on different medical images by the five types of moments have shown the ability of these types to improve and speed up the medical image reconstruction.

DeepOrganNet: On-the-Fly Reconstruction and Visualization of 3D / 4D Lung Models from Single-View Projections by Deep Deformation Network.

This paper introduces a deep neural network based method, i.e., DeepOrganNet, to generate and visualize fully high-fidelity 3D / 4D organ geometric models from single-view medical images with complicated background in real time. Traditional 3D / 4D medical image reconstruction requires near hundreds of projections, which cost insufferable computational time and deliver undesirable high imaging / radiation dose to human subjects. Moreover, it always needs further notorious processes to segment or extract the accurate 3D organ models subsequently. The computational time and imaging dose can be reduced by decreasing the number of projections, but the reconstructed image quality is degraded accordingly. To our knowledge, there is no method directly and explicitly reconstructing multiple 3D organ meshes from a single 2D medical grayscale image on the fly. Given single-view 2D medical images, e.g., 3D / 4D-CT projections or X-ray images, our end-to-end DeepOrganNet framework can efficiently and effectively reconstruct 3D / 4D lung models with a variety of geometric shapes by learning the smooth deformation fields from multiple templates based on a trivariate tensor-product deformation technique, leveraging an informative latent descriptor extracted from input 2D images. The proposed method can guarantee to generate high-quality and high-fidelity manifold meshes for 3D / 4D lung models; while, all current deep learning based approaches on the shape reconstruction from a single image cannot. The major contributions of this work are to accurately reconstruct the 3D organ shapes from 2D single-view projection, significantly improve the procedure time to allow on-the-fly visualization, and dramatically reduce the imaging dose for human subjects. Experimental results are evaluated and compared with the traditional reconstruction method and the state-of-the-art in deep learning, by using extensive 3D and 4D examples, including both synthetic phantom and real patient datasets. The efficiency of the proposed method shows that it only needs several milliseconds to generate organ meshes with 10K vertices, which has great potential to be used in real-time image guided radiation therapy (IGRT).

FPGA as a Hardware Accelerator for Computation Intensive Maximum Likelihood Expectation Maximization Medical Image Reconstruction Algorithm

The major requirements of a good tomographic reconstruction algorithm are a reduction in radiation dosage, accurate reconstruction, detail enhancement, and rapid reconstruction time. Some of these factors are covered by many algorithms but are not collectively addressed in one. While the Maximum Likelihood Expectation Maximization (MLEM) algorithm fares well on many of these factors, it is difficult to apply this algorithm in real-time due to its long execution time. Our predetermined goal is to reduce the execution time to a large extent so that the MLEM’s advantages can be leveraged by using hardware accelerators such as Field Programmable Gate Arrays (FPGA). The FPGAs are becoming especially popular as hardware accelerators and are well known for their programmability, configurability, and massive parallelism through a large number of Configurable Logic Blocks (CLBs). Although FPGAs are extremely versatile, they require complex languages like Verilog or VHDL to program them, incorporating changes in the design level at a later stage in FPGAs demands increased effort. Here, in this paper, for the first time, we present a parallel structure for hardware acceleration of the MLEM on the mammoth Virtex 7 $VC$ 709 FPGA. Using available tools, we also present a programming flow to design the algorithmic acceleration hardware architecture. The proposed flow does not require prior knowledge of the traditionally cumbersome Hardware Description Languages (HDLs) and this makes post design changes very easy to incorporate and validate. The parallel architecture is implemented on an FPGA operating at 220 MHz and we have achieved a $288\times $ performance compared to an optimized software execution on an Intel Xeon workstation with 12-cores 3.1 GHz 32 GB RAM and 12 MB Cache architecture.

Applications of generative adversarial networks in medical image processing

In recent years, researchers have introduced various methods in many domains into medical image processing so that its effectiveness and efficiency can be improved to some extent. The applications of generative adversarial networks (GAN) in medical image processing are evolving very fast. In this paper, the state of the art in this area has been reviewed. Firstly, the basic concepts of the GAN were introduced. And then, from the perspectives of the medical image denoising, detection, segmentation, synthesis, reconstruction and classification, the applications of the GAN were summarized. Finally, prospects for further research in this area were presented.

PET Image Reconstruction Using Deep Image Prior.

Recently, deep neural networks have been widely and successfully applied in computer vision tasks and have attracted growing interest in medical imaging. One barrier for the application of deep neural networks to medical imaging is the need for large amounts of prior training pairs, which is not always feasible in clinical practice. This is especially true for medical image reconstruction problems, where raw data are needed. Inspired by the deep image prior framework, in this paper, we proposed a personalized network training method where no prior training pairs are needed, but only the patient's own prior information. The network is updated during the iterative reconstruction process using the patient-specific prior information and measured data. We formulated the maximum-likelihood estimation as a constrained optimization problem and solved it using the alternating direction method of multipliers algorithm. Magnetic resonance imaging guided positron emission tomography reconstruction was employed as an example to demonstrate the effectiveness of the proposed framework. Quantification results based on simulation and real data show that the proposed reconstruction framework can outperform Gaussian post-smoothing and anatomically guided reconstructions using the kernel method or the neural-network penalty.

Maximum accurate medical image demosaicing using WRGB based Newton Gregory interpolation method

Color image demosaicing is an important image processing technique for obtaining high quality image. This operation is complicated for digital camera, since it converts a Bayer mosaic image to a full color image, which estimates the output image quality of the camera. Nowadays, many techniques are developed to reduce the size and weight of electronic gadgets. In this way, to reduce the size of camera Color Filter Array is used which has only one sensor for three primary colors (Red, Green and Blue). So many demosaicing techniques are developed which reconstruct the full color image from the incomplete color samples in CFA. But till today there is no demosaicing algorithm for medical image re-construction is available in existing literature. In this paper a hybrid medical image demosaicing technique is developed for obtaining better image quality, higher sensitivity as well as higher resolution for high quality medical diagnosis. To improve the medical image quality, Newton Gregory Interpolation is used. This technique mainly focused on Green channel and produce interpolated Red, Green and Blue channels and for enhancing the sensitivity and resolution, updating White channel texture component with RGB channels. In post processing Cross Multilateral Filter is used which decomposes the smoothness component from the texture component. The proposed newton Gregory interpolation based medical image Demosaicing achieves 42.31 db for brain database of NIH and 39.52 db for Iris database of UPOL.

Alternative methods for virtual heart transplant-Size matching for pediatric heart transplantation with and without donor medical images available.

Listed pediatric heart transplant patients have the highest solid-organ waitlist mortality rate. The donor-recipient body weight (DRBW) ratio is the clinical standard for allograft size matching but may unnecessarily limit a patient's donor pool. To overcome DRBW ratio limitations, two methods of performing virtual heart transplant fit assessments were developed that account for patient-specific nuances. Method 1 uses an allograft total cardiac volume (TCV) prediction model informed by patient data wherein a matched allograft 3-D reconstruction is selected from a virtual library for assessment. Method 2 uses donor images for a direct virtual transplant assessment. Assessments were performed in medical image reconstruction software. The allograft model was developed using allometric/isometric scaling assumptions and cross-validation. The final predictive model included gender, height, and weight. The 25th-, 50th-, and 75th-percentiles for TCV percentage errors were -13% (over-prediction), -1%, and 8% (under-prediction), respectively. Two examples illustrating the potential of virtual assessments are presented. Transplant centers can apply these methods to perform their virtual assessments using existing technology. These techniques have potential to improve organ allocation. With additional experience and refinement, virtual transplants may become standard of care for determining suitability of donor organ size for an identified recipient.

Hierarchical Temporal Memory Network for Medical Image Processing

Medical image segmentation is a basic step in medical image analysis, especially for medical image sequences such as CT sequences. Automated segmentation of different objects in the medical image sequences is of great significance to the 3D reconstruction of medical images. A novel image recognition method which can be implemented in automated medical image segmentation is introduced. In contrast with other algorithm, HTM (hierarchical temporal memory) is a network using a spatio-temporary hierarchy that works as our neocortex. The algorithm refereed in this paper consists of three main steps. Firstly, a four level hierarchical structure is established. Secondly, create frames by animating gray images to train the HTM network. During the learning phase, the nodes in HTM network build its representations spatial pooler and temporal pooler for inputs. Thirdly, test with dataset to get the inference result for classification. The results show that the proposed method can recognize the “middle slice” for different given objects when process the medical image sequences.

OpenCLIPER: An OpenCL-Based C++ Framework for Overhead-Reduced Medical Image Processing and Reconstruction on Heterogeneous Devices.

Medical image processing is often limited by the computational cost of the involved algorithms. Whereas dedicated computing devices (GPUs in particular) exist and do provide significant efficiency boosts, they have an extra cost of use in terms of housekeeping tasks (device selection and initialization, data streaming, synchronization with the CPU, and others), which may hinder developers from using them. This paper describes an OpenCL-based framework that is capable of handling dedicated computing devices seamlessly and that allows the developer to concentrate on image processing tasks. The framework handles automatically device discovery and initialization, data transfers to and from the device and the file system and kernel loading and compiling. Data structures need to be defined only once independently of the computing device; code is unique, consequently, for every device, including the host CPU. Pinned memory/buffer mapping is used to achieve maximum performance in data transfers. Code fragments included in the paper show how the computing device is almost immediately and effortlessly available to the users algorithms, so they can focus on productive work. Code required for device selection and initialization, data loading and streaming and kernel compilation is minimal and systematic. Algorithms can be thought of as mathematical operators (called processes), with input, output and parameters, and they may be chained one after another easily and efficiently. Also for efficiency, processes can have their initialization work split from their core workload, so process chains and loops do not incur in performance penalties. Algorithm code is independent of the device type targeted.

VIDOSAT: High-Dimensional Sparsifying Transform Learning for Online Video Denoising.

Techniques exploiting the sparsity of images in a transform domain are effective for various applications in image and video processing. In particular, transform learning methods involve cheap computations and have been demonstrated to perform well in applications, such as image denoising and medical image reconstruction. Recently, we proposed methods for online learning of sparsifying transforms from streaming signals, which enjoy good convergence guarantees and involve lower computational costs than online synthesis dictionary learning. In this paper, we apply online transform learning to video denoising. We present a novel framework for online video denoising based on high-dimensional sparsifying transform learning for spatio-temporal patches. The patches are constructed either from corresponding 2D patches in successive frames or using an online block matching technique. The proposed online video denoising requires little memory and offers efficient processing. Numerical experiments evaluate the performance of the proposed video denoising algorithms on multiple video data sets. The proposed methods outperform several related and recent techniques, including denoising with 3D DCT, prior schemes based on dictionary learning, non-local means, background separation, and deep learning, as well as the popular VBM3D and VBM4D.

Learning deconvolutional deep neural network for high resolution medical image reconstruction

Super resolution reconstruction can be used to recover a high resolution image from a low resolution image and is particularly beneficial for clinically significant medical images in diagnosis, treatment, and research applications. However, super resolution is a challenging inverse problem due to its ill-posed nature. In this paper, inspired by recent developments in deep learning, a super resolution algorithm (SR-DCNN) is proposed for medical images that is based on a neural network and employs a deconvolution operation. The purpose of the deconvolution is to effectively establish an end-to-end mapping between the low and high resolution images. First, training data consisting of 1500 medical images of the lung, brain, heart, and spine, was collected, down-sampled, and input into the neural network. Then, patch-based image features were extracted using a set of filters and the parametric rectified linear unit (PReLU) was subsequently applied as the activation function. Finally, these extracted image features were used to reconstruct high resolution images by minimizing the loss between the predicted output image and the original high resolution image. Various network structures and hyper parameter settings were explored to achieve a good trade-off between performance and computational efficiency, based on which a four-layer network was found to achieve the best result in terms of the peak signal-to-noise ratio (PSNR), structural similarity measure (SSIM), information entropy (IE), and execution speed. The network was then validated on test data, and it was demonstrated that the proposed SR-DCNN algorithm quantitatively and qualitatively outperformed the current state-of-the-art methods.

Multi-Modality Imaging: A Software Fusion and Image-Guided Therapy Perspective

With the introduction of computers in medical imaging, which were popularized with the presentation of Hounsfield’s ground-breaking work in 1971, numerical image reconstruction and analysis of medical images became a vital part of medical imaging research. While mathematical aspects of reconstruction dominated research in the beginning, a growing body of literature attests to the progress made over the past 30 years in image fusion. This article describes the historical development of non-deformable software-based image co-registration and it’s role in the context of hybrid imaging and provides an outlook on future developments.