Image Reconstruction Approach Research Articles

An affinity propagated clustering aided computerized Inherent Seeded Region Growing and Deep learned Marching Cubes Algorithm (ISRG‐DMCA) based three dimensional image reconstruction approach

AbstractIn this manuscript, the novel three dimensional (3D) image reconstruction approach based on affinity propagated clustering aided computerized Inherent Seeded Region Growing and Deep learned Marching Cubes Algorithm (ISRG‐DMCA) is proposed. The major purpose of this manuscript is to divide the brain tumor based on Shapelets. Here, the information about the shape/depth that can be obtained in every two dimensional (2D) image on the image stack is handled to acquire a 3D reconstruction, which provides high accurate 3D view of tumor Region of Interest (ROI). Then, the 3D model is rendered with the help of the proposed Deep learned Marching Cubes Algorithm (Deep MCA) at 3D reconstruction technique. The performance of the proposed method is executed in MATLAB. The simulation results show that the proposed ISRG‐DMCA algorithm attains a higher detection rate 14.117%, 5.435%, higher accuracy rate 9.556%, 26.41% and lower execution time 66.667%, 75%, compared with the existing methods, like Improved Marching Cubes Algorithm (IMCA), Improved CNN‐CRF method, respectively. In the proposed ISRG‐DMCA method, the volume of the tumor has a length of 2.56 mm. Finally, the simulation outcomes demonstrate that the proposed ISRG‐DMCA method can be able to find the optimal solutions efficiently and accurately.

Three-dimensional modelling and terahertz imaging of malignant cells with convolutional time-reversed FDTD method

ABSTRACT A novel image reconstruction approach to terahertz pulsed imaging for detecting malignant and fibrous cells within healthy tissue is presented. The non-ionizing effect of the terahertz radiation on healthy tissue makes it a future diagnosis method in medical imaging. The implemented method is based on the convolutional time-reversed FDTD algorithm in the terahertz range. It is computationally efficient and accurately reconstructs images of malignancies from reflected terahertz signals. Besides, the technique is capable of differentiating malignant, fibrous, and fatty cells through the variations of their electrical properties at the terahertz frequency range and well reconstructing their images in 3D. The images created in this method are in high resolution, even for cell-sized malignancies. Furthermore, the proposed technique is compared with conventional terahertz imaging methods and found to have more sophisticated outcomes.

Applying self-supervised learning for semantic cloud segmentation of all-sky images

Abstract. Semantic segmentation of ground-based all-sky images (ASIs) can provide high-resolution cloud coverage information of distinct cloud types, applicable for meteorology-, climatology- and solar-energy-related applications. Since the shape and appearance of clouds is variable, and there is high similarity between cloud types, a clear classification is difficult. Therefore, most state-of-the-art methods focus on the distinction between cloudy and cloud-free pixels without taking into account the cloud type. On the other hand, cloud classification is typically determined separately at the image level, neglecting the cloud's position and only considering the prevailing cloud type. Deep neural networks have proven to be very effective and robust for segmentation tasks; however they require large training datasets to learn complex visual features. In this work, we present a self-supervised learning approach to exploit many more data than in purely supervised training and thus increase the model's performance. In the first step, we use about 300 000 ASIs in two different pretext tasks for pretraining. One of them pursues an image reconstruction approach. The other one is based on the DeepCluster model, an iterative procedure of clustering and classifying the neural network output. In the second step, our model is fine-tuned on a small labeled dataset of 770 ASIs, of which 616 are used for training and 154 for validation. For each of them, a ground truth mask was created that classifies each pixel into clear sky or a low-layer, mid-layer or high-layer cloud. To analyze the effectiveness of self-supervised pretraining, we compare our approach to randomly initialized and pretrained ImageNet weights using the same training and validation sets. Achieving 85.8 % pixel accuracy on average, our best self-supervised model outperforms the conventional approaches of random (78.3 %) and pretrained ImageNet initialization (82.1 %). The benefits become even more evident when regarding precision, recall and intersection over union (IoU) of the respective cloud classes, where the improvement is between 5 and 20 percentage points. Furthermore, we compare the performance of our best model with regards to binary segmentation with a clear-sky library (CSL) from the literature. Our model outperforms the CSL by over 7 percentage points, reaching a pixel accuracy of 95 %.

Identification of Orthotropic Elastic Properties of Wood by a Synthetic Image Approach Based on Digital Image Correlation.

This work aims to determine the orthotropic linear elastic constitutive parameters of Pinus pinaster Ait. wood from a single uniaxial compressive experimental test, under quasi-static loading conditions, based on two different specimen configurations: (a) on-axis rectangular specimens oriented on the radial-tangential plane, (b) off-axis specimens with a grain angle of about 60° (radial-tangential plane). Using digital image correlation (DIC), full-field displacement and strain maps are obtained and used to identify the four orthotropic elastic parameters using the finite element model updating (FEMU) technique. Based on the FE data, a synthetic image reconstruction approach is proposed by coupling the inverse identification method with synthetically deformed images, which are then processed by DIC and compared with the experimental results. The proposed methodology is first validated by employing a DIC-levelled FEA reference in the identification procedure. The impact of the DIC setting parameters on the identification results is systematically investigated. This influence appears to be stronger when the parameter is less sensitive to the experimental setup used. When using on-axis specimen configuration, three orthotropic parameters of Pinus pinaster (, and ) are correctly identified, while the shear modulus () is robustly identified when using off-axis specimen configuration.

Multimodal Image Reconstruction of Electrical Impedance Tomography Using Kernel Method

The inverse problem of electrical impedance tomography (EIT) is nonlinear and severely ill-posed, resulting in low image quality, which explicitly involves the aspects of structure preservation and conductivity contrast differentiation. This article reports a kernel method-based multimodal EIT image reconstruction approach to tackle this challenge. The kernel method performs image-level segmentation-free information fusion and incorporates the structural information of an auxiliary high-resolution image into the EIT inversion process through the kernel matrix, leading to an unconstrained least square problem. We describe this approach in a general way so that the high-resolution images from various imaging modalities can be adopted as the auxiliary image if they contain sufficient structural information. Compared with some state-of-the-art algorithms, the proposed kernel method generates superior EIT images on challenging simulation and experimental phantoms. It also presents the advantage of suppressing the interference of imaging-irrelevant objects in the auxiliary image. Simulation and experiment results suggest the kernel method has great potential to be applied to more complex tissue engineering applications.

Reconstructing Clear Image for High-Speed Motion Scene With a Retina-Inspired Spike Camera

Conventional digital cameras typically accumulate all the photons within an exposure period to form a snapshot image. It requires the scene to be quite still during the imaging time, otherwise it would result in blurry image for the moving objects. Recently, a retina-inspired spike camera has been proposed and shown great potential for recording high-speed motion scenes. Instead of capturing the visual scene by a single snapshot, the spike camera records the dynamic light intensity variation continuously. Each pixel on spike camera sensor accumulates the incoming photons independently and persistently, which fires a spike and restarts the photon accumulation immediately once the dispatch threshold is reached, producing a continuous stream of spikes recorded at very high temporal resolution. To recover the dynamic scene from captured spike stream, this paper presents an image reconstruction approach for spike camera. In order to generate high-quality reconstruction, we investigate the temporal correlation along motion trajectories and exploit it via adaptive temporal filtering. In particular, we present a hierarchical motion-aligned temporal filtering scheme, combining short-term filtering with long-term filtering to take advantage of long-term temporal correlation with low model complexity. Experimental results demonstrate that the proposed scheme outperforms the existing schemes significantly, producing much better objective and subjective qualities for spike camera image reconstruction.

Evaluation of Ultra-low-dose (ULD) Lung Computed Tomography (CT) Using Deep-learning: A Phantom Study

As an electromagnetic wave, X-rays are used to acquire diagnostic CT images. The aim of this phantom study was to evaluate the image quality of ultra-low-dose (ULD) lung computed tomography (CT) achieved using a deep-learning based image reconstruction method. The chest phantom was scanned with a tube voltage of 100 kV for various CT dose index (CTDIvol) conditions: 0.4 mGy for ultra-low-dose (ULD), 0.6 mGy for low-dose (LD), 2.7 mGy for standard (SD), and 7.1 mGy for large size (LS). The signal-to-noise ratio (SNR) and noise values in reconstructions produced via filtered back projection (FBP), iterative reconstruction (IR), and deep convolutional neural network (DCNN) were computed for comparison. The quantitative results of both the SNR and noise indicate that the adoption of the DCNN makes the image reconstruction in the ULD setting more stable and robust, achieving a higher image quality when compared with the FBP algorithm in the SD condition. Compared with the conventional FBP and IR, the proposed deep learning-based image reconstruction approach can improve the ULD CT image quality while significantly reducing the patient dose.

Regularized Multiframe Super-Resolution Image Reconstruction Using Linear and Nonlinear Filters

The primary goal of the multiframe super-resolution image reconstruction is to produce an image with a higher resolution by integrating information extracted from a set of corresponding images with low resolution, which is used in various fields. However, super-resolution image reconstruction approaches are typically affected by annoying restorative artifacts, including blurring, noise, and staircasing effect. Accordingly, it is always difficult to balance between smoothness and edge preservation. In this paper, we intend to enhance the efficiency of multiframe super-resolution image reconstruction in order to optimize both analysis and human interpretation processes by improving the pictorial information and enhancing the automatic machine perception. As a result, we propose new approaches that firstly rely on estimating the initial high-resolution image through preprocessing of the reference low-resolution image based on median, mean, Lucy-Richardson, and Wiener filters. This preprocessing stage is used to overcome the degradation present in the reference low-resolution image, which is a suitable kernel for producing the initial high-resolution image to be used in the reconstruction phase of the final image. Then, L2 norm is employed for the data-fidelity term to minimize the residual among the predicted high-resolution image and the observed low-resolution images. Finally, bilateral total variation prior model is utilized to restrict the minimization function to a stable state of the generated HR image. The experimental results of the synthetic data indicate that the proposed approaches have enhanced efficiency visually and quantitatively compared to other existing approaches.

Unsupervised CT Metal Artifact Learning Using Attention-Guided β-CycleGAN.

Metal artifact reduction (MAR) is one of the most important research topics in computed tomography (CT). With the advance of deep learning approaches for image reconstruction, various deep learning methods have been suggested for metal artifact reduction, among which supervised learning methods are most popular. However, matched metal-artifact-free and metal artifact corrupted image pairs are difficult to obtain in real CT acquisition. Recently, a promising unsupervised learning for MAR was proposed using feature disentanglement, but the resulting network architecture is so complicated that it is difficult to handle large size clinical images. To address this, here we propose a simple and effective unsupervised learning method for MAR. The proposed method is based on a novel β -cycleGAN architecture derived from the optimal transport theory for appropriate feature space disentanglement. Moreover, by adding the convolutional block attention module (CBAM) layers in the generator, we show that the metal artifacts can be more focused so that it can be effectively removed. Experimental results confirm that we can achieve improved metal artifact reduction that preserves the detailed texture of the original image.

Image reconstruction approach for a high space-bandwidth product structured illumination microscopy system

Conventional structured illumination microscopy (SIM) utilizes a sinusoidal excitation pattern of frequency within the detection passband and provides a maximum of twofold resolution enhancement over the diffraction limit. A transmission approach proposed in an earlier publication [J. Phys. D 53, 044006 (2019)JPAPBE0022-372710.1088/1361-6463/ab4e68] to further improve the lateral resolution requires sequential higher frequency illumination patterns. However, the existing reconstruction algorithms fail to deliver appropriate reconstruction when the excitation frequency lies far from the passband boundary. Here, we present a correlation-based SIM reconstruction approach for sequential high-frequency illumination patterns even if the pattern frequency lies far from the passband limit. The scheme can be suitably implemented in a variety of custom-built systems where illumination frequency lies beyond the passband support (e.g., non-linear SIM and plasmonic SIM).

ADMM approach for efficient iterative tomographic deconvolution reconstruction of 3D quantitative phase images.

Tomographic deconvolution phase microscopy (TDPM) is a promising approach for 3D quantitative imaging of phase objects such as biological cells and optical fibers. In the present work, the alternating direction method of multipliers (ADMM) is applied to TDPM to shorten its image acquisition and processing times while simultaneously improving its accuracy. ADMM-TDPM is used to optimize the image fidelity by minimizing Gaussian noise and by using total variation regularization with the constraints of nonnegativity and known zeros. ADMM-TDPM can reconstruct phase objects that are shift variant in three spatial dimensions. ADMM-TDPM achieves speedups of 5x in image acquisition time and greater than 10x in image processing time with accompanying higher accuracy compared to TDPM.

Multilayer residual sparsifying transform (MARS) model for low-dose CT image reconstruction.

Signal models based on sparse representations have received considerable attention in recent years. On the other hand, deep models consisting of a cascade of functional layers, commonly known as deep neural networks, have been highly successful for the task of object classification and have been recently introduced to image reconstruction. In this work, we develop a new image reconstruction approach based on a novel multilayer model learned in an unsupervised manner by combining both sparse representations and deep models. The proposed framework extends the classical sparsifying transform model for images to a Multilayer residual sparsifying transform (MARS) model, wherein the transform domain data are jointly sparsified over layers. We investigate the application of MARS models learned from limited regular-dose images for low-dose CT reconstruction using penalized weighted least squares (PWLS) optimization. We propose new formulations for multilayer transform learning and image reconstruction. We derive an efficient block coordinate descent algorithm to learn the transforms across layers, in an unsupervised manner from limited regular-dose images. The learned model is then incorporated into the low-dose image reconstruction phase. Low-dose CT experimental results with both the XCAT phantom and Mayo Clinic data show that the MARS model outperforms conventional methods such as filtered back-projection and PWLS methods based on the edge-preserving (EP) regularizer in terms of two numerical metrics (RMSE and SSIM) and noise suppression. Compared with the single-layer learned transform (ST) model, the MARS model performs better in maintaining some subtle details. This work presents a novel data-driven regularization framework for CT image reconstruction that exploits learned multilayer or cascaded residual sparsifying transforms. The image model is learned in an unsupervised manner from limited images. Our experimental results demonstrate the promising performance of the proposed multilayer scheme over single-layer learned sparsifying transforms. Learned MARS models also offer better image quality than typical nonadaptive PWLS methods.

Coronary Magnetic Resonance Angiography in Chronic Coronary Syndromes.

Cardiovascular disease is the leading cause of mortality worldwide, with atherosclerotic coronary artery disease (CAD) accounting for the majority of cases. X-ray coronary angiography and computed tomography coronary angiography (CCTA) are the imaging modalities of choice for the assessment of CAD. However, the use of ionising radiation and iodinated contrast agents remain drawbacks. There is therefore a clinical need for an alternative modality for the early identification and longitudinal monitoring of CAD without these associated drawbacks. Coronary magnetic resonance angiography (CMRA) could be a potential alternative for the detection and monitoring of coronary arterial stenosis, without exposing patients to ionising radiation or iodinated contrast agents. Further advantages include its versatility, excellent soft tissue characterisation and suitability for repeat imaging. Despite the early promise of CMRA, widespread clinical utilisation remains limited due to long and unpredictable scan times, onerous scan planning, lower spatial resolution, as well as motion related image quality degradation. The past decade has brought about a resurgence in CMRA technology, with significant leaps in image acceleration, respiratory and cardiac motion estimation and advanced motion corrected or motion-resolved image reconstruction. With the advent of artificial intelligence, great advances are also seen in deep learning-based motion estimation, undersampled and super-resolution reconstruction promising further improvements of CMRA. This has enabled high spatial resolution (1 mm isotropic), 3D whole heart CMRA in a clinically feasible and reliable acquisition time of under 10 min. Furthermore, latest super-resolution image reconstruction approaches which are currently under evaluation promise acquisitions as short as 1 min. In this review, we will explore the recent technological advances that are designed to bring CMRA closer to clinical reality.

Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data

Optical coherence tomography (OCT) is a widely used non-invasive biomedical imaging modality that can rapidly provide volumetric images of samples. Here, we present a deep learning-based image reconstruction framework that can generate swept-source OCT (SS-OCT) images using undersampled spectral data, without any spatial aliasing artifacts. This neural network-based image reconstruction does not require any hardware changes to the optical setup and can be easily integrated with existing swept-source or spectral-domain OCT systems to reduce the amount of raw spectral data to be acquired. To show the efficacy of this framework, we trained and blindly tested a deep neural network using mouse embryo samples imaged by an SS-OCT system. Using 2-fold undersampled spectral data (i.e., 640 spectral points per A-line), the trained neural network can blindly reconstruct 512 A-lines in 0.59 ms using multiple graphics-processing units (GPUs), removing spatial aliasing artifacts due to spectral undersampling, also presenting a very good match to the images of the same samples, reconstructed using the full spectral OCT data (i.e., 1280 spectral points per A-line). We also successfully demonstrate that this framework can be further extended to process 3× undersampled spectral data per A-line, with some performance degradation in the reconstructed image quality compared to 2× spectral undersampling. Furthermore, an A-line-optimized undersampling method is presented by jointly optimizing the spectral sampling locations and the corresponding image reconstruction network, which improved the overall imaging performance using less spectral data points per A-line compared to 2× or 3× spectral undersampling results. This deep learning-enabled image reconstruction approach can be broadly used in various forms of spectral-domain OCT systems, helping to increase their imaging speed without sacrificing image resolution and signal-to-noise ratio.

Frontotemporal activation differs between perception of simulated cochlear implant speech and speech in background noise: An image-based fNIRS study

In this study we used functional near-infrared spectroscopy (fNIRS) to investigate neural responses in normal-hearing adults as a function of speech recognition accuracy, intelligibility of the speech stimulus, and the manner in which speech is distorted. Participants listened to sentences and reported aloud what they heard. Speech quality was distorted artificially by vocoding (simulated cochlear implant speech) or naturally by adding background noise. Each type of distortion included high and low-intelligibility conditions. Sentences in quiet were used as baseline comparison. fNIRS data were analyzed using a newly developed image reconstruction approach. First, elevated cortical responses in the middle temporal gyrus (MTG) and middle frontal gyrus (MFG) were associated with speech recognition during the low-intelligibility conditions. Second, activation in the MTG was associated with recognition of vocoded speech with low intelligibility, whereas MFG activity was largely driven by recognition of speech in background noise, suggesting that the cortical response varies as a function of distortion type. Lastly, an accuracy effect in the MFG demonstrated significantly higher activation during correct perception relative to incorrect perception of speech. These results suggest that normal-hearing adults (i.e., untrained listeners of vocoded stimuli) do not exploit the same attentional mechanisms of the frontal cortex used to resolve naturally degraded speech and may instead rely on segmental and phonetic analyses in the temporal lobe to discriminate vocoded speech.

Image reconstruction reveals the impact of aging on face perception.

Extensive work has demonstrated an age-related decline in face recognition, but the nature and the extent of aging-related alterations in face representations remain unclear. Here, we address these issues using an image reconstruction approach to reveal the content of visual representations. Healthy young and older adults provided similarity judgments for pairs of face images. Facial shape and surface features were subsequently derived and combined into image reconstructions of facial appearance. Both objective and experimental evaluations revealed that reconstructions were successful for every participant. Critically, shape and surface properties, such as eye shape and skin tone, were less accurately represented in older than young individuals. Yet, age-related differences in face representations, though significant, were less pronounced than those due to individual variability. Our results provide novel insights into age-related changes in visual perception and demonstrate the utility of image reconstruction to uncovering internal representations across a variety of populations. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

TENIS — ThErmal Neutron Imaging System for use in BNCT

A thermal neutron flux measurement tool with two perpendicular sets of plastic scintillator arrays was designed and simulated (Ghal-Eh and Green, 2016) with the MCNPX code (Version 2.6.0, with ENDF/B-VII cross section library (ENDF, 2011)). The proposed system aimed to provide a thermal neutron map based on the detection of 2.22 MeV gamma-rays resulting from 1H(nth, γ)2D reactions. In the present work, using Monte Carlo code FLUKA and its scintillation light transport capability, several important upgrades were carried out to include the light transport modeling in the response of plastic scintillators, analyze the cross-talk phenomenon, optimize the system geometry, and also provide a new approach in thermal neutron image reconstruction. The results showed that the last two cases played a significant role in improving the longitudinal profile of thermal neutron flux.

Event Encryption for Neuromorphic Vision Sensors: Framework, Algorithm, and Evaluation.

Nowadays, our lives have benefited from various vision-based applications, such as video surveillance, human identification and aided driving. Unauthorized access to the vision-related data greatly threatens users’ privacy, and many encryption schemes have been proposed to secure images and videos in those conventional scenarios. Neuromorphic vision sensor (NVS) is a brand new kind of bio-inspired sensor that can generate a stream of impulse-like events rather than synchronized image frames, which reduces the sensor’s latency and broadens the applications in surveillance and identification. However, the privacy issue related to NVS remains a significant challenge. For example, some image reconstruction and human identification approaches may expose privacy-related information from NVS events. This work is the first to investigate the privacy of NVS. We firstly analyze the possible security attacks to NVS, including grayscale image reconstruction and privacy-related classification. We then propose a dedicated encryption framework for NVS, which incorporates a 2D chaotic mapping to scramble the positions of events and flip their polarities. In addition, an updating score has been designed for controlling the frequency of execution, which supports efficient encryption on different platforms. Finally, extensive experiments have demonstrated that the proposed encryption framework can effectively protect NVS events against grayscale image reconstruction and human identification, and meanwhile, achieve high efficiency on various platforms including resource-constrained devices.

Phase-contrast THz-CT for non-destructive testing.

A new approach for image reconstruction in THz computed tomography (THz-CT) is presented. Based on a geometrical optics model containing the THz signal amplitude and phase, a novel algorithm for extracting an average phase from the measured THz signals is derived. Applying the algorithm results in a phase-contrast sinogram, which is further used for image reconstruction. For experimental validation, a fast THz time-domain spectrometer (THz-TDS) in transmission geometry is employed, enabling CT measurements within several minutes. Quantitative evaluation of reconstructed 3D printed plastic profiles reveals the potential of our approach in non-destructive testing of plastic profiles.

Two-step training deep learning framework for computational imaging without physics priors.

Deep learning (DL) is a powerful tool in computational imaging for many applications. A common strategy is to use a preprocessor to reconstruct a preliminary image as the input to a neural network to achieve an optimized image. Usually, the preprocessor incorporates knowledge of the physics priors in the imaging model. One outstanding challenge, however, is errors that arise from imperfections in the assumed model. Model mismatches degrade the quality of the preliminary image and therefore affect the DL predictions. Another main challenge is that many imaging inverse problems are ill-posed and the networks are over-parameterized; DL networks have flexibility to extract features from the data that are not directly related to the imaging model. This can lead to suboptimal training and poorer image reconstruction results. To solve these challenges, a two-step training DL (TST-DL) framework is proposed for computational imaging without physics priors. First, a single fully-connected layer (FCL) is trained to directly learn the inverse model with the raw measurement data as the inputs and the images as the outputs. Then, this pre-trained FCL is fixed and concatenated with an un-trained deep convolutional network with a U-Net architecture for a second-step training to optimize the output image. This approach has the advantage that does not rely on an accurate representation of the imaging physics since the first-step training directly learns the inverse model. Furthermore, the TST-DL approach mitigates network over-parameterization by separately training the FCL and U-Net. We demonstrate this framework using a linear single-pixel camera imaging model. The results are quantitatively compared with those from other frameworks. The TST-DL approach is shown to perform comparable to approaches which incorporate perfect knowledge of the imaging model, to be robust to noise and model ill-posedness, and to be more robust to model mismatch than approaches which incorporate imperfect knowledge of the imaging model. Furthermore, TST-DL yields better results than end-to-end training while suffering from less overfitting. Overall, this TST-DL framework is a flexible approach for image reconstruction without physics priors, applicable to diverse computational imaging systems.