Image Reconstruction Error Research Articles

Self-Supervised Monocular Depth Estimation of Untextured Indoor Rotated Scenes

Self-supervised deep learning methods have leveraged stereo images for training monocular depth estimation. Although these methods show strong results on outdoor datasets such as KITTI, they do not match performance of supervised methods on indoor environments with camera rotation. Indoor, rotated scenes are common for less constrained applications and pose problems for two reasons: abundance of low texture regions and increased complexity of depth cues for images under rotation. In an effort to extend self-supervised learning to more generalised environments we propose two additions. First, we propose a novel Filled Disparity Loss term that corrects for ambiguity of image reconstruction error loss in textureless regions. Specifically, we interpolate disparity in untextured regions, using the estimated disparity from surrounding textured areas, and use L1 loss to correct the original estimation. Our experiments show that depth estimation is substantially improved on low-texture scenes, without any loss on textured scenes, when compared to Monodepth by Godard et al. Secondly, we show that training with an application's representative rotations, in both pitch and roll, is sufficient to significantly improve performance over the entire range of expected rotation. We demonstrate that depth estimation is successfully generalised as performance is not lost when evaluated on test sets with no camera rotation. Together these developments enable a broader use of self-supervised learning of monocular depth estimation for complex environments.

Phase and Amplitude Calibration of Rotating Equispaced Circular Array for Geostationary Microwave Interferometric Radiometers—Simulation Results and Discussion

In the theoretical part of this work, a calibration strategy for both phase and amplitude calibration of rotating equispaced circular array based on redundant space calibration was introduced. It avoids the challenging implementation of conventional noise-injection networks, especially for a large-scale array, by using the strong self-consistency accomplished by array rotation. This study presents comprehensive simulation results of the proposed calibration strategy, where calibration residuals and image reconstruction errors are evaluated. The phase calibration shows outstanding performance, while the amplitude calibration is found to be biased due to the logarithms used to linearize the system of calibration equations, and the resulting image reconstruction bias is scene-dependent. A bias correction method is proposed accordingly, which requires an a priori regional brightness temperature within the observation scene. The performance of both phase and amplitude calibration take advantage of increasing the number of elements, which promises the system scalability for high spatial resolution applications.

Adaptive hyperparameter updating for training restricted Boltzmann machines on quantum annealers

Restricted Boltzmann Machines (RBMs) have been proposed for developing neural networks for a variety of unsupervised machine learning applications such as image recognition, drug discovery, and materials design. The Boltzmann probability distribution is used as a model to identify network parameters by optimizing the likelihood of predicting an output given hidden states trained on available data. Training such networks often requires sampling over a large probability space that must be approximated during gradient based optimization. Quantum annealing has been proposed as a means to search this space more efficiently which has been experimentally investigated on D-Wave hardware. D-Wave implementation requires selection of an effective inverse temperature or hyperparameter (beta ) within the Boltzmann distribution which can strongly influence optimization. Here, we show how this parameter can be estimated as a hyperparameter applied to D-Wave hardware during neural network training by maximizing the likelihood or minimizing the Shannon entropy. We find both methods improve training RBMs based upon D-Wave hardware experimental validation on an image recognition problem. Neural network image reconstruction errors are evaluated using Bayesian uncertainty analysis which illustrate more than an order magnitude lower image reconstruction error using the maximum likelihood over manually optimizing the hyperparameter. The maximum likelihood method is also shown to out-perform minimizing the Shannon entropy for image reconstruction.

Image reconstruction using the modified texture synthesis algorithm

The article discusses a method for image reconstruction based on the search for similar blocks using a texture synthesis algorithm. The effectiveness of the new approach is shown using several examples for various areas with lost pixels. The subject of the research is methods and algorithms for processing space-time reconstruction of two-dimensional signals based on a geometric model of images. The object of research is a set of test static images. The result of the study is a modification of the image reconstruction method based on the search for similar blocks in order to reduce the error in image reconstruction. The novelty of the work is an algorithm that improves the quality of image restoration. The results obtained make it possible to reduce the root mean square error.

Compressible Latent-Space Invertible Networks for Generative Model-Constrained Image Reconstruction.

There remains an important need for the development of image reconstruction methods that can produce diagnostically useful images from undersampled measurements. In magnetic resonance imaging (MRI), for example, such methods can facilitate reductions in data-acquisition times. Deep learning-based methods hold potential for learning object priors or constraints that can serve to mitigate the effects of data-incompleteness on image reconstruction. One line of emerging research involves formulating an optimization-based reconstruction method in the latent space of a generative deep neural network. However, when generative adversarial networks (GANs) are employed, such methods can result in image reconstruction errors if the sought-after solution does not reside within the range of the GAN. To circumvent this problem, in this work, a framework for reconstructing images from incomplete measurements is proposed that is formulated in the latent space of invertible neural network-based generative models. A novel regularization strategy is introduced that takes advantage of the multiscale architecture of certain invertible neural networks, which can result in improved reconstruction performance over classical methods in terms of traditional metrics. The proposed method is investigated for reconstructing images from undersampled MRI data. The method is shown to achieve comparable performance to a state-of-the-art generative model-based reconstruction method while benefiting from a deterministic reconstruction procedure and easier control over regularization parameters.

Self-Supervised Monocular Depth Estimation With Extensive Pretraining

Although depth estimation is a key technology for three-dimensional sensing applications involving motion, active sensors such as LiDAR and depth cameras tend to be expensive and bulky. Here, we explore the potential of monocular depth estimation (MDE) using a self-supervised approach. MDE is a promising technology, but supervised learning suffers from a need for accurate ground-truth depth data. Recent studies have enabled self-supervised training on an MDE model with only monocular image sequences and image-reconstruction errors. We pretrained networks using multiple datasets, including monocular and stereo image sequences. The main challenges posed by the self-supervised MDE model were occlusions and dynamic objects. We proposed novel loss functions to handle these problems in the form of min-over-all and min-with-flow losses, both based on the per-pixel minimum reprojection error of Monodepth2 and extended to stereo images and optical flow. With extensive pretraining and novel losses, our model outperformed existing unsupervised approaches in quantitative depth estimation and the ability to distinguish small objects against a background, as evaluated by KITTI 2015.

Solution of 2D MIT Forward Problem by Considering Skin and Proximity Effects in Coils

Previous studies on the forward problem of magnetic induction tomography (MIT) have used simplified Maxwell's equations which assume a constant and position-independent total current density (TCD) inside the coils (ignoring skin effect). Moreover, they assume that TCD is independent of relative position of the coils (ignoring proximity effect). This article presents an improved finite element (FE) modeling for the two-dimensional (2D) forward problem of MIT by incorporating skin and proximity effects in the exciter and sensor coils. Consideration of skin and proximity effects requires the use of a position-dependent TCD in Maxwell's equations inside the coil domain. The FE method implementation of the improved forward method is validated with a simple analytical test problem. To evaluate the performance of the improved method in possible medical and industrial applications with low conductivity regions, a 16 coils 2D MIT system is modeled and the FE method is employed to solve the forward problem using a synthetic phantom. Results show the difference between the real parts of induced voltages obtained from the early and improved method falls into the range which is meaningful in terms of achievable conductivity contrast from the improved one. This discrepancy can generate considerable errors in image reconstruction. By considering skin and proximity effects, the improved method generates lower voltages in the coils suggesting the use of more precise hardware to detect inclusion. Findings show the importance and necessity of using improved method for modeling 2D MIT coils especially at low conductivity applications where measured signals are inherently weak.

Ultrasound elastography using a regularized modified error in constitutive equations (MECE) approach: a comprehensive phantom study

Many of the current techniques in transient elastography, such as shear wave elastography (SWE) assume a dominant planar shear wave propagating in an infinite medium. This underlying assumption, however, can be easily violated in real scenarios in vivo, leading to image artifacts and reconstruction errors. Other approaches that are not bound to planar shear wave assumption, such solutions based on the partial differential equation, can potentially overcome the shortcomings of the conventional SWE. The main objective of this paper is to demonstrate the advantages of the modified error in constitutive equations (MECE) formulation with total variation regularization (MECE + TV) over SWE in reconstructing the elastic moduli of different tissue-mimicking phantoms. Experiments were conducted on phantoms with inclusions of well-defined shapes to study the reconstruction of specific features relevant to practical applications. We compared the performances of MECE + TV and SWE in terms of quantitative metrics to estimate reconstruction accuracy, inclusion shape recovery, edge preservation and edge sharpness, inclusion size representation, and shear elasticity and contrast accuracies. The results indicate that the MECE + TV approach outperforms SWE based on several of these metrics. It is concluded that, with further development, the proposed method may offer elastography reconstructions that are superior to SWE in clinical applications.

Posterior Information-Based Image Measurement Matrix Optimization

In order to enhance the robustness of image compression sensing system and reduce the mutual coherence between measurement matrix and sparse basis, this paper proposes an image measurement matrix optimization algorithm based on posterior information. Based on the traditional measurement matrix optimization model, the proposed algorithm considers the image reconstruction error from the OMP algorithm and uses it as a regular term. Matrix F-norm expansion and singular value decomposition are used to reduce the computational complexity and ensure the convergence of algorithm. Besides, the gradient matrix method is used to iteratively solve the measurement matrix. The proposed measurement matrix optimization model makes full use of the reconstruction error information of the image itself, not only improves the robustness of image compression sensing system, but also reduces the mutual coherence between the measurement matrix and the sparse basis. Experiments results show that compared with the state-of-the-art measurement matrix optimization algorithms, the proposed algorithm can reduce the average correlation coefficient more effectively, and the peak signal-to-noise ratio of the image can be increased by up to 1.2 dB.

Unified cross-modality feature disentangler for unsupervised multi-domain MRI abdomen organs segmentation.

Our contribution is a unified cross-modality feature disentagling approach for multi-domain image translation and multiple organ segmentation. Using CT as the labeled source domain, our approach learns to segment multi-modal (T1-weighted and T2-weighted) MRI having no labeled data. Our approach uses a variational auto-encoder (VAE) to disentangle the image content from style. The VAE constrains the style feature encoding to match a universal prior (Gaussian) that is assumed to span the styles of all the source and target modalities. The extracted image style is converted into a latent style scaling code, which modulates the generator to produce multi-modality images according to the target domain code from the image content features. Finally, we introduce a joint distribution matching discriminator that combines the translated images with task-relevant segmentation probability maps to further constrain and regularize image-to-image (I2I) translations. We performed extensive comparisons to multiple state-of-the-art I2I translation and segmentation methods. Our approach resulted in the lowest average multi-domain image reconstruction error of 1.34±0.04. Our approach produced an average Dice similarity coefficient (DSC) of 0.85 for T1w and 0.90 for T2w MRI for multi-organ segmentation, which was highly comparable to a fully supervised MRI multi-organ segmentation network (DSC of 0.86 for T1w and 0.90 for T2w MRI).

A portable system for two dimensional magnetic particle imaging

Magnetic Particle Imaging (MPI) is a novel approach which outperforms other powerful imaging techniques such as Magnetic Resonance Imaging (MRI) in terms of image resolution and image reconstruction execution time. However it uses high intensity electric current to generate relatively strong magnetic field which hinders its suitability to be used in hazardous areas and where portability is required. This paper suggests a new design of a portable and safe MPI system dedicated for process instrumentation and medical imaging applications. The apparatus consists of an array of Neodymium Iron Boron (NdFeB) magnets, evenly distributed according to Halbach arrangement, to generate a relatively strong homogeneous magnetic flux intensity. The aim of the design is to enhance the value of the magnetic flux density within the field of view (FOV) above the saturation value of the magnetic field corresponding to the paramagnetic particles. In addition, an increase of the FOV volume and a minimization of the number of pixels within the Free Field Line (FFL) with the lowest possible total weight are sought to improve the performance of the image reconstruction algorithm. This multivariable optimization problem which considers simultaneously the size of permanent magnets and the gap between adjacent magnets as design parameters was solved using two dimensional finite element method (2D-FEM) package. Both trapezoid and cuboid magnets elements design alternatives were considered during the optimization process using the Particle Swarm Optimization (PSO) algorithm. FEM simulation results indicate that an average magnetic flux density (Bavg = 0.658 T) could be achieved using 12 permanent cuboid magnets elements of 20 mm length on each side. A hardware prototype using the optimized variables was constructed and the associated results match to a good extend the FEM simulation results, where an average magnetic flux density within the FOV region was found to be equal to 0.634 T could be obtained. Further assessment of the design on both phantom and real images using Kaczmarz image reconstruction algorithm demonstrates the effectiveness of the design where an image reconstruction error of less than 9% could be achieved using cuboid magnets using a magnet array of total weight around 1.8 kg. This may suggest that the proposed design can be a tangible alternative for next generation portable MPI systems which can be used either for biomedical applications or in hazardous areas associated to hydrocarbon pipelines within which traditional high current coils used to generate the static magnetic flux density are not allowed to be deployed.

Composite Sensitivity Matrix for Reducing the Influence of Medium Electrification on Electrical Capacitance Tomography

The electrical capacitance tomography (ECT) is an image-based technology which has been widely used for gas–solid two-phase flow measurement. The solid-phase electrification is an inevitable phenomenon during pneumatic conveying. To reduce the influence of medium electrification on the ECT image reconstruction, a composite sensitivity matrix (CSM) which takes into account the effect of charge electrification is developed in this paper, so that the image reconstruction error can be well suppressed. This paper first analyzes how the medium electrification affects ECT sensitivity field from the perspective of electromagnetic theory. The CSM is then derived accordingly. Simulations and physical experiments are carried out to validate the proposed method. The experimental results show that the relative error and the correlative coefficient of the reconstructed images have been significantly improved comparing with that in traditional approaches. The results support the effectiveness of the proposed method.

DVSNet: deep variance‐stabilised network robust to spatially variant characteristics in imaging

In real imaging systems, unwanted noises or artefacts can be caused by an image reconstruction error during photon-to-digital conversion. The generated noises typically tend to have spatially variant characteristics in an acquired image due to their signal dependencies. The noise characteristics via a preliminary study are analytically introduced and an image restoration scheme based on deep variance-stabilised network is proposed. Specifically, to improve the robustness of restoration performance to the noise properties, variance-stabilising transformation and binning priors are properly combined with a deep neural network as a layer structure particularly designed for denoising. Experimental results show the suggested network model outperforms existing state-of-the-art denoising methods based on convolutional neural network.

CT imaging parameters for precision models using additive manufacturing

Additive manufacturing (AM), generally known as 3D Printing, is one of the manufacturing methods to develop medical models. The AM physical medical models are manufactured by virtual Computer-Aided Design (CAD) models. These CAD models are generated using medical scan data such as, Computed Tomography (CT) and Magnetic Resonance Imaging (MRI), etc. In the process of CT data acquisition and transformation from CT to CAD data, dimensional and volumetric errors occur. In this work, the influence of several CT image reconstruction parameters on the 3D CAD model was evaluated experimentally. The dry mandible has been considered as a phantom from construction of a 3D CAD model. The linear dimensional and volumetric errors in the CT image reconstruction were compared from the dry mandible to the 3D CAD model of the CT images. Further, the CT image reconstruction parameters are optimized by Taguchi and Gray relational analysis methods. The optimized parameter of a 3D CAD model was used to develop STereo Lithography (STL) file and Fused Deposition Modeling (FDM) model by AM process. The scaling factor of each axis is determined by considering the ratio of STL to FDM model linear dimensions. Finally, the scaled STL model was manufactured by FDM technique and investigated that the dimensional error of the FDM model was minimized.

F-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks.

Obtaining expert labels in clinical imaging is difficult since exhaustive annotation is time-consuming. Furthermore, not all possibly relevant markers may be known and sufficiently well described a priori to even guide annotation. While supervised learning yields good results if expert labeled training data is available, the visual variability, and thus the vocabulary of findings, we can detect and exploit, is limited to the annotated lesions. Here, we present fast AnoGAN (f-AnoGAN), a generative adversarial network (GAN) based unsupervised learning approach capable of identifying anomalous images and image segments, that can serve as imaging biomarker candidates. We build a generative model of healthy training data, and propose and evaluate a fast mapping technique of new data to the GAN's latent space. The mapping is based on a trained encoder, and anomalies are detected via a combined anomaly score based on the building blocks of the trained model - comprising a discriminator feature residual error and an image reconstruction error. In the experiments on optical coherence tomography data, we compare the proposed method with alternative approaches, and provide comprehensive empirical evidence that f-AnoGAN outperforms alternative approaches and yields high anomaly detection accuracy. In addition, a visual Turing test with two retina experts showed that the generated images are indistinguishable from real normal retinal OCT images. The f-AnoGAN code is available at https://github.com/tSchlegl/f-AnoGAN.

Automatic Orbit Selection for a Radio Interferometric Spacecraft Constellation

A constellation of radio telescope spacecraft can leverage interferometry to accurately image distant objects throughout the universe, but mission design must balance among many interrelated constraints. In particular, the number of spacecraft and their time-varying orbital parameters determine what interferometric baselines are feasible for each target, which in turn drives the imaging capabilities of the constellation. The large combinatorics of dynamic constellation configuration and the numerous competing engineering concerns present a challenge that is not well addressed by labor-intensive manual mission design processes. This paper describes search-based optimization methods that direct mission design effort toward promising constellation geometries: those that achieve broad interferometric coverage but remain cost-effective and resilient to failures. Six families of automatic optimization algorithms with complementary search strategies were created to explore among explicit constellation configuration plans. Evaluation of each candidate constellation plan was accelerated by efficiently combining precomputed caches of orbital and interferometric data. Comparative results indicate that leveraging automated optimization for constellation mission design is practical and useful. Optimized constellations demonstrated target image reconstruction errors 10% better than a manually designed constellation and up to 35% better than random solutions.

Unsupervised Multi-Object Detection for Video Surveillance Using Memory-Based Recurrent Attention Networks

Nowadays, video surveillance has become ubiquitous with the quick development of artificial intelligence. Multi-object detection (MOD) is a key step in video surveillance and has been widely studied for a long time. The majority of existing MOD algorithms follow the “divide and conquer” pipeline and utilize popular machine learning techniques to optimize algorithm parameters. However, this pipeline is usually suboptimal since it decomposes the MOD task into several sub-tasks and does not optimize them jointly. In addition, the frequently used supervised learning methods rely on the labeled data which are scarce and expensive to obtain. Thus, we propose an end-to-end Unsupervised Multi-Object Detection framework for video surveillance, where a neural model learns to detect objects from each video frame by minimizing the image reconstruction error. Moreover, we propose a Memory-Based Recurrent Attention Network to ease detection and training. The proposed model was evaluated on both synthetic and real datasets, exhibiting its potential.

A sparsity-based Bayesian approach for hyperspectral unmixing using normal compositional model

A new Bayesian-based method is developed for unmixing of hyperspectral images. Endmembers are assumed variable based on the Gaussian distribution. A semi-supervised scenario is considered, and as a practical aspect, the abundance vectors are assumed sparse. We propose the Dirichlet prior to represent the sparsity and derive the corresponding posteriors in Bayesian sense. Numerical results are used to evaluate different methods for both simulated and real data. It is shown that the proposed method achieves a lower error in abundance estimation and image reconstruction.

Methods for multiple-telescope beam imaging and guiding in the near-infrared

Atmospheric turbulence and precise measurement of the astrometric baseline vector between any two telescopes are two major challenges in implementing phase referenced interferometric astrometry and imaging. They limit the performance of a fibre-fed interferometer by degrading the instrument sensitivity and astrometric measurements precision and by introducing image reconstruction errors due to inaccurate phases. A multiple beam acquisition and guiding camera was built to meet these challenges for a recently commissioned four beam combiner instrument, GRAVITY, at the ESO Very Large Telescope Interferometer. For each telescope beam it measures: a) field tip-tilts by imaging stars in the sky; b) telescope pupil shifts by imaging pupil reference laser beacons installed on each telescope using a $2 \times 2$ lenslet; c) higher order aberrations using a $9 \times 9$ Shack-Hartmann. The telescope pupils are imaged for a visual monitoring while observing. These measurements enable active field and pupil guiding by actuating a train of tip-tilt mirrors placed in the pupil and field planes, respectively. The Shack-Hartmann measured quasi-static aberrations are used to focus the Auxiliary Telescopes and allow the possibility of correcting the non-common path errors between the Unit Telescopes adaptive optics systems and GRAVITY. The guiding stabilizes light injection into single-mode fibres, increasing sensitivity and reducing the astrometric and image reconstruction errors. The beam guiding enables to achieve astrometric error less than $50\,\mu$as. Here, we report on the data reduction methods and laboratory tests of the multiple beam acquisition and guiding camera and its performance on-sky.

Decoupled dynamic magnetic field measurements improves diffusion-weighted magnetic resonance images

Field probes are miniature receiver coils with localized NMR-active samples inside. They are useful in monitoring magnetic field. This information can be used to improve magnetic resonance image quality. While field probes are coupled to each other marginally in most applications, this coupling can cause incorrect resonance frequency estimates and image reconstruction errors. Here, we propose a method to reduce the coupling between field probes in order to improve the accuracy of magnetic field estimation. An asymmetric sensitivity matrix describing the coupling between channels of field probes and NMR active droplets within field probes was empirically measured. Localized signal originating from each probe was derived from the product of the inverse of the sensitivity matrix and the coupled probe measurements. This method was used to estimate maps of dynamic magnetic fields in diffusion weighted MRI. The estimated fields using decoupled probe measurement led to images more robust to eddy currents caused by diffusion sensitivity gradients along different directions.