Imaging Problems Research Articles

Bilevel inverse problems in neuromorphic imaging

Event or neuromorphic cameras are novel biologically inspired sensors that record data based on the change in light intensity at each pixel asynchronously. They have a temporal resolution of microseconds. This is useful for scenes with fast moving objects that can cause motion blur in traditional cameras, which record the average light intensity over an exposure time for each pixel synchronously. This paper presents a bilevel inverse problem framework for neuromorphic imaging. Existence of solution to the inverse problem is established. Second order sufficient conditions are derived under special situations for this nonconvex problem. A second order Newton type solver is derived to solve the problem. The efficacy of the approach is shown on several examples.

Two-dimensional biaxial magnetic field imaging with millisecond resolution

Multi-axial dynamic magnetic field imaging with femto-Tesla sensitivity paves a promising route for acquiring comprehensive information of the extremely-weak magnetic source. Here we propose a two-dimensional (2D) scanning imaging configuration that can realize rapid biaxial magnetic field measurement. Based on the Bragg diffraction, the configuration utilizes two orthogonally-arranged acousto-optic modulators to scan the probe laser temporally and spatially, realizing a 2D (7 × 7 pixels) magnetic field detection. To solve the multi-axial dynamic magnetic field imaging problem, a non-modulated time-sharing technique is theoretically and experimentally verified. Profiting from the non-modulated biaxial working mode that avoids long demodulation time, the magnetometer realizes millisecond temporal resolution (17.15 ms) for each sensitive axis. Operation in the spin-exchange relaxation-free regime, the single-pixel measurement sensitivity reaches around 2.6 fT/Hz1/2 (y-axis) and 7.5 fT/Hz1/2 (z-axis). Furthermore, weak pump and probe lasers power (500 μW and 350 μW) make it a low-power imaging device.

Reconstruction of Convex Sets from One or Two X-rays

We consider a class of problems of Discrete Tomography which has been deeply investigated in the past: the reconstruction of convex lattice sets from their horizontal and/or vertical X-rays, i.e. from the number of points in a sequence of consecutive horizontal and vertical lines. The reconstruction of the HV-convex polyominoes works usually in two steps, first the filling step consisting in filling operations, second the convex aggregation of the switching components. We prove three results about the convex aggregation step: (1) The convex aggregation step used for the reconstruction of HV-convex polyominoes does not always provide a solution. The example yielding to this result is called the bad guy and disproves a conjecture of the domain. (2) The reconstruction of a digital convex lattice set from only one X-ray can be performed in polynomial time. We prove it by encoding the convex aggregation problem in a Directed Acyclic Graph. (3) With the same strategy, we prove that the reconstruction of fat digital convex sets from their horizontal and vertical X-rays can be solved in polynomial time. Fatness is a property of the digital convex sets regarding the relative position of the left, right, top and bottom points of the set. The complexity of the reconstruction of the digital convex sets which are not fat remains an open question.

Self-supervised learning of hologram reconstruction using physics consistency

Existing applications of deep learning in computational imaging and microscopy mostly depend on supervised learning, requiring large-scale, diverse and labelled training data. The acquisition and preparation of such training image datasets is often laborious and costly, leading to limited generalization to new sample types. Here we report a self-supervised learning model, termed GedankenNet, that eliminates the need for labelled or experimental training data, and demonstrate its effectiveness and superior generalization on hologram reconstruction tasks. Without prior knowledge about the sample types, the self-supervised learning model was trained using a physics-consistency loss and artificial random images synthetically generated without any experiments or resemblance to real-world samples. After its self-supervised training, GedankenNet successfully generalized to experimental holograms of unseen biological samples, reconstructing the phase and amplitude images of different types of object using experimentally acquired holograms. Without access to experimental data, knowledge of real samples or their spatial features, GedankenNet achieved complex-valued image reconstructions consistent with the wave equation in free space. The GedankenNet framework also shows resilience to random, unknown perturbations in the physical forward model, including changes in the hologram distances, pixel size and illumination wavelength. This self-supervised learning of image reconstruction creates new opportunities for solving inverse problems in holography, microscopy and computational imaging.

3D Radio Imaging Under Low-Rank Constraint

Radio frequency (RF) imaging has been of interest for years due to its ability to create images of targets without being affected by weather and light conditions. The existing heuristic solutions such as back projection (BP), suffer from the sidelobe interference due to limited antenna aperture. In this paper, we propose a super-resolution 3-D imaging algorithm to enhance imaging performance. Mathematically, the reconstruction problem is an inverse problem that can be formulated as an optimization problem. The low-rank property of the object is exploited to regularize the imaging problem by nuclear norm minimization. We also develop a coarse-to-fine scan structure to reduce the computational complexity of the proposed algorithm. We evaluate the proposed algorithm with both simulations and measurements. With a frequency band range from 2.7-4.1 GHz and a 16×6 multiple-input-multiple-output (MIMO) antenna array, simulation results show high imaging quality with a median boundary keypoint precision of 2 cm, and experimental results validate the feasibility of the proposed algorithm in a real-world environment.

Preliminary processing of data from a satellite navigation receiver in the problem of three-dimensional upper ionosphere tomography

Preliminary processing of data from a satellite navigation receiver in the problem of three-dimensional upper ionosphere tomography

Spatial-spectral attention-enhanced Res-3D-OctConv for corn and weed identification utilizing hyperspectral imaging and deep learning

Corn production is an important basis to ensure the world food security, and weeds in the field will cause corn production decline. Therefore, in order to quickly recognize corn and weed in the field, a model was proposed by combining hyperspectral image with deep learning method. However, there are some problems in hyperspectral image, such as high redundancy of adjacent spectra and insufficient feature information extraction. In order to solve the above problems, the four principal components based on principal component analysis (PCA) were firstly extracted in this paper, so as to decrease the information redundancy between adjacent spectra. Secondly, the residual three-dimensional octave convolution (Res-3D-OctConv) was used to excavate the spatial information from the frequency components, while taking into account the spectral information. Finally, spatial and spectral attention models were introduced to highlight important spatial information and spectral information. At the same time, the spatial information and spectral information was integrated by cross fusion. Experimental results show that the recognition accuracy of the proposed model is 98.56 %, which is 8.65 % and 10.20 % higher than that of k-nearest neighbor (KNN) and support vector machine (SVM) respectively. The recognition result of the proposed model is further compared with that of 3D residual network (3D-ResNet) and 3D convolutional neural network (3D-CNN), and the recognition precision of the proposed model in this paper is increased by 1.40 % and 1.02 % compared with 3D-CNN and 3D-ResNet, respectively. The results show that the proposed model can better recognize the hyperspectral images of corn and weed.

CMLocate: A cross‐modal automatic visual geo‐localization framework for a natural environment without GNSS information

AbstractIn this paper, a new approach to visual geo‐localization for natural environments is proposed. The digital elevation model (DEM) data in virtual space is rendered and construct a panoramic skyline database is constructed. By combining the skyline database with real‐world image data (used as the “queries” to be localized), visual geo‐localization is treated as a cross‐modal image retrieval problem for panoramic skyline images, creating a unique new visual geo‐localization benchmark for the natural environment. Specifically, the semantic segmentation model named LineNet is proposed, for skyline extractions from query images, which has proven to be robust to a variety of complex natural environments. On the aforementioned benchmarks, the fully automatic method is elaborated for large‐scale cross‐modal localization using panoramic skyline images. Finally, the compound index is delicately designed to reduce the storage space of the positioning global descriptors and improve the retrieval efficiency. Moreover, the proposed method is proven to outperform most state‐of‐the‐art methods.

Efficient Bayesian Computation for Low-Photon Imaging Problems

Efficient Bayesian Computation for Low-Photon Imaging Problems

Inverse problem for a nonlocal diffuse optical tomography equation

In this article a nonlocal analogue of an inverse problem in diffuse optical tomography is considered. We show that whenever one has given two pairs of diffusion and absorption coefficients , , such that there holds in the measurement set W and they generate the same DN data, then they are necessarily equal in and Ω, respectively. Additionally, we show that the condition is optimal in the sense that without this restriction one can construct two distinct pairs , generating the same DN data.

On Krylov methods for large-scale CBCT reconstruction.

Krylov subspace methods are a powerful family of iterative solvers for linear systems of equations, which are commonly used for inverse problems due to their intrinsic regularization properties. Moreover, these methods are naturally suited to solve large-scale problems, as they only require matrix-vector products with the system matrix (and its adjoint) to compute approximate solutions, and they display a very fast convergence. Even if this class of methods has been widely researched and studied in the numerical linear algebra community, its use in applied medical physics and applied engineering is still very limited. e.g. in realistic large-scale computed tomography (CT) problems, and more specifically in cone beam CT (CBCT). This work attempts to breach this gap by providing a general framework for the most relevant Krylov subspace methods applied to 3D CT problems, including the most well-known Krylov solvers for non-square systems (CGLS, LSQR, LSMR), possibly in combination with Tikhonov regularization, and methods that incorporate total variation regularization. This is provided within an open source framework: the tomographic iterative GPU-based reconstruction toolbox, with the idea of promoting accessibility and reproducibility of the results for the algorithms presented. Finally, numerical results in synthetic and real-world 3D CT applications (medical CBCT andμ-CT datasets) are provided to showcase and compare the different Krylov subspace methods presented in the paper, as well as their suitability for different kinds of problems.

Enhancing fine retinal vessel segmentation: Morphological reconstruction and double thresholds filtering strategy.

Eye diseases such as diabetic retinopathy are progressive with various changes in the retinal vessels, and it is difficult to analyze the disease for future treatment. There are many computerized algorithms implemented for retinal vessel segmentation, but the tiny vessels drop off, impacting the performance of the overall algorithms. This research work contains the new image processing techniques such as enhancement filters, coherence filters and binary thresholding techniques to handle the different color retinal fundus image problems to achieve a vessel image that is well-segmented, and the proposed algorithm has improved performance over existing work. Our developed technique incorporates morphological techniques to address the center light reflex issue. Additionally, to effectively resolve the problem of insufficient and varying contrast, our developed technique employs homomorphic methods and Wiener filtering. Coherent filters are used to address the coherence issue of the retina vessels, and then a double thresholding technique is applied with image reconstruction to achieve a correctly segmented vessel image. The results of our developed technique were evaluated using the STARE and DRIVE datasets and it achieves an accuracy of about 0.96 and a sensitivity of 0.81. The performance obtained from our proposed method proved the capability of the method which can be used by ophthalmology experts to diagnose ocular abnormalities and recommended for further treatment.

Few-shot person re-identification based on Feature Set Augmentation and Metric Fusion

Person re-identification identifies pedestrians by analyzing image information from surveillance videos. However, it faces challenges like occlusion, changing lighting, and costly annotation. Thus, it is often performed in a few-shot environment with limited images. In response to the problem of insufficient available pedestrian images in person re-identification, a few-shot person re-identification method based on feature set augmentation and metric fusion is proposed. In this work, firstly, a feature augmentation method is introduced into the feature embedding module. This method introduces multi-head self-attention in different feature extraction layers and spatial attention in feature fusion of different feature extraction layers, which can extract more diverse and discriminative pedestrian features. Secondly, a dual metric method combining Euclidean and cosine distance is proposed in the metric module to comprehensively measure the absolute spatial distance and directional difference of pedestrian features. In this way, the reliability of pedestrian similarity measurement is improved. Then, pedestrian feature similarity scores are obtained separately using the dual metric and relation metric methods. Finally, the combined metric score is obtained by weighted fusion, and the combined metric score is used to construct the joint loss to realize the overall optimization and training of the network. Experimental results on three small datasets, Market-mini, Duke-mini, and MSMT17-mini, show that the proposed method significantly improves recognition performance compared to other few-shot learning algorithms. Specifically, in scenarios 5-way 1-shot and 5-way 5-shot, the average recognition accuracies are 92.54% and 96.99%, 87.93% and 96.08%, and 71.68% and 84.51%, respectively.

Object Recognition for Millimeter Wave MIMO-SAR Images Based on High-Resolution Feature Recursive Alignment Fusion Network

There are several complex situations in recognizing concealed objects from millimeter wave multiple-input multiple-output synthetic aperture radar (MIMO-SAR) security images, such as incomplete imaging of objects, partially occluded objects, and overlapping objects, which are detrimental to the accurate recognition of concealed objects. To solve these problems, a concealed object detection method based on a high-resolution feature recursive alignment fusion network (HR-FRAFnet) is proposed. The HR-FRAFnet can segment the object area from the grayscale image with complex human background and complete the recognition. The overall architecture of the HR-FRAFnet follows the encoder-decoder framework. Specifically, in the encoder stage, a deep parallel feature extraction network (DPFEN) connects the multi-resolution feature maps in parallel and repeats multi-scale feature fusion. This approach suppresses the background noise flowing and retains more recognizable target characteristics. Then, in the decoder stage, a feature recursive alignment fusion module (FRAFM) is designed to enhance the perception of object edges. The FRAFM effectively improves the segmentation accuracy of objects whereas decreasing the computational complexity of the network. Besides, we employ a combined loss function to alleviate the foreground-background imbalance problem in MIMO-SAR images. Homemade human security screening image datasets are used for evaluation. The experimental results show that the proposed method outperforms existing semantic segmentation methods in Mean Intersection over Union (mIoU) and reduces the incidence of missed and error detection of targets.

Efficient-Lightweight YOLO: Improving Small Object Detection in YOLO for Aerial Images.

The most significant technical challenges of current aerial image object-detection tasks are the extremely low accuracy for detecting small objects that are densely distributed within a scene and the lack of semantic information. Moreover, existing detectors with large parameter scales are unsuitable for aerial image object-detection scenarios oriented toward low-end GPUs. To address this technical challenge, we propose efficient-lightweight You Only Look Once (EL-YOLO), an innovative model that overcomes the limitations of existing detectors and low-end GPU orientation. EL-YOLO surpasses the baseline models in three key areas. Firstly, we design and scrutinize three model architectures to intensify the model's focus on small objects and identify the most effective network structure. Secondly, we design efficient spatial pyramid pooling (ESPP) to augment the representation of small-object features in aerial images. Lastly, we introduce the alpha-complete intersection over union (α-CIoU) loss function to tackle the imbalance between positive and negative samples in aerial images. Our proposed EL-YOLO method demonstrates a strong generalization and robustness for the small-object detection problem in aerial images. The experimental results show that, with the model parameters maintained below 10 M while the input image size was unified at 640 × 640 pixels, the APS of the EL-YOLOv5 reached 10.8% and 10.7% and enhanced the APs by 1.9% and 2.2% compared to YOLOv5 on two challenging aerial image datasets, DIOR and VisDrone, respectively.

Crystal Analyzer Based Multispectral Microtomography Using CCD-Sensor.

To solve the problems of spectral tomography, an X-ray optical scheme was proposed, using a crystal analyzer in Laue geometry between the sample and the detector, which allowed for the selection of predetermined pairs of wavelengths from the incident polychromatic radiation to obtain projection images. On a laboratory X-ray microtomography setup, an experiment was carried out for the first time where a mixture of micro-granules of sodium chloride NaCl, silver behenate AgC22H43O2, and lithium niobate LiNbO3 was used as a test sample to identify their spatial arrangement. The elements were chosen based on the presence of absorption edges in two of the elements in the energy range of the polychromatic spectrum of the probing radiation. The method of projection distortion correction was used to preprocess the obtained projections. To interpret the obtained reconstruction results, the segmentation method based on the analysis of joint histograms was used. This allowed us to identify each of the three substances. To compare the results obtained, additional "reference" tomographic measurements were performed: one in polychromatic and two in monochromatic (MoKα-, MoKβ-lines) modes. It took three times less time for the tomographic experiment with the crystal analyzer, while the reconstruction accuracy was comparable to that of the "reference" tomography.

Numerical and experimental investigation of many-objective optimization for efficient temperature and velocity fields reconstruction via acoustic tomography

Accurate and high-quality measurement of temperature and velocity fields is important in combustion and flow diagnosis, especially for the control of boiler operations and optimization of combustion processes. Nonlinear acoustic tomography (NAT) is an appealing approach to simultaneously monitor temperature and velocity fields by observing the changes in sound speed they induce. However, the inherent ill-posedness of the tomographic inverse problem and the lack of a priori information may lead to poor robustness, low precision, and the presence of artifacts. On the other hand, the inclusion of a priori information, such as smoothness and box constraints, may require the use of hyperparameters that hinder quasi real-time reconstruction and may degrade performance in actual applications. In this study, we aim to ensure quasi real-time reconstruction and alleviate the ill-posedness of the NAT problem by eliminating the regularization parameters and transforming the inverse problem into a many-objective optimization problem with four objectives. The knee point-driven evolutionary algorithm with improved environmental selection strategy (KEA-IES) is proposed to simultaneously reconstruct the arbitrary inhomogeneous temperature and aerodynamic fields in a furnace. Apart from the investigation of the performance of KEA-IES and the influence of various factors, including measurement noise, function evaluation number, population size, and functional norms, on the quality of the reconstruction, an experimental AT system with independent 16 T/R channels is developed to evaluate the proposed method. The results show that combustion flame temperature and velocity fields can be monitored simultaneously by using the proposed KEA-IES with more accuracy, less consumed time, and better noise immunity compared with the state-of-the-art algorithms. The proposed method can provide valuable guidance in the development of a non-intrusive real-time pyrometry and velocimetry system, particularly for applications with large velocity fields and temperature gradients. Furthermore, its capability in complex combustion and flow diagnosis will offer benefits in terms of energy conservation, emission reduction, boiler operations supervision, and combustion process optimization.

DEGAN: Decompose-Enhance-GAN Network for Simultaneous Low-Light Image Lightening and Denoising

Images taken in low-light situations frequently have a significant quality reduction. Taking care of these degradation problems in low-light images is essential for raising their visual quality and enhancing high-level visual task performance. However, because of the inherent information loss in dark images, conventional Retinex-based approaches for low-light image enhancement frequently fail to accomplish real denoising. This research introduces DEGANet, a revolutionary deep-learning framework created particularly for improving and denoising low-light images. To overcome these restrictions, DEGANet makes use of the strength of a Generative Adversarial Network (GAN). The Decom-Net, Enhance-Net, and an Adversarial Generative Network (GAN) are three linked subnets that make up our novel Retinex-based DEGANet architecture. The Decom-Net is in charge of separating the reflectance and illumination components from the input low-light image. This decomposition enables Enhance-Net to effectively enhance the illumination component, thereby improving the overall image quality. Due to the complicated noise patterns, fluctuating intensities, and intrinsic information loss in low-light images, denoising them presents a significant challenge. By incorporating a GAN into our architecture, DEGANet is able to effectively denoise and smooth the enhanced image as well as retrieve the original data and fill in the gaps, producing an output that is aesthetically beautiful while maintaining key features. Through a comprehensive set of studies, we demonstrate that DEGANet exceeds current state-of-the-art methods in both terms of image enhancement and denoising quality.

Semi-device-dependent blind quantum tomography

Extracting tomographic information about quantum states is a crucial task in the quest towards devising high-precision quantum devices. Current schemes typically require measurement devices for tomography that are a priori calibrated to high precision. Ironically, the accuracy of the measurement calibration is fundamentally limited by the accuracy of state preparation, establishing a vicious cycle. Here, we prove that this cycle can be broken and the dependence on the measurement device's calibration significantly relaxed. We show that exploiting the natural low-rank structure of quantum states of interest suffices to arrive at a highly scalable `blind' tomography scheme with a classically efficient post-processing algorithm. We further improve the efficiency of our scheme by making use of the sparse structure of the calibrations. This is achieved by relaxing the blind quantum tomography problem to the de-mixing of a sparse sum of low-rank matrices. We prove that the proposed algorithm recovers a low-rank quantum state and the calibration provided that the measurement model exhibits a restricted isometry property. For generic measurements, we show that it requires a close-to-optimal number of measurement settings. Complementing these conceptual and mathematical insights, we numerically demonstrate that robust blind quantum tomography is possible in a practical setting inspired by an implementation of trapped ions.

Motion detection in diffraction tomography by common circle methods

The method of common lines is a well-established reconstruction technique in cryogenic electron microscopy (cryo-EM), which can be used to extract the relative orientations of an object given tomographic projection images from different directions. In this paper, we deal with an analogous problem in optical diffraction tomography. Based on the Fourier diffraction theorem, we show that rigid motions of the object, i.e., rotations and translations, can be determined by detecting common circles in the Fourier-transformed data. We introduce two methods to identify common circles. The first one is motivated by the common line approach for projection images and detects the relative orientation by parameterizing the common circles in the two images. The second one assumes a smooth motion over time and calculates the angular velocity of the rotational motion via an infinitesimal version of the common circle method. Interestingly, using the stereographic projection, both methods can be reformulated as common line methods, but these lines are, in contrast to those used in cryo-EM, not confined to pass through the origin and allow for a full reconstruction of the relative orientations. Numerical proof-of-the-concept examples demonstrate the performance of our reconstruction methods.