Sparse Image Research Articles

Modeling the impact of coincidence loss on count rate statistics and noise performance in counting detectors for imaging applications

Coincidence loss can have detrimental effects on the image quality provided by pixelated counting detectors, especially in dose-sensitive applications like cryoEM where the information extracted from the recorded signal needs to be maximized. In this work, we investigate the impact of coincidence loss phenomena on the recorded statistics in counting detectors producing sparse binary images. First, we derive exact analytical expressions for the mean and the variance of the recorded counts as a function of the incoming event rate. Second, we address the problem of the mean and variance of the recorded events (i.e., pixel clusters identified as individual incoming events), which also acts as a function of the incoming event rate. In this frame, we review previous studies from different disciplines on approximated two-dimensional models, and we critically reinterpret them in our context and evaluate the suitability of their adoption in the present case. The knowledge of the first two momenta of the recorded statistics allows inferring about the signal-to-noise ratio (SNR) and the detective quantum efficiency at zero frequency (DQE0). Analytical results are validated through comparison with numerical data obtained with a custom-made Monte Carlo code. We chose a realistic case study for cryoEM application consisting of a 25-µm-thick MAPS detector featuring a pixel size of 10 µm and illuminated with electrons of 300 keV energy over a wide range of incoming rate.

Few-Shot Remote Sensing Novel View Synthesis with Geometry Constraint NeRF

Novel View Synthesis (NVS) is an important task for 3D interpretation in remote sensing scenes, which also benefits vicinagearth security by enhancing situational awareness capabilities. Recently, NVS methods based on Neural Radiance Fields (NeRFs) have attracted increasing attention for self-supervised training and highly photo-realistic synthesis results. However, it is still challenging to synthesize novel view images in remote sensing scenes, given the complexity of land covers and the sparsity of input multi-view images. In this paper, we propose a novel NVS method named FReSNeRF, which combines Image-Based Rendering (IBR) and NeRF to achieve high-quality results in remote sensing scenes with sparse input. We effectively solve the degradation problem by adopting the sampling space annealing method. Additionally, we introduce depth smoothness based on the segmentation mask to constrain the scene geometry. Experiments on multiple scenes show the superiority of our proposed FReSNeRF over other methods.

On the faecal detection of Halicephalobus gingivalis in equines in Iran.

The life-cycle of the parasitic forms of Halicephalobus gingivalis is yet to be fully understood. In cases where there is evidence to support the presence of eggs from this parasite or its DNA in the faeces of equines, a thorough investigation of the clinical status and gastro-intestinal tract of the affected animals is warranted, as well as detailed descriptions of the employed coprological technique. Since reports of the identification of H. gingivalis eggs in faeces are sparse, objective measurements and high-quality images must be provided, in order to ensure optimal criteria for classification. Regarding research concerning H. gingivalis, no piece of information should be regarded as superfluous.

LRPS‐GCN: A millimeter wave sparse imaging algorithm based on graph signal

AbstractAiming at the problems of slow speed and poor accuracy of traditional millimeter wave sparse imaging, a sparse imaging algorithm based on graph convolution model is proposed from the perspective of sparse signal recovery. The graph signal model is constructed by combining the low‐rank and piecewise smoothing(LRPS) regular terms, based on which the proximal operator is replaced by the denoising graph convolution network, and the graph convolution sparse reconstruction network LRPS‐GCN is constructed, and the recovered target image is obtained by iterating with the optimal non‐linear sparse variation. For the proposed algorithm, simulation experiments are carried out using synthetic datasets under different target densities, iteration times and noise environments, and compared with the traditional graph signal reconstruction algorithm and the deep compressed sensing reconstruction algorithm, and then use the measured data with varying degrees of sparsity to validate. The experimental results show that the reconstructed images by this algorithm have better performance in terms of normalised mean square error, target to background ratio, reconstruction time and memory usage.

MOX-NET: Multi-stage deep hybrid feature fusion and selection framework for monkeypox classification

Background:Monkeypox virus has quickly expanded throughout several nations, raising serious public health concerns. Lack of precautionary measures raises concerns about the possibility of global pandemic. As a result, early detection of monkeypox is essential for efficient diagnosis and treatment before broad community transmission. Artificial intelligence (AI) and machine learning (ML) have recently shown tremendous promise in early monkeypox diagnosis using image-based diagnoses methods. But, challenges such as low contrast lesions, overloaded datasets, high memory complexity, and redundant feature extraction still need to be addressed. Methods:This work proposes a coalesced CAD system for monkeypox classification using a deep learning framework. The proposed approach includes multiple stages: First, the given dataset images are pre-processed by a proposed fusion-based contrast enhancement method. Second, six pre-trained deep learning models i.e., Vision Transformers (ViT), Shifted Windows (Swin) Transformers, ResNet-50, ResNet-101, EfficientNetV2, and ConvNeXt-V2 are modified and transfer learning is used for training. Third, from all the deep learning networks, the deep feature vectors are acquired and integrated through the convolutional sparse image decomposition fusion strategy. Subsequently, an entropy-controlled firefly feature selection approach is applied to identify the most suitable features for classification. Lastly, the multi-class support vector machine is utilized to classify the input data based on the selected features. Results:The proposed algorithm utilized the MSLID dataset and obtained an accuracy of 98.64% which is better than other state-of-the-art works. Conclusions:Experimental outcomes reveals that the proposed monkeypox classification algorithm outperforms well-established existing algorithms in both visual and quantitative evaluations, resulting in improved accuracy.

Automatic segmentation and implicit surface representation of dynamic cardiac data

Segmentation of anatomical structures on 2D images of cardiac exams is necessary for performing 3D volumetric analysis, enabling the computation of parameters for diagnosing cardiovascular disease. In this work, we present robust algorithms to automatically segment cardiac imaging data and generate a volumetric anatomical reconstruction of a patient-specific heart model by propagating active contour output within a patient stack through a self-supervised learning model. Contour initializations are automatically generated, then output segmentations on sparse image slices are transferred and merged across a stack of images within the same heart data set during the segmentation process. We demonstrate whole-heart segmentation and compare the results with ground truth manual annotations. Additionally, we provide a framework to represent segmented heart data in the form of implicit surfaces, allowing interpolation operations to generate intermediary models of heart sections and volumes throughout the cardiac cycle and to estimate ejection fraction.

Hyperspectral-multispectral image fusion using subspace decomposition and Elastic Net Regularization

ABSTRACT The fusion of hyperspectral and multispectral images presents a challenge as it involves blending a low-resolution hyperspectral image (HSI) with a corresponding multispectral image (MSI) to produce a high-resolution hyperspectral image (HRI). A number of existing techniques have limitations; for instance, matrix decomposition-based approaches fail to retain adequate spatial and spectral image information during fusion, while tensor decomposition-based processes have high computational overhead. In this paper, we propose a novel method for fusing hyperspectral and multispectral images. Our method leverages the strong correlation among the spectral bands of hyperspectral images and employs the SVD technique to extract spectral feature subspaces. This approach results in a more compact and representative feature space for fusion. Secondly, the proposed method utilizes Elastic Net regularization in combination with L 1 and L 2 regularization for effective feature selection of highly covariate features. Weighted group sparse regularization is employed to enhance the fusion effect, enabling better representation of the image’s structure and features. The algorithm is subsequently evaluated on multiple datasets to confirm its effectiveness. The results of the experiment suggest that the suggested algorithm greatly enhances the spatial resolution and visual clarity of HSI images while preserving the spectral characteristics when compared to conventional methods for fusion of hyperspectral images. Additionally, the constraint for regulation can competently reduce any noise or artefacts, thereby boosting image discrimination. To summarize, the utilization of a sparse tensor-based hyperspectral image fusion algorithm with subspace learning provides an efficacious approach for processing hyperspectral imagery. This method is capable of improving spatial resolution, extracting advantageous features from hyperspectral imagery, and ultimately supports the process of remote sensing image analysis and application.

A Multi-Shot Approach for Spatial Resolution Improvement of Multispectral Images from an MSFA Sensor.

Multispectral imaging technology has advanced significantly in recent years, allowing single-sensor cameras with multispectral filter arrays to be used in new scene acquisition applications. Our camera, developed as part of the European CAVIAR project, uses an eight-band MSFA to produce mosaic images that can be decomposed into eight sparse images. These sparse images contain only pixels with similar spectral properties and null pixels. A demosaicing process is then applied to obtain fully defined images. However, this process faces several challenges in rendering fine details, abrupt transitions, and textured regions due to the large number of null pixels in the sparse images. Therefore, we propose a sparse image composition method to overcome these challenges by reducing the number of null pixels in the sparse images. To achieve this, we increase the number of snapshots by simultaneously introducing a spatial displacement of the sensor by one to three pixels on the horizontal and/or vertical axes. The set of snapshots acquired provides a multitude of mosaics representing the same scene with a redistribution of pixels. The sparse images from the different mosaics are added together to get new composite sparse images in which the number of null pixels is reduced. A bilinear demosaicing approach is applied to the composite sparse images to obtain fully defined images. Experimental results on images projected onto the response of our MSFA filter show that our composition method significantly improves image spatial resolution and minimizes reconstruction errors while preserving spectral fidelity.

SonoMyoNet: A Convolutional Neural Network for Predicting Isometric Force from Highly Sparse Ultrasound Images.

Ultrasound imaging or sonomyography has been found to be a robust modality for measuring muscle activity due to its ability to image deep-seated muscles directly while providing superior spatiotemporal specificity compared to surface electromyography-based techniques. Quantifying the morphological changes during muscle activity involves computationally expensive approaches for tracking muscle anatomical structures or extracting features from brightness-mode (B-mode) images and amplitude-mode (A-mode) signals. This paper uses an offline regression convolutional neural network (CNN) called SonoMyoNet to estimate continuous isometric force from sparse ultrasound scanlines. SonoMyoNet learns features from a few equispaced scanlines selected from B-mode images and utilizes the learned features to estimate continuous isometric force accurately. The performance of SonoMyoNet was evaluated by varying the number of scanlines to simulate the placement of multiple single-element ultrasound transducers in a wearable system. Results showed that SonoMyoNet could accurately predict isometric force with just four scanlines and is immune to speckle noise and shifts in the scanline location. Thus, the proposed network reduces the computational load involved in feature tracking algorithms and estimates muscle force from the global features of sparse ultrasound images.

Deep-learning-based decomposition of overlapping-sparse images: application at the vertex of simulated neutrino interactions

Image decomposition plays a crucial role in various computer vision tasks, enabling the analysis and manipulation of visual content at a fundamental level. Overlapping and sparse images pose unique challenges for decomposition algorithms due to the scarcity of meaningful information to extract components. Here, we present a solution based on deep learning to accurately extract individual objects within multi-dimensional overlapping-sparse images, with a direct application to the decomposition of overlaid elementary particles obtained from imaging detectors. Our approach allows us to identify and measure independent particles at the vertex of neutrino interactions, where one expects to observe images with indiscernible overlapping charged particles. By decomposing the image of the detector activity at the vertex through deep learning, we infer the kinematic parameters of the low-momentum particles and enhance the reconstructed energy resolution of the neutrino event. Finally, we combine our approach with a fully-differentiable generative model to improve the image decomposition further and the resolution of the measured parameters. This improvement is crucial to search for asymmetries between matter and antimatter.

Realistic reconstruction of trees from sparse images in volumetric space

The realistic reconstruction of real-world trees is a challenging task in the community of computer graphics because natural trees have complex structures of branches and leaves. Existing terrestrial laser scanning (TLS) system is able to capture dense and precise tree point clouds, yet the TLS system is expensive and not easy to carry around. An alternative low-cost and portable way is the reconstruction of tree point cloud from multiple view images. However, it is usually difficult to reconstruct a complete tree point cloud because of the texture similarity of branches and leaves as well as the lack of a sufficient number of images. Thus, we propose a new approach for reconstructing tree point clouds and geometries from sparse images. We first infer the camera parameters of each image, and then calculate the bounding volume of a tree from the camera parameters. Next, we set the mask of each image and the resolution of voxel, and then project each voxel in 3D space to all the mask images to determine the validity of the voxel. To alleviate the miss deletion of valid voxel, we utilize a boundary threshold and adjust mask resolution for robust point cloud reconstruction. Finally, an efficient tree reconstruction method is proposed to generate plausible tree geometries. We tested 6 different tree species that contain deciduous and evergreen trees, and the results showed that our approach is able to generate complete tree point cloud and realistic tree models even from a few number of images.

Three-Dimensional Array SAR Sparse Imaging Based on Hybrid Regularization

Three-Dimensional Array SAR Sparse Imaging Based on Hybrid Regularization

Georeferencing of Satellite Images with Geocoded Image Features

Abstract. Currently, using digital orthophoto map (DOM) and digital elevation model (DEM) as reference to achieve geometric positioning of newly acquired satellite images has become a popular photogrammetric approach. However, this method relies on DOM and DEM data which requires a lot of storage space in practical applications. In addition, for geometric positioning of satellite images, only sparse image feature points are needed as control points. Consequently, for the sake of convenience, the compression of control data emerges as a necessity with significant practical implications. This paper investigates a "cloud control" photogrammetry method based on geocoded image features. The method extracts SIFT feature points from DOMs, and obtains their ground coordinates, then constructs geocoded image feature library instead of DOM and DEM data as control, thus realizing the compression of control data. Experiments conducted on the Tianhui-1, Ziyuan-3 and Gaofen-2 satellite images demonstrate that the proposed method can achieve high-precision geometric positioning of satellite images and greatly reduce the size of the control data. Specifically, with the reduction of the reference data from 180~1248 MB 2 m DOM and 30 m DEM to 5~10 MB geocoded image features, the geopositional accuracies of the test Tianhui-1, Ziyuan-3 and Gaofen-2 images are improved from 3.12 pixels to 1.74 pixels, 3.69 pixels to 1.09 pixels, and 150.93 pixels to 2.67 pixels, respectively.

A sampling greedy average regularized Kaczmarz method for tensor recovery

AbstractRecently, a regularized Kaczmarz method has been proposed to solve tensor recovery problems. In this article, we propose a sampling greedy average regularized Kaczmarz method. This method can be viewed as a block or mini‐batch version of the regularized Kaczmarz method, which is based on averaging several regularized Kaczmarz steps with a constant or adaptive extrapolated step size. Also, it is equipped with a sampling greedy strategy to select the working tensor slices from the sensing tensor. We prove that our new method converges linearly in expectation and show that the sampling greedy strategy can exhibit an accelerated convergence rate compared to the random sampling strategy. Numerical experiments are carried out to show the feasibility and efficiency of our new method on various signal/image recovery problems, including sparse signal recovery, image inpainting, and image deconvolution.

A novel RCACycleGAN model is proposed for the high-precision reconstruction of sparse TFM images

The sparse total focusing method (TFM) has been shown to enhance the computational efficacy of ultrasound imaging but the image quality of ultrasound regrettably deteriorates with an increase in the sparsity rate of array elements. Deep learning has made remarkable advancements in image processing and cycle-consistent generative adversarial networks (CycleGANs) have been extensively employed to reconstruct diverse image categories. However, due to the incomplete extraction of image feature information by the generator and discriminator in a CycleGAN, high-quality sparse TFM images cannot be directly reconstructed using CycleGANs. There is also a risk of losing crucial feature information related to minor defects. As a result, this paper modifies the generator and discriminator in the CycleGAN to construct a new relativistic discriminator and coordinate attention CycleGAN (RCACycleGAN) model, which enables high-precision reconstruction of sparse TFM images. The addition of the coordinate attention module to the CycleGAN enhances the defective feature representation by fully considering the channel and spatial correlation between regions and using the fusion of spatially perceived feature maps in different directions. It solves the problem of easy loss of defective key feature information. The relativistic discriminator replaces the PatchGAN discriminator in the CycleGAN and evaluates the quality of both real and sparse TFM reconstructed images to ensure a relative image quality evaluation. This process solves the problem of unstable image quality of the sparse TFM reconstructed image. Experimental results demonstrate that RCACycleGAN can stably reconstruct sparse TFM images even in small sample dataset scenarios. The proposed network model reconstructs images with better accuracy, including in terms of structural similarity, defect roundness and area, and has a shorter training time than several existing network models.

S‐LASSIE: Structure and smoothness enhanced learning from sparse image ensemble for 3D articulated shape reconstruction

AbstractIn computer vision, the task of 3D reconstruction from monocular sparse images poses significant challenges, particularly in the field of animal modelling. The diverse morphology of animals, their varied postures, and the variable conditions of image acquisition significantly complicate the task of accurately reconstructing their 3D shape and pose from a monocular image. To address these complexities, we propose S‐LASSIE, a novel technique for 3D reconstruction of quadrupeds from monocular sparse images. It requires only 10–30 images of similar breeds for training. To effectively mitigate depth ambiguities inherent in monocular reconstructions, S‐LASSIE employs a multi‐angle projection loss function. In addition, our approach, which involves fusion and smoothing of bone structures, resolves issues related to disjointed topological structures and uneven connections at junctions, resulting in 3D models with comprehensive topologies and improved visual fidelity. Our extensive experiments on the Pascal‐Part and LASSIE datasets demonstrate significant improvements in keypoint transfer, overall 2D IOU and visual quality, with an average keypoint transfer and overall 2D IOU of 59.6% and 86.3%, respectively, which are superior to existing techniques in the field.

NeRF-NQA: No-Reference Quality Assessment for Scenes Generated by NeRF and Neural View Synthesis Methods.

Neural View Synthesis (NVS) has demonstrated efficacy in generating high-fidelity dense viewpoint videos using a image set with sparse views. However, existing quality assessment methods like PSNR, SSIM, and LPIPS are not tailored for the scenes with dense viewpoints synthesized by NVS and NeRF variants, thus, they often fall short in capturing the perceptual quality, including spatial and angular aspects of NVS-synthesized scenes. Furthermore, the lack of dense ground truth views makes the full reference quality assessment on NVS-synthesized scenes challenging. For instance, datasets such as LLFF provide only sparse images, insufficient for complete full-reference assessments. To address the issues above, we propose NeRF-NQA, the first no-reference quality assessment method for densely-observed scenes synthesized from the NVS and NeRF variants. NeRF-NQA employs a joint quality assessment strategy, integrating both viewwise and pointwise approaches, to evaluate the quality of NVS-generated scenes. The viewwise approach assesses the spatial quality of each individual synthesized view and the overall inter-views consistency, while the pointwise approach focuses on the angular qualities of scene surface points and their compound inter-point quality. Extensive evaluations are conducted to compare NeRF-NQA with 23 mainstream visual quality assessment methods (from fields of image, video, and light-field assessment). The results demonstrate NeRF-NQA outperforms the existing assessment methods significantly and it shows substantial superiority on assessing NVS-synthesized scenes without references. An implementation of this paper are available at https://github.com/VincentQQu/NeRF-NQA.

DPP: deep phase prior for parallel imaging with wave encoding

Objective. In Magnetic Resonance (MR) parallel imaging with virtual channel-expanded Wave encoding, limitations are imposed on the ability to comprehensively and accurately characterize the background phase. These limitations are primarily attributed to the calibration process relying solely on center low-frequency Auto-Calibration Signals (ACS) data for calibration. Approach. To tackle the challenge of accurately estimating the background phase in wave encoding, a novel deep neural network model guided by deep phase priors is proposed with integrated virtual conjugate coil (VCC) extension. Concretely, within the proposed framework, the background phase is implicitly characterized by employing a carefully designed decoder convolutional neural network, leveraging the inherent characteristics of phase smoothness and compact support in the transformed domain. Furthermore, the proposed model with wave encoding benefits from additional priors, which incorporate transmission sparsity of the latent image and coil sensitivity smoothness. Main results. Ablation experiments were conducted to ascertain the proposed method’s capability to implicitly represent CSM and the background phase. Subsequently, the superiority of the proposed method is demonstrated through confidence comparisons with competing methods, employing 4-fold and 5-fold acceleration experiments. In achieving 4-fold and 5-fold acceleration, the optimal quantitative metrics (PSNR/SSIM/NMSE) are 44.1359 dB/0.9863/0.0008 (4-fold) and 41.2074/0.9846/0.0017 (5-fold), respectively. Furthermore, the generalizability of the proposed method is further validated by conducting acceleration experiments with T1, T2, T2*, and various undersampling patterns. In addition, the DPP delivered much better performance than the conventional methods by exploring accelerated phase-sensitive SWI imaging. In SWI accelerated imaging, it also surpasses the optimal competing method in terms of (PSNR/SSIM/NMSE) with 0.096%/0.009%/0.0017%. Significance. The proposed method enables precise characterization of the background phase in the integrated VCC and wave encoding framework, supported via theoretical analysis and empirical findings. Our code is available at: https://github.com/sober235/DPP.

Improved Magnetic Resonance Image Reconstruction using Compressed Sensing and Adaptive Multi Extreme Particle Swarm Optimization Algorithm

One powerful technique that can offer a thorough examination of the body's internal structure is magnetic resonance imaging (MRI). MRI's lengthy acquisition times, however, may restrict its clinical usefulness, particularly in situations where time is of the essence. Compressed sensing (CS) has emerged as a potentially useful method for cutting down on MRI acquisition times; nevertheless, the effectiveness of CS-MRI is dependent on the selection of the sparsity-promoting algorithm and sampling scheme. This research paper presents a novel method based on adaptive multi-extreme particle swarm optimization (AMEPSO) and dual tree complex wavelet transform (DTCWT) for fast image acquisition in magnetic resonance. The method uses AMEPSO in order to maximize the sampling pattern and minimize reconstruction error, while also exploiting the sparsity of MR images in the DTCWT domain to improve directional selectivity and shift invariance. MATLAB software was used for simulation of the proposed method. In comparison with the particle swarm optimized-DTCWT (PSODTCWT) and DTCWT algorithms, respectively, the results demonstrated an improvement in the peak signal-to-noise ratio of 8.92% and 15.92% and a higher structural similarity index measure of 3.69% and 7.5%. Based on these improvements, the proposed method could potentially make high-quality, real-time MRI imaging possible, which might improve detection and treatment of medical conditions and increase the throughput of MRI machines.

Increasing SAR Imaging Precision for Burden Surface Profile Jointly Using Low-Rank and Sparsity Priors

The synthetic aperture radar (SAR) imaging technique for a frequency-modulated continuous wave (FMCW) has attracted wide attention in the field of burden surface profile measurement. However, the imaging data are virtually under-sampled due to the severely restricted scan time, which prevents the antenna being exposed to high temperatures and heavy dust in the blast furnace (BF) for an extended period. In traditional SAR imaging algorithm research, the insufficient accumulation of scattered energy in reconstructing the burden surface profile leads to lower imaging precision, and the harsh smelting increases the probability of distortion in shape detection. In this study, to address these challenges, a novel rotating SAR imaging algorithm based on the constructed mechanical swing radar system is proposed. This algorithm is inspired by the low-rank property of the sampled signal matrix and the sparsity of burden surface profile images. First, the sparse FMCW signal is modeled, and the position transform matrix, calculated according to the BF dimensions, is embedded into the dictionary matrix. Then, the low-rank and sparsity priors are considered and reformulated as split variables in order to establish a convex optimization problem. Lastly, the augmented Lagrange multiplier (ALM) is employed to solve this problem under double constraints, and the imaging results are obtained using the alternating direction method of multipliers (ADMM). The experimental results demonstrate that, in the subsequent shape detection, the root mean square error (RMSE) is 15.38% lower than the previous algorithm and 15.63% lower under low signal-to-noise (SNR) conditions. In both enclosed and harsh environments, the proposed algorithm is able to achieve higher imaging precision even under high noise. It will be further optimized for speed and reliability, with plans to extend its application to 3D measurements in the future.