Low-quality Images Research Articles

Anatomical sites identification in both ordinary and capsule gastroduodenoscopy via deep learning

Anatomical sites recognition is a basic requirement for gastroenterologists. But there is not a unified framework for anatomical sites identification in both ordinary and capsule endoscopes. Deep learning (DL), especially vision transformer (ViT), is promising in the field of medical imaging, however, the performance of them is not comprehensively compared. The retrospective cohort study included 322 patients who visited Friendship hospital for capsule endoscopy and 556 patients who visited Minhang hospital for ordinary endoscopy. Convolutional neural network (CNN) and two types of ViT (B/16 and L/32) that was trained to identify qualified and low-quality images (the first model), then the qualified images were used to train the second model for distinguishing different anatomical sites. 62,850 images from capsule endoscopy and 17,434 images from ordinary endoscopy were included in developing models. In internal cross-validation, CNN achieved average area under receiver operating characteristic curve (AUROC) of 0.9844 (95 % confidence interval [CI] 0.9640–0.9960) in distinguishing qualified and low-quality images, and 0.9251 average accuracy (95 % CI 0.9133–0.9369) in distinguishing different anatomical sites. Besides, the performance of ViT did not surpass the performance of CNN. 18,636 images from 355 patients who received capsule endoscopy and 15,949 images of 501 patients who received ordinary endoscopy were prospectively collected. The AUROC of CNN reached 0.8715 (95 % CI 0.8674–0.8754) in the first model, and 0.8376 accuracy (95 % CI 0.8336–0.8414) for the second model, respectively. The performance of CNN is better than ViT with the same hyperparameter setting for sifting out the unqualified images and distinguishing anatomical sites effectively.

Data-driven coordinated attention deep learning for high-fidelity brain imaging denoising and inpainting.

Deep learning offers promise in enhancing low-quality images by addressing weak fluorescence signals, especially in deep in vivo mouse brain imaging. However, current methods struggle with photon scarcity and noise within in vivo deep mouse brains, and often neglecting tissue preservation. In this study, we propose an innovative in vivo cortical fluorescence image restoration approach, combining signal enhancement, denoising, and inpainting. We curated a deep brain cortical image dataset and developed a novel deep brain coordinate attention restoration network (DeepCAR), integrating coordinate attention with optimized residual networks. Our method swiftly and accurately restores deep cortex images exceeding 800 μm, preserving small-scale tissue structures. It boosts the peak signal-to-noise ratio (PSNR) by 6.94 dB for weak signals and 11.22 dB for large noisy images. Crucially, we validate the effectiveness on external datasets with diverse noise distributions, structural features compared to those in our training data, showcasing real-time high-performance image restoration capabilities.

BEAS-Net: a Shape-Prior-Based Deep Convolutional Neural Network for Robust Left Ventricular Segmentation in 2D Echocardiography.

Left ventricle (LV) segmentation of 2D echocardiography images is an essential step in the analysis of cardiac morphology and function and - more generally - diagnosis of cardiovascular diseases. Several deep learning (DL) algorithms have recently been proposed for the automatic segmentation of the LV, showing significant performance improvement over the traditional segmentation algorithms. However, unlike the traditional methods, prior information about the segmentation problem, e.g. anatomical shape information, is not usually incorporated for training the DL algorithms. This can degrade the generalization performance of the DL models on unseen images if their characteristics are somewhat different from those of the training images, e.g. low-quality testing images. In this study, a new shape-constrained deep convolutional neural network (CNN) - called BEAS-Net - is introduced for automatic LV segmentation. The BEAS-Net learns how to associate the image features, encoded by its convolutional layers, with anatomical shape-prior information derived by the B-spline explicit active surface (BEAS) algorithm to generate physiologically meaningful segmentation contours when dealing with artifactual or low-quality images. The performance of the proposed network was evaluated using three different in-vivo datasets and was compared a deep segmentation algorithm based on the U-Net model. Both networks yielded comparable results when tested on images of acceptable quality, but the BEAS-Net outperformed the benchmark DL model on artifactual and low-quality images.

Flexible Iris Matching Based on Spatial Feature Reconstruction

In an iris recognition-at-a-distance (IAAD) system used in surveillance scenarios, the camera usually captures a large number of low-quality images. These images exhibit partial occlusions due to eyelids and eyelashes, specular reflections, and severe deformations caused by pupil dilations and contractions. Recognizing these low-quality images is a challenging yet dominant problem in IAAD. To mitigate this issue, current iris recognition systems mostly filter out low-quality images by using strict criteria based on image quality evaluation. This strategy, however, wastes device capabilities and produces low throughput of subjects. Other systems require highly cooperative users. In this work, we propose a novel occlusion-robust, deformation-robust, and alignment-free framework for low-quality iris matching, which integrates the merits of deep features and sparse representation in an end-to-end learning process known as iris spatial feature reconstruction (ISFR). Here each probe image can be sparsely reconstructed on the basis of appropriate feature maps from gallery high-quality images. ISFR uses the error from robust reconstruction over spatial pyramid features to measure similarities between two iris images, which naturally avoids the time-consuming alignment step. In summary, the distinctiveness of deep features, the robustness of sparse reconstruction, and the flexibility of multiscale matching strategy are unified in a general framework to attain more accurate and reasonable iris matching. Extensive experimental results on four public iris image databases demonstrate that the proposed method significantly outperforms both traditional and deep learning-based iris recognition methods.

Deep Power-aware Tunable Weighting for Ultrasound Microvascular Imaging.

Ultrasound microvascular imaging (UMI), including ultrafast power Doppler imaging (uPDI) and ultrasound localization microscopy (ULM), obtains blood flow information through plane wave transmissions at high frame rates. However, low signal-to-noise ratio of plane waves causes low image quality. Adaptive beamformers have been proposed to suppress noise energy to achieve higher image quality accompanied by increasing computational complexity. Deep learning (DL) leverages powerful hardware capabilities to enable rapid implementation of noise suppression at the cost of flexibility. To enhance the applicability of DL-based methods, in this work, we propose a deep power-aware tunable (DPT) weighting (i.e., postfilter) for delay-and-sum (DAS) beamforming to improve UMI by enhancing plane wave images. The model, called Yformer is a hybrid structure combining convolution and Transformer. With the DAS beamformed and compounded envelope image as input, Yformer can estimate both noise power and signal power. Furthermore, we utilize the obtained powers to compute pixel-wise weights by introducing a tunable noise control factor, which is tailored for improving the quality of different UMI applications. In vivo experiments on the rat brain demonstrate that Yformer can accurately estimate the powers of noise and signal with the structural similarity index (SSIM) higher than 0.95. The performance of the DPT weighting is comparable to that of superior adaptive beamformer in uPDI with low computational cost. The DPT weighting was then applied to four different datasets of ULM, including public simulation, public rat brain, private rat brain, and private rat liver datasets, showing excellent generalizability using the model trained by the private rat brain dataset only. In particular, our method indirectly improves the resolution of liver ULM from 25.24 μm to 18.77 μm by highlighting small vessels. In addition, the DPT weighting exhibits more details of blood vessels with faster processing, which has the potential to facilitate the clinical applications of high-quality UMI.

Accelerated multiple-quantum-filtered sodium magnetic resonance imaging using compressed sensing at 7 T

PurposeMultiple-quantum-filtered (MQF) sodium magnetic resonance imaging (MRI), such as enhanced single-quantum and triple-quantum-filtered imaging of 23Na (eSISTINA), enables images to be weighted towards restricted sodium, a promising biomarker in clinical practice, but often suffers from clinically infeasible acquisition times and low image quality. This study aims to mitigate the above limitation by implementing a novel eSISTINA sequence at 7 T with the application of compressed sensing (CS) to accelerate eSISTINA acquisitions without a noticeable loss of information. MethodsA novel eSISTINA sequence with a 3D spiral-based sampling scheme was implemented at 7 T for the application of CS. Fully sampled datasets were obtained from one phantom and ten healthy subjects, and were then retrospectively undersampled by various undersampling factors. CS undersampled reconstructions were compared to fully sampled and undersampled nonuniform fast Fourier transform (NUFFT) reconstructions. Reconstruction performance was evaluated based on structural similarity (SSIM), signal-to-noise ratio (SNR), weightings towards total and compartmental sodium, and in vivo quantitative estimates. ResultsCS-based phantom and in vivo images have less noise and better structural delineation while maintaining the weightings towards total, non-restricted (predominantly extracellular), and restricted (primarily intracellular) sodium. CS generally outperforms NUFFT with a higher SNR and a better SSIM, except for the SSIM in TQ brain images, which is likely due to substantial noise contamination. CS enables in vivo quantitative estimates with <15% errors at an undersampling factor of up to two. ConclusionsSuccessful implementation of an eSISTINA sequence with an incoherent sampling scheme at 7 T was demonstrated. CS can accelerate eSISTINA by up to twofold at 7 T with reduced noise levels compared to NUFFT, while maintaining major structural information, reasonable weightings towards total and compartmental sodium, and relatively reliable in vivo quantification. The associated reduction in acquisition time has the potential to facilitate the clinical applicability of MQF sodium MRI.

Deep Learning-Based Cell Tracking in Deforming Organs and Moving Animals.

Cell tracking is an essential step in extracting cellular signals from moving cells, which is vital for understanding the mechanisms underlying various biological functions and processes, particularly in organs such as the brain and heart. However, cellsin living organisms often exhibit extensive and complex movements caused by organ deformation and whole-body motion. These movementspose a challenge in obtaining high-quality time-lapse cell images and tracking the intricate cell movements in thecaptured images. Recent advances in deep learning techniques provide powerful tools for detecting cells in low-quality images with densely packed cell populations, as well as estimating cell positions for cells undergoing large nonrigid movements. This chapter introduces the challenges of cell tracking in deforming organs and moving animals, outlinesthe solutions to these challenges, and presents a detailed protocol for data preparation, as well as for performing cell segmentation and tracking using the latest version of 3DeeCellTracker. This protocol is expected to enable researchers to gain deeper insights into organ dynamics and biological processes.

Learning Cruxes to Push for Object Detection in Low-Quality Images

Learning Cruxes to Push for Object Detection in Low-Quality Images

Texture-guided transfer learning for low-quality face recognition.

Although many advanced works have achieved significant progress for face recognition with deep learning and large-scale face datasets, low-quality face recognition remains a challenging problem in real-word applications, especially for unconstrained surveillance scenes. We propose a texture-guided (TG) transfer learning approach under the knowledge distillation scheme to improve low-quality face recognition performance. Unlike existing methods in which distillation loss is built on forward propagation; e.g., the output logits and intermediate features, in this study, the backward propagation gradient texture is used. More specifically, the gradient texture of low-quality images is forced to be aligned to that of its high-quality counterpart to reduce the feature discrepancy between the high-and low-quality images. Moreover, attention is introduced to derive a soft-attention (SA) version of transfer learning, termed as SA-TG, to focus on informative regions. Experiments on the benchmark low-quality face DB's TinyFace and QMUL-SurFace confirmed the superiority of the proposed method, especially more than 6.3% Rank1 accuracy improvement is achieved on TinyFace.

Quantization Step Estimation of Color Images Based on Res2Net-C With Frequency Clustering Prior Knowledge

The quantization step is a crucial parameter in JPEG compression, that can reveal the compression history of a JPEG image. Estimating the quantization steps for single compressed and recompressed images is attracting considerable interest in the field of image forensics and steganalysis. Several effective methods have been proposed, but the performance of these methods still needs to be improved on small-sized and low-quality images. To solve the above problems, feature enrichment is performed on images in the frequency domain, resulting in clustering discrete cosine transform (DCT) coefficients of the same frequency. Then, we construct a hierarchical connection within the residual blocks of the network to represent multi-scale features, enabling the network to learn deep features of the image. At the same time, we use multiple small-sized convolution kernels instead of one large-sized convolution kernel to minimize the impact of block artifacts. Based on the above two ideas, we construct a network model, Res2Net-C, to discover information about the quantization steps in the frequency domain. The integration of multi-channel information of color images is achieved by multi-channel convolution, and the quantization steps of the chrominance and luminance channels of the color images are estimated. The experimental results show that the accuracy of the proposed method for estimating the quantization steps is 29.97% better than that of the existing algorithm with a single compressed dataset and 4.87% better than that of the existing algorithm with a recompressed image dataset. In addition, the method has good performance with mixed datasets that contain both single compressed and recompressed images.

AIROGS: Artificial Intelligence for Robust Glaucoma Screening Challenge.

The early detection of glaucoma is essential in preventing visual impairment. Artificial intelligence (AI) can be used to analyze color fundus photographs (CFPs) in a cost-effective manner, making glaucoma screening more accessible. While AI models for glaucoma screening from CFPs have shown promising results in laboratory settings, their performance decreases significantly in real-world scenarios due to the presence of out-of-distribution and low-quality images. To address this issue, we propose the Artificial Intelligence for Robust Glaucoma Screening (AIROGS) challenge. This challenge includes a large dataset of around 113,000 images from about 60,000 patients and 500 different screening centers, and encourages the development of algorithms that are robust to ungradable and unexpected input data. We evaluated solutions from 14 teams in this paper and found that the best teams performed similarly to a set of 20 expert ophthalmologists and optometrists. The highest-scoring team achieved an area under the receiver operating characteristic curve of 0.99 (95% CI: 0.98-0.99) for detecting ungradable images on-the-fly. Additionally, many of the algorithms showed robust performance when tested on three other publicly available datasets. These results demonstrate the feasibility of robust AI-enabled glaucoma screening.

Enhancing bladder cancer diagnosis through transitional cell carcinoma polyp detection and segmentation: an artificial intelligence powered deep learning solution.

Bladder cancer, specifically transitional cell carcinoma (TCC) polyps, presents a significant healthcare challenge worldwide. Accurate segmentation of TCC polyps in cystoscopy images is crucial for early diagnosis and urgent treatment. Deep learning models have shown promise in addressing this challenge. We evaluated deep learning architectures, including Unetplusplus_vgg19, Unet_vgg11, and FPN_resnet34, trained on a dataset of annotated cystoscopy images of low quality. The models showed promise, with Unetplusplus_vgg19 and FPN_resnet34 exhibiting precision of 55.40 and 57.41%, respectively, suitable for clinical application without modifying existing treatment workflows. Deep learning models demonstrate potential in TCC polyp segmentation, even when trained on lower-quality images, suggesting their viability in improving timely bladder cancer diagnosis without impacting the current clinical processes.

Assessment of renal quality with quantitative contrast-enhanced ultrasound (CEUS) for differentiating kidney histopathology before procurement.

Purpose: This study aimed to investigate the use of contrast-enhanced ultrasonography (CEUS) to assess the kidneys' quality before procurement. Methods: This prospective study included 74 donors and 148 recipients of kidneys. 119 kidneys underwent quantitative analysis. Before organ procurement, potential kidney donors underwent CEUS, though organ procurement involved a zero-point puncture biopsy. CEUS parameters of the renal cortex and medulla were evaluated, including rise time (RT), time to peak (TTP), the area under the curve (AUC), wash-in slope (WIS), peak intensity (PI), and mean transit time (MTT). Donors' kidneys were classified based on their pathological. Additionally, short-term clinical indicators of renal recipients were collected and analyzed to determine whether the patients had delayed recovery of renal allograft function. Results: This experiment included 148 cases of kidney information, divided into two groups based on the Remuzzi score of the kidneys. However, 29 kidneys were excluded from the quantitative analysis due to loss or low quality of CEUS images. Comparing the time-intensity curve (TIC) of renal cortical region of interest (ROI), we found that the group with lower pathological scores exhibited higher PI (P=0.002), AUC(P=0.003), and WIS (P=0.009). TIC comparison results for renal medulla ROI revealed that the group with lower pathological scores had higher PI (P=0.010), AUC (P=0.023), and WIS (P=0.024). Conclusions: This study highlighted the potential of CEUS as a non-invasive, safe, and real-time examination method that correlates with the Remuzzi score and renal pathology. Therefore, it can be used as a prospective preoperative non-invasive evaluation method for the donor's kidney.

Investigating the Impact of Compressed Sensing Techniques and IoT in Medical Imaging

This research paper examines compressed sensing's impact on medical imaging. Math and signal processing inspired compressed sensing. Future picture-capturing will be radically different. The paper focuses on adaptive random sampling (ARS), iterative shrinkage-thresholding algorithms (ISTA), and temporal compressed sensing (TCS). These approaches were rigorously tested using MRIs, X-rays, and dynamic imaging patterns. Low scan times, picture quality, and dynamic imaging were the main test criteria. The technologies considerably reduced scan time, demonstrating their potential to speed up imaging procedures. The reconstructed photos had higher SNRs and SSIs than those obtained using normal techniques, indicating greater accuracy. The TCS algorithm's dynamic imaging skills, especially evident in heart and musculoskeletal imaging, eliminated motion defects while exhibiting real-time physiological changes. The study was expanded to incorporate customized treatment, and the recommended procedures have proven amazing adaptability to each patient's demands. This adaptability fits current medical treatments, making unique imaging technologies viable.

Explainable AI for Medical Image Analysis in Medical Cyber-Physical Systems: Enhancing Transparency and Trustworthiness of IoMT.

Medical image analysis plays a crucial role in healthcare systems of Internet of Medical Things (IoMT), aiding in the diagnosis, treatment planning, and monitoring of various diseases. With the increasing adoption of artificial intelligence (AI) techniques in medical image analysis, there is a growing need for transparency and trustworthiness in decision-making. This study explores the application of explainable AI (XAI) in the context of medical image analysis within medical cyber-physical systems (MCPS) to enhance transparency and trustworthiness. To this end, this study proposes an explainable framework that integrates machine learning and knowledge reasoning. The explainability of the model is realized when the framework evolution target feature results and reasoning results are the same and are relatively reliable. However, using these technologies also presents new challenges, including the need to ensure the security and privacy of patient data from IoMT. Therefore, attack detection is an essential aspect of MCPS security. For the MCPS model with only sensor attacks, the necessary and sufficient conditions for detecting attacks are given based on the definition of sparse observability. The corresponding attack detector and state estimator are designed by assuming that some IoMT sensors are under protection. It is expounded that the IoMT sensors under protection play an important role in improving the efficiency of attack detection and state estimation. The experimental results show that the XAI in the context of medical image analysis within MCPS improves the accuracy of lesion classification, effectively removes low-quality medical images, and realizes the explainability of recognition results. This helps doctors understand the logic of the system's decision-making and can choose whether to trust the results based on the explanation given by the framework.

Fundus Image Enhancement Through Direct Diffusion Bridges.

We propose FD3, a fundus image enhancement method based on direct diffusion bridges, which can cope with a wide range of complex degradations, including haze, blur, noise, and shadow. We first propose a synthetic forward model through a human feedback loop with board-certified ophthalmologists for maximal quality improvement of low-quality in-vivo images. Using the proposed forward model, we train a robust and flexible diffusion-based image enhancement network that is highly effective as a stand-alone method, unlike previous diffusion model-based approaches which act only as a refiner on top of pre-trained models. Through extensive experiments, we show that FD3 establishes superior quality not only on synthetic degradations but also on in vivo studies with low-quality fundus photos taken from patients with cataracts or small pupils. To promote further research in this area, we open-source all our code and data used for this research at https://github.com/heeheee888/FD3.

Robust Fast Inter-Bin Image Registration for Undersampled Coronary MRI Based on a Learned Motion Prior.

To propose a 3D nonrigid registration method that accurately estimates the 3D displacement field from artifact-corrupted Coronary Magnetic Resonance Angiography (CMRA) images. We developed a novel registration framework for registration of artifact-corrupted images based on a 3D U-Net initializer and a deep unrolling network. By leveraging a supervised learning framework with training labels estimated from fully-sampled images, the unrolling network learns a task-specific motion prior which reduces motion estimation biases caused by undersampling artifacts in the source images. We evaluated the proposed method, UNROLL, against an iterative Free-Form Deformation (FFD) registration method and a recently proposed Respiratory Motion Estimation network (RespME-net) for 6-fold (in-distribution) and 11-fold (out-of-distribution) accelerated CMRA. Compared to the baseline methods, UNROLL improved both the accuracy of motion estimation and the quality of motion-compensated CMRA reconstruction at 6-fold acceleration. Furthermore, even at 11-fold acceleration, which was not included during training, UNROLL still generated more accurate displacement fields than the baseline methods. The computational time of UNROLL for the whole 3D volume was only 2 seconds. By incorporating a learned respiratory motion prior, the proposed method achieves highly accurate motion estimation despite the large acceleration. This work introduces a fast and accurate method to estimate the displacement field from low-quality source images. It has the potential to significantly improve the quality of motion-compensated reconstruction for highly accelerated 3D CMRA.

SSP-Net: A Siamese-Based Structure-Preserving Generative Adversarial Network for Unpaired Medical Image Enhancement.

Recently, unpaired medical image enhancement is one of the important topics in medical research. Although deep learning-based methods have achieved remarkable success in medical image enhancement, such methods face the challenge of low-quality training sets and the lack of a large amount of data for paired training data. In this article, a dual input mechanism image enhancement method based on Siamese structure (SSP-Net) is proposed, which takes into account the structure of target highlight (texture enhancement) and background balance (consistent background contrast) from unpaired low-quality and high-quality medical images. Furthermore, the proposed method introduces the mechanism of the generative adversarial network to achieve structure-preserving enhancement by jointly iterating adversarial learning. Experiments comprehensively illustrate the performance in unpaired image enhancement of the proposed SSP-Net compared with other state-of-the-art techniques.

HQG-Net: Unpaired Medical Image Enhancement With High-Quality Guidance.

Unpaired medical image enhancement (UMIE) aims to transform a low-quality (LQ) medical image into a high-quality (HQ) one without relying on paired images for training. While most existing approaches are based on Pix2Pix/CycleGAN and are effective to some extent, they fail to explicitly use HQ information to guide the enhancement process, which can lead to undesired artifacts and structural distortions. In this article, we propose a novel UMIE approach that avoids the above limitation of existing methods by directly encoding HQ cues into the LQ enhancement process in a variational fashion and thus model the UMIE task under the joint distribution between the LQ and HQ domains. Specifically, we extract features from an HQ image and explicitly insert the features, which are expected to encode HQ cues, into the enhancement network to guide the LQ enhancement with the variational normalization module. We train the enhancement network adversarially with a discriminator to ensure the generated HQ image falls into the HQ domain. We further propose a content-aware loss to guide the enhancement process with wavelet-based pixel-level and multiencoder-based feature-level constraints. Additionally, as a key motivation for performing image enhancement is to make the enhanced images serve better for downstream tasks, we propose a bi-level learning scheme to optimize the UMIE task and downstream tasks cooperatively, helping generate HQ images both visually appealing and favorable for downstream tasks. Experiments on three medical datasets verify that our method outperforms existing techniques in terms of both enhancement quality and downstream task performance. The code and the newly collected datasets are publicly available at https://github.com/ChunmingHe/HQG-Net.

CNNs trained with adult data are useful in pediatrics. A pneumonia classification example.

The scarcity of data for training deep learning models in pediatrics has prompted questions about the feasibility of employing CNNs trained with adult images for pediatric populations. In this work, a pneumonia classification CNN was used as an exploratory example to showcase the adaptability and efficacy of such models in pediatric healthcare settings despite the inherent data constraints. To develop a curated training dataset with reduced biases, 46,947 chest X-ray images from various adult datasets were meticulously selected. Two preprocessing approaches were tried to assess the impact of thoracic segmentation on model attention outside the thoracic area. Evaluation of our approach was carried out on a dataset containing 5,856 chest X-rays of children from 1 to 5 years old. An analysis of attention maps indicated that networks trained with thorax segmentation placed less attention on regions outside the thorax, thus eliminating potential bias. The ensuing network exhibited impressive performance when evaluated on an adult dataset, achieving a pneumonia discrimination AUC of 0.95. When tested on a pediatric dataset, the pneumonia discrimination AUC reached 0.82. The results of this study show that adult-trained CNNs can be effectively applied to pediatric populations. This could potentially shift focus towards validating adult models over pediatric population instead of training new CNNs with limited pediatric data. To ensure the generalizability of deep learning models, it is important to implement techniques aimed at minimizing biases, such as image segmentation or low-quality image exclusion.