Image Degradation Research Articles

Optimal transmit apodization for the maximization of lag-one coherence with applications to aberration delay estimation

Phase aberration is one of the major sources of image degradation in medical ultrasound imaging. One of the earliest and simplest techniques to correct for phase aberration involves nearest-neighbor cross correlation to estimate delays between neighboring receive channels and the compensation of aberration delays in a delay-and-sum beamformer. The main challenge is that neighboring receive channels may not have sufficient signal correlation to accurately estimate the aberration delays. Although algorithms such as the translating transmit aperture or the common midpoint gather are designed to perfectly maximize signal correlations between received signals, these algorithms require the use of different transmit apertures for each received signal. Instead, this work proposes the use of a single globally-applicable transmit apodization function that optimizes the lag-one coherence based on the van Cittert–Zernike theorem. For the application to phase aberration correction, it is shown across 20 different zero-mean Gaussian-random aberrators that the proposed optimal apodization function reduces the estimation error in the aberration delay profile from 22.85% to 15.72%.

Spatial aliasing quantification and sampling frequency selection in imaging sensors.

Sampling, whether it be spatial or temporal, is a common occurrence. A result of this fact is the need for an anti-aliasing filter, which effectively limits high frequencies and prevents them from folding over and appearing as a low(er) frequency when sampled. In typical imaging sensors, such as optics plus focal plane detector(s), the optical transfer function (OTF) acts as a spatial anti-aliasing filter. However, decreasing this anti-aliasing cutoff frequency (or lowering the curve in general) via the OTF is tantamount to image degradation. On the other hand, the lack of high-frequency attenuation produces aliasing within the image, which is another form of image degradation. In this work, aliasing is quantified, and a method for sampling frequency selection is brought forth.

Correction of B0 and linear eddy currents: Impact on morphological and quantitative ultrashort echo time double echo steady state (UTE-DESS) imaging.

The purpose of the current study was to investigate the effects of B0 and linear eddy currents on ultrashort echo time double echo steady state (UTE-DESS) imaging and to determine whether eddy current correction (ECC) effectively resolves imaging artifacts caused by eddy currents. 3D UTE-DESS sequences based on either projection radial or spiral cones trajectories were implemented on a 3-T clinical MR scanner. An off-isocentered thin-slice excitation approach was used to measure eddy currents. The measurements were repeated four times using two sets of tested gradient waveforms with opposite polarities and two different slice locations to measure B0 and linear eddy currents simultaneously. Computer simulation was performed to investigate the eddy current effect. Finally, a phantom experiment, an ex vivo experiment with human synovium and ankle samples, and an in vivo experiment with human knee joints, were performed to demonstrate the effects of eddy currents and ECC in UTE-DESS imaging. In a computer simulation, the two echoes (S+ and S-) in UTE-DESS imaging exhibited strong distortion at different orientations in the presence of B0 and linear eddy currents, resulting in both image degradation as well as misalignment of pixel location between the two echoes. The same phenomenon was observed in the phantom, ex vivo, and in vivo experiments, where the presence of eddy currents degraded S+, S-, echo subtraction images, and T2 maps. The implementation of ECC dramatically improved both the image quality and image registration between the S+ and S- echoes. It was concluded that ECC is crucial for reliable morphological and quantitative UTE-DESS imaging.

Effective method for low-light image enhancement based on the JND and OCTM models.

Low-light images always suffer from dim overall brightness, low contrast, and low dynamic ranges, thus result in image degradation. In this paper, we propose an effective method for low-light image enhancement based on the just-noticeable-difference (JND) and the optimal contrast-tone mapping (OCTM) models. First, the guided filter decomposes the original images into base and detail images. After this filtering, detail images are processed based on the visual masking model to enhance details effectively. At the same time, the brightness of base images is adjusted based on the JND and OCTM models. Finally, we propose a new method to generate a sequence of artificial images to adjust the brightness of the output, which has a better performance in image detail preservation compared with other single-input algorithms. Experiments have demonstrated that the proposed method not only achieves low-light image enhancement, but also outperforms state-of-the-art methods qualitatively and quantitatively.

Dual image degradation repair network based on adaptive multifeature fusion

摘要 当前图像修复方法大多局限于处理某个单一特定任务,比如:超分辨率、去噪、着色等,很少有网络模型同时具备处理双重退化的能力.而现存可以解决多重图像退化问题的算法普遍结构复杂、训练时间长、人力成本较大.本文提出一种基于自适应多特征融合的双重图像退化修复网络——AMFNet(Adaptive Multi-Feature Fusion Dual Degradation Restoration Network),利用自引导模块(SGM)融合图像的多尺度信息,有效去除了图像中的部分缺陷;使用带有空洞卷积的编码解码器模块巩固图像的语义信息,实现了中间图像的着色;引入带有自适应多特征融合模块(AMF)的中间信息传输机制(ITM)链接以上两大结构,自适应选择保留网络递进过程的图像特征以避免有用信息的丢失.实验结果表明,基于自适应多特征融合的双重图像退化修复网络模型视觉生成效果最优,通过在CelebA和Landscape数据集上的测试分析,其结构相似度(SSIM)与感知图像补丁相似度(LPIPS)优于同类方法,而峰值信噪比(PSNR)则远超同类方法高达5dB.

Framework of degraded image restoration and simultaneous localization and mapping for multiple bad weather conditions

Image degradation caused by bad weather, such as haze, rain, and snow, significantly degrades a vision system’s performance, imposing challenges to the autonomous motion of mobile robots. Therefore, it is crucial to restore the degraded images for mobile robots operating outdoors. Spurred by this concern, a framework that restores degraded images and affords simultaneous localization and mapping (SLAM) under multiple bad weather conditions is presented. Specifically, the developed architecture combines the advantages of convolutional neural network features and weather features. It improves the accuracy of identifying the degradation type by developing a weather inference module based on ensemble learning. According to the internal mechanism of image degradation, a degraded image restoration method is proposed utilizing physical models, with a subsequent operation refining the preliminary restored results. To improve our method’s adaptability to real scenes, unpaired real-world weather images are introduced into the degradation removal algorithm through generative adversarial networks. The proposed weather persistent assumption combines weather inference and degradation restoration modules in the SLAM system to capture multiple weather conditions, which improves the system’s accuracy and running speed. Comprehensive experiments evaluating the developed framework and its components highlight that the proposed weather inference and degraded image restoration methods achieve a highly appealing effect. The final experimental results demonstrate that our framework autonomously identifies weather types, triggers the corresponding restoration, and realizes accurate localization in multiple bad weather conditions. It affords SLAM accuracy under multiple bad weather conditions that is close to raw SLAM in clear weather.

Aircraft Target Detection in Low Signal-to-Noise Ratio Visible Remote Sensing Images

With the increasing demand for the wide-area refined detection of aircraft targets, remote sensing cameras have adopted an ultra-large area-array detector as a new imaging mode to obtain broad width remote sensing images (RSIs) with higher resolution. However, this imaging technology introduces new special image degradation characteristics, especially the weak target energy and the low signal-to-noise ratio (SNR) of the image, which seriously affect the target detection capability. To address the aforementioned issues, we propose an aircraft detection method for RSIs with low SNR, termed L-SNR-YOLO. In particular, the backbone is built blending a swin-transformer and convolutional neural network (CNN), which obtains multiscale global and local RSI information to enhance the algorithm’s robustness. Moreover, we design an effective feature enhancement (EFE) block integrating the concept of nonlocal means filtering to make the aircraft features significant. In addition, we utilize a novel loss function to optimize the detection accuracy. The experimental results demonstrate that our L-SNR-YOLO achieves better detection performance in RSIs than several existing advanced methods.

Study of compatibility between a 3T MR system and detector modules for a second-generation RF-penetrable TOF-PET insert for simultaneous PET/MRI.

Simultaneous positron emission tomography/magnetic resonance imaging (PET/MRI) has shown promise in acquiring complementary multiparametric information of disease. However, designing these hybrid imaging systems is challenging due to the propensity for mutual interference between the PET and MRI subsystems. Currently, there are integrated PET/MRI systems for clinical applications. For neurologic imaging, a brain-dedicated PET insert provides superior spatial resolution and sensitivity compared to body PETscanners. Our first-generation prototype brain PET insert ("PETcoil") demonstrated RF-penetrability and MR-compatibility. In the second-generation PETcoil system, all analog silicon photomultiplier (SiPM) signal digitization is moved inside the detectors, which results in substantially better PET detector performance, but presents a greater technical challenge for achieving MR-compatibility. In this paper, we report results from MR-compatibility studies of two fully assembled second-generation PET insert detectormodules. We studied the effect of the presence of the two second-generation TOF-PET insert detectors on parameters that affect MR image quality and evaluated TOF-PET detector performance under different MRI pulse sequenceconditions. With the presence of operating PET detectors, no RF noise peaks were induced in the MR images, but the relative average noise level was increased by 15%, which led to a 3.1 to 4.2-dB degradation in MR image signal-to-noise ratio (SNR). The relative homogeneity of MR images degraded by less than 1.5% with the two operating TOF-PET detectors present. The reported results also indicated that ghosting artifacts (percent signal ghosting (PSG) ⩽ 1%) and MR susceptibility artifacts (0.044 ppm) were insignificant. The PET detector data showed a relative change of less than 5% in detector module performance between running outside and within the MR bore under different MRI pulse sequences except for energy resolution in EPI sequence (13% relative difference). The PET detector operation did not cause any significant artifacts in MR images and the performance and time-of-flight (TOF) capability of the former were preserved under different tested MRconditions.

UHA‐CycleGAN: Unpaired hybrid attention network based on CycleGAN for terahertz image super‐resolution

AbstractIn recent years, terahertz imaging technology has been widely used in security, medicine, and other fields. However, the image resolution is low due to the limits of imaging equipment and diffraction. Traditional super‐resolution methods based on machine learning use artificial paired image datasets, and their image degradation process is quite different from the actual terahertz imaging mechanism. In the paper, an unpaired hybrid attention network is proposed, using the real unpaired high‐resolution and low‐resolution terahertz images as the original input, two sub‐networks are trained: degenerate reconstruction network and super‐resolution network. The degenerate reconstruction network is used to learn the degradation process of the real terahertz imaging system, and reconstruct the generated degradation images into the fixed mapping down‐sampled image of high‐resolution terahertz image. The super‐resolution network uses the generated paired image with fixed correspondence to train the super‐resolution network of paired terahertz images, to improve the imaging resolution of the network in the super‐resolution work of real low‐resolution terahertz images. In addition, the no‐reference image quality evaluation system is introduced to objectively evaluate the network performance. The experimental results show that the UHA‐CycleGAN network outperforms the traditional paired super‐resolution network on both open‐source and self‐built datasets.

An Improved YOLOv5-Based Underwater Object-Detection Framework

To date, general-purpose object-detection methods have achieved a great deal. However, challenges such as degraded image quality, complex backgrounds, and the detection of marine organisms at different scales arise when identifying underwater organisms. To solve such problems and further improve the accuracy of relevant models, this study proposes a marine biological object-detection architecture based on an improved YOLOv5 framework. First, the backbone framework of Real-Time Models for object Detection (RTMDet) is introduced. The core module, Cross-Stage Partial Layer (CSPLayer), includes a large convolution kernel, which allows the detection network to precisely capture contextual information more comprehensively. Furthermore, a common convolution layer is added to the stem layer, to extract more valuable information from the images efficiently. Then, the BoT3 module with the multi-head self-attention (MHSA) mechanism is added into the neck module of YOLOv5, such that the detection network has a better effect in scenes with dense targets and the detection accuracy is further improved. The introduction of the BoT3 module represents a key innovation of this paper. Finally, union dataset augmentation (UDA) is performed on the training set using the Minimal Color Loss and Locally Adaptive Contrast Enhancement (MLLE) image augmentation method, and the result is used as the input to the improved YOLOv5 framework. Experiments on the underwater datasets URPC2019 and URPC2020 show that the proposed framework not only alleviates the interference of underwater image degradation, but also makes the mAP@0.5 reach 79.8% and 79.4% and improves the mAP@0.5 by 3.8% and 1.1%, respectively, when compared with the original YOLOv8 on URPC2019 and URPC2020, demonstrating that the proposed framework presents superior performance for the high-precision detection of marine organisms.

Optimizing microdisplay requirements for pancake VR applications

AbstractVirtual reality (VR) devices use imaging optics to magnify the microdisplay images for providing an immersive viewing experience. A microdisplay is preferred to have high‐resolution density and high dynamic range to meet the demanding requirements of human vision system (HVS), for example, visual acuity >60 pixels per degree and grayscale depth >10 bits. However, increasing resolution density and dynamic range often lead to a reduced optical efficiency and sophisticated fabrication process of the microdisplay panels. In this paper, we systematically analyze the image degradation mechanisms of VR devices caused by both imaging optics and microdisplay and find that the image degradation caused by imaging optics significantly unleash the requirements of microdisplay, such as contrast ratio, number of local dimming zones, and resolution density. For example, aberrations of the imaging optics reduce the resolution density requirement of the microdisplay, and stray light of the imaging optics relieves the contrast ratio requirement of the microdisplay. These results help prevent excessive design of microdisplay, for example, mini‐light‐emitting diode (LED) backlit liquid‐crystal displays (LCDs), organic LEDs, or micro‐LEDs, in a pancake lens‐based VR headset.

Effect of acoustic noise reduction technology on image quality: a multivendor study

The purpose of this study was to clarify the appropriate use of a combination of pulse sequences and acoustic noise reduction technology in general-purpose brain magnetic resonance imaging. Five pulse sequences commonly used in brain magnetic resonance imaging examinations-turbo spin-echo T2-weighted imaging, T1-weighted fluid-attenuated inversion recovery, T2-weighted fluid-attenuated inversion recovery, diffusion-weighted imaging, and magnetic resonance angiography-were performed on healthy participants at three vendors where acoustic noise reduction technology was available. The results showed that acoustic noise reduction technology reduced sound pressure levels and altered image quality in all pulse sequences across all vendors' magnetic resonance imaging scanners. Although T2-weighted imaging and T1-weighted fluid-attenuated inversion recovery resulted in little image quality degradation, T2-weighted fluid-attenuated inversion recovery, diffusion-weighted imaging, and magnetic resonance angiography had significant image degradation. Therefore, acoustic noise reduction technology should be used with caution.

Multi-frame image fusion using a machine learning-based weight mask predictor for turbulence-induced image degradation

Imaging through degraded visual environments is a challenging task in remote sensing missions. Image degradation may come from the loss of contrast due to particle scattering and/or distortion due to turbulence-induced effects. The problem is especially challenging when imaging from moving platforms such as autonomous underwater vehicles. One potential approach to address these issues is to use multiple images and employ a multi-frame image fusion technique to aid in the recovery of corrupted image detail and quality. A machine learning (ML)-based image enhancement and fusion technique is investigated to restore images distorted by underwater turbulence. The main contributions include the incorporation of a ML-based image weight predictor to predict the ideal weight maps to be used in an image fusion process. This network is trained using a generative adversarial network framework and a synthetically generated image dataset, which is created according to an analytical image degradation model. In addition, the image loss function for the weight map predictor is determined by the final fused image, resulting in a balanced fusion technique that can reduce image distortion, recover crisp image detail, and reduce the overall noise figure. Another key contribution of this paper is the adoption of an image loss function that incorporates an innovative combined correntropy and Fourier space loss function to reinforce the network in both the spatial and frequency domain. The performance of the proposed algorithm is evaluated using a synthetic validation dataset of images and several real datasets captured at the Naval Research Lab’s Simulated Turbulence and Turbidity Environment under various controlled turbulence intensities.

Image quality and computer vision performance: assessing the effects of image distortions and modeling performance relationships using the general image quality equation

Substantial research has explored methods to optimize convolutional neural networks (CNNs) for tasks such as image classification and object detection, but research into the image quality drivers of computer vision performance has been limited. Additionally, there are indications that image degradations such as blur and noise affect human visual interpretation and machine interpretation differently. The general image quality equation (GIQE) predicts overhead image quality for human analysis using the National Image Interpretability Rating Scale, but no such model exists to predict image quality for interpretation by CNNs. Here, we assess the relationship between image quality variables and CNN performance. Specifically, we examine the impacts of resolution, blur, and noise on CNN performance for models trained with in-distribution and out-of-distribution distortions. Using two datasets, we observe that while generalization remains a significant challenge for CNNs faced with out-of-distribution image distortions, CNN performance against low visual quality images remains strong with appropriate training, indicating the potential to expand the design trade space for sensors providing data to computer vision systems. Additionally, we find that CNN performance predictions using the functional form of the GIQE can predict CNN performance as a function of image degradation, but we observe that the legacy form of the GIQE (from GIQE versions 3 and 4) does a better job of modeling the impact of blur/relative edge response in our experiments.

An efficient image compression architecture based on optimized 9/7 wavelet transform with hybrid post processing and entropy encoder module

Satellite communication is popular due to the advancement in storing and transmitting satellite images. Image compression can be defined as the science of lowering the number of bits necessary to represent an image. In satellite image processing, image degradation is considered a challenging problem. The Consultative Committee for Space Data Systems (CCSDS) Image Data Compression (IDC) standard CCSDS-122.0-B-1 is the transformation-dependent image compression method developed especially for usage in on-board space platforms. It holds de-correlation, quantization and entropy encoding phases. This paper introduces a new optimized memory organization for Discrete Wavelet Transform (DWT) to perform spatial de-correlation with fewer memory requirements on an FPGA device. Also, the proposed optimized DWT is integrated with a hybrid post-processing and entropy encoder module to reduce the spatial redundancies between the wavelet coefficients and compress the de-correlated data with high compression performance. A high-throughput hardware implementation of the Binary arithmetic entropy Coder (BAEC) is also provided to perform lossless compression with low implementation complexity. To provide the convenience and compactness, the proposed system will be implemented in the Xilinx working platform by developing a Verilog code. The proposed model is then evaluated on the Arty Z7–20 development board. The proposed approach is analyzed on different performance parameters such as throughput and frequency. The proposed design allowed a maximum operating frequency of 250 MHz, leading to a throughput of 156.25 Msamples/sec on Zynq. In addition, the structure of the proposed method overtakes the conventional design in terms of memory requirements and area.

Joint face completion and super-resolution using multi-scale feature relation learning

Previous research on face restoration often focused on repairing specific types of low-quality facial images such as low-resolution (LR) or occluded facial images. However, in the real world, both the above-mentioned forms of image degradation often coexist. Therefore, it is important to design a model that can repair images that are LR and occluded simultaneously. This paper proposes a multi-scale feature graph generative adversarial network (MFG-GAN) to carry out face restoration in contexts in which both LR and occluded degradation modes coexist, and also to repair images with a single type of degradation. Based on the GAN, the MFG-GAN integrates the graph convolution and feature pyramid networks to restore occluded low-resolution face images to non-occluded high-resolution face images. The MFG-GAN uses a set of customized losses to ensure that high-quality images are generated. In addition, we designed the network in an end-to-end format. We conduct experiments on general face image restoration and facial expression restoration. Experimental results on the public-domain databases show that the proposed approach outperforms state-of-the-art methods in performing face super resolution (up to 4× or 8×) and face completion simultaneously and can recover better facial expression details.

Mask‐guided image person removal with data synthesis

AbstractAs a special case of common object removal, image person removal is playing an increasingly important role in social media and criminal investigation domains. Due to the integrity of person area and the complexity of human posture, person removal has its own dilemmas. In this paper, a novel idea is proposed to tackle these problems from the perspective of data synthesis. Concerning the lack of a dedicated dataset for image person removal, two dataset production methods are proposed to automatically generate images, masks and ground truths, respectively. Then, a learning framework similar to local image degradation is proposed so that the masks can be used to guide the feature extraction process and more texture information can be gathered for final prediction. A coarse‐to‐fine training strategy is further applied to refine the details. The data synthesis and learning framework combine well with each other. Experimental results verify the effectiveness of the method quantitatively and qualitatively, and the trained network proves to have good generalization ability either on real or synthetic images.

A deep learning-based framework for retinal fundus image enhancement.

Low-quality fundus images with complex degredation can cause costly re-examinations of patients or inaccurate clinical diagnosis. This study aims to create an automatic fundus macular image enhancement framework to improve low-quality fundus images and remove complex image degradation. We propose a new deep learning-based model that automatically enhances low-quality retinal fundus images that suffer from complex degradation. We collected a dataset, comprising 1068 pairs of high-quality (HQ) and low-quality (LQ) fundus images from the Kangbuk Samsung Hospital's health screening program and ophthalmology department from 2017 to 2019. Then, we used these dataset to develop data augmentation methods to simulate major aspects of retinal image degradation and to propose a customized convolutional neural network (CNN) architecture to enhance LQ images, depending on the nature of the degradation. Peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), r-value (linear index of fuzziness), and proportion of ungradable fundus photographs before and after the enhancement process are calculated to assess the performance of proposed model. A comparative evaluation is conducted on an external database and four different open-source databases. The results of the evaluation on the external test dataset showed an significant increase in PSNR and SSIM compared with the original LQ images. Moreover, PSNR and SSIM increased by over 4 dB and 0.04, respectively compared with the previous state-of-the-art methods (P < 0.05). The proportion of ungradable fundus photographs decreased from 42.6% to 26.4% (P = 0.012). Our enhancement process improves LQ fundus images that suffer from complex degradation significantly. Moreover our customized CNN achieved improved performance over the existing state-of-the-art methods. Overall, our framework can have a clinical impact on reducing re-examinations and improving the accuracy of diagnosis.

X-ray scattering and attenuation cross-sections and coefficients of bone, brain, lung, fat, and soft tissue for applications in dosimetry, cancer detection, and treatment

The Klein-Nishina electronic and atomic cross-sections and Compton mass attenuation coefficients of the bone, lung, soft tissue, brain, and fat are calculated for X-ray or γ-ray energies of 50 keV, 140 keV, 364 keV, 1.25 MeV, 4.784 MeV, and 6.0 MeV. The Klein-Nishina formula was used in the entire calculation. The results show that the Klein-Nishina electronic cross-section (eσ) decreases with increasing photon energy. Due to the complex variation in the effective charge number Z of the tissues, the Klein-Nishina atomic cross-section (aσ) first increases and then decreases for the investigated tissues. The Compton mass attenuation coefficient (σ/ρ) has even more complexity when the calculated values of σ/ρ are analyzed as a function of increasing Z/A for the investigated tissues. An overall trend of increase in σ/ρ is observed for increasing Z/A for all tissues except the cortical bone. In the case of the cortical bone, the values of σ/ρ decrease since cortical bone is a relatively high Z tissue as compared to other tissues where a rapid increase in the effective mass number A occurs when Z is increased. Overall, the results are very useful in calculating and delivering precise and optimum doses to avoid image quality degradation in radiographic images and to treat a tumor more effectively while keeping the normal tissues safe in radiation treatment planning.

Hybrid connected attentional lightweight network for gangue intelligent segmentation in top-coal caving face

The estimation of gangue content is the main basis for intelligent top coal caving mining by computer vision, and the automatic segmentation of gangue is crucial to computer vision analysis. However, it is still a great challenge due to the degradation of images and the limitation of computing resources. In this paper, a hybrid connected attentional lightweight network (HALNet) with high speed, few parameters and high accuracy is proposed for gangue intelligent segmentation on the conveyor in the top-coal caving face. Firstly, we propose a deep separable dilation convolution block (DSDC) combining deep separable convolution and dilation convolution, which can provide a larger receptive field to learn more information and reduce the size and computational cost of the model. Secondly, a bridging residual learning framework is designed as the basic unit of encoder and decoder to minimize the loss of semantic information in the process of feature extraction. An attention fusion block (AFB) with skip pathway is introduced to capture more representative and distinctive features through the fusion of high-level and low-level features. Finally, the proposed network is trained through the expanded dataset, and the gangue image segmentation results are obtained by pixel-by-pixel classification method. The experimental results show that the proposed HALNet reduces about 57 percentage parameters compared with U-Net, and achieves state-of-the art performance on dataset.