Image Restoration Research Articles

Edge-preserving image restoration based on a weighted anisotropic diffusion model

Partial differential equation-based methods have been widely applied in image restoration. The anisotropic diffusion model has a good noise removal capability without affecting significant edges. However, existing anisotropic diffusion-based models closely depend on the diffusion coefficient function and threshold parameter. This paper proposes a new weighted anisotropic diffusion coefficient model with multiple scales, and it has a higher speed of closing to X-axis and exploits adaptive threshold parameters. Meanwhile, the proposed algorithm is verified to be suitable for multiple types of noise. Numerical metrics and visual comparison of simulation experiments show the proposed model has significant superiority in edge-preserving and staircase artifacts reducing over the existing anisotropic diffusion-based techniques.

DIVA: Deep unfolded network from quantum interactive patches for image restoration

This paper presents a deep neural network called DIVA unfolding a baseline adaptive denoising algorithm (DeQuIP), relying on the theory of quantum many-body physics. Furthermore, it is shown that with very slight modifications, this network can be enhanced to solve more challenging image restoration tasks such as image deblurring, super-resolution and inpainting. Despite a compact and interpretable (from a physical perspective) architecture, the proposed deep learning network outperforms several recent algorithms from the literature, designed specifically for each task. The key ingredients of the proposed method are on one hand, its ability to handle non-local image structures through the patch-interaction term and the quantum-based Hamiltonian operator, and, on the other hand, its flexibility to adapt the hyperparameters patch-wisely, due to the training process.

Enhancing trabecular CT scans based on deep learning with multi-strategy fusion

Trabecular bone analysis plays a crucial role in understanding bone health and disease, with applications like osteoporosis diagnosis. This paper presents a comprehensive study on 3D trabecular computed tomography (CT) image restoration, addressing significant challenges in this domain. The research introduces a backbone model, Cascade-SwinUNETR, for single-view 3D CT image restoration. This model leverages deep layer aggregation with supervision and capabilities of Swin-Transformer to excel in feature extraction. Additionally, this study also brings DVSR3D, a dual-view restoration model, achieving good performance through deep feature fusion with attention mechanisms and Autoencoders. Furthermore, an Unsupervised Domain Adaptation (UDA) method is introduced, allowing models to adapt to input data distributions without additional labels, holding significant potential for real-world medical applications, and eliminating the need for invasive data collection procedures. The study also includes the curation of a new dual-view dataset for CT image restoration, addressing the scarcity of real human bone data in Micro-CT. Finally, the dual-view approach is validated through downstream medical bone microstructure measurements. Our contributions open several paths for trabecular bone analysis, promising improved clinical outcomes in bone health assessment and diagnosis.

MWformer: a novel low computational cost image restoration algorithm

MWformer: a novel low computational cost image restoration algorithm

Computational single fundus image restoration techniques: a review.

Fundus cameras are widely used by ophthalmologists for monitoring and diagnosing retinal pathologies. Unfortunately, no optical system is perfect, and the visibility of retinal images can be greatly degraded due to the presence of problematic illumination, intraocular scattering, or blurriness caused by sudden movements. To improve image quality, different retinal image restoration/enhancement techniques have been developed, which play an important role in improving the performance of various clinical and computer-assisted applications. This paper gives a comprehensive review of these restoration/enhancement techniques, discusses their underlying mathematical models, and shows how they may be effectively applied in real-life practice to increase the visual quality of retinal images for potential clinical applications including diagnosis and retinal structure recognition. All three main topics of retinal image restoration/enhancement techniques, i.e., illumination correction, dehazing, and deblurring, are addressed. Finally, some considerations about challenges and the future scope of retinal image restoration/enhancement techniques will be discussed.

Design of optimized fourth order PDE filter for restoration and enhancement of Microbiopsy images of breast Cancer

Design of optimized fourth order PDE filter for restoration and enhancement of Microbiopsy images of breast Cancer

Interactive Deep Image Colorization of Quality

Deep Image Colonization is a pioneering project aimed to revolutionizing the field of automated image colorization, particularly focusing on enhancing grayscale photographs' visual appeal and historical significance. Leveraging advanced deep learning models like VGG16 and UNET GAN, the project seeks to accurately and faithfully Add images in black and white some color. Through meticulous evaluation and comparison of different colorization algorithms, including real-time display of results and batch processing capabilities, the project strives to provide users with a seamless and intuitive experience. Beyond aesthetic enhancement, the project explores the implications of automated image colorization in various domains, from historical image restoration to creative visual storytelling. By evaluating colorization accuracy and refining models for real-world usage, the project aims to contribute to the advancement of image processing technologies. Ultimately, "Interactive Deep Image Colonization of Quality" endeavour to fill the void left by the past and the present, providing monochromatic imagery through vibrant hues and precision colorization techniques.

A modified Fletcher-Reeves conjugate gradient method for unconstrained optimization with applications in image restoration

A modified Fletcher-Reeves conjugate gradient method for unconstrained optimization with applications in image restoration

Importing Diffusion and Re-Designed Backward Process for Image De-Raining.

In recent years, with the increasing demand for high-quality images in various fields, more and more attention has been focused on noise removal techniques for image processing. The effective elimination of unwanted noise plays a crucial role in improving image quality. To meet this challenge, many noise removal methods have been proposed, among which the diffusion model has become one of the focuses of many researchers. In order to make the restored image closer to the real image and retain more features of the image, this paper proposes a DIR-SDE method with reference to the diffusion models of IR-SDE and IDM, which improve the feature retention of the image in the de-raining process, and then improve the realism of the image for the image de-raining task. In this study, IR-SDE was used as the base structure of the diffusion model, IR-SDE was improved, and DINO-ViT was combined to enhance the image features. During the diffusion process, the image features were extracted using DINO-ViT, and these features were fused with the original images to enhance the learning effect of the model. The model was also trained and validated with the Rain100H dataset. Compared with the IR-SDE method, it improved 0.003 in the SSIM, 0.003 in the LPIPS, and 1.23 in the FID. The experimental results show that the diffusion model proposed in this study can effectively improve the image restoration performance.

Computational Polarization Imaging In Vivo through Surgical Smoke Using Refined Polarization Difference.

In surgery, the surgical smoke generated during tissue dissection and hemostasis can degrade the image quality, affecting tissue visibility and interfering with the further image processing. Developing reliable and interpretable computational imaging methods for restoring smoke-affected surgical images is crucial, as typical image restoration methods relying on color-texture information are insufficient. Here a computational polarization imaging method through surgical smoke is demonstrated, including a refined polarization difference estimation based on the discrete electric field direction, and a corresponding prior-based estimation method, for better parameter estimation and image restoration performance. Results and analyses for ex vivo, the first in vivo animal experiments, and human oral cavity tests show that the proposed method achieves visibility restoration and color recovery of higher quality, and exhibits good generalization across diverse imaging scenarios with interpretability. The method is expected to enhance the precision, safety, and efficiency of advanced image-guided and robotic surgery.

Imaging simulation and restoration for a mobile-based long-focus camera with optical stabilization

Image stabilization technology of the mobile-based long-focus camera is an influential factor that limits the imaging quality. Among them, the periscope lens system is a common choice and is capable of compensating for the external vibration by rotating its prism. To further investigate the impact of residual vibration during the optical image stabilization process, we propose an imaging simulation method to quantify the degradation and construct the corresponding dataset for training. We decompose the degradation into translation perpendicular to the optical axis on the image plane and defocus parallel to the optical axis. The defocus blur pattern is modified to better reproduce the spatially variant deterioration caused by defocus. Constructing synthetic datasets through our proposed imaging simulation method significantly reduces the cost of training the post-processing network while enhancing the imaging quality. Extensive experiments demonstrate the accuracy of our proposed method and the superiority of our synthetic dataset.

Research on image restoration of [formula omitted] porous foam ceramics media using Transformer architecture

Pore structure images of foam ceramics play a crucial role in the field of refractory materials science and engineering, facilitating the observation and analysis of the material’s pore structure and unique characteristics. However, traditional image restoration techniques have limitations in handling background noise, complex texture restoration, deformation, and dislocation correction, as well as automation and intelligence aspects in restoring material pore structure images. The latest Transformer architecture has shown significant potential in image processing, particularly in image restoration research tasks. Nevertheless, its application in restoring pore structure images of materials remains unexplored. This paper aims to address this gap by focusing on the image restoration of foam ceramic pore structure images using a Transformer-based algorithm for super-resolution (SR) processing. Experimental comparisons with state-of-the-art methods demonstrate a remarkable improvement in Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM), underscoring the effectiveness of the proposed approach.

Meta-learning based blind image super-resolution approach to different degradations

Although recent studies on blind single image super-resolution (SISR) have achieved significant success, most of them typically require supervised training on synthetic low resolution (LR)-high resolution (HR) paired images. This leads to re-training necessity for different degradations and restricted applications in real-world scenarios with unfavorable inputs. In this paper, we propose an unsupervised blind SISR method with input underlying different degradations, named different degradations blind super-resolution (DDSR). It formulates a Gaussian modeling on blur degradation and employs a meta-learning framework for solving different image degradations. Specifically, a neural network-based kernel generator is optimized by learning from random kernel samples, referred to as random kernel learning. This operation provides effective initialization for blur degradation optimization. At the same time, a meta-learning framework is proposed to resolve multiple degradation modelings on the basis of alternative optimization between blur degradation and image restoration, respectively. Differing from the pre-trained deep-learning methods, the proposed DDSR is implemented in a plug-and-play manner, and is capable of restoring HR image from unfavorable LR input with degradations such as partial coverage, noise addition, and darkening. Extensive simulations illustrate the superior performance of the proposed DDSR approach compared to the state-of-the-arts on public datasets with comparable memory load and time consumption, yet exhibiting better application flexibility and convenience, and significantly better generalization ability towards multiple degradations. Our code is available at https://github.com/XYLGroup/DDSR.

Optimizing Underwater Image Restoration and Depth Estimation with Light Field Images

Methods based on light field information have shown promising results in depth estimation and underwater image restoration. However, improvements are still needed in terms of depth estimation accuracy and image restoration quality. Previous work on underwater image restoration employed an image formation model (IFM) that overlooked the effects of light attenuation and scattering coefficients in underwater environments, leading to unavoidable color deviation and distortion in the restored images. Additionally, the high blurriness and associated distortions in underwater images make depth information extraction and estimation very challenging. In this paper, we refine the light propagation model and propose a method to estimate the attenuation and backscattering coefficients of the underwater IFM. We simplify these coefficients into distance-related functions and design a relationship between distance and the darkest channel to estimate the water coefficients, effectively suppressing color deviation and distortion in the restoration results. Furthermore, to increase the accuracy of depth estimation, we propose using blur cues to construct a cost for refocusing in the depth direction, reducing the impact of high signal-to-noise ratio environments on depth information extraction, and effectively enhancing the accuracy and robustness of depth estimation. Finally, experimental comparisons show that our method achieves more accurate depth estimation and image restoration closer to real scenes compared to state-of-the-art methods.

Restoration of Vector Magnetization Image from Vector Scanning-SQUID Microscope Measurement

Abstract A generalized mathematical framework to treat image data measured by the scanning superconducting quantum interference device (SQUID) microscope using a three-dimensional vector pickup coil is presented. The blurring of the images originating from the effects of diamagnetism due to the superconductivity of the sensor, the non-zero sensor size, and the finite sensor-to-sample separation are numerically reduced. We use a lattice model of the measurement system, and singular value decomposition and the Moore-Penrose pseudo-inverse matrix are employed to handle ill-conditioned matrices we encounter in the numerical processes. Based on a numerical model, measurement of the vector magnetization distributed on a sample surface, and the image restoration using the present procedure are demonstrated.

Weather Translation via Weather-Cue Transferring.

In this article, the weather translation task is proposed, which aims to transfer the weather type of the image from one category to another. Weather translation is a complicated image weather editing task that changes the weather cue of an image across multiple weather types, and it is related to image restoration, image editing, and photographic style transfer tasks. Although lots of approaches have been developed for traditional image translation and restoration tasks, only few of them are capable of handling the multicategory weather types problem with a single network due to the rich categories and highly complicated semantic structures of weather images. Especially, it is difficult to change the weather cue while preserving the weather-invariant area. To solve these issues, we developed a weather-cue guided multidomain translation approach based on StarGAN v2, termed WeatherGAN. In the proposed model, the core generator is redesigned to transfer the weather cue according to the target weather type. The weather segmentation module is first introduced to acquire the weather semantic structure of images in a weakly supervised multitask manner. In addition, a weather clues module is presented to reprocess the weather segmentation into a weather-specific clues map, which identifies the weather-invariant and weather-cue areas clearly. Extensive studies and evaluations show that our approach outperforms the state of the art. The data and source code will be publicly available soon after the manuscript is accepted.

Cold Diffusion Model for Seismic Denoising

AbstractSeismic waves contain information about the earthquake source, the geologic structure they traverse, and many forms of noise. Separating the noise from the earthquake is a difficult task because optimal parameters for filtering noise typically vary with time and, if chosen inappropriately, may strongly alter the original seismic waveform. Diffusion models based on Deep Learning have demonstrated remarkable capabilities in restoring images and audio signals. However, those models assume a Gaussian distribution of noise, which is not the case for typical seismic noise. Motivated by the effectiveness of “cold” diffusion models in speech enhancement, medical anomaly detection, and image restoration, we present a cold variant for seismic data restoration. We describe the first Cold Diffusion Model for Seismic Denoising (CDiffSD), including key design aspects, model architecture, and noise handling. Using metrics to quantify the performance of CDiffSD models compared to previous works, we demonstrate that it provides a new standard in performance. CDiffSD significantly improved the Signal to Noise Ratio by about 18% compared to previous models. It also enhanced Cross‐correlation by 6%, showing a better match between denoised and original signals. Moreover, testing revealed a 50% increase in the recall of P‐wave picks for seismic picking. Our work show that CDiffSD outperforms existing benchmarks, further underscoring its effectiveness in seismic data denoising and analysis. Additionally, the versatility of this model suggests its potential applicability across a range of tasks and domains, such as GNSS, Lab Acoustic Emission, and Distributed Acoustic Sensing data, offering promising avenues for further utilization.

Generalized nonconvex nonsmooth four-directional total variation with overlapping group sparsity for image restoration

In this paper, we address the challenge of image deblurring in the presence of Gaussian noise. To achieve high-quality image restoration, we introduce an optimization framework that integrates four-directional total variation regularization, overlapping group sparsity, and a range of nonconvex penalties. This novel model effectively mitigates the staircase artifacts associated with total variation regularization and enhances restoration quality by leveraging domain-specific information about image pixels. Compared to the ℓ1 norm, the nonconvex penalty applied to overlapping groups promotes sparsity in image gradients at the group level. To solve this nonconvex optimization problem, we propose a proximal alternating reweighted minimization algorithm, which has a proximal alternating scheme with a reweighted approximation of its subproblem. Theoretically, we establish that the sequences generated by the proposed algorithm converge to a critical point using the Kurdyka–Łojasiewicz property. Experimental results validate the superiority of our approach over competing methods.

Generation diffusion degradation: Simple and efficient design for blind super-resolution

In the domain of computer vision, blind super-resolution is a key area focused on generating high-resolution images with enhanced visual quality from low-resolution counterparts affected by indeterminate degradation factors. This area is primarily advanced through self-supervised learning techniques utilizing GANs. Despite their prominence, GAN-based methods encounter challenges including unstable training dynamics and limited diversity, compounded by the intricate necessity to configure degradation models to mimic various blur effects and noise types. Lately, denoising diffusion models have shown promising results in image restoration, yet their sampling efficiency constraints impede their deployment in real-time scenarios. This study introduces the Generation Diffusion Degradation (GDD) model, a novel and efficient technique for replicating image degradation by applying random Gaussian noise in a sequential manner via a parametric Markov chain, followed by a progressive reconstruction of the initial image through a U-net-based noise predictor. This method adeptly mirrors the inherent degradation distribution observed in actual degraded images. Furthermore, we present an innovative training strategy that utilizes a composite loss function to train the GDD model, ensuring stable training, improving the authenticity of the generated degraded images, and precisely reflecting the degradation patterns of target images. Extensive experimental analyses underscore the superior performance of the proposed GDD model, both in objective metrics and subjective visual quality. The code is available at https://github.com/lgylab/GDD.

A Global Self-Attention Memristive Neural Network for Image Restoration

A Global Self-Attention Memristive Neural Network for Image Restoration