Image Inpainting Technique Research Articles

Improving the Segmentation Accuracy of Ovarian-Tumor Ultrasound Images Using Image Inpainting.

Diagnostic results can be radically influenced by the quality of 2D ovarian-tumor ultrasound images. However, clinically processed 2D ovarian-tumor ultrasound images contain many artificially recognized symbols, such as fingers, crosses, dashed lines, and letters which assist artificial intelligence (AI) in image recognition. These symbols are widely distributed within the lesion's boundary, which can also affect the useful feature-extraction-utilizing networks and thus decrease the accuracy of lesion classification and segmentation. Image inpainting techniques are used for noise and object elimination from images. To solve this problem, we observed the MMOTU dataset and built a 2D ovarian-tumor ultrasound image inpainting dataset by finely annotating the various symbols in the images. A novel framework called mask-guided generative adversarial network (MGGAN) is presented in this paper for 2D ovarian-tumor ultrasound images to remove various symbols from the images. The MGGAN performs to a high standard in corrupted regions by using an attention mechanism in the generator to pay more attention to valid information and ignore symbol information, making lesion boundaries more realistic. Moreover, fast Fourier convolutions (FFCs) and residual networks are used to increase the global field of perception; thus, our model can be applied to high-resolution ultrasound images. The greatest benefit of this algorithm is that it achieves pixel-level inpainting of distorted regions without clean images. Compared with other models, our model achieveed better results with only one stage in terms of objective and subjective evaluations. Our model obtained the best results for 256 × 256 and 512 × 512 resolutions. At a resolution of 256 × 256, our model achieved 0.9246 for SSIM, 22.66 for FID, and 0.07806 for LPIPS. At a resolution of 512 × 512, our model achieved 0.9208 for SSIM, 25.52 for FID, and 0.08300 for LPIPS. Our method can considerably improve the accuracy of computerized ovarian tumor diagnosis. The segmentation accuracy was improved from 71.51% to 76.06% for the Unet model and from 61.13% to 66.65% for the PSPnet model in clean images.

Contour–Context Joint Blind Image Inpainting Network for Molecular Sieve Particle Size Measurement of SEM Images

Scanning electron microscope (SEM) images generated in the experiment of high-throughput molecular sieves catalysts contain particle size information of various molecular sieves. However, a certain proportion of molecular sieves in SEM images are occluded due to their characteristics of cluster distribution, which cause difficulties to measure the particle size of these occluded molecular sieves. In this paper, we leverage the image inpainting technique to reconstruct missing regions, and then calculate the particle size. Aiming at the problem that the generated regions of recent image inpainting methods exist blurriness in contour and context, we propose a contour-context joint blind image inpainting network, which is specifically a two-stage model with full utilization of edge information to reconstruct the contour and context of these occluded molecular sieve images. In the first stage, the edge of occluded region is reconstructed. In the second stage, the reconstructed edge is introduced through down sampling and skip connection to generate images with complete contour. Meanwhile, a novel encoder loss is proposed to motivate the encoder to extract more reasonable features. Compared with the existing methods, our model achieved more integral results visually and higher quantitative metrics. Thus, our approach can accurately predict the particle size from reconstructed molecular sieve images.

Panoptic blind image inpainting

In autonomous driving, scene understanding is a critical task in recognizing the driving environment or dangerous situations. Here, a variety of factors, including foreign objects on the lens, cloudy weather, and light blur, often reduce the accuracy of scene recognition. In this paper, we propose a new blind image inpainting model that accurately reconstructs images in a real environment where there is no ground truth for restoration. To this end, we first introduce a panoptic map to represent content information in detail and design an encoder–decoder structure to predict the panoptic map and the corrupted region mask. Then, we construct an image inpainting model that utilizes the information of the predicted map. Lastly, we present a mask refinement process to improve the accuracy of map prediction. To evaluate the effectiveness of the proposed model, we compared the restoration results of various inpainting methods on the cityscapes and coco datasets. Experimental results show that the proposed model outperforms other blind image inpainting models in terms of L1/L2 losses, PSNR and SSIM, and achieves similar performance to other image inpainting techniques that utilize additional information.

A comparison of deep-learning-based inpainting techniques for experimental X-ray scattering.

The implementation is proposed of image inpainting techniques for the reconstruction of gaps in experimental X-ray scattering data. The proposed methods use deep learning neural network architectures, such as convolutional autoencoders, tunable U-Nets, partial convolution neural networks and mixed-scale dense networks, to reconstruct the missing information in experimental scattering images. In particular, the recovered pixel intensities are evaluated against their corresponding ground-truth values using the mean absolute error and the correlation coefficient metrics. The results demonstrate that the proposed methods achieve better performance than traditional inpainting algorithms such as biharmonic functions. Overall, tunable U-Net and mixed-scale dense network architectures achieved the best reconstruction performance among all the tested algorithms, with correlation coefficient scores greater than 0.9980.

The Andromeda gamma-ray excess: background systematics of the millisecond pulsars and dark matter interpretations

ABSTRACT Since the discovery of an excess in gamma rays in the direction of M31, its cause has been unclear. Published interpretations focus on dark matter or stellar related origins. Studies of a similar excess in the Milky Way centre motivate a correlation of the spatial morphology of the signal with the distribution of stellar mass in M31. However, a robust determination of the best theory for the observed excess emission is challenging due to uncertainties in the astrophysical gamma-ray foreground model. We perform a spectro-morphological analysis of the M31 gamma-ray excess using state-of-the-art templates for the distribution of stellar mass in M31 and novel astrophysical foreground models for its sky region. We construct maps for the old stellar populations of M31 based on data from the PAndAS survey and carefully remove the foreground stars. We also produce improved astrophysical foreground models via novel image inpainting techniques based on machine learning methods. Our stellar maps, mimicking the location of a population of millisecond pulsars in the bulge of M31, reach a 5.4σ significance, making them as strongly favoured as the simple phenomenological models usually considered in the literature, e.g. disc-like templates. This detection is robust to generous variations of the astrophysical foreground model. Once the stellar templates are included in the astrophysical model, we show that the dark matter annihilation interpretation of the signal is unwarranted. We demonstrate that about one million unresolved millisecond pulsars naturally explain the observed gamma-ray luminosity per stellar mass, energy spectrum, and stellar bulge-to-disc flux ratio.

Image Retrieval Using Digital Image Inpainting Techniques

Image retrieval is an inverse problem in digital image processing. In this paper, the authors deal with restoration of image using digitally image inpainting methods. In this inpainting technique, one can extract a missing an important part or can fill unwanted parts of an image, so as to recover original information without loss. They are using semi-automatic and fast image inpainting methods and compared their result in terms of image quality assessment parameters such as PSNR, MSE, as well as required timing constraints. The observational as well as statistical results show that the fast image inpainting technique is superior in all respects for any kinds of multimedia image application.

A topography-aware approach to the automatic generation of urban road networks

Existing deep-learning tools for road network generation have limited applications in flat urban areas due to their overreliance on the geometric and spatial configurations of street networks and inadequate considerations of topographic information. This paper proposes a new method of street network generation based on a generative adversarial network by designing a pre-positioned geo-extractor module and a geo-merging bypath. The two improvements employ the complementary use of geometric configurations and topographic features to automate street network generation in both flat and hilly urban areas. Our experiments demonstrate that the improved model yields a more realistic prediction of street configurations than conventional image inpainting techniques. The model’s effectiveness is further enhanced when generating streets in hilly areas. Furthermore, the geo-extractor module provides insights from the computer vision perspective in recognizing when topographic information should be considered and which topographic information should receive more attention.

AuSR1: Authentication and self-recovery using a new image inpainting technique with LSB shifting in fragile image watermarking

With the rapid development of multimedia technology, editing and manipulating digital images have become more accessible than ever. This paper proposed color image authentication based on blind fragile image watermarking for tamper detection and self-recovery named AuSR1. The AuSR1 divides each channel of the cover image into non-overlapping blocks with the size of 2 × 2 pixels. The authentication data is embedded into the original block location, while the recovery data is embedded into the distant location from the original location based on the block mapping algorithm. The watermark data is then embedded into the 2 LSB to achieve high quality of the recovered image under tampering attacks. In addition, the permutation algorithm is applied to ensure the security of the watermark data. The AuSR1 utilizes a three-layer authentication algorithm to achieve a high detection rate.The experimental results show that the scheme produced a PSNR value of 45.57 dB and an SSIM value of 0.9972 of the watermarked images. Furthermore, the AuSR1 detected the tampered area of the images with a high precision value of 0.9943. In addition, the recovered image achieved a PSNR value of 27.64 dB and an SSIM value of 0.9339 on a 50% tampering rate.

IISTD: Image Inpainting-Based Small Target Detection in a Single Infrared Image

Small target detection is a crucial and challenging task in infrared search and track system. Background estimation-based methods is an effective and important approach for infrared small target detection. Affected by the target pixels, existing background estimation methods may reconstruct an inaccurate background. Based on image inpainting technique, we propose a novel two-stage approach to predict more accurate backgrounds. At the first stage, the inner and outer window-based image inpainting (IOWII) is used to obtain a rough background estimation. Then a mask of candidate target region is automatically obtained by calculating and evaluating the difference between raw image and rough background. In the second stage, the final accurate background is predicted by mask-based image inpainting (MII). It recovers the removed candidate target area using the information of surrounding background pixels, avoiding target pixels to participate in the calculation of background reconstruction. Finally, the target saliency map is obtained by subtracting the final estimated background from the original image, and a simple adaptive threshold is used to segment the target. Experimental results on real infrared images and sequences demonstrate that the proposed method outperforms other state-of-the-arts. It is simple and effective, with strong robustness and good real-time performance.

A Survey on Various Image Inpainting T arious Image Inpainting Techniques echniques

Over the decades' researchers have studied the image inpainting problem intensively due to its high significance and effectiveness in various image processing applications such as people and object security, object removal, face editing applications. Image inpainting is defined as the process of completing or removing a missing region in images. It is considered one of the most challenging topics in the image processing field, although, it requires a deep understanding of the image details in terms of texture and structure. In this paper, a survey of most image inpainting techniques is presented and summarized with comparisons including the merits and demerits of each technique which could help researchers in evaluating their proposed techniques against.

Medical image inpainting with edge and structure priors

Image inpainting techniques are widely applied to repair distorted or missing regions in the image. However, unreasonable results are usually generated due to lack of structural information. In order to repair the structure rationally, we propose a model based on edge and structure information for medical image inpainting (ESMII). In addition, for overcoming the degradation problem of the deep network, multi-scale residual blocks (MRB) are used to extract features from the image at three scales. Meanwhile, the multi-level loss is proposed to motivate the network synthesizing more significant structural textures. Experimental results on COVID CT, Abdominal CT and Abdominal MRI datasets indicate that ESMII can generate more reasonable structures and fine textures compared to other models. Furthermore, ablation studies for justification verify the effectiveness of different modules in the network. We show that our approach can effectively solve the distortion of medical images.

Image Inpainting Using Image Interpolation - An Analysis

Image inpainting is a computer technique of filling lost regions of an image by using available information from the surrounding area. This digital image inpainting technique has wider applications like image restoration, dis-occlusion and image/video compression. The traditional image inpainting approaches Partial Differential Equation (PDE) based method and Exemplar based method mainly focus on the size of the target region to be filled. PDE is a pixel oriented method and works well if the region to be filled is small. On the other hand exemplar based method is a patch based method and works well if the region to be filled is large. This paper proposes another new technique called image interpolation technique and describes how this technique can be used effectively for image inpainting without giving much focus on the size of the target region.

Specular highlight region restoration using image clustering and inpainting

A novel specular highlight restoration algorithm has been proposed to remove the specular highlight in a real-time vision system. In this paper, the specular highlight region has been detected and it has been restored by image inpainting method. Most of specular highlight detection algorithms are effective only for a single image. And the auto-threshold algorithm has been implemented in real time, but, it has still low reliability with a heavy computational cost. The proposed system detects pixels corresponding to the specular highlight region in the HSI color space through a newly defined classification table. So, Specular highlight can be detected quickly with these simplified classification table. In addition, it has versatility because it can be combined with various existing high-performance image inpainting method. The superiority of the proposed algorithm is compared with the conventional specular highlight removal algorithm by combining proposed algorithm with two high-performance image inpainting techniques.

Image inpainting using frequency-domain priors

In this paper, we present a novel image inpainting technique using frequency domain information. Prior works on image inpainting predict the missing pixels by training neural networks using only the spatial domain information. However, these methods still struggle to reconstruct high-frequency details for real complex scenes, leading to a discrepancy in color, boundary artifacts, distorted patterns, and blurry textures. To alleviate these problems, we investigate if it is possible to obtain better performance by training the networks using frequency domain information (Discrete Fourier Transform) along with the spatial domain information. To this end, we propose a frequency-based deconvolution module that enables the network to learn the global context while selectively reconstructing the high-frequency components. We evaluate our proposed method on the publicly available datasets CelebA, Paris Streetview, and DTD texture dataset, and show that our method outperforms current state-of-the-art image inpainting techniques both qualitatively and quantitatively.

Mask-guided GAN for robust text editing in the scene

Text editing in the scene aims to replace the source text with target text while maintaining the original text style and background, which involves text detection, style transfer and image inpainting techniques. Due to diversified text styles in the real scene, such as colorful outlines and shadows, it is quite challenging to edit complicated text. To address this problem, a mask-guided GAN method is proposed to adequately use the body, outline, and shadow of text to guide the task. First, a mask generating module is designed to detect the body, outline, and shadow regions in the text image. The generated mask in the source image is then used for the proposed text inpainting module and background inpainting module to extract source stylish text and restore the background, respectively. Next, a shape transfer module is designed to infer the target mask that depicts the text structural style from the source mask, which guides a style transfer module to transfer the text style onto the target image. Finally, a fusion module is developed to fuse the target text and background. Each module fulfills its own functional role and collaborates with each other, thus decomposing the complex task into easy-to-learn ones. Experiments and comparisons demonstrate the effectiveness of the proposed method.

Generative Image Steganography Based on Digital Cardan Grille

In this paper, a generative image steganography algorithm based on digital Cardan Grille is proposed. Combining the ideas of traditional Cardan Grille and the semantic image inpainting technique, the stego image are driven by secret messages directly. The algorithm first embeds the information based on digital Cardan Grille, and then uses generative adversarial network (GANs) to complete the damaged image. The adversarial game not only reconstruct the corrupted image, but also generate a stego image which contains the logic rationality of image content. The experimental results verify the feasibility of the proposed method.

An improved image quality algorithm for exemplar-based image inpainting

Image inpainting is a common technique for repairing image regions that are scratched or damaged. This process involves reconstructing damaged parts and filling-in regions in which data/colour information is missing. There are many potential applications for image inpainting, such as repairing old images, repairing scratched images, removing unwanted objects, and filling-in missing areas. This paper develops an exemplar-based algorithm, one of the most important and popular image inpainting techniques, to fill-in missing regions caused by removing unwanted objects, image compression, scratches, or image transformation via the Internet. The proposed algorithm includes two phases of searching to select the best-matching information. In the first phase, the searching mechanism uses the entire image to find and select the most similar patches using the Euclidean distance. The second phase measures the distance between the location of the selected patches and the location of the patch to be filled. The performance of the proposed approach is evaluated through comprehensive experiments on several well-known images used in this area of research. The experimental results demonstrate the superior performance of the proposed approach over some state-of-the-art approaches in terms of quality in terms of both objective (using the peak signal-to-noise ratio (PSNR) as well as the structural similarity index method (SSIM)) and subjective (i.e., visual) measures.

Overview of Digital Image Inpainting Technology

As an important part of image processing, digital image inpainting is more and more widely used in daily life. Its main repair method is to automatically recover lost information based on the existing information of the damaged image. The article first explains the concept and application of image inpainting; secondly, it outlines the current main image inpainting techniques and main evaluation methods; and finally, the prospect of image inpainting techniques.

A method for concrete crack detection using U-Net based image inpainting technique

A method for concrete crack detection using U-Net based image inpainting technique

Measuring scour level based on spatial and temporal image analyses

Scour monitoring is an important measurement process to determine the soil erosion level at the pillar of bridge. Image-based approach is attractive because it allows monitoring process to be conducted continuously without halting the flow of the water during experiment. Scour images provide abundance of information from a single source of camera sensor. However, this information may appear in different features, orientation, size and brightness. Therefore, it is important to detect and recognise features that are related to scour monitoring and filtered out irrelevant features like image noises and artefacts. This paper presents implementation of image processing techniques to extract various information from scour images. Image inpainting technique is used to separate information of scour level and scale markers into two different images. A proposed gradient of marginal histogram technique is used to detect the horizontal scale line markers and scour level. Backpropagation neural network is used to recognise scale text markers and convert the measurement of the scour level from a pixel unit into a centimetre unit. Interpolation techniques are used to connect scour points and delineate the boundary indicating the level of the scour. Time series acquisition of scour images allows observation of the temporal variation of the scour levels. Results show that the proposed approach achieved higher accuracy than existence method. This approach allows the detection of the scour level for even and uneven sediments, contributing to the high accurate results at the spatial and temporal measurements, thus potentially offering continuous scour monitoring solution.