Image Blending Research Articles

Autonomous Single-Image Dehazing: Enhancing Local Texture with Haze Density-Aware Image Blending

Single-image dehazing is an ill-posed problem that has attracted a myriad of research efforts. However, virtually all methods proposed thus far assume that input images are already affected by haze. Little effort has been spent on autonomous single-image dehazing. Even though deep learning dehazing models, with their widely claimed attribute of generalizability, do not exhibit satisfactory performance on images with various haze conditions. In this paper, we present a novel approach for autonomous single-image dehazing. Our approach consists of four major steps: sharpness enhancement, adaptive dehazing, image blending, and adaptive tone remapping. A global haze density weight drives the adaptive dehazing and tone remapping to handle images with various haze conditions, including those that are haze-free or affected by mild, moderate, and dense haze. Meanwhile, the proposed approach adopts patch-based haze density weights to guide the image blending, resulting in enhanced local texture. Comparative performance analysis with state-of-the-art methods demonstrates the efficacy of our proposed approach.

Seamless Panoramic Image Stitching Based onInvariant Feature Detector and Image Blending

Seamless Panoramic Image Stitching Based onInvariant Feature Detector and Image Blending

Dual-Split CT to Simulate Multiple Radiation Doses From a Single Scan-Liver Lesion Detection Compared With Dose-Matched Single-Energy CT.

The aim of this study was to evaluate the potential use of simulated radiation doses from a dual-split CT scan for dose optimization by comparing their lesion detectability to dose-matched single-energy CT acquisitions at different radiation dose levels using a mathematical model observer. An anthropomorphic abdominal phantom with liver lesions (5-10 mm, both hyperattenuating and hypoattenuating) was imaged using a third-generation dual-source CT in single-energy dual-source mode at 100 kVp and 3 radiation doses (5, 2.5, 1.25 mGy). The tube current was 67% for tube A and 33% for tube B. For each dose, 5 simulated radiation doses (100%, 67%, 55%, 45%, 39%, and 33%) were generated through linear image blending. The phantom was also imaged using traditional single-source single-energy mode at equivalent doses. Each setup was repeated 10 times. Image noise texture was evaluated by the average spatial frequency (fav) of the noise power spectrum. Liver lesion detection was measured by the area under the receiver operating curve (AUC), using a channelized Hotelling model observer with 10 dense Gaussian channels. Fav decreased at lower radiation doses and differed between simulated and single-energy images (eg, 0.16 mm-1 vs 0.14 mm-1 for simulated and single-energy images at 1.25 mGy), indicating slightly blotchier noise texture for dual-split CT. For hyperattenuating lesions, the mean AUC ranged between 0.92-0.99, 0.81-0.96, and 0.68-0.89 for single-energy, and between 0.91-0.99, 0.78-0.91, and 0.70-0.85 for dual-split at 5 mGy, 2.5 mGy, and 1.25 mGy, respectively. For hypoattenuating lesions, the AUC ranged between 0.90-0.98, 0.75-0.93, and 0.69-0.86 for the single-energy, and between 0.92-0.99, 0.76-0.87, and 0.67-0.81 for dual-split at 5 mGy, 2.5 mGy, and 1.25 mGy, respectively. AUC values were similar between both modes at 5 mGy, and slightly lower, albeit not significantly, for the dual-split mode at 2.5 and 1.25 mGy. Lesion detectability was comparable between multiple simulated radiation doses from a dual-split CT scan and dose-matched single-energy CT. Noise texture was slightly blotchier in the simulated images. Simulated doses using dual-split CT can be used to assess the impact of radiation dose reduction on lesion detectability without the need for repeated patient scans.

SC-AOF: A Sliding Camera and Asymmetric Optical-Flow-Based Blending Method for Image Stitching.

Parallax processing and structure preservation have long been important and challenging tasks in image stitching. In this paper, an image stitching method based on sliding camera to eliminate perspective deformation and asymmetric optical flow to solve parallax is proposed. By maintaining the viewpoint of two input images in the mosaic non-overlapping area and creating a virtual camera by interpolation in the overlapping area, the viewpoint is gradually transformed from one to another so as to complete the smooth transition of the two image viewpoints and reduce perspective deformation. Two coarsely aligned warped images are generated with the help of a global projection plane. After that, the optical flow propagation and gradient descent method are used to quickly calculate the bidirectional asymmetric optical flow between the two warped images, and the optical-flow-based method is used to further align the two warped images to reduce parallax. In the image blending, the softmax function and registration error are used to adjust the width of the blending area, further eliminating ghosting and reducing parallax. Finally, by comparing our method with APAP, AANAP, SPHP, SPW, TFT, and REW, it has been proven that our method can not only effectively solve perspective deformation, but also gives more natural transitions between images. At the same time, our method can robustly reduce local misalignment in various scenarios, with higher structural similarity index. A scoring method combining subjective and objective evaluations of perspective deformation, local alignment and runtime is defined and used to rate all methods, where our method ranks first.

Alignment and fusion for adaptive domain nighttime semantic segmentation

In the field of autonomous driving technology, both daytime and nighttime scenes are common. However, due to the poor illumination and difficulty in manual annotation of nighttime images, semantic segmentation of nighttime scenes is more challenging compared to daytime scenes. Therefore, achieving significant progress in nighttime semantic segmentation would greatly enhance the effectiveness of the application of autonomous driving scenarios. In this work, we investigate the problem of domain-adaptive nighttime semantic segmentation (DANSS). The problem aims to learn semantic segmentation in nighttime scenes by leveraging a labeled Cityscapes dataset and unlabeled but roughly aligned day-night image pairs. To address this, we propose an Align and Fusion Network (AAFnet), a network for adaptive domain nighttime semantic segmentation. AAFnet utilizes a novel DAFormer as the backbone to separately compute features for dynamic and static objects during the training process. It also employs methods such as small category sampling and image blending to improve learning effectiveness. Additionally, we propose utilizing local image patches to enhance the results during the testing process. Experimental results demonstrate a significant improvement of 10.4% compared to the previous method, DANNet. Extensive experiments conducted on the Dark Zurich dataset and Nightdriving dataset validate the effectiveness of the proposed approach. Our method outperforms previous backbones and achieves top-ranking results.

A DOMAIN-DECOMPOSED VAE METHOD FOR BAYESIAN INVERSE PROBLEMS

Bayesian inverse problems are often computationally challenging when the forward model is governed by complex partial differential equations (PDEs). This is typically caused by expensive forward model evaluations and highdimensional parameterization of priors. This paper proposes a domain-decomposed variational autoencoder Markov chain Monte Carlo (DD-VAE-MCMC) method to tackle these challenges simultaneously. Through partitioning the global physical domain into small subdomains, the proposed method first constructs local deterministic generative models based on local historical data, which provide efficient local prior representations. Gaussian process models with active learning address the domain decomposition interface conditions. Then inversions are conducted on each subdomain independently in parallel and in low-dimensional latent parameter spaces. The local inference solutions are postprocessed through the Poisson image blending procedure to result in an efficient global inference result. Numerical examples are provided to demonstrate the performance of the proposed method.

Combining Gaussian Process Regression with Poisson Blending for Seamless Cloud Removal from Optical Remote Sensing Imagery for Cropland Monitoring

Constructing optical image time series for cropland monitoring requires a cloud removal method that accurately restores cloud regions and eliminates discontinuity around cloud boundaries. This paper describes a two-stage hybrid machine learning-based cloud removal method that combines Gaussian process regression (GPR)-based predictions with image blending for seamless optical image reconstruction. GPR is employed in the first stage to generate initial prediction results by quantifying temporal relationships between multi-temporal images. GPR predictive uncertainty is particularly combined with prediction values to utilize uncertainty-weighted predictions as the input for the next stage. In the second stage, Poisson blending is applied to eliminate discontinuity in GPR-based predictions. The benefits of this method are illustrated through cloud removal experiments using Sentinel-2 images with synthetic cloud masks over two cropland sites. The proposed method was able to maintain the structural features and quality of the underlying reflectance in cloud regions and outperformed two existing hybrid cloud removal methods for all spectral bands. Furthermore, it demonstrated the best performance in predicting several vegetation indices in cloud regions. These experimental results indicate the benefits of the proposed cloud removal method for reconstructing cloud-contaminated optical imagery.

Research of different feature detection and matching algorithms on panoramic image

Image stitching is the process of combining numerous photos to make a panorama. The technique of image stitching has rapidly advanced and grown to be a significant area of digital image processing. Many image stitching methods have been proposed and studied in prior study. In this paper, the image stitching process is implemented using different algorithms. For keypoints detection, the algorithms of Harris corner detection, SIFT(Scale-Invariant Feature Transform), SURF(Speeded Up Robust Feature) and ORB(Oriented FAST and Rotated BRIEF) algorithms are applied, then use different methods (i.e., Brute Force, etc.) for feature matching. The RANSAC(Random Sample Consensus) method is used to calculate a homography matrix from matched feature vectors and use it to warp the images. Image blending and cropping methods are proposed to enhance the image quality. Given groups of the self-captured images, experiments have been down to shown the performance of different techniques.

Inpainting GAN-Based Image Blending with Adaptive Binary Line Mask

Inpainting GAN-Based Image Blending with Adaptive Binary Line Mask

A Straightforward Bifurcation Pattern-Based Fundus Image Registration Method.

Fundus image registration is crucial in eye disease examination, as it enables the alignment of overlapping fundus images, facilitating a comprehensive assessment of conditions like diabetic retinopathy, where a single image's limited field of view might be insufficient. By combining multiple images, the field of view for retinal analysis is extended, and resolution is enhanced through super-resolution imaging. Moreover, this method facilitates patient follow-up through longitudinal studies. This paper proposes a straightforward method for fundus image registration based on bifurcations, which serve as prominent landmarks. The approach aims to establish a baseline for fundus image registration using these landmarks as feature points, addressing the current challenge of validation in this field. The proposed approach involves the use of a robust vascular tree segmentation method to detect feature points within a specified range. The method involves coarse vessel segmentation to analyze patterns in the skeleton of the segmentation foreground, followed by feature description based on the generation of a histogram of oriented gradients and determination of image relation through a transformation matrix. Image blending produces a seamless registered image. Evaluation on the FIRE dataset using registration error as the key parameter for accuracy demonstrates the method's effectiveness. The results show the superior performance of the proposed method compared to other techniques using vessel-based feature extraction or partially based on SURF, achieving an area under the curve of 0.526 for the entire FIRE dataset.

A Novel High-Speed and Low-PDP Approximate Full Adder Cell for Image Blending

This paper presents a new and high-performance inaccurate Full Adder Cell utilizing the Carbon Nanotube Field Effect Transistor (CNFET) technology. Comprehensive simulations are performed at the transistor and application levels to justify the performance of our design. Simulations performed using the HSPICE tool confirm the significant improvement in the performance of the proposed circuit delay, power-delay product (PDP) and energy-delay product (EDP) compared to competitor designs. Additionally, via a MATLAB tool, the image blending (alpha blending) application uses inaccurate Full Adder cells. Software simulations confirm the suitable quality of the final images according to the image quality evaluation criteria.

Computational Integral Imaging Reconstruction via Elemental Image Blending without Normalization

This paper presents a novel computational integral imaging reconstruction (CIIR) method using elemental image blending to eliminate the normalization process in CIIR. Normalization is commonly used in CIIR to address uneven overlapping artifacts. By incorporating elemental image blending, we remove the normalization step in CIIR, leading to decreased memory consumption and computational time compared to those of existing techniques. We conducted a theoretical analysis of the impact of elemental image blending on a CIIR method using windowing techniques, and the results showed that the proposed method is superior to the standard CIIR method in terms of image quality. We also performed computer simulations and optical experiments to evaluate the proposed method. The experimental results showed that the proposed method enhances the image quality over that of the standard CIIR method, while also reducing memory usage and processing time.

CWAN: Self-supervised learning for deep grape disease image composition

Grape leaf diseases seriously affect grape yield and directly lead to the loss of the industrial economy. An accurate diagnosis is crucial for effective spraying and control. Presently, the combination of image blending and a deep recognition model is the most effective method for accurately identifying grape leaf diseases. Existing methods of image blending depend primarily on single-channel and multi-scale channel blending, computing and optimizing the Poisson equation, and the iterative optimization of learning-based methods for simple composite results. Although these methods have had some success, they are not able to simultaneously address lesion and leaf gap size, color bleeding, and edge smoothness. In this paper, we propose a novel image blending method named cascade wavelet attention network (CWAN). Specifically, CWAN consists of a cascade wavelet attention (CWA) and a blending autoencoder. First, we build a CWA to decompose the image into a multi-level and multi-scale frequency domain image. Then, the detailed features of the source and target domains are extracted, and the edges are smoothed. Following this, a shared decoder is designed to fuse these features in the corresponding frequency-domain layers, and the result serves as the attention feature. Finally, we design a blending autoencoder with a feature fusion mechanism to seamlessly blend lesions and leaves enhanced by CWA. We also construct a weighted blended loss function to avoid leaf color infiltration into the lesion in the blended image. Comprehensive experiments on two grape leaf disease datasets. Qualitative and quantitative results demonstrate the superiority and effectiveness of our model.

MT-GAN: toward realistic image composition based on spatial features

The purpose of image composition is to combine the visual elements of different natural images to produce a natural image. The performance of most existing image composition methods drops significantly when they solve multiple issues, such as image harmonization, image blending, shadow generation, object placement, and spatial transformation. To address this problem, we propose a multitask GAN for image compositing based on spatial features, aiming to simultaneously address the geometric and appearance inconsistency. We use three related learning objective functions to train the network. Moreover, a new dataset including 7756 images with RoI region annotations is contributed to help evaluate the multitask image compositing results. Extensive experiments demonstrate that our proposed method is effective on our dataset.

Terrain methods on spectral analysis for paleoclimate interpretations: A novel visualization technique using python

Spectral analysis techniques are valuable tools at the disposal of paleoclimate scientists in their research of cyclical phenomena potentially responsible for past climatic fluctuations. Advances in computing and an increased availability of climate time series have helped to consolidate this approach. Yet the visual representation of spectral analysis results has not improved at the same pace. Time-frequency analysis aims to identify periodic signals that vary over time using 2-D color graphs, depicting spectral bands theoretically discernible from the image background. The paleoclimate literature is full of examples such as the continuous wavelet analysis, the evolutionary fast Fourier transform (FFT) spectrogram, or even the more recent synchrosqueezing transform. Our approach is based on the stack of non-evolutive (assuming stationary behaviour) spectral analysis results from thousands of fixed interval time series, previously split from a longer and unevenly spaced (irregular sampling) paleoclimate time series. As illustrative examples, the targeted time series are derived from the LR04 Global Pliocene-Pleistocene Benthic δ18O stack and from the 65 °N summer insolation for the last 5.3 Myr, by means of the Lomb-Scargle periodogram technique. Enhanced and clearer visualization is achieved through the novel incorporation of the terrain analysis techniques: slope, hillshading and color mapping, and posterior blending of the individual images, using Python code. The result consists of a high resolution graphical output, allowing for better qualitative and quantitative interpretations of the obtained cyclicity, as the code incorporates the import of the achieved confidence levels from the spectral technique and the option to obscure the pixels under a certain threshold value. The application of terrain analysis techniques on visualization of spectral analysis results has the purpose of improving previous graphical representations of paleoclimate time-series, especially those time-frequency aspects of the involved phenomena. New developments of our approach may be applied to time-frequency analysis directly, supporting present and future paleoclimate studies.

Efficient computational technique of de-convolution and image blending for motion blurring problems in medical imaging

Efficient computational technique of de-convolution and image blending for motion blurring problems in medical imaging

Improved Automatic Classification of Litho-Geomorphological Units by Using Raster Image Blending, Vipava Valley (SW Slovenia)

Automatic landslide classification based on digital elevation models has become a powerful complementary tool to field mapping. Many studies focus on the automatic classification of landslides’ geomorphological features, such as their steep main scarps, but in many cases, the scarps and other morphological features are difficult for algorithms to detect. In this study, we performed an automatic classification of different litho-geomorphological units to differentiate slope mass movements in field maps by using Maximum Likelihood Classification. The classification was based on high-resolution lidar-derived DEM of the Vipava Valley, SW Slovenia. The results show an improvement over previous approaches as we used a blended image (VAT, which included four different raster layers with different weights) along with other common raster layers for morphometric analysis of the surface (e.g., slope, elevation, aspect, TRI, curvature, etc.). The newly created map showed better classification of the five classes we used in the study and recognizes alluvial deposits, carbonate cliffs (including landslide scarps), carbonate plateaus, flysch, and slope deposits better than previous studies. Multivariate statistics recognized the VAT layer as the most important layer with the highest eigenvalues, and when combined with Aspect and Elevation layers, it explained 90% of the total variance. The paper also discusses the correlations between the different layers and which layers are better suited for certain geomorphological surface analyses.

A Content-adaptive Visibility Predictor for Perceptually Optimized Image Blending

The visibility of an image semi-transparently overlaid on another image varies significantly, depending on the content of the images. This makes it difficult to maintain the desired visibility level when the image content changes. To tackle this problem, we developed a perceptual model to predict the visibility of the blended results of arbitrarily combined images. Conventional visibility models cannot reflect the dependence of the suprathreshold visibility of the blended images on the appearance of the pre-blended image content. Therefore, we have proposed a visibility model with a content-adaptive feature aggregation mechanism, which integrates the visibility for each image feature (i.e., such as spatial frequency and colors) after applying weights that are adaptively determined according to the appearance of the input image. We conducted a large-scale psychophysical experiment to develop the visibility predictor model. Ablation studies revealed the importance of the adaptive weighting mechanism in accurately predicting the visibility of blended images. We have also proposed a technique for optimizing the image opacity such that users can set the visibility of the target image to an arbitrary level. Our evaluation revealed that the proposed perceptually optimized image blending was effective under practical conditions.

Virtual Hairstyle Service Using GANs & Segmentation Mask (Hairstyle Transfer System)

The virtual hair styling service, which now is necessary for cosmetics companies and beauty centers, requires significant improvement efforts. In the existing technologies, the result is unnatural as the hairstyle image is serviced in the form of a ‘composite’ on the face image, image, extracts and synthesizing simple hair images. Because of complicated interactions in illumination, geometrical, and occlusions, that generate pairing among distinct areas of an image, blending features from numerous photos is extremely difficult. To compensate for the shortcomings of the current state of the art, based on GAN-Style, we address and propose an approach to image blending, specifically for the issue of visual hairstyling to increase accuracy and reproducibility, increase user convenience, increase accessibility, and minimize unnaturalness. Based on the extracted real customer image, we provide a virtual hairstyling service (Live Try-On service) that presents a new approach for image blending with maintaining details and mixing spatial features, as well as a new embedding approach-based GAN that can gradually adjust images to fit a segmentation mask, thereby proposing optimal styling and differentiated beauty tech service to users. The visual features from many images, including precise details, can be extracted using our system representation, which also enables image blending and the creation of consistent images. The Flickr-Faces-HQ Dataset (FFHQ) and the CelebA-HQ datasets, which are highly diversified, high quality datasets of human faces images, are both used by our system. In terms of the image evaluation metrics FID, PSNR, and SSIM, our system significantly outperforms the existing state of the art.

Digital Image Blending Using Inaccurate Addition

Inaccurate computing is found to be a high-speed, low-power and energy-efficient alternative to accurate computing for error-tolerant applications. In this context, this paper analyzes the usefulness of inaccurate computing for a digital image processing application, viz. digital image blending, which has been less explored. We analyze the use of inaccurate addition for image blending used in applications such as photo editing and computer graphics. For experimentation, we considered blending two digital images using accurate and inaccurate addition separately. We considered many inaccurate addition architectures which are suitable for implementation in both FPGA and ASIC design environments to perform a comparative analysis. We found that an inaccurate addition with an optimum inaccuracy produces a similar quality of blended image as obtained using accurate addition. The quality of blended images is quantified using standard metrics such as the peak signal-to-noise ratio and the structural similarity index measure. In particular, an inaccurate adder, M-HERLOA, was found to be preferable for image blending from the combined perspectives of quality of blended image, error metrics and design metrics. We implemented the accurate and inaccurate adders corresponding to an optimum inaccuracy in FPGA and ASIC design environments. We considered a Xilinx Artix-7 FPGA for an FPGA-based implementation and a 32/28 nm CMOS standard digital cell library for an ASIC type standard cell-based implementation. The results show that, for an FPGA-based implementation, M-HERLOA enables a reduction in delay by 13%, requires 50% fewer LUTs and 49% fewer registers and consumes 24.3% less on-chip power compared to the accurate high-speed FPGA adder while yielding a blended image of good quality. For an ASIC type standard cell-based implementation, M-HERLOA enables a reduction in delay by 64.6% (24.4%), requires 43% (82.4%), less area and dissipates 63% (67.8%) less power than the accurate carry ripple adder (carry look-ahead adder).