Shadow Removal Research Articles

Enhanced osmosis model with bilateral total variation for effective shadow removal

This research paper presents an innovative technique for shadow images removal. The method involves redefining a contemporary osmosis model by incorporating bilateral total variation (TV) operators. This integration allows to take advantage of robust anisotropic diffusion, resulting in improved image restoration. The paper also outlines new mathematical derivation of the bilateral TV and a combination with nonlinear anisotropic transport term. The experimental results substantiate the effectiveness of the anisotropic osmosis model, showcasing its superior qualitative and quantitative performance when compared to current state-of-the-art techniques.

Frequency‐Aware Facial Image Shadow Removal through Skin Color and Texture Learning

AbstractExisting facial image shadow removal methods predominantly rely on pre‐extracted facial features. However, these methods often fail to capitalize on the full potential of these features, resorting to simplified utilization. Furthermore, they tend to overlook the importance of low‐frequency information during the extraction of prior features, which can be easily compromised by noises. In our work, we propose a frequency‐aware shadow removal network (FSRNet) for facial image shadow removal, which utilizes the skin color and texture information in the face to help recover illumination in shadow regions. Our FSRNet uses a frequency‐domain image decomposition network to extract the low‐frequency skin color map and high‐frequency texture map from the face images, and applies a color‐texture guided shadow removal network to produce final shadow removal result. Concretely, the designed fourier sparse attention block (FSABlock) can transform images from the spatial domain to the frequency domain and help the network focus on the key information. We also introduce a skin color fusion module (CFModule) and a texture fusion module (TFModule) to enhance the understanding and utilization of color and texture features, promoting high‐quality result without color distortion and detail blurring. Extensive experiments demonstrate the superiority of the proposed method. The code is available at https://github.com/laoxie521/FSRNet.

Two-stage deep image restoration network with application to single image shadow removal

In this paper, we introduce a two-stage deep learning-based image restoration network and its application to remove shadow information from a single image, named by ESCNet (Encoder-decoder based Shadow removal with Colorization Network). Most existed single image-based shadow removal methods may suffer from that the shadow contains multiple regions of different colors or rich image details. To tackle with the problems, our key idea is to first remove shadow(s) from an image followed by repainting the shadow-removed region(s) in this image. To accomplish this, we present a deep two-stage network, cascading a shadow removal network (SRN) and a colorization network (CN). The presented encoder-decoder-based SRN with fusion of global and local feature information is used to remove the shadow(s) in the grayscale domain of the input image while recovering the image details for the shadow-removed region(s). Then the proposed CN aims at repainting the removed shadow region(s) via re-colorization. The proposed deep model has been well trained and well evaluated on the two well-known public datasets, i.e., ISTD (Image Shadow Triplets Dataset) and SRD (Shadow Removal Dataset). Experimental results have shown that the proposed method outperforms the compared state-of-the-art (SOTA) shadow removal approaches quantitatively and qualitatively.

CRFormer: A cross-region transformer for shadow removal

Image shadow removal is a fundamental task in computer vision, which aims to restore damaged signals caused by shadows, thereby improving image quality and scene understanding. Recently, transformers have demonstrated strong capabilities in various applications by capturing global pixel interactions, a capability highly desirable for shadow removal. However, applying transformers to promote shadow removal is non-trivial for the following two reasons: 1) The patchify operation is not suitable for shadow removal due to irregular shadow shapes; 2) Shadow removal only requires one-way interaction from the non-shadow region to the shadow region instead of the common two-way interactions among all pixels in the image. In this paper, we propose a novel Cross-Region transFormer (CRFormer) for shadow removal which differs from existing transformers by only considering the pixel interactions from the non-shadow region to the shadow region without splitting images into patches. This is achieved by a carefully designed region-aware cross-attention mechanism that aggregates the recovered shadow region features, conditioned on the non-shadow region features. Extensive experiments on the ISTD, AISTD, SRD, and Video Shadow Removal datasets demonstrate the superiority of our method compared to other state-of-the-art methods.

An improved EnlightenGAN shadow removal framework for images of cracked concrete

The concrete crack images in engineering are usually obscured by unexpected shadow and their removal is usually required which is always a challenging task due to its complexity and diversity. An improved Enlighten Generative Adversarial Networks (IEnlightenGAN) is developed in a proposed framework to improve shadow removal results. The Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) is firstly developed to improve the concrete image resolution. Residual-in-residual-dense blocks module and Atrous Spatial Pyramid Pooling module are then integrated into the IEnlightenGAN to improve its shadow removal capability. A quality assessment methodology on the shadow removal results is also proposed. The crack contour is extracted based on U-Net, and denoising is conducted based on morphological closing. Image segmentation metrics are used for the evaluation. The proposed IEnlightenGAN and the evaluation method are trained and verified using a dataset containing both normal and shadowed concrete crack images with comparison of results from the state-of-the-art models. The proposed shadow removal framework is found able to improve the quality of shadowed crack images, and the proposed quality assessment method is also found better than existing approach.

GR-GAN: A unified adversarial framework for single image glare removal and denoising

In this paper, we introduce the single image glare removal (SIGR) task. SIGR aims to eliminate glare or light caused by external environment in lighting scene. To efficiently improve natural image quality and usability, many research tasks, such as image deraining and shadow removal, have been investigated a lot. However, SIGR is still underexplored. Therefore, we propose to construct a dataset and explore deep learning-based models for SIGR. Our contributions can be summarized as follows: (1) We establish a new benchmark dataset for SIGR, termed De-Glare, aiming to propel research of SIGR. This dataset comprises pairs of {glare, glare-free} images sourced from both real-world and synthetic data, utilized for training and evaluating models. (2) We conduct a comprehensive benchmarking of extensive state-of-the-art (SoTA) methods on the constructed De-Glare dataset and provide insightful analyses based on the results. (3) An innovative approach for single image glare removal employing a multi-scale generative adversarial network (GR-GAN) model is proposed. Glare images typically exhibit irregular glare shapes and cluttered backgrounds. To address these irregular glare patterns, we introduce a deformable convolution-based glare attention detector (GAD) designed to generate an attention map which specifics glare spots or rays in the input image. In pursuit of enhancing the perceptual quality of output image, GR-GAN adaptively filters out irrelevant noises and enhances salient features through a generator with cascaded pyramid neck (CPN) network. This work can provide useful insights for developing better SIGR models. Without specific tuning, our method achieves the SoTA results on multiple computer vision tasks, including the image deraining and image shadow removal.

Shadow-aware decomposed transformer network for shadow detection and removal

Shadow detection and removal are important yet challenging computer vision tasks. Existing methods simultaneously contend with the brightness and color information of input image while treating different regions of input images equally. We argue that these operations fail to effectively extract the relationship between shadow and non-shadow regions. To relieve these problems, this paper proposes a shadow-aware decomposed transformer network for shadow detection and removal. The network decomposes the input image into brightness and color maps using its bright channel, which are concatenated with the original image as the input data, enabling the network to pay balanced attention to the brightness and color information when calculating the relationship between regions. Additionally, given that the correlation matrix of the transformer measures the relative dependency between regions, the proposed shadow-aware transformer block can extract the relationship between shadow and non-shadow regions more effectively by retaining the specific elements of the correlation matrix. We conduct extensive experiments on three shadow detection benchmark datasets and two shadow removal benchmark datasets. Experimental results show that the proposed method performs favorably against state-of-the-art methods. Codes have been made available at https://github.com/XIAOWANG914/SADT.

Shadow removal method for high-resolution aerial remote sensing images based on region group matching

Shadow removal is a critical step in preprocessing high-resolution aerial remote sensing images. The presence of shadows reduces the information in high-resolution aerial remote sensing images, lowers the image quality, and seriously interferes with the work of feature classification, object detection, and information extraction. Therefore, this paper proposes a high-resolution aerial remote sensing image shadow removal method based on region group matching. The method starts from both spatial information and color information, and is able to take into account the whole and the local, and recover the detailed features of different features inside the shadows. Firstly, this paper proposes an irregular region color transfer method based on three-dimensional color space, which can effectively restore the original color and texture information of the features in the shadow region. Secondly, considering the random and variable direction of feature distribution in remote sensing images, the direction of texture feature extraction cannot be set in advance. Therefore, in this paper, the texture details of the features are extracted based on the rotationally invariant light-independent texture feature extraction method to exclude the abnormal interference. Then, the shadow and light regions of the image are segmented internally and grouped according to the type of features to avoid subsequent matching anomalies due to the small size of the image blocks. Next, construct an average texture feature vector for each group of image blocks for grouping matching. The matched light groups are then used to guide the shadow groups for local feature information enhancement, which ensures that on top of the overall shadow removal, the detail recovery of different local features is enhanced. Finally, for the boundary effect after shadow removal, a boundary optimization method with dynamically assigned weights is proposed, which can ensure the natural coherence of the resultant image in the boundary part. The experimental results show that the proposed method balances both overall and local shadow removal effects and can effectively achieve high-resolution aerial remote sensing image shadow removal. Compared with existing methods, the proposed method achieves the best quality of shadow removal results and restores more realistic ground features in shadow region.

Towards enhancing shadow removal from images

Towards enhancing shadow removal from images

Training a shadow removal network using only 3D primitive occluders

Training a shadow removal network using only 3D primitive occluders

Global context aware dual channel pyramid model for robust image shadow removal

Deep learning has made significant advancements in shadow removal. However, current methods primarily focus on local operations, leading to artifacts around shadow boundaries and inconsistent lighting between shadow and non-shadow regions. To address this issue, a global context aware dual channel pyramid model is proposed for robust image shadow removal in this paper. The model leverages a multi-scale channel attention framework based on transformers to capture global information. It consists of a shadow detection module and a dual-channel shadow interaction module to utilize non-shadow regions in aiding shadow restoration. Additionally, Winograd-based separable convolution attention shadow interaction attention is proposed to effectively expand the perceptual field and facilitate comprehensive utilization of contextual relevance between shadow and non-shadow regions. Extensive experiments are conducted on several popular public datasets such as ISTD, SRD and ISTD + to evaluate the effectiveness of our model.

Shadow Detection and Elimination for Robot and Machine Vision Applications

Shadow removal is crucial for robot and machine vision as the accuracy of object detection is greatly influenced by the uncertainty and ambiguity of the visual scene. In this paper, we introduce a new algorithm for shadow detection and removal based on different shapes, orientations, and spatial extents of Gaussian equations. Here, the contrast information of the visual scene is utilized for shadow detection and removal through five consecutive processing stages. In the first stage, contrast filtering is performed to obtain the contrast information of the image. The second stage involves a normalization process that suppresses noise and generates a balanced intensity at a specific position compared to the neighboring intensities. In the third stage, the boundary of the target object is extracted, and in the fourth and fifth stages, respectively, the region of interest (ROI) is highlighted and reconstructed. Our model was tested and evaluated using realistic scenarios which include outdoor and indoor scenes. The results reflect the ability of our approach to detect and remove shadows and reconstruct a shadow free image with a small error of approximately 6%.

An Improved Multi-Scale Region of Interest Deep Learning Model for Fruit Disease Classification

One of the main sources of nutrients is fruits, yet a variety of diseases appear that harmfully impact the fruit's quality and productivity. To avoid this issue, Deep Learning (DL) models have been developed, which identify and classify fruit diseases earlier. Amongst, a new Attribute registration and Shadow removal Adversarial Network with Multi-scale Region-Of-Interest (ROI)-based Deep Learning (ArSAN-MRDL) model creates more shadow-less fruit images to classify multiple diseases in a single fruit image and localizes the diseased regions precisely. In contrast, the standard convolution filters in the MRDL may lose the discrimination information for similar fruit diseases, which degrades the classification accuracy. Hence, this article designs the ArSAN with an improved MRDL (ArSAN-iMRDL) model to improve the accuracy of classifying similar diseases in the single shadow-less fruit image. Important advances made in this paper include (i) isolating the channel and spatial interdependency prediction of downsampling to prevent information loss, (ii) replacing the convolution in MRDL's Residual Blocks (ResBlocks) with a pyramid convolution with dilated convolution at many dilation rates to enrich the finite-size representation, and (iii) replacing the convolution in MRDL's Residual Blocks (ResBlocks. This iMRDL increases the robustness of classifying and localizing multiple fruit diseases. At last, extensive experiments show that the ArSAN-iMRDL model on apple, citrus, and tomato fruit images reaches 95.15%, 95.6%, and 95.43% accuracy, correspondingly, compared to the existing CNN models.

Estimating intrinsic characteristics of images for shadow removal

Existing works show that shadow removal tasks can benefit from the physical illumination model on the formation of shadows. Inspired by prior works that recover the intrinsic characteristics by decomposing an image for its reflectance and illumination components, we study a variant of shadow illumination model that can better reflect the complexity in the real world — in this model, shadow-free pixels can be expressed by a translation formed of reflectance and illumination components. Based on the new illumination model, we develop a new LR-ShadowNet, which contains two sub-nets for estimating the illumination and reflectance components, respectively, and one sub-net for refining the shadow-removal result. Besides, several mask guidance module are incorporated into LR-ShadowNet for guiding components estimation and result refinement based on shadow region information. The whole network is trained in an end-to-end fashion guided by the shadow masks. Extensive experiments on the ISTD dataset and SBU-Timelapse dataset show that the proposed LR-ShadowNet achieves competitive performance with less computational cost and strong generalisation ability.

Recasting Regional Lighting for Shadow Removal

Removing shadows requires an understanding of both lighting conditions and object textures in a scene. Existing methods typically learn pixel-level color mappings between shadow and non-shadow images, in which the joint modeling of lighting and object textures is implicit and inadequate. We observe that in a shadow region, the degradation degree of object textures depends on the local illumination, while simply enhancing the local illumination cannot fully recover the attenuated textures. Based on this observation, we propose to condition the restoration of attenuated textures on the corrected local lighting in the shadow region. Specifically, We first design a shadow-aware decomposition network to estimate the illumination and reflectance layers of shadow regions explicitly. We then propose a novel bilateral correction network to recast the lighting of shadow regions in the illumination layer via a novel local lighting correction module, and to restore the textures conditioned on the corrected illumination layer via a novel illumination-guided texture restoration module. We further annotate pixel-wise shadow masks for the public SRD dataset, which originally contains only image pairs. Experiments on three benchmarks show that our method outperforms existing state-of-the-art shadow removal methods. Project page in: yuhaoliu7456.github.io/RRL-Net.

Design and analysis of shadow detection and removal scheme in real-time still images

This research’s main objective is to study and evaluate the detection and removal of undesired shadows from still images since these shadows might mask important information caused by light sources and other obstructions. A variety of methods for detecting and eliminating shadows as well as object tracking approaches based on movement estimation and identification are investigated. This includes shadow removal methods like background subtraction, which are intended to improve obstacle recognition of the source item and increase the accuracy of shadow removal from objects. When new items enter the frame, they are first distinguished from the background using a reference frame. The tracking procedure is made more difficult by the merging of the shadow with the foreground object. The approach highlights the difficulties in object detection owing to frequent occurrences of obstacles by using morphological procedures for shadow identification and removal. The proposed approach uses feature extraction is also discussed, highlighting its importance in image processing research and the use of suggested methods to get over obstacles in image sequences. The proposed method for shadow identification and removal offers a novel approach to improve image processing when dealing with still images. The purpose of this technique is to better detect and remove shadows from images, which will increase the precision of object tracking and detection. Depending on the type of images being processed, the process begins with initializing a background model, which is based on a static image background.

Triple shadow multilinear unmixing for near-ground hyperspectral vegetation canopy shadow removal

High spatial resolution hyperspectral images are full of spatial and spectral information, which is beneficial to the identification and extraction of small changes in vegetation. The shape of spectra in the vegetation canopy region is changed by illumination and shadow, which affects the accuracy of parameter retrieval and environmental monitoring. However, existing shadow removal methods primarily target buildings, whose three-dimensional structure differs from that of the vegetation canopy. There is a requirement for research on methods of shadow removal for near-ground hyperspectral vegetation canopy regions, as existing models also suffer from over-compensation and mis-compensation when applied to these areas. Therefore, this study proposed a triple shadow multilinear mixing model-based shadow removal method (triple-SMLM). The triple-SMLM method contains three steps. For the first unmixing, the extraction of vegetation and soil endmembers from the sunlit regions served as input for the SMLM model to detect shadowed regions. For the second unmixing, the shadowed endmembers were selected using the Euclidean distance and unmixed based on the SMLM model. The shadowed regions were divided into two parts based on their abundance and vegetation pixel extraction index (VPEI): information-rich and information-poor shadowed regions. For the third unmixing, the former was compensated by spectral reconstruction based on the unmixed parameters of the previous step, while the latter was compensated by spectral reconstruction in combination with the VPEI indices after unmixing in the SMLM model. The results of the experiment are as follows: (1) The accuracy evaluation shows that the STD and RMSE of the shadow-removed image for the triple-SMLM method are 0.47 and 4.93%, and the results are more precise. (2) Compared with other methods, there is a great improvement in the shadowed boundaries of vegetation and the information-poor shadowed regions of the soil. The shadow removal performance of the triple-SMLM has potential in the vegetation canopy region of near-ground hyperspectral imagery.

Projection mapping technologies: A review of current trends and future directions.

This study summarizes current trends and future directions in projection mapping technologies. Projection mapping seamlessly merges the virtual and real worlds through projected imagery onto physical surfaces, creating an augmented reality environment. Beyond traditional applications in advertising, art, and entertainment, various fields, including medical surgery, product design, and telecommunications, have embraced projection mapping. This study categorizes recent techniques that address technical challenges in accurately replicating desired appearances on physical surfaces through projected imagery into four groups: geometric registration, radiometric compensation, defocus compensation, and shadow removal. It subsequently introduces unconventional projectors developed to resolve specific technical issues and discusses two approaches for overcoming the inherent limitations of projector hardware, such as the inability to display images floating above physical surfaces. Finally, this study concludes the discussion with possible future directions for projection mapping technologies.

Enhancing Urban Above-Ground Vegetation Carbon Density Mapping: An Integrated Approach Incorporating De-Shadowing, Spectral Unmixing, and Machine Learning

Accurately mapping urban above-ground vegetation carbon density presents challenges due to fragmented landscapes, mixed pixels, and shadows induced by buildings and mountains. To address these issues, a novel methodological framework is introduced, utilizing a linear spectral unmixing analysis (LSUA) for shadow removal and vegetation information extraction from mixed pixels. Parametric and nonparametric models, incorporating LSUA-derived vegetation fraction, are compared, including linear stepwise regression, logistic model-based stepwise regression, k-Nearest Neighbors, Decision Trees, and Random Forests. Applied in Shenzhen, China, the framework integrates Landsat 8, Pleiades 1A & 1B, DEM, and field measurements. Among the key findings, the shadow removal algorithm is effective in mountainous areas, while LSUA-enhanced models improve urban vegetation carbon density mapping, albeit with marginal gains. Integrating kNN and RF with LSUA reduces errors, and Decision Trees, especially when integrated with LSUA, outperform other models. This study underscores the potential of the proposed framework, particularly the integration of Decision Trees with LSUA, for advancing the accuracy of urban vegetation carbon density mapping.

Synthetic Document Images with Diverse Shadows for Deep Shadow Removal Networks.

Shadow removal for document images is an essential task for digitized document applications. Recent shadow removal models have been trained on pairs of shadow images and shadow-free images. However, obtaining a large, diverse dataset for document shadow removal takes time and effort. Thus, only small real datasets are available. Graphic renderers have been used to synthesize shadows to create relatively large datasets. However, the limited number of unique documents and the limited lighting environments adversely affect the network performance. This paper presents a large-scale, diverse dataset called the Synthetic Document with Diverse Shadows (SynDocDS) dataset. The SynDocDS comprises rendered images with diverse shadows augmented by a physics-based illumination model, which can be utilized to obtain a more robust and high-performance deep shadow removal network. In this paper, we further propose a Dual Shadow Fusion Network (DSFN). Unlike natural images, document images often have constant background colors requiring a high understanding of global color features for training a deep shadow removal network. The DSFN has a high global color comprehension and understanding of shadow regions and merges shadow attentions and features efficiently. We conduct experiments on three publicly available datasets, the OSR, Kligler's, and Jung's datasets, to validate our proposed method's effectiveness. In comparison to training on existing synthetic datasets, our model training on the SynDocDS dataset achieves an enhancement in the PSNR and SSIM, increasing them from 23.00 dB to 25.70 dB and 0.959 to 0.971 on average. In addition, the experiments demonstrated that our DSFN clearly outperformed other networks across multiple metrics, including the PSNR, the SSIM, and its impact on OCR performance.