Shadow Compensation Method Research Articles

Adaptive Shadow Compensation Method in Hyperspectral Images via Multi-Exposure Fusion and Edge Fusion

Shadows in hyperspectral images lead to reduced spectral intensity and changes in spectral characteristics, significantly hindering analysis and applications. However, current shadow compensation methods face the issue of nonlinear attenuation at different wavelengths and unnatural transitions at the shadow boundary. To address these challenges, we propose a two-stage shadow compensation method based on multi-exposure fusion and edge fusion. Initially, shadow regions are identified through color space conversion and an adaptive threshold. The first stage utilizes multi-exposure, generating a series of exposure images through adaptive exposure coefficients that reflect spatial shadow intensity variations. Fusion weights for exposure images are determined based on exposure, contrast, and spectral variance. Then, the exposure sequence and fusion weights are constructed as Laplacian pyramids and Gaussian pyramids, respectively, to obtain a weighted fused exposure sequence. In the second stage, the previously identified shadow regions are smoothly reintegrated into the original image using edge fusion based on the p-Laplacian operator. To further validate the effectiveness and spectral fidelity of our method, we introduce a new hyperspectral image dataset. Experimental results on the public dataset and proposed dataset demonstrate that our method surpasses other mainstream shadow compensation methods.

Balanced Cloud Shadow Compensation Method in High-Resolution Image Combined with Multi-Level Information

As clouds of different thicknesses block sunlight, large areas of cloud shadows with varying brightness can appear on the ground. Cloud shadows in high-resolution remote sensing images lead to uneven loss of image feature information. However, cloud shadows still retain feature information, and how to compensate for and restore unbalanced cloud shadow occlusion is of great significance in improving image quality. Though traditional shadow compensation methods can enhance the shaded brightness, the results are inconsistent in a single shadow region with over-compensated or insufficient compensation problems. Thus, this paper proposes a shadow-balanced compensation method combined with multi-level information. Multi-level information comprising the information of a shadow pixel, a local super-pixel centered with the pixel, the global cloud shadow region, and the global non-shadow region information, to comply with the cloud shadow’s internal difference. First, the original image is detected via the cloud shadow detection method and post-processing. The initial shadow is detected combined with designed complex shadow features and morphological shadow index features with threshold methods. Then, post-processing considering shadow area and morphological operation is applied to remove the small, non-cloud-shadow objects. Meanwhile, the initial image is also divided into super-pixel homogeneity regions using the super-pixel segmentation principle. A super-pixel region is between the pixel and the shadow area. Different from pixel and other window regions, it can provide a different measurement levels considering object homogeneity. Thus, a balanced compensation model is designed by combining the feature value of a shadow pixel and the mean and variance of a super-pixel, shadow region, and non-shadow region on the basis of the linear correlation correction principle. The super-pixel around the shadow pixel provides a local reliable homogenous region. It can reflect the internal difference inside the shadow region. Therefore, introducing a super-pixel in the proposed model can effectively compensate for the shaded information in a balanced way. Compared to those of only using pixel and shadow region information, the compensated results introduce super-pixel information, can deal with the homogenous region as a global one, and can be adaptive to the illustration differences in a cloud shadow. The experimental results show that compared to that of other reference methods, the quality of the proposed compensation result is better. The proposed method can enhance brightness and recover detailed information in shadow regions in a more balanced way. The issue of over-compensation and insufficient compensation inside a single shadow region can be resolved. Thus, the total result is similar to that of a non-shadow region. The proposed method can be used to recover the cloud shadow information more self-adaptively to improve image quality and usage in other applications.

Shadow Compensation from UAV Images Based on Texture-Preserving Local Color Transfer

The lack of color information and texture information in the shadow region seriously affect the recognition and interpretation of remote sensing image information. The commonly used methods focus on the restoration of texture information, but it is often easy to overcompensate, resulting in color distortion of the shadow region. However, some methods only ensure accurate correction of color information, and tend to cause texture blurring. In order to not lose the texture information and to accurately compensate the color information in the shadow region of the image, we propose a shadow compensation method from UAV images based on texture-preserving local color transfer in this paper. Firstly, homogeneous regions are extracted from UAV images and homogeneous subregion segmentation is performed on the basis of homogeneous regions using the mean shift method. Secondly, in combination with the shadow mask, each shadow subregion is matched with the corresponding non-shadow subregion based on its texture features and spatial distance. Then, the matched non-shadow subregion is used as the reference region, and the color transfer based on preserving texture is performed on the shadow subregion. Finally, pixel-by-pixel width shadow compensation is applied to the penumbra region. The results of the qualitative and quantitative analysis validate the accuracy and effectiveness of the proposed methodology to compensate for the color and texture details of the shadow regions.

Hyperspectral image shadow compensation via cycle-consistent adversarial networks

Illumination variance and shadows are challenging problems in remote sensing and hyperspectral imaging applications. Shadow compensation can effectively enhance the accuracy of object detection and material classification. Most shadow compensation methods either require preprocessing to detect the shadow region, or extra knowledge collected from additional sensors. Supervised deep learning based methods require paired samples to train the network. To overcome these restrictions, this work proposes an effective cycle-consistent adversarial network for shadow compensation (SC-CycleGAN). This unsupervised method is able to automatically transfer spectra in shadow region to their nonshadow counterparts, without requiring paired training samples and the step of shadow detection. The superiority of the proposed scheme is confirmed with both laboratory-created labeled data and real airborne data.

Automatic Shadow Compensation Based on Improved Wallis Model for High Resolution Remote Sensing Images

Objectives Shadows in high-resolution remote sensing images will cause objects information loss and image quality decline, which is not beneficial for relative applications. Current shadow compensation methods often take advantage of non-shadow information around the shadow area to increase the brightness, whereas there is an unsolved problem that the contrast cannot be enhanced well enough and self-adaptively. Wallis filter principle has been used in image dodging. However, when it is used in shadow compensation, contrast improvement is not as good as other methods, leading to poor compensation results. Therefore, an improved Wallis model compensation method is proposed in this paper to enhance the brightness and contrast better to restore the shaded information. Methods First, by adding compensation strength and stretch parameters, an improved Wallis model is designed. The strength parameter is positive to the brightness and contrast, and the stretch parameter is sensitive to the contrast. Therefore, the improved Wallis model is more efficient to adjust brightness and contrast. Moreover, an automatic parameters calculation strategy is further explored to customize a suitable compensation model for each shadow area. On one hand, the brightness average and deviation of the adjacent non-shadow region are calculated and used as the compensation target values. On the other hand, based on searching the same kinds of points around shadow boundaries, a series of non-shadow and shadow feature points are matched. Assumed the feature value of the non-shadow point is the approximate value of its responding shadow point, they can be used to calculate strength and stretch parameters automatically. Lastly, the brightness of each pixel in different shadow regions will be compensated by customized models to cover the shaded information self-adaptively. Results In this paper, three sets of comparative experiments are set up to analyze and compare the part compensation methods algorithm, original Wallis method in three images with ordinary object shadows and cloud shadows, respectively. The differences in brightness average and gradient average between the compensated value and the non-shadow target value are used to evaluate the compensation quality. The experimental results show that: (1) Original Wallis model is useful to enhance brightness to some extent, while it is insufficient to improve contrast. As a consequence, the visual compensated results are not acceptable as the other two methods. (2) Part compensation method can improve brightness and contrast effectively. However, it cannot adjust self-adaptively according to each shadow regions condition, leading to some over-compensated and some insufficient compensation results in the same image. (3) The proposed method results indicate the best compensation quality in different shadow regions because the improved Wallis model is more pointed to improve contrast and the automatically calculated parameters values are suitable to customize the compensation model for each shadow region. Conclusions Aiming at the contrast improvement of the current algorithm of automatic shadow compensation, an automatic compensation method based on an improved Wallis model is proposed in this paper. The experimental results show that the newly designed model is more effective to enhance brightness and contrast, which is useful for finding suitable parameters values. Combining with the strategy of automatic parameters calculation, its customized model can adaptively compensate each shadow. However, it should be pointed out that there are still some limitations about the shadow border compensation and the internal difference in one shadow area, which need to be further studied.

Shadow Detection and Compensation from Remote Sensing Images under Complex Urban Conditions

Due to the block of high-rise objects and the influence of the sun’s altitude and azimuth, shadows are inevitably formed in remote sensing images particularly in urban areas, which causes missing information in the shadow region. In this paper, we propose a new method for shadow detection and compensation through objected-based strategy. For shadow detection, the shadow was highlighted by an improved shadow index (ISI) combined color space with an NIR band, then ISI was reconstructed by the objects acquired from the mean-shift algorithm to weaken noise interference and improve integrity. Finally, threshold segmentation was applied to obtain the shadow mask. For shadow compensation, the objects from segmentation were treated as a minimum processing unit. The adjacent objects are likely to have the same ambient light intensity, based on which we put forward a shadow compensation method which always compensates shadow objects with their adjacent non-shadow objects. Furthermore, we presented a dynamic penumbra compensation method (DPCM) to define the penumbra scope and accurately remove the penumbra. Finally, the proposed methods were compared with the stated-of-art shadow indexes, shadow compensation method and penumbra compensation methods. The experiments show that the proposed method can accurately detect shadow from urban high-resolution remote sensing images with a complex background and can effectively compensate the information in the shadow region.

Irradiance Restoration Based Shadow Compensation Approach for High Resolution Multispectral Satellite Remote Sensing Images.

Numerous applications are hindered by shadows in high resolution satellite remote sensing images, like image classification, target recognition and change detection. In order to improve remote sensing image utilization, significant importance appears for restoring surface feature information under shadow regions. Problems inevitably occur for current shadow compensation methods in processing high resolution multispectral satellite remote sensing images, such as color distortion of compensated shadow and interference of non-shadow. In this study, to further settle these problems, we analyzed the surface irradiance of both shadow and non-shadow areas based on a satellite sensor imaging mechanism and radiative transfer theory, and finally develop an irradiance restoration based (IRB) shadow compensation approach under the assumption that the shadow area owns the same irradiance to the nearby non-shadow area containing the same type features. To validate the performance of the proposed IRB approach for shadow compensation, we tested numerous images of WorldView-2 and WorldView-3 acquired at different sites and times. We particularly evaluated the shadow compensation performance of the proposed IRB approach by qualitative visual sense comparison and quantitative assessment with two WorldView-3 test images of Tripoli, Libya. The resulting images automatically produced by our IRB method deliver a good visual sense and relatively low relative root mean square error (rRMSE) values. Experimental results show that the proposed IRB shadow compensation approach can not only compensate information of surface features in shadow areas both effectively and automatically, but can also well preserve information of objects in non-shadow regions for high resolution multispectral satellite remote sensing images.

An Automatic Shadow Compensation Method via a New Model Combined Wallis Filter with LCC Model in High Resolution Remote Sensing Images

Current automatic shadow compensation methods often suffer because their contrast improvement processes are not self-adaptive and, consequently, the results they produce do not adequately represent the real objects. The study presented in this paper designed a new automatic shadow compensation framework based on improvements to the Wallis principle, which included an intensity coefficient and a stretching coefficient to enhance contrast and brightness more efficiently. An automatic parameter calculation strategy also is a part of this framework, which is based on searching for and matching similar feature points around shadow boundaries. Finally, a final compensation combination strategy combines the regional compensation with the local window compensation of the pixels in each shadow to improve the shaded information in a balanced way. All these strategies in our method work together to provide a better measurement for customizing suitable compensation depending on the condition of each region and pixel. The intensity component I also is automatically strengthened through the customized compensation model. Color correction is executed in a way that avoids the color bias caused by over-compensated component values, thereby better reflecting shaded information. Images with clouds shadows and ground objects shadows were utilized to test our method and six other state-of-the-art methods. The comparison results indicate that our method compensated for shaded information more effectively, accurately, and evenly than the other methods for customizing suitable models for each shadow and pixel with reasonable time-cost. Its brightness, contrast, and object color in shaded areas were approximately equalized with non-shaded regions to present a shadow-free image.

Deep learning based noise reduction method for automatic 3D segmentation of the anterior of lamina cribrosa in optical coherence tomography volumetric scans

A deep-learning (DL) based noise reduction algorithm, in combination with a vessel shadow compensation method and a three-dimensional (3D) segmentation technique, has been developed to achieve, to the authors best knowledge, the first automatic segmentation of the anterior surface of the lamina cribrosa (LC) in volumetric ophthalmic optical coherence tomography (OCT) scans. The present DL-based OCT noise reduction algorithm was trained without the need of noise-free ground truth images by utilizing the latest development in deep learning of de-noising from single noisy images, and was demonstrated to be able to cover more locations in the retina and disease cases of different types to achieve high robustness. Compared with the original single OCT images, a 6.6 dB improvement in peak signal-to-noise ratio and a 0.65 improvement in the structural similarity index were achieved. The vessel shadow compensation method analyzes the energy profile in each A-line and automatically compensates the pixel intensity of locations underneath the detected blood vessel. Combining the noise reduction algorithm and the shadow compensation and contrast enhancement technique, medical experts were able to identify the anterior surface of the LC in 98.3% of the OCT images. The 3D segmentation algorithm employs a two-round procedure based on gradients information and information from neighboring images. An accuracy of 90.6% was achieved in a validation study involving 180 individual B-scans from 36 subjects, compared to 64.4% in raw images. This imaging and analysis strategy enables the first automatic complete view of the anterior LC surface, to the authors best knowledge, which may have the potentials in new LC parameters development for glaucoma diagnosis and management.

The impact of shadows on partitioning of radiometric temperature to canopy and soil temperature based on the contextual two-source energy balance model (TSEB-2T).

Tests of the most recent version of the two-source energy balance model have demonstrated that canopy and soil temperatures can be retrieved from high-resolution thermal imagery captured by an unmanned aerial vehicle (UAV). This work has assumed a linear relationship between vegetation indices (VIs) and radiometric temperature in a square grid (i.e., 3.6 m × 3.6 m) that is coarser than the resolution of the imagery acquired by the UAV. In this method, with visible, near infrared (VNIR), and thermal bands available at the same high-resolution, a linear fit can be obtained over the pixels located in a grid, where the x-axis is a vegetation index (VI) and the y-axis is radiometric temperature. Next, with an accurate VI threshold that separates soil and vegetation pixels from one another, the corresponding soil and vegetation temperatures can be extracted from the linear equation. Although this method is simpler than other approaches, such as TSEB with Priestly-Taylor (TSEB-PT), it could be sensitive to VIs and the parameters that affect VIs, such as shadows. Recent studies have revealed that, on average, the values of VIs, such as normalized difference vegetation index (NDVI) and leaf area index (LAI), that are located in sunlit areas are greater than those in shaded areas. This means that involving or compensating for shadows will affect the linear relationship parameters (slope and bias) between radiometric temperature and VI, as well as thresholds that separate soil and vegetation pixels. This study evaluates the impact of shadows on the retrieval of canopy and soil temperature data from four UAV images before and after applying shadow compensation techniques. The retrieved temperatures, using the TSEB-2T approach, both before and after shadow correction, are compared to the average temperature values for both soil and canopy in each grid. The imagery was acquired by the Utah State University AggieAir UAV system over a commercial vineyard located in California as part of the USDA Agricultural Research Service Grape Remote sensing Atmospheric Profile and Evapotranspiration Experiment (GRAPEX) Program during 2014 to 2016. The results of this study show when it is necessary to employ shadow compensation methods to retrieve vegetation and soil temperature directly.

Behavior of vegetation/soil indices in shaded and sunlit pixels and evaluation of different shadow compensation methods using UAV high-resolution imagery over vineyards.

In high-resolution imagery, shadows may cause problems in object segmentation and recognition due to their low reflectance. For instance, the spectral reflectance of shadows and water are similar, particularly in the visible band. In precision agriculture, the vegetation condition in terms of plant water use, plant water stress, and chlorophyll content can be estimated using vegetation indices. Normalized Difference Vegetation Index (NDVI), Leaf Area Index (LAI) and Enhanced Vegetation Index (EVI) are widely used vegetation indices for characterizing the condition of the vegetation. In addition, many soil indices have been developed for describing soil characteristics, such as Soil-Adjusted Vegetation Index (SAVI). However, shadows can have an influence on the performance of these vegetation and soil indices.Moreover, enhancing spatial resolution heightens the impact of shadows in the imagery. In this study, the behavior of vegetation and soil indices are evaluated using four sets of high-resolution imagery captured by the Utah State University AggieAir unmanned aerial vehicle (UAV) system. These indices were obtained from flights conducted in 2014, 2015, and 2016 over a commercial vineyard located in California for the USDA Agricultural Research Service Grape Remote sensing Atmospheric Profile and Evapotranspiration Experiment (GRAPEX) Program. Different shadow restoration methods are used to alleviate the impact of shadows in information products that might be developed from the high-resolution imagery. The histogram pattern of vegetation and soil indices before and after shadow compensation, are compared using sanalysi of variance (ANOVA). The results of this study indicate how shadows can affect the vegetation/soil indices and whether shadow compensation methods are able to remove the statistical difference between sunlit and shadowed vegetation/soil indices.

Deshadowing of Urban Airborne Imagery Based on Object-Oriented Automatic Shadow Detection and Regional Matching Compensation

With the increase of the spatial resolution of aerial images, the shadow problem is more prominent. The shadows affect the applications such as object recognition, image dense matching, and object classification. Existing shadow detection methods can acquire results with high accuracy, but usually need much manual intervention and the traditional stretch models generally lead to color distortion and undercompensation. Therefore, we propose an object-oriented automatic shadow detection method without manual intervention and a shadow compensation method by regional matching. In the proposed method, pixel-based soft shadow detection, which uses Gaussian mixture model to simulate the gray distribution and refines soft shadow map with guiled filtering, is combined with image segmentation result to obtain accurate shadow regions with complete shape and no hole. Then shadow regions are compensated, with less loss of details and brightness imbalance, referring to their optimal homogeneous nonshadow region obtained by regional matching based on Bag-of-Words. The total variation model is used to decrease the noise amplified by the pixel-based stretch and boundary effect in the result. Experiments are performed on three publicly available high-resolution aerial images to demonstrate the superiority of our proposed methods. It shows that the proposed method can accurately detect shadows from urban high-resolution aerial images with an overall rate of over 88%. The compensation results display excellent visual effects compared with the state-of-the-art methods, consistent with the true color of ground objects.

A Comparison of 4 Shadow Compensation Techniques for Land Cover Classification of Shaded Areas from High Radiometric Resolution Aerial Images

Very high resolution imagery offers great possibilities for land cover/use mapping. Unfortunately, very high resolution imagery leads to significant shadowy pixels. In recent years, aerial image devices have produced high radiometric resolution data (12-bit or higher), providing more radiometric detail of potential use in classification or interpretation of land cover of shadow areas. This study evaluated ADS-40 high radiometric resolution aerial images to determine the feasibility of classifying shadow areas. We analyzed the spectral properties of land cover in shadow areas and conducted shadow-image classification comparing 4 shadow compensation methods: Method 1, no treatment, used 13-bit spectral information in shaded areas for classification; Method 2 used linear correlation correction (LCC) before the classification; Method 3 used histogram matching (HM) before the classification; and Method 4 used multisource data fusion (MSDF) to aid in classification of shadows. Subsequently, we developed a practical proposal for the processing of shadow areas. The results of the 4 shadow processing methods of image classification indicated that Method 1 and Method 2 produced good results (Method 1, overall accuracy = 90.33%; Method 2, overall accuracy = 92.33%). Therefore, the high radiometric resolution images show potential for the classification of shadow areas.Résumé. L’imagerie à très haute résolution offre de grandes possibilités pour la cartographie de la couverture terrestre ou de l’utilisation du sol. Malheureusement, les images à très haute résolution conduisent à beaucoup de pixels ombragés. Au cours des dernières années, l’instrumentation pour l’imagerie aérienne a permis la mesure de données radiométriques à haute résolution (12 bits ou plus). Ceux-ci fournissent plus de détails radiométriques potentiellement utiles dans la classification ou l’interprétation de la couverture terrestre des zones ombragées. Cette étude a évalué des images aériennes de haute résolution radiométriques ADS-40 pour déterminer la faisabilité de classification des zones ombragées. Nous avons analysé les propriétés spectrales de la couverture terrestre dans les zones ombragées et également réalisé la classification des images d’ombre en comparant 4 méthodes de compensation de l’ombre (méthode 1, aucun traitement, utilisant l’information spectrale 13-bits dans les zones ombragées pour la classification; méthode 2 utilisant la correction de corrélation linéaire «linear correlation correction» (LCC) avant la classification; méthode 3, utilisant mise en correspondance des histogrammes «histogram matching» (HM) avant la classification; et la méthode 4 utilisant la fusion de données multisources «multisource data fusion» (MSDF) pour aider à la classification des ombres). Par la suite, nous avons développé une méthode pratique pour le traitement des zones ombragées. Les résultats des 4 méthodes de traitement de l’ombre pour la classification d’image ont indiqué que la méthode 1 et la méthode 2 donnent de bons résultats (méthode 1, précision globale = 90,33 %; méthode 2, précision globale = 92,33 %). Par conséquent, les images avec une résolution radiométrique élevée permettent potentiellement la classification des zones ombragées.

Shadow determination and compensation for face recognition

Illumination variation that occurs on face images will significantly influence the performance of the recognition. Based on the low-dimensional intrinsic of face images, we design a novel two-step shadow compensation method for face recognition. Three indexes are proposed and employed to distinguish the shaded-images. Then, we compensate the shadows adaptively by using a modified Robust PCA result. Experimental results on Yale database and Yale B database demonstrate that the proposed approach can improve the recognition rate. Showing our method is suitable for face recognition with illumination variations.

Shadow compensation and illumination normalization of face image

This study proposes a novel shadow compensation and illumination normalization method under uncontrolled light conditions. First, we decompose the face image into two images based on the Lambertian theory, which corresponds to the large- and small-scale features, respectively. Then, the threshold minimum-and-maximum filter on the small-scale features to smooth the shadow edge is applied. After that, the robust Principal Component Analysis and some normalization methods are used to remove the shadow and normalize the face image on the large-scale features. In the end, the normalized face image is obtained by combining both results from the large- and small-scale features. Our main contribution is that a more reliable shadow compensation approach is found, which can get a better normalized face image. Experiments on the Extended Yale B, CMU-PIE and FRGC 2.0 (Face Recognition Grand Challenge) face datasets show that not only the recognition performance is significantly improved, but also much better visual quality is achieved.

Face Recognition under Illumination Variation Using Shadow Compensation and Pixel Selection

We propose a robust face recognition method under illumination variation. By using shadow compensation methods, we restore the image of a facial image taken under arbitrary illumination into an image that is similar to the image taken with frontal illumination. Then we apply a pixel selection method to these restored images in order to reduce the noise components, which can interfere with the extraction of discriminant features for face recognition. The experimental results for the CMU-PIE, Yale B and Multi-PIE databases show that the proposed method results in the improvement of recognition performance under illumination variation.

An Efficient Face Recognition System based on the Hybridization of Pose Invariant and Illumination Process

In the previous decade, one of the most effectual applications of image analysis and indulgent that attracted significant consideration is the human face recognition. One of the diverse techniques used for identifying an individual is the Face recognition. Normally the image variations for the reason that of the change in face identity are less than the variations between the images of the same face under different illumination and viewing angle. Among several factors that manipulate face recognition, illumination and pose are the two major challenges. Pose and illumination variations harshly affect the performance of face recognition. Considerably less effort has been taken to deal with the problem of mutual variations of pose and illumination in face recognition, while several algorithms have been proposed for face recognition from fixed points. In this paper we intend a face recognition method that is forceful to pose and illumination variations. We first put forward a simple pose estimation method based on 2D images, which uses a proper classification rule and image representation to classify a pose of a face image. After that, the image can be assigned to a pose class by a classification rule in a low-dimensional subspace constructed by a feature extraction method. We offer a shadow compensation method that compensates for illumination variation in a face image so that the image can be predictable by a face recognition system designed for images under normal illumination condition. Starting the accomplishment result, it is obvious that our projected technique based on the hybridization system recognizes the face images effectively.

Shadow Compensation Using Fourier Analysis With Application to Face Recognition

Shadows that occur on face images due to illumination variation can change the appearance of a face and degrade face recognition performance. In this paper, we propose a new shadow compensation method based on the Fourier analysis for handling illumination variation. The face image deteriorated by shadow is restored by using the auxiliary magnitude, which compensates for the magnitude of the Fourier transform distorted due to the illumination variation. The experimental results for the CMU-PIE and Yale B databases show that the proposed method results in improved recognition performance under illumination variation.