Specular Reflection Removal Research Articles

Enhancing Meibography Image Analysis Through Artificial Intelligence-Driven Quantification and Standardization for Dry Eye Research.

This study enhances Meibomian gland (MG) infrared image analysis in dry eye (DE) research through artificial intelligence (AI). It is comprised of two main stages: automated eyelid detection and tarsal plate segmentation to standardize meibography image analysis. The goal is to address limitations of existing assessment methods, bridge the curated and real-world dataset gap, and standardize MG image analysis. The approach involves a two-stage process: automated eyelid detection and tarsal plate segmentation. In the first stage, an AI model trained on curated data identifies relevant eyelid areas in non-curated datasets. The second stage refines the eyelid area in meibography images, enabling precise comparisons between normal and DE subjects. This approach also includes specular reflection removal and tarsal plate mask refinement. The methodology achieved a promising instance-wise accuracy of 80.8% for distinguishing meibography images from 399 DE and 235 non-DE subjects. By integrating diverse datasets and refining the area of interest, this approach enhances meibography feature extraction accuracy. Dimension reduction through Uniform Manifold Approximation and Projection (UMAP) allows feature visualization, revealing distinct clusters for DE and non-DE phenotypes. The AI-driven methodology presented here quantifies and classifies meibography image features and standardizes the analysis process. By bootstrapping the model from curated datasets, this methodology addresses real-world dataset challenges to enhance the accuracy of meibography image feature extraction. The study presents a standardized method for meibography image analysis. This method could serve as a valuable tool in facilitating more targeted investigations into MG characteristics.

EndoSRR: a comprehensive multi-stage approach for endoscopic specular reflection removal.

Specular reflections in endoscopic images not only disturb visual perception but also hamper computer vision algorithm performance. However, the intricate nature and variability of these reflections, coupled with a lack of relevant datasets, pose ongoing challenges for removal. We present EndoSRR, a robust method for eliminating specular reflections in endoscopic images. EndoSRR comprises two stages: reflection detection and reflection region inpainting. In the reflection detection stage, we adapt and fine-tune the segment anything model (SAM) using a weakly labeled dataset, achieving an accurate reflection mask. For reflective region inpainting, we employ LaMa, a fast Fourier convolution-based model trained on a 4.5M-image dataset, enabling effective inpainting of arbitrarily shaped reflection regions. Lastly, we introduce an iterative optimization strategy for dual pre-trained models to refine the results of specular reflection removal, named DPMIO. Utilizing the SCARED-2019 dataset, our approach surpasses state-of-the-art methods in both qualitative and quantitative evaluations. Qualitatively, our method excels in accurately detecting reflective regions, yielding more natural and realistic inpainting results. Quantitatively, our method demonstrates superior performance in both segmentation evaluation metrics (IoU, E-measure, etc.) and image inpainting evaluation metrics (PSNR, SSIM, etc.). The experimental results underscore the significance of proficient endoscopic specular reflection removal for enhancing visual perception and downstream tasks. The methodology and results presented in this study are poised to catalyze advancements in specular reflection removal, thereby augmenting the accuracy and safety of minimally invasive surgery.

Multispectral Depolarization Based Living-Skin Detection: A New Measurement Principle.

Camera-based photoplethysmographic imaging enabled the segmentation of living-skin tissues in a video, but it has inherent limitations to be used in real-life applications such as video health monitoring and face anti-spoofing. Inspired by the use of polarization for improving vital signs monitoring (i.e. specular reflection removal), we observed that skin tissues have an attractive property of wavelength-dependent depolarization due to its multi-layer structure containing different absorbing chromophores, i.e. polarized light photons with longer wavelengths (R) have deeper skin penetrability and thus experience thorougher depolarization than those with shorter wavelengths (G and B). Thus we proposed a novel dual-polarization setup and an elegant algorithm (named "MSD") that exploits the nature of multispectral depolarization of skin tissues to detect living-skin pixels, which only requires two images sampled at the parallel and cross polarizations to estimate the characteristic chromaticity changes (R/G) caused by tissue depolarization. Our proposal was verified in both the laboratory and hospital settings (ICU and NICU) focused on anti-spoofing and patient skin segmentation. The clinical experiments in ICU also indicate the potential of MSD for skin perfusion analysis, which may lead to a new diagnostic imaging approach in the future.

Endometriosis detection and localization in laparoscopic gynecology

Endometriosis is a common gynecologic condition typically treated via laparoscopic surgery. Its visual versatility makes it hard to identify for non-specialized physicians and challenging to classify or localize via computer-aided analysis. In this work, we take a first step in the direction of localized endometriosis recognition in laparoscopic gynecology videos using region-based deep neural networks Faster R-CNN and Mask R-CNN. We in particular use and further develop publicly available data for transfer learning deep detection models according to distinctive visual lesion characteristics. Subsequently, we evaluate the performance impact of different data augmentation techniques, including selected geometrical and visual transformations, specular reflection removal as well as region tracking across video frames. Finally, particular attention is given to creating reasonable data segmentation for training, validation and testing. The best performing result surprisingly is achieved by randomly applying simple cropping combined with rotation, resulting in a mean average segmentation precision of 32.4% at 50-95% intersection over union overlap (64.2% for 50% overlap).

A New Polarization-Based Vegetation Index to Improve the Accuracy of Vegetation Health Detection by Eliminating Specular Reflection of Vegetation

Monitoring chlorophyll content changes in the plant via remote sensing is of great significance for understanding plant growth, monitoring vegetation pests and diseases, which is an important method to study the global climate change. However, the monitored information is often interfered by leaf specular reflection, resulting in reduced accuracy of chlorophyll content inversion. In this article, to eliminate the interference of specular reflection in vegetation remote sensing, a polarized multispectral imaging system (PMSIS) used in the different-light-level situation to observe vegetation was developed, and a new specular reflection removal vegetation index (NSRVI) was proposed to better detect the vegetation health condition under specular reflection interference. Based on previous studies, several vegetation indices (Simple ratio index (SR), Normalized difference vegetation index (NDVI); mSR, mNDVI (ref. [41]); pSR, pNDVI (ref. [46]); and NSRVI) were established, and the impact of specular reflection on vegetation health detection was evaluated. Correlation analysis was done on Relative Chlorophyll content (SPAD), SR, NDVI, mSR, mNDVI, pSR, pNDVI and NSRVI to understand their potential ability to eliminate specular interference. The results show that SR and NDVI have the highest sensitivity to specular reflection, and the other three methods can alleviate the adverse effects of specular reflection to varying degrees. It was observed that NSRVI was well correlated with SPAD (coefficient of determination (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) = 0.899, RMSE=6.16), highlighting the potential of NRSVI in eliminating specular reflection interference and identifying vegetation health condition. In summary, this method can effectively eliminate specular interference and improve the detection accuracy of vegetation health condition.

Automatic Acetowhite Lesion Segmentation via Specular Reflection Removal and Deep Attention Network.

Automatic acetowhite lesion segmentation in colposcopy images (cervigrams) is essential in assisting gynecologists for the diagnosis of cervical intraepithelial neoplasia grades and cervical cancer. It can also help gynecologists determine the correct lesion areas for further pathological examination. Existing computer-aided diagnosis algorithms show poor segmentation performance because of specular reflections, insufficient training data and the inability to focus on semantically meaningful lesion parts. In this paper, a novel computer-aided diagnosis algorithm is proposed to segment acetowhite lesions in cervigrams automatically. To reduce the interference of specularities on segmentation performance, a specular reflection removal mechanism is presented to detect and inpaint these areas with precision. Moreover, we design a cervigram image classification network to classify pathology results and generate lesion attention maps, which are subsequently leveraged to guide a more accurate lesion segmentation task by the proposed lesion-aware convolutional neural network. We conducted comprehensive experiments to evaluate the proposed approaches on 3045 clinical cervigrams. Our results show that our method outperforms state-of-the-art approaches and achieves better Dice similarity coefficient and Hausdorff Distance values in acetowhite legion segmentation.

Robust specular reflection removal and visibility enhancement of endoscopic images using 3-channel thresholding technique and image inpainting

Specular reflections create artifacts in endoscopic images, which may lead to misdiagnosis. In this paper, we propose a method for robust removal of specular reflections by using a thresholding technique in each of the RGB channels to segment the specular reflections from images. We further use dilation to ensure full local segmentation and inpainting to replace the areas of reflections with non-specular regions. Our method also provides a visibility enhancement feature to improve the decreased brightness due to the reflection removal by using the gamma-correction, histogram shift, and histogram equalization. On the Iparkmall Clinic dataset, our method has achieved average Peak-to-Signal-Noise Ratio (PSNR) of 42.62dB with a standard deviation of 5.80 dB and a minimum value of 23.3 dB. The average processing time was 219ms, enabling average 4 5 frames per second (FPS) processing speed on an Intel i7 processor.

A fast specular reflection removal based on pixels properties method

Iris recognition has been around for many years due to an extensive research on the uniqueness of human iris. It is well known that the iris is not similar to each other which means every human in the planet has their own iris pattern and cannot be shared. One of the main issues in iris recognition is iris segmentation. One element that can reduce the accuracy of iris segmentation is the presence of specular reflection. Another issue is the speed of specular reflection removal since the iris recognition system needs to process a lot of irises. In this paper, a specular reflection removal method was proposed to achieve a fast and accurate specular reflection removal. Some modifications were implemented on the existing pixels properties method. Based on the results, the proposed method achieved the fastest execution time, the highest segmentation accuracy and the highest SSIM compared to the other methods. This proves that the proposed method is fast and accurate to be implemented in the iris recognition systems.

Classification of retinal fundus image using MS-DRLBP features and CNN-RBF classifier

The most common retinal diseases that are to be diagnosed are Diabetic Retinopathy (DR), Age-related Macular Degeneration (AMD) and Choroidal Neovascularization (CNV). For the people above 60 years of age, detection of these retinal diseases is an important task for treatment that reduces the risk of vision loss. Retinal fundus images play a significant role in the detection of DR, AMD and CNV disease diagnosis and treatment. The existing techniques for the detection of DR, AMD and CNV have not fulfilled with the classification accuracy of the retinal diseases effectively. This research work proposes an efficient classification framework for retinal fundus image recognition to overcome these drawbacks. Initially, the input image from the publicly available STARE database is preprocessed with the following three steps (a) Specular reflection removal and smoothing, (b) contrast enhancement and (c) retinal region expansion. With the preprocessed image, the features are extracted using Multi-Scale Discriminative Robust Local Binary Pattern (MS-DRLBP), based on RGB component selection, Gradient operation, and LBP descriptor. Finally, classification was done using hybrid Convolution Neural Network (CNN) and Radial Basis Function (RBF) model (CNN-RBF) which classifies the retinal fundus images into four classes such as DR, AMD, CNV and Normal (NR). Experimental results of the proposed method gives an accuracy of 97.22% compared with the existing other methodologies.

Medical Image Inpainting Using Multi-Scale Patches and Neural Networks Concepts

In this paper, we consider the problem of medical images inpainting, where the goal is to reconstruct missing or damaged parts of the image. This is a good tool for medical applications such as vascular restoration, removal of specular reflections for endoscopic images, removal of MRI artifacts, etc. The new method combines the search for patches of various sizes and the operation of a pre-trained neural network. The large patches are used to reconstruct homogeneous areas, and then small patches are used to reconstruct structure image details. As a result, the proposed method provides a plausible of medical images inpainting. Experimental results demonstrate the effectiveness of the proposed method in the tasks of medical images inpainting.

ReTouchImg: Fusioning from-local-to-global context detection and graph data structures for fully-automatic specular reflection removal for endoscopic images

Minimally invasive surgical and diagnostic systems are commonly used in clinical practices. However, the accuracy and robustness of these systems depend heavily on computer based processes such as tracking, detecting or segmenting clinically meaningful regions of interest, which are significantly affected by the inherent specular reflections that appear on the organs’ surfaces. Restoration of the acquired data for clinical purposes still presents challenges because of the high texture and color variations across the image. In this work, we propose a novel fully-automated solution for endoscopic image restoration, which we call ReTouchImg. Our approach is designed as a two-step scheme. The first is a detection step that is based on the synergy of a set of color variations and gradient information conditions. For the second step, we introduce an inpainting process which is based on graph data structures for recovering the missing information. We exhaustively evaluate our approach on real endoscopic datasets and compare it against some works from the body of literature. We also demonstrate that our solution deals with complex cases such as strong illumination variation and large affected areas through a careful quantitative evaluation of a range of numerical results.

Computer-Aided Diagnosis of Anterior Segment Eye Abnormalities using Visible Wavelength Image Analysis Based Machine Learning.

Eye disease is a major health problem among the elderly people. Cataract and corneal arcus are the major abnormalities that exist in the anterior segment eye region of aged people. Hence, computer-aided diagnosis of anterior segment eye abnormalities will be helpful for mass screening and grading in ophthalmology. In this paper, we propose a multiclass computer-aided diagnosis (CAD) system using visible wavelength (VW) eye images to diagnose anterior segment eye abnormalities. In the proposed method, the input VW eye images are pre-processed for specular reflection removal and the iris circle region is segmented using a circular Hough Transform (CHT)-based approach. The first-order statistical features and wavelet-based features are extracted from the segmented iris circle and used for classification. The Support Vector Machine (SVM) by Sequential Minimal Optimization (SMO) algorithm was used for the classification. In experiments, we used 228 VW eye images that belong to three different classes of anterior segment eye abnormalities. The proposed method achieved a predictive accuracy of 96.96% with 97% sensitivity and 99% specificity. The experimental results show that the proposed method has significant potential for use in clinical applications.

Efficient road specular reflection removal based on gradient properties

Highlights caused by changes in sunlight throughout any given day cause failure in stereo matching, object recognition, and road segmentation. This is a serious challenge in advanced driver assistance systems (ADAS), because local high brightness and color discontinuities generally result in noticeable blurring of the road surface or object. This paper presents a novel strategy for removing specular reflection from highlight images by gradients distribution to optimize the diffuse image. The dark channel is introduced as a prior to initially estimate and locate the highlight. The threshold filter is then adopted to divide the high-intensity highlight and the weak highlight - the weak highlight affect neither the stereo matching nor road segmentation process. Finally, gradient properties (varying smoothness of specular and diffuse reflections) are presented to optimize the layer separation. Experimental results in speed and accuracy of road segmentation show that proposed method outperforms other techniques for separating highlights from road surfaces.

Polarization-sensitive laser feedback interferometry for specular reflection removal.

Specular reflection from the surface of targets or prepared specimens represents a significant problem in optical microscopy and related optical imaging techniques as usually the surface reflection does not contribute to the desired signal. Solutions exist for many of these imaging techniques; however, remedial techniques for imaging based on laser feedback interferometry (LFI) are absent. We propose a reflection cancellation technique based on crossed-polarization filtering that is tailored for a typical LFI configuration. The technique is validated with three experimental designs, and a significant improvement of about 40dB in the ratio of the diffuse and specular LFI signal is observed. Applications of this principle extend from specular reflection removal to characterization of target materials in industrial to biomedical domains.

Specular reflection removal of ocean surface remote sensing images from UAVs

Images captured by UAVs above the sea surface often contain lots of highlight regions due to the specular reflection of solar radiation on the non-flat sea surface. The existence of a great deal of specular highlight components may cover the objects under the water which is negative to those applications based on the UAV remote sensing images. In this paper, we present a method to remove the specular reflection on the RGB images of ocean surface. The intensity of specular highlight components is much larger than that of diffuse components in the images, simply subtracting the highlight component form the original image will leave a lot of holes. So our method contains two main steps: highlight regions detection and restoration of those regions. We use the method based on the intensity ratio to extract the regions affected by the specular reflection. Then we use the local information around those highlight regions to restore the intensity of those pixels. The experimental results indicate that the proposed method can effectively remove the specular reflection and keep details of ocean surface images.

Removal of specular reflections from image sequences using feature correspondences

The presence of specular highlights can hide underlying features of a scene within an image and can be problematic in many application scenarios. In particular, this poses a significant challenge for applications where image stitching is used to create a single static image of a scene from inspection footage of pipes, gas tubes, train tracks and concrete structures. Furthermore, they can hide small defects in the images causing them to be missed during inspection. We present a method which exploits additional information in neighbouring frames from video footage to reduce specularity from each frame. The technique first automatically determines frames which contain overlapping regions before the relationship that exists between them is exploited in order to suppress the effects of specular reflections. This results in an image that is free from specular highlights provided there is at least one frame present in the sequence where a given pixel is present in a diffuse form. The method is shown to work well on greyscale as well as colour images and effectively reduces specularity and significantly improves the quality of the stitched image, even in the presence of noise. While applied to the challenge of reducing specularity in inspection videos, the method improves upon the state-of-the-art in specularity removal, and its applications are wide-ranging as a general purpose pre-processing tool.

Light Field Imaging Based Accurate Image Specular Highlight Removal

Specular reflection removal is indispensable to many computer vision tasks. However, most existing methods fail or degrade in complex real scenarios for their individual drawbacks. Benefiting from the light field imaging technology, this paper proposes a novel and accurate approach to remove specularity and improve image quality. We first capture images with specularity by the light field camera (Lytro ILLUM). After accurately estimating the image depth, a simple and concise threshold strategy is adopted to cluster the specular pixels into “unsaturated” and “saturated” category. Finally, a color variance analysis of multiple views and a local color refinement are individually conducted on the two categories to recover diffuse color information. Experimental evaluation by comparison with existed methods based on our light field dataset together with Stanford light field archive verifies the effectiveness of our proposed algorithm.

Efficient iris localisation using a guided filter

Biometric information is widely used in user identification systems. Iris is one of the most reliable and accurate biometric information. In an iris recognition system, the iris localisation is one the most important parts because the performance of an iris recognition system is highly dependent on the accuracy of iris localisation. If unreliable iris regions are used in the iris recognition system, the recognition rate may be degraded. Therefore many researchers have studied the iris localisation methods. Especially, localising an iris region from noisy images is one of the hot topics in the iris recognition researches. In this study, the authors are concentrating on the iris localisation, where an efficient iris localisation method for noisy iris images is proposed. The proposed iris localisation method consists of two steps: pupil boundary localisation and iris boundary localisation. To localise a pupil region, an efficient block-based minimum energy detection method is used, in which specular reflection removal is performed as a preprocessing. Iris boundary is localised using a guided filter, the circular Hough transform and an ellipse fitting method. Experimental results with various test image sets show the effectiveness of the proposed method.

Weighted Nuclear Norm Minimization Based Tongue Specular Reflection Removal

In computational tongue diagnosis, specular reflection is generally inevitable in tongue image acquisition, which has adverse impact on the feature extraction and tends to degrade the diagnosis performance. In this paper, we proposed a two-stage (i.e., the detection and inpainting pipeline) approach to address this issue: (i) by considering both highlight reflection and subreflection areas, a superpixel-based segmentation method was adopted for the detection of the specular reflection areas; (ii) by extending the weighted nuclear norm minimization (WNNM) model, a nonlocal inpainting method is proposed for specular reflection removal. Experimental results on synthetic and real images show that the proposed method is accurate in detecting the specular reflection areas and is effective in restoring tongue image with more natural texture information of tongue body.

Separation of specular and diffuse components using tensor voting in color images.

Most methods for the detection and removal of specular reflections suffer from nonuniform highlight regions and/or nonconverged artifacts induced by discontinuities in the surface colors, especially when dealing with highly textured, multicolored images. In this paper, a novel noniterative and predefined constraint-free method based on tensor voting is proposed to detect and remove the highlight components of a single color image. The distribution of diffuse and specular pixels in the original image is determined using tensors' saliency analysis, instead of comparing color information among neighbor pixels. The achieved diffuse reflectance distribution is used to remove specularity components. The proposed method is evaluated quantitatively and qualitatively over a dataset of highly textured, multicolor images. The experimental results show that our result outperforms other state-of-the-art techniques.