Optimization of SWIR Image Capture and Processing for Defect Detection in Photovoltaic Panels

Introduction Burn depth determination is a critical aspect of burn patient care but currently lacks accuracy in clinical practice. We have shown first in animal models and then in a two-patient characterization that Short Wave Infrared (SWIR) imaging, which penetrates tissue better than visual or near-infrared light and is very sensitive to water content, can distinguish between superficial and deep tissue necrosis. Here we present the findings from our pilot study of 10 patients showing the use of multi-spectral SWIR imaging of human burn injury as a potential technique for burn depth determination. Methods Ten patients admitted for mixed depth thermal injuries between 5-40% TBSA were analyzed using our SWIR assessment tool. Prior to burn excision the SWIR system was used to image burn areas and normal skin at 4 different SWIR wavelengths (Figure 1A). Standard photographs from imaged areas were collected and presented for 5 independent, blinded, surgeons’ assessments. Using the visual and SWIR images, 5-15 regions of interest (ROI) were selected from each of the burned areas and normalized to adjacent normal skin and the reflected light intensity in each ROI was averaged at each wavelength. Statistics were done using Mann-Whitney U tests. Results Visual and SWIR images were collected from burn areas in 10 patients for a total of 273 burn ROIs. The ROIs were assessed by the surgeons with ROIs being agreed upon as being superficial or superficial partial thickness (SPT) (n=47), deep partial thickness (DPT) (n=97), or full thickness (FT) (n=129) burns by a majority (60% or above) consensus. As seen in Figure 1B, the reflectance intensities (RIs) from superficial and SPT burns were significantly different than the DPT burns at the 1940 nm wavelength (p=0.05) and significantly different than FT burns at 1200 nm (p=0.008) and 1940 nm (p=0.009). When DPT and FT burns were compared there was a significant difference a 1200 nm (p=0.003) and 2250 nm (p=0.01). Conclusions Here we present a 10 patient SWIR pilot study demonstrating distinct ROIs of SWIR wavelengths for different burn depths based on consensus surgeon assessments. Overall, as burn depth increases, the 1200 and 2250 nm wavelengths show increasing RIs and 1940 nm show decreasing RIs. These results motivate further studies of SWIR imaging including pending histological analysis in the hope to non-invasively and accurately identify operative versus non-operative burns. Applicability of Research to Practice This new SWIR technology may enable us to more objectively assess burn depth, leading to improved surgical decision making and better patient outcomes.

This paper presents a novel approach for vegetation stress assessment using a combination of Short-Wave Infrared (SWIR) and the Red, Green, and Blue bands of the 8-band multispectral (MS) images from the WorldView-3 satellite. The method aims to identify stressed vegetation based on changes in water content within plant leaves. Initially, the RGB image is created from the Red, Green, and Blue bands, while the SWIR image is resampled to ensure pixel-wise correspondence with the RGB image. This process allows accurate representation of RGB pixels in the SWIR domain. Furthermore, the Silhouette Coefficient method is used on the SWIR pixels to determine the optimal number of clusters (k) for the subsequent k-Means clustering step. The Silhouette Coefficient method evaluates the cohesion and separation of clusters before applying k-Means clustering on the SWIR pixels. The SWIR band’s sensitivity to leaf water content enables effective crop health assessment, as it reflects the physiological response of stomatal closure in stressed plants. The method is tested in experimental vineyards (Vitis vinifera L.), with the last two clusters used to pseudo-colour the RGB image, highlighting stress areas in yellow. Additionally, an Unmanned Aerial Vehicle (UAV) equipped with high-resolution visible-spectrum (RGB) and Thermal Infrared (TIR) imaging was deployed to capture the area shortly after the satellite image acquisition. Despite differences in resolution, satellite, and UAV images produced consistent stress detection results. The proposed method demonstrates the potential of integrating SWIR and RGB images for precision agriculture applications, offering a robust framework for assessing and monitoring crop health based on real-time satellite imagery. It contributes to the ongoing efforts in developing remote sensing techniques for efficient crop management and resource optimization.

This paper proposes an image fusion method using a noise-robust contrast discrimination measure for night vision applications with short-wave infrared (SWIR) and long-wave infrared (LWIR) cameras. At night, SWIR images typically contain heavy noise while LWIR images contain less noise. The conventional method, which adapts the fusion ratio on the basis of only local contrasts, does not work appropriately for such SWIR and LWIR images since local contrasts of noisy SWIR images are unreliable due to the heavy noise. The proposed method adapts the fusion ratio on the basis of local contrast and noise variance. The experimental results show that the proposed method can generate visually pleasant fused images even when the SWIR images are severely degraded by noise.

The scale invariant feature transform (SIFT) is a widely used interest operator for supporting tasks such as 3D matching, 3D scene reconstruction, panorama stitching, image registration and motion tracking. Although SIFT is reported to be robust to disparate radiometric and geometric conditions in visible light imagery, using the default input parameters does not yield satisfactory results when matching imagery acquired at non-overlapping wavelengths. In this paper, optimization of the SIFT parameters for matching multi-wavelength image sets is documented. In order to integrate hyperspectral panoramic images with reference imagery and 3D data, corresponding points were required between visible light and short wave infrared images, each acquired from a slightly different position and with different resolutions and geometric projections. The default SIFT parameters resulted in too few points being found, requiring the influence of five key parameters on the number of matched points to be explored using statistical techniques. Results are discussed for two geological datasets. Using the SIFT operator with optimized parameters and an additional outlier elimination method, allowed between four and 22 times more homologous points to be found with improved image point distributions, than using the default parameter values recommended in the literature.

IntroductionDetermining the depth of skin burns in patients is critical for surgical decision making, but currently lacks accuracy in clinical practice. Short-wave infrared (SWIR) light penetrates tissue more than visual or near-infrared light and is very sensitive to water content. We have shown in animal models that imaging of skin burns in the SWIR range distinguishes between superficial and deep tissue necrosis. Here we present the first 2 cases of multi-spectral SWIR imaging of human burn injury as a first step toward a non-invasive, label-free, technique for burn depth determination.MethodsTwo subjects admitted for mixed depth, thermal, 6% and 7% total body surface area (TBSA), burns were studied. Prior to burn excision, a novel system, based on a specialized camera, imaged the burn areas and normal skin at 4 different SWIR bands. Standard photographs from imaged areas were collected and presented for 5 independent, blinded, surgeons’ assessments. In SWIR images, 3-5 regions of interest (ROIs) were selected in burned and adjacent normal skin and the reflected light intensity in each ROI was averaged.ResultsVisual and SWIR images were collected for 9 burn areas in the hands, arms, and shoulder of 2 patients (Panel A). Fifty ROIs from the burn areas were assessed by the surgeons and 30 (60%) ROIs were agreed as being superficial or superficial partial thickness (n=5), deep partial thickness (n=11), or full thickness (n=14) burns by a majority (60% or above consensus together with a possible disagreement only between deep partial and full thickness burn). In Panel B the cumulative SWIR reflectance intensity at the 4 SWIR bands for the 3 burn groups, determined by expert surgeon evaluation, and normal skin are compared. The reflectance from superficial and superficial partial thickness burns (yellow) were 102.7±1.2%, 102.3±0.7% and 103.4±1.4% of the normal skin reflectance for 1200, 1650 and 1940 nm, respectively. On the other hand, the reflectance from deep partial thickness burns (grey) were 96.7±0.1% and 94.7±0.1%, and for full thickness burn (red) were 96.1±1.4% and 93.7±1.6% of normal skin reflectance for 1650 and 1940 nm, respectively.ConclusionsWe present the first human SWIR study demonstrating a distinct reflectance intensity of SWIR wavelengths for different burn depths based on surgeon assessments. The results motivate further studies of SWIR imaging of burns in the hope to non-invasively and accurately identify operative versus non-operative burns.

Short wave infrared (SWIR) is an emerging imaging modality in surveillance applications. It is able to capture clear long range images of a subject in harsh atmospheric conditions and at night time. However, matching SWIR images against a gallery of color images is a very challenging task. The photometric properties of images in these two spectral bands are highly distinct. This work presents a novel cross-spectral face recognition scheme that encodes images filtered with a bank of Gabor filters followed by three local operators: Simplified Weber Local Descriptor, Local Binary Pattern, and Generalized Local Binary Pattern. Both magnitude and phase of filtered images are encoded. Matching encoded face images is performed by using a symmetric I-divergence. We quantify the verification and identification performance of the cross-spectral matcher on two multispectral face datasets. In the first dataset (PRE-TINDERS), both SWIR and visible gallery images are captured at a close distance (about 2 meters). In the second dataset (TINDERS), the probe SWIR images are collected at longer ranges (50 and 106 meters). The results on PRE-TINDERS dataset form a baseline for matching long range data. We also demonstrate the capability of the proposed approach by comparing its performance with the performance of Faceit G8, a commercial face recognition engine distributed by L1. The results show that the designed method outperforms Faceit G8 in terms of verification and identification rates on both datasets.

Imaging technology has been in revolutionary progresses in decades with well-developed semiconductor and memory industries. Silicon sensors are used in most of camera and DV, since silicon is the best material for visible light imaging (wavelength from 400nm∼700nm). Short wave infrared (SWIR) requires indium gallium arsenide (InGaAs), composed of chemical compounds including indium arsenide (InAs) and gallium arsenide (GaAs), to cover SWIR spectrum. Wavelength of typical SWIR is defined between 0.7um and 2.5um; SWIR cameras focus on wavelength between 0.9um∼1.7um (In0.53Ga0.47As). Unlike Mid-Wave IR and Long-Wave IR, SWIR is reflected and absorbed by objects, which advantages SWIR imaging higher resolution due to better contrast. SWIR also has excellent imaging quality in low illumination environment and moon light or star light are good emitters outdoor at night. Another primary characteristic of SWIR is high penetration, providing effective imaging under hazy conditions. An Example for night vision between SWIR.

The Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) consists of three subsystems divided by the wavelength region: Visible and Near-Infrared (VNIR), Shortwave Infrared (SWIR), and Thermal Infrared (TIR) subsystems. The VNIR, SWIR and TIR subsystems have 3, 6, and 5 spectral bands with the spatial resolution of 15, 30, and 90m, respectively. The purpose of this study is to propose an algorithm for generating SWIR and TIR imagery with a 15m resolution based on spectral similarity. In the algorithm, SWIR images are first super-resolved using VNIR images, and TIR images are then super-resolved using VNIR and super-resolved SWIR images. The first step is as follows: 1) degrade the resolution of the VNIR images to 30m by pixel aggregation with the point spread function (PSF) of SWIR, 2) generate a homogeneous pixel map with a 30m resolution from the original VNIR images, 3) generate a multi-way tree for VNIR and SWIR spectra by stepwise clustering for the 30m-resolution VNIR and SWIR images, 4) generate super-resolved SWIR images by allocating the most likely SWIR spectrum to each 15m-resolution pixel based on spectral similarity in VNIR using the 30m-resolution VNIR and SWIR images, and the multi-way tree, and 5) modify the super-resolved SWIR images using the PSF as to be fully restorable to the original images. The second step is similar, except that super-resolved TIR images are derived from both the VNIR and the super-resolved SWIR images. In the latter part of the study, the algorithm is validated using ASTER data.

Machine learning algorithms, both traditional and neuralnetwork-based, have been tested against RGB facial images for years, but these algorithms are prone to fail when illumination conditions are insufficient, for example, at night or when images are taken from long distances. Short-Wave Infra-Red (SWIR) illumination provides a much higher intensity and a much more ambient structure than visible light, which makes it better suited for face recognition in different conditions. However, current neural networks require lots of training data, which is not available in the SWIR domain. In this paper, we examine the ability of a convolutional neural network, specifically, the VGG Face network, which was trained on visible spectrum images, to work on SWIR images. Utilizing a dataset containing both RGB and SWIR images, we hypothesize that the VGG Face network will perform well both on facial images taken in RGB and SWIR wavelengths. We expect that the features extracted with VGG Face are independent of the actual wavelengths that the images were taken with. Thus, face recognition with VGG Face is possible between the RGB and SWIR domains. We find that VGG Face performs reasonable on some of the SWIR wavelengths. We can almost reach the same recognition performance when using composite images built from three SWIR wavelengths probing on RGB.

Shortwave-infrared (SWIR) imaging is reported to yield better contrast in fluorescence-guided surgery than near-infrared (NIR) imaging, due to a reduction in scattering. This benefit of SWIR was shown in animal studies, however not yet in clinical studies with patient samples. We investigate the potential benefit of SWIR to NIR imaging in clinical samples containing cetuximab-IRDye800CW in fluorescence-guided surgery. The potential of the epidermal growth factor-targeted NIR dye cetuximab-IRDye800CW in the shortwave range was examined by recording the absorption and emission spectrum. An ex vivo comparison of NIR and SWIR images using clinical tumor samples of patients with penile squamous cell carcinoma (PSCC) and head and neck squamous cell carcinoma (HNSCC) containing cetuximab-IRDye800CW was performed. The comparison was based on the tumor-to-background ratio and an adapted contrast-to-noise ratio (aCNR) using the standard of care pathology tissue assessment as the golden standard. Based on the emission spectrum, cetuximab-IRDye800CW can be detected in the SWIR range. In clinical PSCC samples, overall SWIR imaging was found to perform similarly to NIR imaging (NIR imaging is better than SWIR in the 2/7 criteria examined, and SWIR is better than NIR in the 3/7 criteria). However, when inspecting HNSCC data, NIR is better than SWIR in nearly all (5/7) examined criteria. This difference seems to originate from background autofluorescence overwhelming the off-peak SWIR fluorescence signal in HNSCC tissue. SWIR imaging using the targeted tracer cetuximab-IRDye800CW currently does not provide additional benefit over NIR imaging in ex vivo clinical samples. Background fluorescence in the SWIR region, resulting in a higher background signal, limits SWIR imaging in HNSCC samples. However, SWIR shows potential in increasing the contrast of tumor borders in PSCC samples, as shown by a higher aCNR over a line.

.The potential to image subsurface fluorescent contrast agents at high spatial resolution has facilitated growing interest in short-wave infrared (SWIR) imaging for biomedical applications. The early but growing literature showing improvements in resolution in small animal models suggests this is indeed the case, yet to date, images from larger animal models that more closely recapitulate humans have not been reported. We report the first imaging of SWIR fluorescence in a large animal model. Specifically, we imaged the vascular kinetics of an indocyanine green (ICG) bolus injection during open craniotomy of a mini-pig using a custom SWIR imaging instrument and a clinical-grade surgical microscope that images ICG in the near-infrared-I (NIR-I) window. Fluorescence images in the SWIR were observed to have higher spatial and contrast resolutions throughout the dynamic sequence, particularly in the smallest vessels. Additionally, vessels beneath a surface pool of blood were readily visualized in the SWIR images yet were obscured in the NIR-I channel. These first-in-large-animal observations represent an important translational step and suggest that SWIR imaging may provide higher spatial and contrast resolution images that are robust to the influence of blood.

The problem of face verification across the short wave infrared spectrum (SWIR) is studied in order to illustrate the advantages and limitations of SWIR face verification. The contributions of this work are two-fold. First, a database of 50 subjects is assembled and used to illustrate the challenges associated with the problem. Second, a set of experiments is performed in order to demonstrate the possibility of SWIR cross-spectral matching. Experiments also show that images captured under different SWIR wavelengths can be matched to visible images with promising results. The role of multispectral fusion in improving recognition performance in SWIR images is finally illustrated. To the best of our knowledge, this is the first time cross-spectral SWIR face recognition is being investigated in the open literature.

A hyperspectral short-wave infrared (SWIR) imager with a HgCdTe focal plane detector usually suffers from interference fringes in spectral and spatial dimension due to the etalon effect. It can cause a modulation of sensitivity higher than 15% when the spectral resolution is higher than 10 nm. In this article, a two-step method is proposed and evaluated to remove interference fringes for hyperspectral SWIR imagers. First, we extract the spectral data, conduct harmonic decomposition to suppress the fringes in spectral dimension, and calculate the correction coefficients. Second, we optimize the correction coefficients according to the distribution of the interference fringes in spatial dimension. Sample SWIR images acquired from the Advanced Hyperspectral Imager (AHSI) on China’s GaoFen-5 satellite are used to evaluate the performance of the algorithm. It shows that the proposed method can well preserve the original spectral pattern and reduce the peak-to-peak amplitude of spatial fringing from ±15% to ±4%. The method is compared with three other previously proposed methods. It turns out that the newly proposed method has strong adaptability, high accuracy, and high efficiency.

The short wave infrared (SWIR) spectral band is an emerging domain thanks to its large potential. Close to VISible/Near Infrared wavelengths, SWIR images interpretation is made easier for the users. In this spectral region, new opportunities can be found in several fields of applications such as defense and security (night vision, active imaging), space (earth observation), transport (automotive safety), or industry (nondestructive process control, food and plastic sorting). In the frame of this paper, two different developments of the InGaAs technology addressing emerging fields of SWIR imaging are described: pixel pitch reduction and multi-spectral imaging. In pixel pitch reduction the obvious objective is to increase the imaging resolution without jeopardizing system cost. Multi-spectral resolution deals, on the other hand, with interposition of pixelated filters in the optical path right onto SWIR focal plane arrays (FPA) to enable a real time spectral analysis of recorded SWIR images.

Face Recognition is the automatic processing of face images with the purpose to recognize individuals. Recognition task becomes especially challenging in surveillance applications, where images are acquired from a long range in the presence of difficult environments. Short Wave Infrared (SWIR) is an emerging imaging modality that is able to produce clear long range images in difficult environments or during night time. Despite the benefits of the SWIR technology, matching SWIR images against a gallery of visible images presents a challenge, since the photometric properties of the images in the two spectral bands are highly distinct. In this dissertation, we describe a cross spectral matching method that encodes magnitude and phase of multi-spectral face images filtered with a bank of Gabor filters. The magnitude of filtered images is encoded with Simplified Weber Local Descriptor (SWLD) and Local Binary Pattern (LBP) operators. The phase is encoded with Generalized Local Binary Pattern (GLBP) operator. Encoded multi-spectral images are mapped into a histogram representation and cross matched by applying symmetric Kullback-Leibler distance. Performance of the developed algorithm is demonstrated on TINDERS database that contains long range SWIR and color images acquired at a distance of 2, 50, and 106 meters. Apart from long acquisition range, other variations and distortions such as pose variation, motion and out of focus blur, and uneven illumination may be observed in multispectral face images. Recognition performance of the face recognition matcher can be greatly affected by these distortions. It is important, therefore, to ensure that matching is performed on high quality images. Poor quality images have to be either enhanced or discarded. This dissertation addresses the problem of selecting good quality samples. The last chapters of the dissertation suggest a number of modifications applied to the cross spectral matching algorithm for matching low resolution color images in near-real time. We show that the method that encodes the magnitude of Gabor filtered images with the SWLD operator guarantees high recognition rates. The modified method (Gabor-SWLD) is adopted in a camera network set up where cameras acquire several views of the same individual. The designed algorithm and software are fully automated and optimized to perform recognition in near-real time. We evaluate the recognition performance and the processing time of the method on a small dataset collected at WVU.