Near-infrared Multispectral Imaging Research Articles

Training State-of-the-Art Deep Learning Algorithms with Visible and Extended Near-Infrared Multispectral Images of Skin Lesions for the Improvement of Skin Cancer Diagnosis

An estimated 60,000 people die annually from skin cancer, predominantly melanoma. The diagnosis of skin lesions primarily relies on visual inspection, but around half of lesions pose diagnostic challenges, often necessitating a biopsy. Non-invasive detection methods like Computer-Aided Diagnosis (CAD) using Deep Learning (DL) are becoming more prominent. This study focuses on the use of multispectral (MS) imaging to improve skin lesion classification of DL models. We trained two convolutional neural networks (CNNs)—a simple CNN with six two-dimensional (2D) convolutional layers and a custom VGG-16 model with three-dimensional (3D) convolutional layers—using a dataset of MS images. The dataset included spectral cubes from 327 nevi, 112 melanomas, and 70 basal cell carcinomas (BCCs). We compared the performance of the CNNs trained with full spectral cubes versus using only three spectral bands closest to RGB wavelengths. The custom VGG-16 model achieved a classification accuracy of 71% with full spectral cubes and 45% with RGB-simulated images. The simple CNN achieved an accuracy of 83% with full spectral cubes and 36% with RGB-simulated images, demonstrating the added value of spectral information. These results confirm that MS imaging provides complementary information beyond traditional RGB images, contributing to improved classification performance. Although the dataset size remains a limitation, the findings indicate that MS imaging has significant potential for enhancing skin lesion diagnosis, paving the way for further advancements as larger datasets become available.

Decipherment of text on foxed documents by Hyperspectral Imaging on VSC 6000

Foxing is a process of deterioration that generates spots and browning on ancient paper documents such as books, monographs, postage stamps, certificates, etc. It is believed that metals and bacteria are the primary causes of foxing production. According to scientific studies, physical, chemical, and biological processes degrade paper. Paper is especially prone to biodeterioration processes due to its organic composition and hygroscopic nature; as a result, foxed paper becomes weaker, more fragile, and more acidic than undamaged paper. In circumstances where the text of a book or other document becomes unreadable owing to foxing, measures must be established to make the content readable. In the present study, efforts were made to understand the unreadable text content of books damaged by foxing, and the capabilities of Hyperspectral Image on VSC 6000 employing Near-Infrared Multispectral imaging analysis were presented. It is a method for optimising spectral differences that is non-invasive. First, microbiological flora (fungal and bacterial) from old books were identified, followed by the selection of the mechanism of VSC. The results revealed a correlation between fungal foxing and bacterial foxing, leading to the conclusion that fungus foxing necessitates distinct wavelength ranges between 600 and 800 nm for the decipherment of text, but bacterial foxing allows for decipherment at 645nm. Purchase Article - $10

Agarose Gel Characterization for the Fabrication of Brain Tissue Phantoms for Infrared Multispectral Vision Systems.

Synthetic phantoms that recreate the characteristics of biological tissues are valuable tools for systematically studying and comprehending physiologies, pathologies, and biological processes related to tissues. The reproduction of mechanical and optical properties allows for the development and evaluation of novel systems and applications in areas such as imaging, optics, ultrasound, or dosimetry, among others. This paper proposes a methodology for manufacturing agarose-based phantoms that mimics the optical properties of healthy brain tissue within the wavelength infrared range of 800 to 820 nm. The fabrication of such phantoms enables the possibility of testing and experimentation in controlled and safe environments toward the design of new near-infrared multispectral imaging systems in neurosurgery. The results of an experimental optical characterization study indicate the validity and reliability of the proposed method for fabricating brain tissue phantoms in a cost-effective and straightforward fashion.

336P Dual-mode near-infrared multispectral imaging system equipped with deep learning models improves the identification of cancer foci in breast cancer specimens

336P Dual-mode near-infrared multispectral imaging system equipped with deep learning models improves the identification of cancer foci in breast cancer specimens

Multispectral Imaging for Vein Localization and Contrast Enhancement.

Intravenous (IV) catheterization is a common procedure. Still, there is a 26% chance of the first attempt catheterization failure due to the changing visibility of veins because of the patient's skin tone and body fat content. Ultrasound assistive devices help locate deeper veins but are not practical in emergencies, and transillumination assistive devices have a low field of view. Commercial near-infrared (NIR) imaging devices are effective in vein localization but are expensive and are not used in low-cost clinical settings. To overcome this, NIR Multispectral Imaging (MSI) was used to find the optimal wavelength that provides the enhanced visualization of veins for all skin types and Body Mass Index (BMI). The band with the highest vein-to-skin contrast ratio was selected and contrast enhancement was done using our proposed method. The primary blocks of the proposed method are Gamma correction, Contrast Limited Adaptive Histogram Equalization (CLAHE), Adaptive Thresholding, and image Fusion. The optimal spectral range was found to be 814-876 nm and our method increased the contrast by 0.41, 0.375, and 0.39 for fair, brown, and dark brown skin types, respectively, with different BMI.Clinical relevance- From the study, we can develop a potentially low-cost vein localization assistive device for training medical and nursing students and use it in emergencies for venous access to improve confidence in IV catheterization.

Over 1000 nm Near-Infrared Multispectral Imaging System for Laparoscopic In Vivo Imaging.

In this study, a laparoscopic imaging device and a light source able to select wavelengths by bandpass filters were developed to perform multispectral imaging (MSI) using over 1000 nm near-infrared (OTN-NIR) on regions under a laparoscope. Subsequently, MSI (wavelengths: 1000–1400 nm) was performed using the built device on nine live mice before and after tumor implantation. The normal and tumor pixels captured within the mice were used as teaching data sets, and the tumor-implanted mice data were classified using a neural network applied following a leave-one-out cross-validation procedure. The system provided a specificity of 89.5%, a sensitivity of 53.5%, and an accuracy of 87.8% for subcutaneous tumor discrimination. Aggregated true-positive (TP) pixels were confirmed in all tumor-implanted mice, which indicated that the laparoscopic OTN-NIR MSI could potentially be applied in vivo for classifying target lesions such as cancer in deep tissues.

Near-infrared fundus camera with a patterned interference filter for the retinal scattering detection

The fundus camera is a system that enables retinal observations for the diagnosis of eye diseases. A near-infrared (NIR) multispectral fundus imaging system can capture color images using NIR light instead of visible light, thus avoiding irritation of the eyes and allowing for easier observations of the fundus. In this study, we developed an NIR fundus camera with a patterned metal mask for retinal patterned illumination. The pattern illuminated fundus images from the experiment prove the feasibility of retinal scattering detection using retinal patterned illumination. The layout of the patterned mask was optimized to prevent the interaction of scattered light and the multi-layer film structures of a patterned interference filter were proposed. The simulated transmission spectra showed that the transmittance of the linear stripe pattern, visual target, and fundus illumination are sufficiently high for the retinal patterned illumination.

Visible to near-infrared multispectral images dataset for image sensors design

We publish two carefully prepared and spectrally measured datasets of paint swatches. The main advantage of these datasets is that a diverse set of paint mixtures are manually prepared the way an artist may create them. The ratio of paints in each mixture is also published. The first set has 286 swatches made from 8 tubes of Old Holland oil paints in different combinations. The second set has 397 swatches made from 9 tubes of Schmincke watercolor paints. We provide exact details about the preparation of our swatches. We analyze the colorimetric and spectral properties of the two datasets in order to show the spread of the colorimetric gamut and the intrinsic, spectral dimensionality of the datasets. The dataset will be available on http://cam.mpi-inf.mpg.de/ and http://www.azadehasadi.com/.

Near-infrared multispectral endoscopic imaging of deep artificial interproximal lesions in extracted teeth.

A safer alternative method to radiographic imaging is needed. We present a multispectral near-infrared scanning fiber endoscope (nirSFE) for dental imaging which is designed to be the smallest imaging probe with near-infrared (NIR) imaging (1200-2000 nm). The prototype nirSFE is designed for wide-field forward viewing of scanned laser illumination at 1310, 1460, or 1550 nm. Artificial lesions with varying sizes and locations were prepared on proximal surfaces of extracted human teeth to examine capability and limitation of this new dental imaging modality. Nineteen artificial interproximal lesions and several natural occlusal lesions on extracted teeth were imaged with nirSFE, OCT, and microCT. Our nirSFE system has a flexible shaft as well as a probe tip with diameter of 1.6 mm and a rigid length of 9 mm. The small form factor and multispectral NIR imaging capability enables multiple viewing angles and reliable detection of lesions that can extend into the dentin. Among nineteen artificial interproximal lesions, the nirSFE reflectance imaging operating at 1460-nm and OCT operating at 1310-nm scanned illumination exhibited high sensitivity for interproximal lesions that were closer to occlusal surface. Diagnosis from a non-blinded trained user by looking at real-time occlusal-side nirSFE videos indicate true positive rate of 78.9%. There were no false positives. This study demonstrates that nirSFE may be used for detecting occlusal lesions and interproximal lesions located less than 4 mm under the occlusal surface. Major advantages of this imaging system include multiple viewing angles due to flexibility and small form factor, as well as the ability to capture real-time video. The multispectral nirSFE has the potential to be employed as a low-cost dental camera for detecting dental lesions without exposure to ionizing radiation. Lasers Surg. Med. 51:459-465, 2019. © 2019 Wiley Periodicals, Inc.

Visible and Extended Near-Infrared Multispectral Imaging for Skin Cancer Diagnosis.

With the goal of diagnosing skin cancer in an early and noninvasive way, an extended near infrared multispectral imaging system based on an InGaAs sensor with sensitivity from 995 nm to 1613 nm was built to evaluate deeper skin layers thanks to the higher penetration of photons at these wavelengths. The outcomes of this device were combined with those of a previously developed multispectral system that works in the visible and near infrared range (414 nm–995 nm). Both provide spectral and spatial information from skin lesions. A classification method to discriminate between melanomas and nevi was developed based on the analysis of first-order statistics descriptors, principal component analysis, and support vector machine tools. The system provided a sensitivity of 78.6% and a specificity of 84.6%, the latter one being improved with respect to that offered by silicon sensors.

Diffuse near-infrared imaging of tissue with picosecond time resolution.

Optical imaging of biological tissue in vivo at multiple wavelengths in the near-infrared (NIR) spectral range can be achieved with picosecond time resolution at high sensitivity by time-correlated single photon counting. Measuring and analyzing the distribution of times of flight of photons randomly propagated through the tissue has been applied for diffuse optical imaging and spectroscopy, e.g. of human breast tissue and of the brain. In this article, we review the main features and the potential of NIR multispectral imaging with picosecond time resolution and illustrate them by exemplar applications in these fields. In particular, we discuss the experimental methods developed at the Physikalisch-Technische Bundesanstalt (PTB) to record optical mammograms and to quantify the absorption and scattering properties from which hemoglobin concentration and oxygen saturation of healthy and diseased breast tissue have been derived by combining picosecond time-domain and spectral information. Furthermore, optical images of functional brain activation were obtained by a non-contact scanning device exploiting the null source-detector separation approach which takes advantage of the picosecond time resolution as well. The recorded time traces of changes in the oxy- and deoxyhemoglobin concentrations during a motor stimulation investigation show a localized response from the brain.

Polymeric nanosystems for near-infrared multispectral photoacoustic imaging: Synthesis, characterization and in vivo evaluation

Photoacoustic imaging (PAI) is a new biomedical imaging modality based on light-triggered ultrasound emission. For in vivo application, materials with good photoacustic response to illumination in the near-infrared spectrum and suited tissue delivery strategies are needed. We developed polymeric, near-infrared responsive nanomaterials tuned for in vivo application based on oxazoline block copolymer chemistry by living cationic polymerization and a related functional transformation, loaded with a new photonic material, hydrophobized phthalocyanine Zinc complex (H-PcZn), that was efficiently encapsulated into the nanoparticles by self-assembly. The resulting nanoparticles P-NPs and N-NPs bear positive, and negative surface charge, respectively. After physicochemical characterization, applicability of the two nanoparticles as photoacoustic contrast agents was evaluated in vitro and in phantom experiments, where they exhibited excellent PAI contrast. In vivo distribution and visualization of P-NPs and N-NPs following i.v. injection imaged by PAI was confirmed by cryosection fluorescence analysis and showed that the materials accumulated in tissues within 1h with differential tissue distribution. This pilot study thus describes synthesis of a novel polymeric photoacoustic nanosystem and demonstrates its potential for multimodal, photoacoustic in vivo imaging and for fluorescence imaging.

Lipid volume fraction in atherosclerotic plaque phantoms classified under saline conditions by multispectral angioscopy at near-infrared wavelengths around 1200 nm.

To identify high-risk atherosclerotic lesions, we require detailed information on the stability of atherosclerotic plaques. In this study, we quantitatively classified the lipid volume fractions in atherosclerotic plaque phantoms by a novel angioscope combined with near-infrared multispectral imaging. The multispectral angioscope was operated at peak absorption wavelengths of lipid in vulnerable plaques (1150, 1200, and 1300 nm) and at lower absorption wavelengths of water. The potential of the multispectral angioscope was demonstrated in atherosclerotic plaque phantoms containing 10-60 vol.% lipid and immersed in saline solution. The acquired multispectral data were processed by a spectral angle mapper algorithm, which enhanced the simulated plaque areas. Consequently, we classified the lipid volume fractions into five categories (0-5, 5-15, 15-30, 30-50, and 50-60 vol.%). Multispectral angioscopy at wavelengths around 1200 nm is a powerful tool for quantitatively evaluating the stability of atherosclerotic plaques based on the lipid volume fractions.

Non-Destructive Inspection of Insects in Chocolate Using near Infrared Multispectral Imaging

Foreign matters in food products strongly affect the commercial viability of food production companies. To monitor and improve the reliability of food production processes, fast and non-destructive online inspection methods are urgently required. In this study, we exploited the high transmittance of near infrared (NIR) light, and designed an NIR transmission-type imaging device that detects low-density foreign matters (such as insects) inside chocolate. Insect parts had consistently lower transmittance than chocolate parts. To enhance the important differences between the two parts, the images were processed using principal component analysis. Furthermore, an algorithm for detecting the insect parts was developed in conjunction with image threshold processing and Laplacian of Gaussian edge detection. A favourable detection rate (93%) was achieved using spectral regions 840–870 nm, 870–900 nm and 900–930 nm. The results suggested that NIR multispectral imaging is useful for detecting low-density matter (insects) in the chocolate manufacturing industry.

Facial Plethora: Modern Technology for Quantifying an Ancient Clinical Sign and Its Use in Cushing Syndrome.

Facial plethora is a clinical sign described since ancient times for a variety of diseases. In the 19th century, it was linked to increased blood volume or flow, but this has never been proven. Facial plethora is also one of the earliest described clinical features of Cushing's syndrome (CS). This study aimed to quantify facial plethora changes in CS as an early assessment of cure after surgery using noninvasive near-infrared multispectral imaging (MSI). The longitudinal cohort study was initiated in August 2012 and completed in August 2014. Clinical research hospital, National Institutes of Health. Thirty-four of the 38 patients who received surgical treatment for CS under protocol 97CH0076 during this period were included. MSI was performed on the right cheek of patients before surgery and 4.9 ± 3.1 days afterward. Average blood volume fraction as measured by MSI and serum cortisol. All but four of the 28 patients (86%) who were assessed as cured by postoperative plasma cortisol measurements of < 3 μg/dL showed a decrease in blood volume fraction (17.7 ± 0.03 vs 15.8 ± 0.03%; P = .0019), whereas an increase was seen in patients with persistent CS (18.5 ± 0.03 vs 21.4 ± 0.04%; P = .0017). Change in blood volume fraction before and after surgery was correlated with postoperative cortisol (rs = 0.58; P = .0003). Clinical data obtained from 34 patients indicate that a decrease in facial plethora after surgery, as evidenced by a decrease in blood volume fraction, is correlated with CS outcome. This novel technology for the first time identified a physiological mechanism associated with an ancient clinical sign. Furthermore, as a proof of principle, MSI is a promising early marker of cure in patients with CS that complements biochemical and clinical data.

Optimal wavelengths for near-infrared multispectral imaging of atherosclerotic plaque

Near-infrared multispectral imaging (NIR-MSI) is a potentially effective technique for evaluation of atherosclerotic plaque. In the present study, the optimal wavelength combinations in the NIR wavelength range for detecting plaque were investigated. Atherosclerotic phantoms were observed using a NIR camera at different wavelengths in the range 1150–1790 nm. It was found that the plaque phantom could be detected using just three suitably chosen wavelengths in the neighborhood of absorption peaks due to the lipid at 1210 and 1730 nm. Although higher quality images could be obtained using the peak at 1730 nm due to the stronger absorption at this wavelength, the peak at 1210 nm offered the advantage of a larger optical penetration depth.

Quantitative Evaluation of Lipid Volume Fraction in Atherosclerotic Plaque Phantoms by Near-infrared Multispectral Imaging at Wavelengths around 1200 nm

Quantitative Evaluation of Lipid Volume Fraction in Atherosclerotic Plaque Phantoms by Near-infrared Multispectral Imaging at Wavelengths around 1200 nm

Histological validation of near-infrared reflectance multispectral imaging technique for caries detection and quantification

Near infrared (NIR) multispectral imaging is a novel noninvasive technique that maps and quantifies dental caries. The technique has the ability to reduce the confounding effect of stain present on teeth. The aim of this study was to develop and validate a quantitative NIR multispectral imaging system for caries detection and assessment against a histological reference standard. The proposed technique is based on spectral imaging at specific wavelengths in the range from 1000 to 1700 nm. A total of 112 extracted teeth (molars and premolars) were used and images of occlusal surfaces at different wavelengths were acquired. Three spectral reflectance images were combined to generate a quantitative lesion map of the tooth. The maximum value of the map at the corresponding histological section was used as the NIR caries score. The NIR caries score significantly correlated with the histological reference standard (Spearman's Coefficient=0.774, p<0.01). Caries detection sensitivities and specificities of 72% and 91% for sound areas, 36% and 79% for lesions on the enamel, and 82% and 69% for lesions in dentin were found. These results suggest that NIR spectral imaging is a novel and promising method for the detection, quantification, and mapping of dental caries.

Visualizing the Effect of Gold Nanocages on Absorption, Imaging, and Lower Critical Solution Temperature Phase Transition of Individual Poly(NiPAM)-Based Hydrogel Particles by Near Infrared Multispectral Imaging Microscopy

We have successfully utilized the near-infrared multispectral imaging (NIR-MSI) microscope to observe and measure directly images and spectra of individual hydrogel particles alone or with added gold nanocages (GNs). The NIR-MSI is suited for this task because it can simultaneously record spectral and spatial information of a sample with high sensitivity (single pixel resolution) and high spatial resolution (∼0.9 μm/pixel). Because both images and spectra of the individual particles can be directly and simultaneously measured by the microscope, it is possible to detect any changes in the spectroscopic properties and/or nature (size, volume) of individual hydrogel particles induced by external factors (e.g., temperature and/or pH). These features make it possible to determine lower critical solution temperature (LCST) values based on monitoring either changes in the NIR spectra or the volume of the hydrogel particle in response to variations in temperature. More importantly, the measured volume transition temperature or LCST value is not of a collection of many hydrogel particles, but rather of individual hydrogel particles. GNs were found to significantly affect not only absorption but also images and properties of individual hydrogel particles. Specifically, GNs were found to enhance absorption of individual hydrogel particles, particularly the C-H band at 1716 nm, by about 25%. Of particular interest is the fact that not all individual hydrogel particles were enhanced by GNs; only about 50% of total number of particles were enhanced by GNs. GNs were also found to make it difficult to observe individual hydrogel particles, i.e., it seems that GNs defocused images of hydrogel particles. The defocusing effect by GNs might be due to photothermal generation of heat and vapor bubbles by the GNs. Of particular interest is the effect of GNs on the volume transition temperature of individual hydrogel particles. It seems that individual hydrogel particles lose their LCST in the presence of GNs, i.e., when heated, they undergo a gradual decrease in the volume but do not exhibit any clear and observable discontinued phase transition temperature.

Determination of Chemical Homogeneity of Fire Retardant Polymeric Nanocomposite Materials by Near-Infrared Multispectral Imaging Microscopy

Polymer nanocomposites containing layered double hydroxide (LDH) additives offer great potential for improving polymer physical properties. Of particular interest is the possibility of improving the fire retardancy and thermal stability of polymers using low loadings of this emerging class of nano-additives. Understanding the relationship between the quality of additive dispersion in the polymer matrix (i.e., chemical homogeneity) and selected flammability properties is a key question for optimizing LDHs for use in fire retardant formulations. We have demonstrated, for the first time, that the near infrared multispectral imaging (NIR-MSI) microscope can be successfully used to characterize the chemical homogeneity of a model system containing a magnesium aluminum hydroxide LDH modified with interlayer undecenoate anions mixed with poly(ethylene). The NIR-MSI is suited for this task because it can simultaneously record spectral and spatial information of a sample with high sensitivity (single pixel resolution) and high spatial resolution (∼0.9 μm/pixel). At 20% added, LDH was found to distributed inhomogeneously in a poly(ethylene) nanocomposite sample on the micron scale.