Medical hyperspectral imaging: a review

Chapter 3.6 - Hyperspectral imaging in medical applications

Hyperspectral imaging (HSI) is an emerging imaging modality for medical applications, especially in disease diagnosis and image-guided surgery. HSI acquires a three-dimensional dataset called hypercube, with two spatial dimensions and one spectral dimension. Spatially resolved spectral imaging obtained by HSI provides diagnostic information about the tissue physiology, morphology, and composition. This review paper presents an overview of the literature on medical hyperspectral imaging technology and its applications. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a specific application.

This review report represents an overview of research and development on medical hyperspectral imaging technology and its applications. Spectral imaging technology is attracting attention as a new imaging modality for medical applications, especially in disease diagnosis and image-guided surgery. Considering the recent advances in imaging, this technology provides an opportunity for two-dimensional mapping of oxygen saturation (SatO2) of blood with high accuracy, spatial spectral imaging, and its analysis and provides detection and diagnostic information about the tissue physiology and morphology. Multispectral imaging also provides information about tissue oxygenation, perfusion, and potential function during surgery. Analytical algorithm has been examined, and indication of accurate map of relative hemoglobin concentration and SatO2 can be indicated with preferable resolution and frame rate. This technology is expected to provide promising biomedical information in practical use. Several studies suggested that blood flow and SatO2 are associated with gastrointestinal disorders, particularly malignant tumor conditions. The use and analysis of spectroscopic images are expected to potentially play a role in the detection and diagnosis of these diseases.

With the continuous progress of development, deep learning has made good progress in the analysis and recognition of images, which has also triggered some researchers to explore the area of combining deep learning with hyperspectral medical images and achieve some progress. This paper introduces the principles and techniques of hyperspectral imaging systems, summarizes the common medical hyperspectral imaging systems, and summarizes the progress of some emerging spectral imaging systems through analyzing the literature. In particular, this article introduces the more frequently used medical hyperspectral images and the pre-processing techniques of the spectra, and in other sections, it discusses the main developments of medical hyperspectral combined with deep learning for disease diagnosis. On the basis of the previous review, tne limited factors in the study on the application of deep learning to hyperspectral medical images are outlined, promising research directions are summarized, and the future research prospects are provided for subsequent scholars.

Hyperspectral imaging (HSI), also called imaging spectrometer, originated from remote sensing. Hyperspectral imaging is an emerging imaging modality for medical applications, especially in disease diagnosis and image-guided surgery. HSI acquires a three-dimensional dataset called hypercube, with two spatial dimensions and one spectral dimension. Spatially resolved spectral imaging obtained by HSI provides diagnostic information about the objects physiology, morphology, and composition. The present work involves testing and evaluating the performance of the hyperspectral imaging system. The methodology involved manually taking reflectance of the object in many images or scan of the object. The object used for the evaluation of the system was cabbage and tomato. The data is further converted to the required format and the analysis is done using machine learning algorithm. The machine learning algorithms applied were able to distinguish between the object present in the hypercube obtain by the scan. It was concluded from the results that system was working as expected. This was observed by the different spectra obtained by using the machine-learning algorithm.

10677 Background: MHSI is a camera-based technique providing spectral data regarding tissue chemistry for each pixel in an image. Over 30% of women suffer local recurrence after resection. Intraoperative assessment of residual tumor & tumor grade would optimize care. Methods: We studied 42 S-D rats w/ breast tumors induced by gavage of DMBA & 15 controls. Tumors were exposed & resected, intentionally leaving ∼1mm residual tumor pieces. Gross examination, histo-pathology & MHSI (total 335) were performed for tumors, tumor beds after partial and total resection & control sites. A visible light MHSI system (HyperMed,Waltham, MA) w/ 40μm resolution & algorithms based on spectral features of the surgical field were developed and implemented for this study. Gross observation at surgery represents truth, as small tumor pieces were left intentionally by the surgeon and recorded. Samples from tumor beds were collected and histopathologically analyzed. When seen, gross tumor was removed from tumor bed by the pathologist. Results: MHSI performed well at identifying tumor. The kappa statistic(κ) for gross vs MHSI (84%) is significantly higher than κ for gross vs histopathology (76%) where for the κ the estimated asymptotic standard error is 3%. MHSI associates more strongly with gross than histopathology does. 81 tissue samples were separated into histologic grade: 0 = normal, 1 = benign tumor, 2 = intraductal Ca, 3 = papillary & cribiform Ca, 4 = papillary & cribiform Ca with invasion &/or comedo Ca. The imaging team (blinded) assigned tumor grade to each MHSI image. Statistical analysis defined 3 histologic groups: 9 normal (grade 0) tissue, 18 benign & intraductal tumors (grades 1–2), 54 advanced tumors (papillary, cribiform with invasion/comedo Ca, grades 3–4). Both histopathology & MHSI identified all 9 normal samples. Of 18 samples in group 2 (benign/intraductal by histopathology), 17 were qualified as benign/intraductal by MHSI (94% sens) & 1 as advanced. Of 54 samples with adv tumors by histopathology, MHSI identified 48 (89% sens) as advanced & 6 as intraductal. Conclusions: MHSI may provide convenient intraoperative, near real-time images with useful data about residual tumor & tumor grade. Human trials are planned. [Table: see text]

Self-supervised pre-training has been shown to improve deep learning networks in various tasks including natural language processing and computer vision. While this approach has shown promise in various fields, more development and translation need to be dedicated to medical imaging applications. Current literature scarcely focuses on thorough assessment of implemented pre-training approaches as well, potentially hindering performance in downstream tasks. In this work, we leverage a state-of-the-art pre-training architecture with hyperspectral imaging (HSI) to effectively encode spatial and spectral features of various ex vivo tissues. We utilize a masked image modeling scheme to perform pre-training on an internal dataset captured with a high-speed hyperspectral laparoscopic imaging system. Our network implements sequential spectral and spatial attention, factorizing the model for efficiency. Evaluation of both pre-training and finetuned classification was performed on a validation dataset unseen in either set to prevent data leakage. Pre-training results are qualitatively assessed through reconstruction visualization and quantitatively assessed with mean absolute error (MAE), achieving a value of 0.0294 on the validation dataset. To test the capabilities of the pre-trained model, we finetuned the network as an abdominal tissue classifier, achieving 87.9% accuracy on 17 classes with frozen model weights. Overall, we present a masked autoencoding framework for the pre-training of hyperspectral images with an emphasis on the evaluation of the network for potential improvements in downstream tasks such as tissue classification and segmentation.

Injuries to the biliary tree during surgical, endoscopic or invasive radiological diagnostic or therapeutic procedures involving the pancreas, liver or organs of the upper gastrointestinal tract give rise to the need to develop a method for clear discrimination of biliary anatomy from surrounding tissue. Hyperspectral imaging (HSI) is an emerging optical technique in disease diagnosis and image-guided surgery with inherent advantages of being a non-contact, non-invasive, and non-ionizing technique. HSI can produce quantitative diagnostic information about tissue pathology, morphology, and chemical composition. HSI was applied in human liver transplantation and compared to porcine model operations to assess the capability of discriminating biliary anatomy from surrounding biological tissue. Absorbance spectra measured from bile ducts, gall bladder, and liver show a dependence on tissue composition and bile concentration, with agreement between human and porcine datasets. The bile pigment biliverdin and structural proteins collagen and elastin were identified as contributors to the bile duct and gall bladder absorbance spectra.

709 Background: Medical Hyperspectral Imaging (MHSI) is a novel, camera-based method of imaging spectroscopy that integrates spatial & spectroscopic data from tissue in a simple image. Despite advances in surgery, adequate breast tumor resection is an issue with over 30% recurring locally. MHSI may provide noninvasive, rapid, & inexpensive evaluation of residual cancer in the tumor bed at the time of resection. Here, we demonstrate proof-of-principle in a preclinical rat model. Methods: Female Sprague-Dawley rats received 50mg/kg DMBA by gastric gavage at age 8wks to induce breast tumors. 21 rats developed multiple tumors. We studied 98 tumors and performed partial resection of 41 tumors, yielding 41 full tumor/partial resection/tumor bed sets for analysis. Tumor was exposed & resected, intentionally leaving a small (0.5mm) piece of residual tumor in the bed. Gross examination, MHSI & histopathology were recorded. MHSI using visible light system (HyperMed, Inc., Watertown, MA) provided 40μm resolution (field of view 4X6cm). Algorithms based on spectral characteristics of tissue type were developed to distinguish tumor & normal tissues. Primary endpoint was designation of tumor/normal tissue found to remain in the tumor bed by gross exam, MHSI algorithm, & histology. Results: Interim analysis on 16 full sets of tumor bed data (gross/MHSI/hist.) showed that MHSI correctly identified tumor in all cases & detected tumor in the resection bed of 0.5mm whereas standard histopathology failed to do so in 2 cases (agreement kappa = 0.75). Conclusions: MHSI may provide reliable data in near-real time with a convenient device for the surgeon in the operating room. MHSI shows promise for increasing sensitivity of detection of residual tumor over current surgical tissue sampling techniques. Follow-on studies in patients are planned. Supported by: CDMRP Concept Award, DAMD17-03-1-0767 BC024521, NIH-NIEHS I-P01 ES11624–03 Author Disclosure Employment or Leadership Consultant or Advisory Role Stock Ownership Honoraria Research Funding Expert Testimony Other Remuneration HyperMed HyperMed HyperMed HyperMed

Accurate medical image segmentation is essential for disease diagnosis, treatment planning, and surgery guidance. Hyperspectral imaging (HSI) improves medical imaging by providing rich spectral information, leading to improved tissue classification, cancer detection, and visualization of anatomical structures. Although deep learning-based segmentation models like U-Net and SegNet have advanced hyperspectral medical image analysis, they still face challenges in effectively handling spectral-spatial relationships. Recent advanced models, such as spectral transformers, knowledge distillation models (KDM), and dual-stream architecture, attempt to address these issues. However, there is a lack of comparative evaluation of these models on diverse medical hyperspectral datasets, making it difficult to determine the most effective approach. To address this gap, this study conducts a comprehensive evaluation of deep learning-based segmentation models, analyzing their performance across multiple medical hyperspectral datasets. Three different medical HSI datasets were collected: oral and dental, pathology, and brain datasets to evaluate three deep learning models designed to overcome specific segmentation challenges: long-distance dependency handling, and high spectral and spatial redundancy. The selected models were trained and tested on the three datasets and evaluated using the two metrics: the Intersection over Union (IoU) and Dice Similarity Coefficient (DSC). The results indicate that KDM achieves the best performance in the pathology dataset, while as the Dual-Stream with a Feature Pyramid Network [FPN] spatial backbone performs well in the oral and dental data sets, and the Dual-Stream with a DeepLab spatial backbone excels in the brain data set. These findings provide valuable insights into model suitability for different hyperspectral medical imaging applications.Clinical relevance- This study provides valuable insights on the importance of the medical hyperspectral imaging by identifying optimal segmentation models for pathology, oral and dental, and brain imaging, improving disease diagnosis and treatment planning.

.SignificanceHyperspectral reflectance imaging can be used in medicine to identify tissue types, such as tumor tissue. Tissue classification algorithms are developed based on, e.g., machine learning or principle component analysis. For the development of these algorithms, data are generally preprocessed to remove variability in data not related to the tissue itself since this will improve the performance of the classification algorithm. In hyperspectral imaging, the measured spectra are also influenced by reflections from the surface (glare) and height variations within and between tissue samples.AimTo compare the ability of different preprocessing algorithms to decrease variations in spectra induced by glare and height differences while maintaining contrast based on differences in optical properties between tissue types.ApproachWe compare eight preprocessing algorithms commonly used in medical hyperspectral imaging: standard normal variate, multiplicative scatter correction, min–max normalization, mean centering, area under the curve normalization, single wavelength normalization, first derivative, and second derivative. We investigate conservation of contrast stemming from differences in: blood volume fraction, presence of different absorbers, scatter amplitude, and scatter slope—while correcting for glare and height variations. We use a similarity metric, the overlap coefficient, to quantify contrast between spectra. We also investigate the algorithms for clinical datasets from the colon and breast.ConclusionsPreprocessing reduces the overlap due to glare and distance variations. In general, the algorithms standard normal variate, min–max, area under the curve, and single wavelength normalization are the most suitable to preprocess data used to develop a classification algorithm for tissue classification. The type of contrast between tissue types determines which of these four algorithms is most suitable.

Hyperspectral imaging (HSI) is a promising technology that can provide valuable support for the advancement of the medical field. Bibliometrics can analyze a vast number of publications on both macroscopic and microscopic levels, providing scholars with essential foundations to shape future directions. The purpose of this study is to comprehensively review the existing literature on medical hyperspectral imaging (MHSI). Based on the Web of Science (WOS) database, this study systematically combs through literature using bibliometric methods and visualization software such as VOSviewer and CiteSpace to draw scientific conclusions. The analysis yielded 2,274 articles from 73 countries/regions, involving 7,401 authors, 2,037 institutions, 1,038 journals/conferences, and a total of 7,522 keywords. The field of MHSI is currently in a positive stage of development and has conducted extensive research worldwide. This research encompasses not only HSI technology but also its application to diverse medical research subjects, such as skin, cancer, tumors, etc., covering a wide range of hardware constructions and software algorithms. In addition to advancements in hardware, the future should focus on the development of algorithm standards for specific medical research targets and cultivate medical professionals of managing vast amounts of technical information.

Introduction: Adequate evaluation of breast tumor resection at surgery continues to be an important issue in surgical care, as over 30% of postoperative tumors recur locally unless radiation is used to destroy remaining tumor cells in the field. Medical Hyperspectral Imaging (MHSI) delivers near-real time images of biomarkers in tissue, providing an assessment of pathophysiology and the potential to distinguish different tissues based on spectral characteristics.Method: We have used an experimental DMBA-induced rat breast tumor model to examine the intraoperative utility of MHSI, in distinguishing tumor from normal breast and other tissues. Rats bearing tumors underwent surgical exposure and MHSI imaging, followed by partial resection of the tumors, then MHSI imaging of the resection bed, and finally total resection of tumors and of grossly normal-appearing glands. Resected tissue underwent gross examination, MHSI imaging, and histopathological evaluation.Results: An algorithm based on spectral characteristics of tissue types was developed to distinguish between tumor and normal tissues. Tissues including tumor, blood vessels, muscle, and connective tissue were clearly identified and differentiated by MHSI. Fragments of residual tumor 0.5 - 1 mm in size intentionally left in the operative bed were readily identified. MHSI demonstrated a sensitivity of 89% and a specificity of 94% for detection of residual tumor, comparable to that of histopathological examination of the tumor bed (85% and 92%, respectively).Conclusion: We conclude that MHSI may be useful in identifying small residual tumor in a tumor resection bed and for indicating areas requiring more extensive resection and more effective biopsy locations to the surgeon. 3

Hyperspectral imaging (HSI) is a technology able to measure information about the spectral reflectance or transmission of light from the surface. The spectral data, usually within the ultraviolet and infrared regions of the electromagnetic spectrum, provide information about the interaction between light and different materials within the image. This fact enables the identification of different materials based on such spectral information. In recent years, this technology is being actively explored for clinical applications. One of the most relevant challenges in medical HSI is the information extraction, where image processing methods are used to extract useful information for disease detection and diagnosis. In this chapter, we provide an overview of the information extraction techniques for HSI. First, we introduce the background of HSI, and the main motivations of its usage for medical applications. Second, we present information extraction techniques based on both light propagation models within tissue and machine learning approaches. Then, we survey the usage of such information extraction techniques in HSI biomedical research applications. Finally, we discuss the main advantages and disadvantages of the most commonly used image processing approaches and the current challenges in HSI information extraction techniques in clinical applications.

Hyperspectral imaging (HSI) is a powerful imaging technique for biomedical applications, such as disease detection, diagnosis, and surgery assistance. HSI provides a three-dimensional dataset (two spatial and one spectral), which allows to obtain spectral curve at each pixel in acquired images. Spectral imaging data offers diagnostic information on physiology, morphology, and composition of the biological tissues. Light in the near-infrared (NIR) region, ranging from 800–2500 nm, is useful for probing deep parts of tissue due to its lower absorption and scattering, and the absorption spectra in the NIR region convey fingerprint data due to overtone or combination vibrations of chemical bonds. This enables to investigate the distribution of chemical composition within a sample, as well as to provide key information for detecting cancer that locates beneath submucosal layer. This chapter reviews the basics of HSI, HSI acquisition systems, and biomedical NIR-HSI applications.