Multispectral Unmixing Research Articles

Clutter rejection in passive non-line-of-sight imaging via blind multispectral unmixing.

Passive non-line-of-sight imaging methods that utilize scattered light to "look around corners" are often hindered by unwanted sources that overwhelm the weaker desired signal. Recent approaches to mitigate these "clutter" sources have exploited dependencies in the spectral content, or color, of the scattered light. A particularly successful method utilized blind source separation methods to isolate the desired imaging signal with minimal prior information. This current paper quantifies the efficacy of several preconditioning and unmixing algorithms when blind source separation methods are employed for passive multispectral non-line-of-sight imaging. Using an OLED television monitor as the source of both the desired signals and clutter, we conducted multiple controlled experiments to test these methods under a variety of scene conditions. We conclude that the preconditioner is a vital component as it greatly decreases the power and correlation of the clutter. Additionally, the choice of unmixing algorithm significantly impacts the reconstruction quality. By optimizing these two components, we find that effective image retrieval can be obtained even when the clutter signals are as much as 670 times stronger than the desired image.

Superpixel linear independent preprocessing for endmember extraction

ABSTRACT One of the limitations of remote sensing is the low spatial resolution of the open-access multispectral sensors, generating a mixture of spatial information. The mixed pixels can be modelled as the linear combination of fundamental components, called endmember, with a weighted contribution or abundance. The development of linear unmixing algorithms considering spatial and spectral information has recently increased. Some unmixing methods have relied on segmentation to integrate spatial data, and one of the most used is superpixel-based segmentation. However, previous work in superpixel-based unmixing focuses on using superpixels as uniform regions. Commonly, linear unmixing is used on hyperspectral imagery, and limited literature is found with multispectral images. This paper aims to propose a new preprocessing approach for multispectral linear unmixing called Superpixel Linear Independent Preprocessing. The proposed approach generates a set of candidates to endmembers based on spatial-spectral information; these are the input of traditional endmember extraction methods for multispectral unmixing. Experimental results show that the proposed preprocessing improves the performance of endmember extraction.

A robust multiplex immunofluorescence and digital pathology workflow for the characterisation of the tumour immune microenvironment.

Multiplex immunofluorescence is a powerful tool for the simultaneous detection of tissue-based biomarkers, revolutionising traditional immunohistochemistry. The Opal methodology allows up to eight biomarkers to be measured concomitantly without cross-reactivity, permitting identification of different cell populations within the tumour microenvironment. In this study, we aimed to validate a multiplex immunofluorescence workflow in two complementary multiplex panels and evaluate the tumour immune microenvironment in colorectal cancer (CRC) formalin-fixed paraffin-embedded tissue. We stained CRC and tonsil samples using Opal multiplex immunofluorescence on a Leica BOND RX immunostainer. We then acquired images on an Akoya Vectra Polaris and performed multispectral unmixing using inform. Antibody panels were validated on tissue microarray sections containing cores from six normal tissue types, using qupath for image analysis. Comparisons between chromogenic immunohistochemistry and multiplex immunofluorescence on consecutive sections from the same tissue microarray showed significant correlation (rs >0.9, P-value<0.0001), validating both panels. We identified many factors that influenced the quality of the acquired fluorescent images, including biomarker co-expression, staining order, Opal-antibody pairing, sample thickness, multispectral unmixing and biomarker detection order during image analysis. Overall, we report the optimisation and validation of a multiplex immunofluorescence process, from staining to image analysis, ensuring assay robustness. Our multiplex immunofluorescence protocols permit the accurate detection of multiple immune markers in various tissue types, using a workflow that enables rapid processing of samples, above and beyond previous workflows.

Photoacoustic staging of nodal metastases using SPIOs: Comparison between in vivo, in toto and ex vivo imaging in a rat model

Background and objectives: To determine prognosis and treatment, accurate nodal staging is essential in many tumor types. After injection of clinical grade superparamagnetic iron oxide (SPIO) nanoparticles, it has been shown that metastatic lymph nodes can be distinguished from benign specimens using MR imaging. However, MR does not benefit per-operative nodal staging which requires a non-ionizing, small volume, high resolution, fast imaging technique. In vivo non-invasive photoacoustic (PA) imaging of lymph nodes might facilitate nodal staging during surgery, thereby benefiting both surgeon and patient. Materials and methods: In order to investigate the feasibility of an in vivo nodal staging approach using photoacoustics, six Mat-lylu inoculated Copenhagen rats were photo-acoustically imaged after injection of a new Class IIa medical device SPIO magnetic tracer (Sienna+). Lymph nodes were imaged in vivo, in toto (after euthanization) and ex vivo using multiple wavelength illumination. Results were compared with MRI, immunohistochemistry and photographs of the sectioned nodes. Results: These experiments demonstrate that in an ex vivo setting, the PA contrast of Sienna+ is able to facilitate a distinction between metastatic and benign nodes. A non-invasive distinction between both groups is partially impeded by the low amount of PA contrast generated by the SPIO particles compared to that of endogenous absorbers such as hemoglobins. Conclusions: This comparison between in vivo, in toto and ex vivo PA imaging of lymph nodes after SPIO injection demonstrates that the clinical potential of combined PA/SPIO staging should initially be exploited in an ex vivo setting. Improved distinction between chromophores by for example multi-spectral unmixing might in the near future enable non-invasive assessment of nodal involvement.

Quantitation of Intra-peritoneal Ovarian Cancer Metastasis

Epithelial ovarian cancer (EOC) is the leading cause of death from gynecologic malignancy in the United States. Mortality is due to diagnosis of 75% of women with late stage disease, when metastasis is already present. EOC is characterized by diffuse and widely disseminated intra-peritoneal metastasis. Cells shed from the primary tumor anchor in the mesothelium that lines the peritoneal cavity as well as in the omentum, resulting in multi-focal metastasis, often in the presence of peritoneal ascites. Efforts in our laboratory are directed at a more detailed understanding of factors that regulate EOC metastatic success. However, quantifying metastatic tumor burden represents a significant technical challenge due to the large number, small size and broad distribution of lesions throughout the peritoneum. Herein we describe a method for analysis of EOC metastasis using cells labeled with red fluorescent protein (RFP) coupled with in vivo multispectral imaging. Following intra-peritoneal injection of RFP-labelled tumor cells, mice are imaged weekly until time of sacrifice. At this time, the peritoneal cavity is surgically exposed and organs are imaged in situ. Dissected organs are then placed on a labeled transparent template and imaged ex vivo. Removal of tissue auto-fluorescence during image processing using multispectral unmixing enables accurate quantitation of relative tumor burden. This method has utility in a variety of applications including therapeutic studies to evaluate compounds that may inhibit metastasis and thereby improve overall survival.

Quantitation of Intra-peritoneal Ovarian Cancer Metastasis.

Epithelial ovarian cancer (EOC) is the leading cause of death from gynecologic malignancy in the United States. Mortality is due to diagnosis of 75% of women with late stage disease, when metastasis is already present. EOC is characterized by diffuse and widely disseminated intra-peritoneal metastasis. Cells shed from the primary tumor anchor in the mesothelium that lines the peritoneal cavity as well as in the omentum, resulting in multi-focal metastasis, often in the presence of peritoneal ascites. Efforts in our laboratory are directed at a more detailed understanding of factors that regulate EOC metastatic success. However, quantifying metastatic tumor burden represents a significant technical challenge due to the large number, small size and broad distribution of lesions throughout the peritoneum. Herein we describe a method for analysis of EOC metastasis using cells labeled with red fluorescent protein (RFP) coupled with in vivo multispectral imaging. Following intra-peritoneal injection of RFP-labelled tumor cells, mice are imaged weekly until time of sacrifice. At this time, the peritoneal cavity is surgically exposed and organs are imaged in situ. Dissected organs are then placed on a labeled transparent template and imaged ex vivo. Removal of tissue auto-fluorescence during image processing using multispectral unmixing enables accurate quantitation of relative tumor burden. This method has utility in a variety of applications including therapeutic studies to evaluate compounds that may inhibit metastasis and thereby improve overall survival.

COMPRESSIVE SAMPLING MULTISPECTRAL IMAGING AND UNMIXING METHOD FOR FLUORESCENT IMAGING

Multispectral imaging is an important tool for understanding composite materials in many disciplines. Spectral unmixing enables the determination of individual fluorophore distributions. Due to the dispersive nature of biomaterials the observed spectra of fluorescent dyes is unknown. Spectral unmixing can be accomplished for unknown endmember spectra using minimum volume simplex analysis (MVSA). Compressive sampling (CS) is a method to reduce the computational cost of operating on sparse data sets and can be performed efficiently using NESTA based on Nesterov’s algorithm. Here we demonstrate that NESTA and MVSA can be combined with a denoising threshold to create a compressive sampling and multispectral unmixing (CSMIU) method that enables efficient bioimaging and unmixing with high levels of accuracy (spectral angle distances (SADs) < 0.05). This CSMIU method may potentially enable broadband and in vivo bioimaging modalities. © 2016, Electromagnetics Academy. All rights reserved.

Quantitative Multispectral Analysis Following Fluorescent Tissue Transplant for Visualization of Cell Origins, Types, and Interactions

With the desire to understand the contributions of multiple cellular elements to the development of a complex tissue; such as the numerous cell types that participate in regenerating tissue, tumor formation, or vasculogenesis, we devised a multi-colored cellular transplant model of tumor development in which cell populations originate from different fluorescently colored reporter gene mice and are transplanted, engrafted or injected in and around a developing tumor. These colored cells are then recruited and incorporated into the tumor stroma. In order to quantitatively assess bone marrow derived tumor stromal cells, we transplanted GFP expressing transgenic whole bone marrow into lethally irradiated RFP expressing mice as approved by IACUC. 0ovarian tumors that were orthotopically injected into the transplanted mice were excised 6-8 weeks post engraftment and analyzed for bone marrow marker of origin (GFP) as well as antibody markers to detect tumor associated stroma using multispectral imaging techniques. We then adapted a methodology we call MIMicc- Multispectral Interrogation of Multiplexed cellular compositions, using multispectral unmixing of fluoroprobes to quantitatively assess which labeled cell came from which starting populations (based on original reporter gene labels), and as our ability to unmix 4, 5, 6 or more spectra per slide increases, we've added additional immunohistochemistry associated with cell lineages or differentiation to increase precision. Utilizing software to detect co-localized multiplexed-fluorescent signals, tumor stromal populations can be traced, enumerated and characterized based on marker staining.

Detection of Truncated HER2 Forms in Formalin-Fixed, Paraffin-Embedded Breast Cancer Tissue Captures Heterogeneity and Is Not Affected by HER2-Targeted Therapy

Truncated forms of HER2, previously identified in subsets of HER2-positive breast cancer, originate from proteolytic extracellular domain (ECD) cleavage or alternative translation initiation. They lack ECD but may retain intracellular domain functionality, potentially associated with unfavorable prognosis, metastasis, and decreased sensitivity to antibody-based HER2-targeted therapy. To study the distribution of truncated HER2 in breast cancer, we detected loss of membrane-bound ECD independently of its molecular origin in paraffin sections, combining multispectral unmixing of chromogenic duplex IHC for HER2 ECD and intracellular domain with advanced image analysis. HER2 C-terminal fragment 611-transfected MCF7 and 4-aminophenylmercuric acetate-treated SKBR3 cell lines were used as controls. Applying a prototype work flow to whole sections, paired surgical resection/core needle biopsy samples, and paired samples from 69 patients of a phase 2 neoadjuvant clinical trial, we observed unexpected heterogeneity of ECD loss at the single-cell level, and in different areas of individual tumors, indicating that extent and localization of HER2 ECD loss add relevant information to averaging truncated HER2 across whole sections. We show acceptable run-to-run variation (coefficient of variation, <0.15), image analysis results in moderate agreement with conventional slide assessment (Cohen's κ = 0.59), and no obvious interference with previous HER2-ECD-targeted therapy. We conclude that duplex IHC and digital image processing extend current approaches of truncated HER2 detection.

Discriminating urban vegetation from a metropolitan matrix through partial unmixing with hyperspectral AVIRIS data

In the semi-arid western United States, urbanization transforms landscapes from sparsely vegetated grasslands into matrices of asphalt, concrete, turf grass, and multistrata wooded stands. Such land-cover change affects ecological processes, such as carbon (C) storage. This research investigates quantifying the vegetated and anthropogenic components of urbanized landscapes, in order to contribute ultimately to further ecological process studies, through a case study of Boulder, Colorado. Rather than map the urban land-cover types, the primary motivation was to extract biophysical information from hyperspectral imagery to understand how urbanization shifts regional biomass. Using convex geometry and partial unmixing algorithms with Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) imagery, major landscape elements were identified, including five vegetation endmembers that comprised cultivated and natural vegetation (both herbaceous and woody), soil, water, and five impervious surfaces. Urban vegetation equalled or exceeded surrounding vegetation fractional abundance. Trees were detected in the city center, and trees and grass intermingled in neighborhoods. Thus, this study expands the multispectral unmixing of the vegetation - impervious surface - soil model and demonstrates a viable method for mapping the composition of urban areas. Shifts in vegetation due to urbanization are best detected through biophysical remote sensing of actual ground components.

Noninvasive Vascular Imaging in Fluorescent Tumors Using Multispectral Unmixing

Noninvasive imaging of tumor vascularization in animal models provides an important tool for studying the biology of tumor angiogenesis as well as monitoring the effects of antiangiogenic therapies. Through the use of in vivo multispectral fluorescent imaging, we have discovered a distinct spectral signature associated with blood vessels present in fluorescent tumors in mice. This unique spectral signature allows for the tumor vasculature to be imaged and quantified without the use of vascular imaging probes. This noninvasive vascular imaging technique allows for real-time analysis of tumor vascularization, which provides a powerful and efficient tool for monitoring the effect of antiangiogenic therapies in preclinical animal models.