Linear Unmixing Algorithm Research Articles

Two-site microendoscopic imaging probe for simultaneous three-dimensional imaging at two anatomic locations in tissues.

Systems that can image in three dimensions at cellular resolution and across different locations within an organism may enable insights into complex biological processes, such as immune responses, for which a single location measurement may be insufficient. In this Letter, we describe an in vivo two-site imaging probe (TIP) that can simultaneously image two anatomic sites with a maximum separation of a few centimeters. The TIP consists of two identical bendable graded index (GRIN) lenses and is demonstrated by a two-photon two-color fluorescence imaging system. Each GRIN lens has a field of view of 162 × 162 × 170 µm3, a nominal numerical aperture of 0.5, a magnification of 0.7, and working distances of 0.2 mm in air for both ends. A blind linear unmixing algorithm is applied to suppress bleedthrough between channels. We use this system to successfully demonstrate two-site two-photon two-color imaging of two biomedically relevant samples, i.e., (1) a mixture of two autofluorescent anti-cancer drugs and (2) a live hybrid tumor consisting of two spectrally distinct fluorescent cell lines.

Superpixel linear independent preprocessing for endmember extraction

ABSTRACT One of the limitations of remote sensing is the low spatial resolution of the open-access multispectral sensors, generating a mixture of spatial information. The mixed pixels can be modelled as the linear combination of fundamental components, called endmember, with a weighted contribution or abundance. The development of linear unmixing algorithms considering spatial and spectral information has recently increased. Some unmixing methods have relied on segmentation to integrate spatial data, and one of the most used is superpixel-based segmentation. However, previous work in superpixel-based unmixing focuses on using superpixels as uniform regions. Commonly, linear unmixing is used on hyperspectral imagery, and limited literature is found with multispectral images. This paper aims to propose a new preprocessing approach for multispectral linear unmixing called Superpixel Linear Independent Preprocessing. The proposed approach generates a set of candidates to endmembers based on spatial-spectral information; these are the input of traditional endmember extraction methods for multispectral unmixing. Experimental results show that the proposed preprocessing improves the performance of endmember extraction.

Caged luciferins enable rapid multicomponent bioluminescence imaging.

Bioluminescence is a sensitive technique for imaging biological features over time. Historically, though, the modality has been challenging to employ for multiplexed tracking due to a lack of resolvable luciferase-luciferin pairs. Recent years have seen the development of numerous orthogonal probes for multi-parameter imaging. While successful, generating such tools often requires complex syntheses and lengthy enzyme evolution campaigns. This work showcases an alternative strategy for multiplexed bioluminescence that takes advantage of already-orthogonal caged luciferins and established uncaging enzymes. These probes generate unique bioluminescent signals that can be distinguished via a linear unmixing algorithm. Caged luciferins enabled two- and three-component imaging on the minutes time scale. We further showed that the tools can be used in conjunction with endogenous enzymes for multiplexed studies. Collectively, this approach lowers the barrier to multicomponent bioluminescence imaging and can be readily adopted by the broader community.

Analysis of Floating Macroalgae Distribution around Japan Using Global Change Observation Mission-Climate/Second-Generation Global Imager Data

Floating macroalgae information is required to manage coastal environments and fishery resources effectively. In situ observations and analyses can result in under-sampling, thereby challenging the comprehension of the floating macroalgae abundance and spatiotemporal alterations. This study reports the spatiotemporal variation of floating macroalgae distribution around Japan from 2018 to 2021 using Global Change Observation Mission-Climate/second-generation Global Imager data. We employed the floating algae index (FAI) scaled from local ocean FAI to minimize the effect of spatial variation in ocean color. Fractional macroalgae coverage in a pixel was determined using a linear unmixing algorithm with lower and upper thresholds. The lower threshold was determined using the cumulative frequency of the scaled FAI, and the upper threshold was modified based on the more precise Sentinel-2 data. The results revealed that monthly macroalgae coverage varies spatially and seasonally, peaking in the spring and summer in the southwestern area. The macroalgae distribution particularly expanded from the East China Sea to west Japan during spring. In 2018–2021, the total biomass of the offshore floating macroalgae was estimated to be 8880–133,790, 8460–141,900, 3910–70,380, and 4620–61,870 tons. The findings of this study validated the empirical knowledge about specific locations and can serve as a reference to analyze temporal and spatial variations in future studies.

Fusion of airborne hyperspectral and LiDAR canopy-height data for estimating fractional cover of tall woody plants, herbaceous vegetation, and other soil cover types in a semi-arid savanna ecosystem

ABSTRACT Vegetation in semi-arid regions is often a spatially heterogeneous mix of bare soil, herbaceous vegetation, and woody plants. Detecting the fractional cover of woody plants versus herbaceous vegetation, in addition to non-photosynthetically active cover types such as biological soil crusts, dry grasses, and bare ground, is crucial for understanding spatial patterns and temporal changes in semi-arid savanna ecosystems. With and without the inclusion of LiDAR-derived canopy height information, we tested different configurations of unsupervised linear unmixing classification of hyperspectral data to estimate fractional cover of tall woody plants, herbaceous vegetation, and other non-photosynthetically active cover types. To perform this analysis, we used 1 m resolution hyperspectral and LiDAR data collected by the NEON airborne observation platform at the Santa Rita Experimental Range in Arizona. Our results show that both hyperspectral and canopy height information are needed in the linear unmixing algorithm to correctly identify tall woody plants versus herbaceous vegetation. However, including height data is not always sufficient to be able to separate the two vegetation types: selecting the optimum number of endmembers is also crucial for separating these two types of photosynthetically active vegetation. In comparison to our reference dataset, fractional cover accuracy was highest for the woody plant class based on two accuracy assessment methods, although the fractional cover of dense woody plant is likely underestimated because linear unmixing also detects sub-canopy elements. The accuracy of the herbaceous vegetation class was lower than the other classes, likely because of the presence of both active and senesced grasses. Our study presents the first use of both hyperspectral and canopy height information to separately detect fractional cover of woody plants versus herbaceous vegetation in addition to other bare soil cover types that should be useful for fractional cover classification in other semi-arid savanna ecosystems where hyperspectral and LiDAR data are available.

Triple-Color STED Nanoscopy: Sampling Absorption Spectra Differences for Efficient Linear Species Unmixing.

Stimulated emission depletion (STED) in confocal fluorescence microscopy enables a visualization of biological structures within cells far below the optical diffraction limit. To meet the demand in the field for simultaneous investigations of multiple species within a cell, a couple of different STED techniques have been proposed, each with their own challenges. By systemically exploiting spectral differences in the absorption of fluorescent labels, we present a novel, beneficial approach to multispecies STED nanoscopy. By using three excitation wavelengths in nanosecond pulsed interleaved excitation (PIE) mode, we probe quasi simultaneously multiple species with fluorescent labels having absorption maxima as close as 13 nm. The acquired image is decomposed into its single species contributions by application of a linear unmixing algorithm based on present reference patterns. For multispecies images containing single species regions, we introduce the image correlation map (ICM). Here, the single species regions easily can be identified in order to generate the necessary single species reference patterns. This avoids the otherwise cumbersome and artifact prone preparation and recording of additional reference samples. The power of the proposed imaging scheme persists in species separation quality at high speed shown for up to three species with established reference samples and dyes commonly used for cellular STED imaging.

Use of digital photography to identify neoplastic skin lesions after labelling by ALA-derived protoporphyrin

Actinic keratosis is a premalignant skin lesion that develops into non-melanoma skin cancer. Various imaging techniques have been developed to find the actinic keratosis lesion. In this clinical study, the feasibility of a nonspectroscopic fluorescence imaging system is investigated for spatial assessment of the actinic keratosis lesion. Six patients between the ages of 70 and 80 years old are diagnosed with actinic keratosis by a board-certified dermatologist to obtain biopsy-proven clinical images. The patients were treated with 5-aminolevulinic acid, which is transformed into the protoporphyrin IX. After illuminating ultraviolet-A light on facial lesions, the protoporphyrin IX produces the exogenous fluorescence. The fluorescence is measured using both a hyperspectral camera and an RGB color camera to obtain spectroscopic and nonspectroscopic fluorescence images, respectively. It is found that fluorescence intensity of the actinic keratosis lesion is higher than that of normal skin. Based on combined fluorescence and physiological characteristics, the actinic keratosis lesion is distinguished from the adjacent normal skin area. For delineation of the actinic keratosis lesion, a linear unmixing algorithm is applied to spectroscopic image data and an erythema index is calculated from nonspectroscopic image data. Then, two extracted actinic keratosis lesions are compared for cross-validation. As a result, both spectroscopic and nonspectroscopic fluorescence images demarcate an identical lesion of actinic keratosis. Given the affordability and simplicity, an RGB camera and a 5-ALA photosensitizer can be used as a cost-effective nonspectroscopic imaging modality for accurate assessment of actinic keratosis margins.

Hyperspectral Unmixing Based on Spectral and Sparse Deep Convolutional Neural Networks

Hyperspectral unmixing refers to the process of obtaining endmembers and abundance vectors through linear or nonlinear models. The traditional linear unmixing model assumes that each mixed pixel can be represented by a linear combination of endmembers. Considering real-world situations, a sparse constraint is normally added to the linear unmixing model. However, the linear model does not take into account that the spectrum of mixed pixels is not simply linearly mixed. To fully study the mixing characteristics of ground object spectra before being imaged by the sensor, we propose a supervised unmixing architecture based on one-dimensional convolutional neural network (CNN) by considering the spectral information and the sparse characteristics in the mixed pixel. Since one-dimensional CNN only consider feature learning, we combine the traditional root mean square error (RMSE) and l_"1" regularization in its loss function to minimize training error. The performance of our proposed unmixing model is assessed by comparing the unmixing results with three traditional linear sparse unmixing algorithms, and the Fuzzy-ARTMAP neural network in a simulated dataset and three real datasets. The RMSE was used to verify the unmixing accuracy of the different methods. The results showed that the RMSE obtained by our proposed CNN-based method was the lowest among the methods on all three real datasets, proving the effectiveness and stability of CNN in unmixing tasks.

Alternating Direction Iterative Nonnegative Matrix Factorization Unmixing for Multispectral and Hyperspectral Data Fusion

Most image fusion algorithms based on hyperspectral unmixing perform worse with the lower spatial resolution of hyperspectral image (HSI) for the reason that the estimated endmembers and abundance deviate from the truth value. Therefore, it is more meaningful to unmix the low spatial resolution hyperspectral image (LRHSI) accurately, which is also helpful to improve the image fusion performance. In order to enhance the spatial resolution of LRHSI, this article proposes an alternating direction iterative nonnegative matrix factorization (ADINMF) based on linear hyperspectral unmixing algorithm. It takes multispectral image as a constraint to improve the spatial resolution of LRHSI. First, we use blind source separation to initialize the endmember and abundance of hyperspectral and multispectral images, respectively. Then, we alternately update the endmembers and abundance in the framework of nonnegative matrix factorization by multiplication iterative algorithm. The updated endmembers and abundance are constrained to each other. We compare the experimental results of simulated dataset and three groups of real datasets. Experimental results show that the proposed method not only accurately extracts the endmembers of LRHSI, but also obtains a significant fusion performance improvement.

Simultaneous Visualization of Multiple Gene Expression in Single Cells Using an Engineered Multicolor Reporter Toolbox and Approach of Spectral Crosstalk Correction.

Synthetic biology aims to make biology easier to engineer and focuses on the design and construction of core components that can be modeled, understood, and tuned to meet specific performance criteria, and the assembly of these smaller parts and devices into larger integrated systems to solve specific problems. Here, we designed and engineered a multicolor fluorescent reporter toolbox to simultaneously monitor the activities of multiple genes in single cells. The toolbox contained standardized and well-characterized genetic building blocks for the convenient and reproducible assembly of multiple promoter-reporter fusions (ranging from 1 to 4) into a single plasmid. Given the common problem of spectral crosstalk among multiple fluorescent proteins, we deciphered multiple spectral signatures within cells through a deduced linear unmixing algorithm. Our approach enabled the quantification of gene expression with direct FP concentrations, instead of mix-contributed fluorescence intensities, thus enabling true signal separation with high confidence. This approach performed well in the imaging of mixing cells with single FP labels. Additionally, combining with the multicolor toolbox, we succeeded in simultaneously monitoring the genetic dynamics of four selected quorum-sensing genes in response to the induction of two exogenously added autoinducers and were able to examine gene regulatory connections within the QS signaling network in Pseudomonas aeruginosa. Overall, this synthetic framework (i.e., the genetic toolbox and the well-evaluated approach of spectral correction) will be useful for applied synthetic biology projects, multicolor imaging, and analyzing interactions of multiple genes of natural genetic networks or assembling synthetic ones.

A Laboratory-Created Dataset With Ground Truth for Hyperspectral Unmixing Evaluation

Spectral unmixing is an important and challenging problem in hyperspectral data processing. This topic has been extensively studied and a variety of unmixing algorithms have been proposed in the literature. However, the lack of publicly available datasets with ground truth makes it difficult to evaluate and compare the performance of unmixing algorithms in a quantitative and objective manner. Most of the existing works rely on the use of numerical synthetic data and an intuitive inspection of the results of real data. To alleviate this dilemma, in this study, we design several experimental scenes in our laboratory, including printed checkerboards, mixed quartz sands, and reflection with a vertical board. A dataset is then created by imaging these scenes with the hyperspectral camera in our laboratory, providing 36 mixtures with more than 130 000 pixels with 256 wavelength bands ranging from 400 to 1000 nm. The experimental settings are strictly controlled so that pure material spectral signatures and material compositions are known. To the best of our knowledge, this dataset is the first publicly available dataset created in a systematic manner with ground truth for spectral unmixing. Some typical linear and nonlinear unmixing algorithms are also tested with this dataset and lead to meaningful results.

Seabed Mapping in Coastal Shallow Waters Using High Resolution Multispectral and Hyperspectral Imagery

Coastal ecosystems experience multiple anthropogenic and climate change pressures. To monitor the variability of the benthic habitats in shallow waters, the implementation of effective strategies is required to support coastal planning. In this context, high-resolution remote sensing data can be of fundamental importance to generate precise seabed maps in coastal shallow water areas. In this work, satellite and airborne multispectral and hyperspectral imagery were used to map benthic habitats in a complex ecosystem. In it, submerged green aquatic vegetation meadows have low density, are located at depths up to 20 m, and the sea surface is regularly affected by persistent local winds. A robust mapping methodology has been identified after a comprehensive analysis of different corrections, feature extraction, and classification approaches. In particular, atmospheric, sunglint, and water column corrections were tested. In addition, to increase the mapping accuracy, we assessed the use of derived information from rotation transforms, texture parameters, and abundance maps produced by linear unmixing algorithms. Finally, maximum likelihood (ML), spectral angle mapper (SAM), and support vector machine (SVM) classification algorithms were considered at the pixel and object levels. In summary, a complete processing methodology was implemented, and results demonstrate the better performance of SVM but the higher robustness of ML to the nature of information and the number of bands considered. Hyperspectral data increases the overall accuracy with respect to the multispectral bands (4.7% for ML and 9.5% for SVM) but the inclusion of additional features, in general, did not significantly improve the seabed map quality.

Multispectral Live-Cell Imaging.

Fluorescent proteins and vital dyes are invaluable tools for studying dynamic processes within living cells. However, the ability to distinguish more than a few different fluorescent reporters in a single sample is limited by the spectral overlap of available fluorophores. Here, we present a protocol for imaging live cells labeled with six fluorophores simultaneously. A confocal microscope with a spectral detector is used to acquire images, and linear unmixing algorithms are applied to identify the fluorophores present in each pixel of the image. We describe the application of this method to visualize the dynamics of six different organelles, and to quantify the contacts between organelles. However, this method can be used to image any molecule amenable to tagging with a fluorescent probe. Thus, multispectral live-cell imaging is a powerful tool for systems-level analysis of cellular organization and dynamics. © 2018 by John Wiley & Sons, Inc.

Application of spatially related MRF model in NMF hyperspectral unmixing

Aiming at Non-negative Matrix Factorization (NMF)’s problem of initialization and "local minima" in hyperspectral unmixing, a NMF linear unmixing algorithm with spatial correlation constrains (SCNMF) based on Markov Random Field (MRF) was proposed. Firstly, Hyperspectral Signal identification by minimum error (HySime) method was adopted to estimate the number of endmembers, initialized endmember matrix and abundance matrix by Vertex Component Analysis (VCA) and Fully Constrained Least Squares (FCLS) respectively. then established energy function to depict the spatial distribution characteristics of ground objects by MRF model. Finally, spatial correlation constraint based on MRF model and NMF standard objective function were combined in the form of altemating iteration to estimate endmember spectrum and abundance of hyperspectral image. Theoretical analysis and experimental results indicated that, the endmember decomposition precision of SCNMF is 10.6% higher than that of Minimum Volume Constrained NMF (MVC-NMF), 12.3% higher than that of Piecewise Smoothness NMF with Sparseness Constraints(PSNMFSC), 14.1% higher than that of NMF with Alternating Projected Subgradients(APS-NMF); the abundance decomposition precision of SCNMF is 14.4% higher than that of MVC-NMF, 15.9% higher than that of PSNMFSC, 15.3% higher than that of APS-NMF.The proposed SCNMF can remedy NMF's deficiency in describing spatial correlation characteristics, and decrease spatial energy distribution error.

Predicting temperate forest stand types using only structural profiles from discrete return airborne lidar

Light detection and ranging (lidar) data have been increasingly used for forest classification due to its ability to penetrate the forest canopy and provide detail about the structure of the lower strata. In this study we demonstrate forest classification approaches using airborne lidar data as inputs to random forest and linear unmixing classification algorithms. Our results demonstrated that both random forest and linear unmixing models identified a distribution of rainforest and eucalypt stands that was comparable to existing ecological vegetation class (EVC) maps based primarily on manual interpretation of high resolution aerial imagery. Rainforest stands were also identified in the region that have not previously been identified in the EVC maps. The transition between stand types was better characterised by the random forest modelling approach. In contrast, the linear unmixing model placed greater emphasis on field plots selected as endmembers which may not have captured the variability in stand structure within a single stand type. The random forest model had the highest overall accuracy (84%) and Cohen’s kappa coefficient (0.62). However, the classification accuracy was only marginally better than linear unmixing. The random forest model was applied to a region in the Central Highlands of south-eastern Australia to produce maps of stand type probability, including areas of transition (the ‘ecotone’) between rainforest and eucalypt forest. The resulting map provided a detailed delineation of forest classes, which specifically recognised the coalescing of stand types at the landscape scale. This represents a key step towards mapping the structural and spatial complexity of these ecosystems, which is important for both their management and conservation.

Adaptive and automatic red blood cell counting method based on microscopic hyperspectral imaging technology

Red blood cell counting, as a routine examination, plays an important role in medical diagnoses. Although automated hematology analyzers are widely used, manual microscopic examination by a hematologist or pathologist is still unavoidable, which is time-consuming and error-prone. This paper proposes a full-automatic red blood cell counting method which is based on microscopic hyperspectral imaging of blood smears and combines spatial and spectral information to achieve high precision. The acquired hyperspectral image data of the blood smear in the visible and near-infrared spectral range are firstly preprocessed, and then a quadratic blind linear unmixing algorithm is used to get endmember abundance images. Based on mathematical morphological operation and an adaptive Otsu’s method, a binaryzation process is performed on the abundance images. Finally, the connected component labeling algorithm with magnification-based parameter setting is applied to automatically select the binary images of red blood cell cytoplasm. Experimental results show that the proposed method can perform well and has potential for clinical applications.

Multiplexed Spectral Imaging of 120 Different Fluorescent Labels.

The number of fluorescent labels that can unambiguously be distinguished in a single image when acquired through band pass filters is severely limited by the spectral overlap of available fluorophores. The recent development of spectral microscopy and the application of linear unmixing algorithms to spectrally recorded image data have allowed simultaneous imaging of fluorophores with highly overlapping spectra. However, the number of distinguishable fluorophores is still limited by the unavoidable decrease in signal to noise ratio when fluorescence signals are fractionated over multiple wavelength bins. Here we present a spectral image analysis algorithm to greatly expand the number of distinguishable objects labeled with binary combinations of fluorophores. Our algorithm utilizes a priori knowledge about labeled specimens and imposes a binary label constraint on the unmixing solution. We have applied our labeling and analysis strategy to identify microbes labeled by fluorescence in situ hybridization and here demonstrate the ability to distinguish 120 differently labeled microbes in a single image.

Quadratic blind linear unmixing: A graphical user interface for tissue characterization

Spectral unmixing is the process of breaking down data from a sample into its basic components and their abundances. Previous work has been focused on blind unmixing of multi-spectral fluorescence lifetime imaging microscopy (m-FLIM) datasets under a linear mixture model and quadratic approximations. This method provides a fast linear decomposition and can work without a limitation in the maximum number of components or end-members. Hence this work presents an interactive software which implements our blind end-member and abundance extraction (BEAE) and quadratic blind linear unmixing (QBLU) algorithms in Matlab. The options and capabilities of our proposed software are described in detail. When the number of components is known, our software can estimate the constitutive end-members and their abundances. When no prior knowledge is available, the software can provide a completely blind solution to estimate the number of components, the end-members and their abundances. The characterization of three case studies validates the performance of the new software: ex-vivo human coronary arteries, human breast cancer cell samples, and in-vivo hamster oral mucosa. The software is freely available in a hosted webpage by one of the developing institutions, and allows the user a quick, easy-to-use and efficient tool for multi/hyper-spectral data decomposition.

A Novel Approach for Efficient <formula formulatype="inline"><tex Notation="TeX">$p$</tex></formula>-Linear Hyperspectral Unmixing

Airborne and spaceborne hyperspectral sensors, due to their limited spatial resolution, often record the spectral response of a mixture of materials. In order to extract the abundances of these materials, linear and nonlinear unmixing algorithms have been developed. In this paper, we focus on nonlinear mixing models that are able to model macro- and microscopic scale interactions. Although very useful, these models may be inverted only by means of optimization techniques, typically impossible to be performed in matrix form. Thereby, only nonlinear mixing models that describe macroscopic effects (e.g., two-reflections schemes) are currently considered as they have lower computational costs. On the other hand, this limitation may result in a loss in terms of description accuracy for the images. In this paper, we propose a new approach for nonlinear unmixing that aims at providing excellent reconstruction performance for arbitrary polynomial nonlinearities making use of the polytope decomposition (POD) method. Additionally, POD transforms nonlinear unmixing into a linear problem, and can be easily implemented in high-performance computing architectures. Results using synthetic and real data confirm the effectiveness and accuracy of the proposed framework. To prove its feasibility for fast computational applications, its complexity is analytically derived and compared with real data analysis.

A Novel Algorithm to Estimate Algal Bloom Coverage to Subpixel Resolution in Lake Taihu

Remote sensing has often been used to monitor the distribution and frequency of floating algae in inland aquatic environments. However, due to the spatial resolution of the most common satellite sensors, accurate determination of algae coverage remains a major technical challenge. Here, a novel algorithm to estimate floating algae area to subpixel scales, denominated the algae pixel-growing algorithm (APA), is developed and evaluated for a series of image data from the Moderate Resolution Imaging Spectroradiometer (MODIS). The algorithm utilizes the Rayleigh-corrected reflectance (Rrc) and a floating algae index (FAI) derived from Rrc in three spectral bands. Comparison with concurrent Landsat TM/ETM+data indicate that the APA provides more accurate estimates of both algal bloom area and algal bloom intensity (i.e., floating algae coverage) (RSE = 15.2 km <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and RE = 9.9%), compared to other commonly used methods such as the linear unmixing algorithm (LA). Furthermore, this study confirms that FAI is abetter index with respect to normalized difference vegetation index (NDVI) and enhanced vegetation index (EVI) for the estimation of algae area coverage, especially when combined with the APA. Finally, the study provides a theoretical basis for the objective assessment of bloom severity in complex inland waterbodies.