Pixel-based Analysis Research Articles

Classification of Citrus Leaf Diseases Using Hyperspectral Reflectance and Fluorescence Imaging and Machine Learning Techniques

Citrus diseases are significant threats to citrus groves, causing financial losses through reduced fruit size, blemishes, premature fruit drop, and tree death. The detection of citrus diseases via leaf inspection can improve grove management and mitigation efforts. This study explores the potential of a portable reflectance and fluorescence hyperspectral imaging (HSI) system for detecting and classifying a control group and citrus leaf diseases, including canker, Huanglongbing (HLB), greasy spot, melanose, scab, and zinc deficiency. The HSI system was used to simultaneously collect reflectance and fluorescence images from the front and back sides of the leaves. Nine machine learning classifiers were trained using full spectra and spectral bands selected through principal component analysis (PCA) from the HSI with pixel-based and leaf-based spectra. A support vector machine (SVM) classifier achieved the highest overall classification accuracy of 90.7% when employing the full spectra of combined reflectance and fluorescence data and pixel-based analysis from the back side of the leaves, whereas a discriminant analysis classifier yielded the best accuracy of 94.5% with the full spectra of combined reflectance and fluorescence data and leaf-based analysis. Among the diseases, control, scab, and melanose were classified most accurately, each with over 90% accuracy. Therefore, the integration of the reflectance and fluorescence HSI with advanced machine learning techniques demonstrated the capability to accurately detect and classify these citrus leaf diseases with high precision.

Satellite measurement data-based assessment of spatiotemporal characteristics of ultraviolet index (UVI) over the state of Johor, Malaysia

The study examines the spatiotemporal and statistical characteristics of the daily ultraviolet index (UVI) over the Johor state of Malaysia. The datasets utilized in this study are the Ozone Monitoring Instrument (OMI)/Aura satellite’s daily UVI observations between October 2004 and March 2023. The innovative trend analysis (ITA) method is employed to identify the statistical trend of daily UVI. Basic statistics of daily, monthly, and seasonal UVI are also carried out to better characterize UVI in the study area. The daily UVI data collected for the analysis exhibit wide variability, with more than 80% of the daily UVI data falling above UVI 8, indicating the severity of UV radiation (UVR) reaching the study area. The monthly and seasonal UVI also display highly varying characteristics, as observed in the daily data. The pixel-based analysis of ITA results indicates a significant increasing and decreasing trend of daily UVI in the region, with values varying between − 9.42 × 10−6 and 3.79 × 10−5. However, for the study area as a whole, UVI shows a significant increasing trend of 4.50 × 10−5. The influence of parameters such as cloud optical thickness (COT), solar zenith angle (SZA), and O3 column on the strength of UVR reaching the study area is also investigated. Based on the results, the study area is characterized by low thickness, broken clouds, and sometimes cloudless conditions. Overall, the results show that UVI in the study area is increasing, and therefore, proper health guidelines and direct UVR exposure regulations need to be implemented to reduce serious health risks associated with UVR exposure.Graphical

Single-cell analysis using Imaging Mass Cytometry spatially resolves immune activity in neurodegenerative brain pathology

Abstract Immune response plays a significant role in the progression of neurodegenerative diseases (NDs) such as Parkinson’s disease (PD), Alzheimer’s disease (AD), and multiple sclerosis (MS). It is paramount to quantify the immune and structural spatial microenvironment of ND brain using research approaches that resolve tissue at a single-cell level. Imaging Mass Cytometry™ (IMC™) is a spatial biology technique that provides quantitative phenotypic evaluation of 40-plus markers. We report the application of two new IMC imaging modes with 40-marker panels for ND research. Tissue mode (TM) was applied to perform a scan of the entire tissue followed by pixel-based analysis (PBA) to uncover spatial distribution of relevant markers. Preview mode (PM) was applied to sample the tissue at low resolution, improving informed selection of areas for high-resolution cell mode (CM) imaging on the same slide. Using TM with PBA, we quantified the spatial distribution of immune cells in tissue compartments of ND tissues. PM and CM highlighted MS lesions devoid of myelinization and a network of α-synuclein in PD tissue. Single-cell analysis identified heterogeneity of amyloid precursor protein aggregates in AD that encompassed total and phosphorylated species of tau and phosphorylated α-synuclein. Overall, whole slide imaging techniques using IMC provide quantitative immunological and structural insights into the spatial biology of ND. For Research Use Only. Not for use in diagnostic procedures.

Detecting immune activity in human tumors with novel high-throughput whole slide imaging approaches using Imaging Mass Cytometry.

Abstract Deciphering the spatial biology of the tumor immune microenvironment (TIME) has potential to inform clinical and translational researchers about the likelihood of immunotherapy success. Imaging Mass Cytometry™ (IMC™) is a spatial biology imaging technique that enables deep characterization of the TIME while fully retaining spatial features of tissues using 40-plus markers simultaneously. We developed two new whole slide imaging methods for IMC. Tissue mode (TM) rapidly acquires an image of the whole tissue that can be used for quantitative analysis of spatial biology. Preview mode (PM) samples the entire tissue at predefined spacing to rapidly capture a low-resolution image of all expressed markers, improving informed identification of areas for high-resolution cell mode (CM) imaging on the same slide. Tumor tissues were stained using a 40-marker panel comprised of the Maxpar™ Human Immuno-Oncology IMC Panel Kit, designed for tumor and immune cell profiling. TM imaging in combination with pixel-based analysis quantified tissue areas containing high or low abundance of myeloid and lymphoid immune, tumor and stromal cell compartments. PM in combination with CM imaging followed by single-cell analysis on selected tumor regions identified cell populations with varying activation states and expression profiles. Overall, we highlight the power of IMC to provide quantitative insights into the spatial biology of the TIME. For Research Use Only. Not for use in diagnostic procedures.

Automated Upper Tract Urothelial Carcinoma Tumor Segmentation During Ureteroscopy Using Computer Vision Techniques.

Introduction: Endoscopic tumor ablation of upper tract urothelial carcinoma (UTUC) allows for tumor control with the benefit of renal preservation but is impacted by intraoperative visibility. We sought to develop a computer vision model for real-time, automated segmentation of UTUC tumors to augment visualization during treatment. Materials and Methods: We collected 20 videos of endoscopic treatment of UTUC from two institutions. Frames from each video (N = 3387) were extracted and manually annotated to identify tumors and areas of ablated tumor. Three established computer vision models (U-Net, U-Net++, and UNext) were trained using these annotated frames and compared. Eighty percent of the data was used to train the models while 10% was used for both validation and testing. We evaluated the highest performing model for tumor and ablated tissue segmentation using a pixel-based analysis. The model and a video overlay depicting tumor segmentation were further evaluated intraoperatively. Results: All 20 videos (mean 36 ± 58 seconds) demonstrated tumor identification and 12 depicted areas of ablated tumor. The U-Net model demonstrated the best performance for segmentation of both tumors (area under the receiver operating curve [AUC-ROC] of 0.96) and areas of ablated tumor (AUC-ROC of 0.90). In addition, we implemented a working system to process real-time video feeds and overlay model predictions intraoperatively. The model was able to annotate new videos at 15 frames per second. Conclusions: Computer vision models demonstrate excellent real-time performance for automated upper tract urothelial tumor segmentation during ureteroscopy.

Cloud-scale molecular gas properties of the ANTENNAE merger: a comparative study with PHANGS-ALMA galaxies and NGC 3256

ABSTRACT We present observations of the central 9 kpc of the Antennae merger (NGC 4038/9) at 55 pc resolution in the CO (2–1) line obtained with the Atacama Large Millimeter/submillimeter Array (ALMA). We use a pixel-based analysis to compare the gas properties in the Antennae to those in 70 nearby spiral galaxies from the PHANGS-ALMA survey, as well as the merger and nearest luminous infrared galaxy NGC 3256. Compared to PHANGS-ALMA galaxies at matched spatial resolution, the molecular gas in the Antennae exhibits some of the highest surface densities, velocity dispersions, peak brightness temperatures, and turbulent pressures. However, the virial parameters in the Antennae are consistent with many of the PHANGS-ALMA galaxies. NGC 3256 has similar gas surface densities but higher nuclear velocity dispersions than the Antennae, as well as higher system-wide peak brightness temperatures and virial parameters. NGC 3256 is at a later stage in the merging process than the Antennae, which may result in more intense merger-driven gas flows that could drive up the turbulence in the gas. The high virial parameters in NGC 3256 may indicate that this increased turbulence is suppressing future star formation as NGC 3256 moves out of the starburst phase. In comparison, the relatively normal virial parameters in the Antennae may imply that it is about to undergo a new burst of star formation.

Abstract 1450: Novel whole slide imaging modes for imaging mass cytometry reveal cellular and structural composition of mouse glioblastoma

Abstract Mouse tumors have become widely used models in translational research of brain malignancies, helping to address the difficulties related to studying the human brain. Mouse brain can be regarded as a miniaturized model of the human brain, permitting visualization of the whole tissue to provide spatial cellular context. Imaging Mass Cytometry™ (IMC™) is a highly relevant tool capable of quantitative evaluation of the multiparametric protein composition in brain tumor microenvironment (TME) without the complications of autofluorescence, tissue degradation, and spectral overlap. The Hyperion XTi™ Imaging System (Standard BioTools™) utilizes IMC technology to simultaneously assess more than 40 individual structural and functional markers in tissue, providing insight into the organization and function of the TME. We demonstrate the whole slide imaging (WSI) application using a 40-marker panel composed of the Maxpar OnDemand™ Mouse Immuno-Oncology IMC Panel Kit and the Maxpar® Neuro Phenotyping IMC Panel Kit on mouse normal and glioblastoma (GBM) brain tissue. The panel was composed of mouse-specific antibodies, which highlight tumor and immune components of the mouse TME. We performed imaging using two new features of Hyperion™ XTi. Ultrafast preview mode (PM) was applied to rapidly screen entire brain sections for marker expression signatures associated with various immuno-oncology processes. This enabled biomarker-guided selection of areas in tumor tissue that were imaged using region of interest-based IMC and analyzed using single-cell analysis (SCA). In parallel, a high-throughput tissue mode (TM) was applied to perform a detailed whole-slide scan of normal and GBM mouse brain tissues that were quantified using pixel-based analysis (PBA) to unravel the composition of the TME. Using TM, we were able to successfully visualize the entire coronal and sagittal sections of normal mouse brain as well as mouse GBM tissue. PBA on TM data provided quantitative spatial expression patterns of structural and immune markers across the whole tissue. In normal tissue, we visualized the highly organized structure of the normal brain and detected neurons, oligodendrocytes, vascular-adjacent astrocytes, and axonal tracks. In GBM tissue, necrotic cores, areas with high immune infiltration, extracellular matrix deposits, and activated tumor cells were detected. SCA of GBM tissue demonstrated expansive vascularization, replicating Olig2+ cells, activation of Ras signaling, and high abundance of infiltrating immune cells. Overall, we demonstrate the successful application of two novel WSI modes and highlight the power of IMC technology to simultaneously explore dozens of relevant biological outputs to better understand the TME of GBM and other tumors. For research use only. Not for use in diagnostic procedures. Citation Format: Qanber Raza, Nick Zabinyakov, Thomas D. Pfister, Nikesh Parsotam, David Howell, Liang Lim, Christina Loh. Novel whole slide imaging modes for imaging mass cytometry reveal cellular and structural composition of mouse glioblastoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 1450.

Abstract 3800: Next generation of spatial biology: High-throughput multiplexed Imaging Mass Cytometry™ with whole slide modes

Abstract Gaining spatial insights into the cellular composition of tumor tissue has tremendous potential to inform clinical and translational researchers about mechanisms behind spatial predictors of immunotherapy success, disease etiology and progression. Imaging Mass Cytometry™ (IMC™) is a high-plex spatial biology imaging technique that enables deep characterization of the diversity and complexity of the tumor microenvironment (TME). IMC supports detailed assessment of cell phenotype and function using 40-plus metal-tagged antibodies simultaneously on a single slide without issues associated with fluorescence-based spectral overlap, tissue autofluorescence or implementation of multiple washing and acquisition cycles. Currently, IMC enables user-defined regions of interest (ROI) in tissues to evaluate cellular and structural composition. To enhance the IMC user experience, we developed two new whole slide imaging (WSI) modes which enable streamlined workflows using ultrafast preview mode (PM) and high-throughput tissue mode (TM). PM samples the entire tissue at predefined spacing to rapidly capture a low-resolution image of all expressed markers in the antibody panel. PM generates an image in minutes to enable informed ROI placements while leaving the stained tissue intact for higher-resolution imaging. PM and TM are designed so acquisitions can easily occur on the same slide without additional processing steps. TM rapidly acquires images of the whole tissue at lower resolution (7 μm pixel size) at a quality that can be used for quantitative analysis of the tissue spatial biology. Specifically designed for high-throughput applications, TM in combination with a newly available 40-slide loader for the Hyperion XTi™ Imaging System permits automated and continuous imaging of more than 40 large tissue samples (400 mm2 per tissue) per week. We showcase the application of WSI modes using the newly developed Maxpar® Human Immuno-Oncology IMC Panel Kit. The 31-marker panel was combined with catalog antibodies to create a 40-plus-marker panel that expands the ability to conduct comprehensive high-plex tumor and immune cell profiling. Tumor tissue microarrays (TMA) and whole tumor tissue sections were stained with the expanded panel. Single-cell analysis of selected ROIs, on whole tumor sections and TMAs, guided by PM data successfully provided quantitative analyses of spatial biology at single-cell resolution. In parallel, TM on whole tumor sections followed by pixel-based analysis provided a spatially resolved quantitative assessment of specific tumor and immune components of the TME. This work demonstrates the expanded capabilities of IMC and establishes it as a reliable high-plex spatial biology imaging platform with high-throughput imaging capabilities ideally suited for translational and clinical applications. For Research Use Only. Not for use in diagnostic procedures. Citation Format: Qanber Raza, Thomas D. Pfister, Jyh Yun Chwee, Liang Lim, David Howell, Nikesh Parsotam, David King, Christina Loh. Next generation of spatial biology: High-throughput multiplexed Imaging Mass Cytometry™ with whole slide modes [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3800.

Abstract 4169: A deep learning approach to guide acquisition region selection for imaging mass cytometry

Abstract The growth in cancer immunotherapy agents requires an understanding of the immune contexture of the tumor microenvironment. This can be aided by high-plex imaging and analysis to obtain phenotypes of specific cells and study their biodistribution and interactions. Imaging Mass Cytometry™ (IMC) is the method of choice for single-step staining and high-plex imaging of tissues, avoiding the complications of autofluorescence and cyclic imaging. IMC has expanded its capabilities with three distinct imaging modes: Preview, Cell, and Tissue. The Preview Mode is a rapid scanning system that captures a comprehensive overview of the stained tissue, mapping out the distribution of over 40 markers and revealing tissue heterogeneity. This enables researchers to make informed decisions about which areas warrant closer examination on the same. Building on this, Cell Mode offers high-resolution imaging for detailed analysis of the Regions of Interest (ROI) identified during Preview, all using the same slide. Tissue Mode complements these by providing a fast acquisition of the entire tissue at a lower resolution, which is optimal for quantitative pixel-based analysis of tissue biology. These modes support automated, continuous imaging of more than 40 large tissue samples (400 mm2) weekly. Following Preview Mode, the selection of ROIs for high-resolution imaging is a critical step, enhanced by automated AI algorithms to ensure it is informed by biomarker expression. Tissue sections of colon adenocarcinoma were stained with a 30-marker IMC panel of structural, tumor, stroma, immune cell, and immune activation markers. Images were acquired on Hyperion XTi™ (Standard BioTools), first in Preview Mode, then in Cell Mode with automatic selection of ROIs using Phenoplex™ (Visiopharm). ROIs were automatically selected based on two criteria: 1) actively proliferating and non-proliferating tumor regions; 2) cold and hot tumor regions as identified by hotspots of lymphoid and myeloid immune markers within stromal or epithelial tumor regions. Single cell analysis of the high-resolution images obtained in Cell Mode, was performed as a multi-step workflow using Phenoplex. Tissue segmentation was used to divide the tissue into epithelial tumor (E-Cadherin/panCK), stromal, and tumor margin regions; cell segmentation was based on Iridium DNA channels; and phenotyping was performed using the guided highplex workflow. These data were used to compare the immune contexture through a series of t-SNE plots partitioned by spatial region and clinical variables. This work demonstrates that the multi-modal features of Hyperion XTi can greatly accelerate the ability of IMC users to gain useful insights from complex biological samples. Phenoplex enables a comprehensive workflow for the analysis of this complex data, providing automated ROI selection, and phenotyping and spatial analyses of high-resolution IMC images for biological assessment. Citation Format: Fabian Schneider, Smriti Kala, Sam Lim, Clinton Hupple, Nina Lane, James Robert Mansfield. A deep learning approach to guide acquisition region selection for imaging mass cytometry [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4169.

Abstract 5501: Novel whole slide imaging modes for imaging mass cytometry unveil extensive cellular heterogeneity in human glioma

Abstract Gliomas present a complex form of cancer that is challenging to diagnose and treat, with a median survival of just over one year after diagnosis for primary glioblastoma (GBM). GBM can occur as multi-lesion, remote, or diffuse tumors, and often contains a tumor microenvironment (TME) that is devoid of peripheral immune cells. Additional hallmark features of GBM include necrosis, hemorrhage, and pseudo palisades, making it a highly heterogeneous disease requiring further investigation. Identification of the cellular and spatial level composition of the TME is vital for interpretation of GBM disease origin, progression, prediction, and treatment. A 40-plus-marker neuro-oncology Imaging Mass Cytometry™ (IMC™) antibody panel was used to determine the cellular and structural landscape of the brain TME. We applied the panel on a tissue microarray (TMA) containing dozens of human glioma cores and uncovered the spatial distribution of over 40 distinct molecular markers. We performed imaging using two new features of the Hyperion XTi™ Imaging System that provide whole slide scanning capabilities. Ultrafast preview mode was applied to rapidly screen tumor cores for expression signatures associated with tumor immuno-oncology processes. This enabled biomarker-guided selection of areas in tumor tissue that were imaged at higher resolution and analyzed using single-cell analysis. In parallel, a high-throughput tissue mode was applied to perform a detailed scan of the brain tumor TMA followed by pixel-based analysis to unravel the spatial composition of the TME. Using tissue mode imaging, we successfully mapped the spatial location of cell populations making up human gliomas, such as neurons, astrocytes, microglia, and oligodendrocytes, across the entire TMA. Various tumor cell phenotypes, resident and infiltrating cells, and resting and activated microglia were detected across all TMA cores. Subsequent single-cell analysis of select regions of interest provided a quantitative assessment of the cellular composition of the brain TME. We classified the distinct states of neurons and quantified myeloid and lymphoid immune cell infiltration across normal, astrocytoma, and GBM tissues. Striking cellular and protein heterogeneity was observed between acquired cores, indicating the complexity of each case that is impossible to capture with low-plex visualization techniques. Therefore, IMC is a highly relevant tool capable of quantitative spatial evaluation of the high-plex protein composition in the brain TME without the complications of autofluorescence, tissue degradation, and spectral overlap. Empowered by the neuro-oncology panel and new whole slide imaging modes, IMC accelerates neurological research and provides insights into the spatial complexity of gliomas and other tumors. For research purpose only. Not for use in diagnostic procedures. Citation Format: Nick Zabinyakov, Qanber Raza, Thomas D. Pfister, Nikesh Parsotam, David Howell, Liang Lim, Christina Loh. Novel whole slide imaging modes for imaging mass cytometry unveil extensive cellular heterogeneity in human glioma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 5501.

Deep learning for three-dimensional segmentation of electron microscopy images of complex ceramic materials

The microstructure is a critical factor governing the functionality of ceramic materials. Meanwhile, microstructural analysis of electron microscopy images of polycrystalline ceramics, which are geometrically complex and composed of countless crystal grains with porosity and secondary phases, has generally been performed manually by human experts. Objective pixel-based analysis (semantic segmentation) with high accuracy is a simple but critical step for quantifying microstructures. In this study, we apply neural network-based semantic segmentation to secondary electron images of polycrystalline ceramics obtained by three-dimensional (3D) imaging. The deep-learning-based models (e.g., fully convolutional network and U-Net) by employing a dataset based on a 3D scanning electron microscopy with a focused ion beam is found to be able to recognize defect structures characteristic of polycrystalline materials in some cases due to artifacts in electron microscopy imaging. Owing to the training images with improved depth accuracy, the accuracy evaluation function, intersection over union (IoU) values, reaches 94.6% for U-Net. These IoU values are among the highest for complex ceramics, where the 3D spatial distribution of phases is difficult to locate from a 2D image. Moreover, we employ the learned model to successfully reconstruct a 3D microstructure consisting of giga-scale voxel data in a few minutes. The resolution of a single voxel is 20 nm, which is higher than that obtained using a typical X-ray computed tomography. These results suggest that deep learning with datasets that learn depth information is essential in 3D microstructural quantifying polycrystalline ceramic materials. Additionally, developing improved segmentation models and datasets will pave the way for data assimilation into operando analysis and numerical simulations of in situ microstructures obtained experimentally and for application to process informatics.

Automatic non-destructive UAV-based structural health monitoring of steel container cranes

Container cranes are of key importance for maritime cargo transportation. The uninterrupted and all-day operation of these container cranes, which directly affects the efficiency of the port, necessitates the continuous inspection of these massive hoisting steel structures. Due to the large size of cranes, the current manual inspections performed by expert climbers are costly, risky, and time-consuming. This motivates further investigations on automated non-destructive approaches for the remote inspection of fatigue-prone parts of cranes. In this paper, we investigate the effectiveness of color space-based and deep learning-based approaches for separating the foreground crane parts from the whole image. Subsequently, three different ML-based algorithms (k-Nearest Neighbors, Random Forest, and Naive Bayes) are employed to detect the rust and repainting areas from detected foreground parts of the crane body. Qualitative and quantitative comparisons of the results of these approaches were conducted. While quantitative evaluation of pixel-based analysis reveals the superiority of the k-Nearest Neighbors algorithm in our experiments, the potential of Random Forest and Naive Bayes for region-based analysis of the defect is highlighted.

A Generic Self-Supervised Learning (SSL) Framework for Representation Learning from Spectral–Spatial Features of Unlabeled Remote Sensing Imagery

Remote sensing data has been widely used for various Earth Observation (EO) missions such as land use and cover classification, weather forecasting, agricultural management, and environmental monitoring. Most existing remote-sensing-data-based models are based on supervised learning that requires large and representative human-labeled data for model training, which is costly and time-consuming. The recent introduction of self-supervised learning (SSL) enables models to learn a representation from orders of magnitude more unlabeled data. The success of SSL is heavily dependent on a pre-designed pretext task, which introduces an inductive bias into the model from a large amount of unlabeled data. Since remote sensing imagery has rich spectral information beyond the standard RGB color space, it may not be straightforward to extend to the multi/hyperspectral domain the pretext tasks established in computer vision based on RGB images. To address this challenge, this work proposed a generic self-supervised learning framework based on remote sensing data at both the object and pixel levels. The method contains two novel pretext tasks, one for object-based and one for pixel-based remote sensing data analysis methods. One pretext task is used to reconstruct the spectral profile from the masked data, which can be used to extract a representation of pixel information and improve the performance of downstream tasks associated with pixel-based analysis. The second pretext task is used to identify objects from multiple views of the same object in multispectral data, which can be used to extract a representation and improve the performance of downstream tasks associated with object-based analysis. The results of two typical downstream task evaluation exercises (a multilabel land cover classification task on Sentinel-2 multispectral datasets and a ground soil parameter retrieval task on hyperspectral datasets) demonstrate that the proposed SSL method learns a target representation that covers both spatial and spectral information from massive unlabeled data. A comparison with currently available SSL methods shows that the proposed method, which emphasizes both spectral and spatial features, outperforms existing SSL methods on multi- and hyperspectral remote sensing datasets. We believe that this approach has the potential to be effective in a wider range of remote sensing applications and we will explore its utility in more remote sensing applications in the future.

Near Real-Time Mapping of Tropical Forest Disturbance Using SAR and Semantic Segmentation in Google Earth Engine

Satellite-based forest alert systems are an important tool for ecosystem monitoring, planning conservation, and increasing public awareness of forest cover change. Continuous monitoring in tropical regions, such as those experiencing pronounced monsoon seasons, can be complicated by spatially extensive and persistent cloud cover. One solution is to use Synthetic Aperture Radar (SAR) imagery acquired by the European Space Agency’s Sentinel-1A and B satellites. The Sentinel 1A and B satellites acquire C-band radar data that penetrates cloud cover and can be acquired during the day or night. One challenge associated with operational use of radar imagery is that the speckle associated with the backscatter values can complicate traditional pixel-based analysis approaches. A potential solution is to use deep learning semantic segmentation models that can capture predictive features that are more robust to pixel-level noise. In this analysis, we present a prototype SAR-based forest alert system that utilizes deep learning classifiers, deployed using the Google Earth Engine cloud computing platform, to identify forest cover change with near real-time classification over two Cambodian wildlife sanctuaries. By leveraging a pre-existing forest cover change dataset derived from multispectral Landsat imagery, we present a method for efficiently developing a SAR-based semantic segmentation dataset. In practice, the proposed framework achieved good performance comparable to an existing forest alert system while offering more flexibility and ease of development from an operational standpoint.

Accounting for intensity variation in image analysis of large-scale multiplexed clinical trial datasets.

Multiplex immunofluorescence (mIF) imaging can provide comprehensive quantitative and spatial information for multiple immune markers for tumour immunoprofiling. However, application at scale to clinical trial samples sourced from multiple institutions is challenging due to pre-analytical heterogeneity. This study reports an analytical approach to the largest multi-parameter immunoprofiling study of clinical trial samples to date. We analysed 12,592 tissue microarray (TMA) spots from 3,545 colorectal cancers sourced from more than 240 institutions in two clinical trials (QUASAR 2 and SCOT) stained for CD4, CD8, CD20, CD68, FoxP3, pan-cytokeratin, and DAPI by mIF. TMA slides were multi-spectrally imaged and analysed by cell-based and pixel-based marker analysis. We developed an adaptive thresholding method to account for inter- and intra-slide intensity variation in TMA analysis. Applying this method effectively ameliorated inter- and intra-slide intensity variation improving the image analysis results compared with methods using a single global threshold. Correlation of CD8 data derived by our mIF analysis approach with single-plex chromogenic immunohistochemistry CD8 data derived from subsequent sections indicates the validity of our method (Spearman's rank correlation coefficients ρ between 0.63 and 0.66, p ≪ 0.01) as compared with the current gold standard analysis approach. Evaluation of correlation between cell-based and pixel-based analysis results confirms equivalency (ρ > 0.8, p ≪ 0.01, except for CD20 in the epithelial region) of both analytical approaches. These data suggest that our adaptive thresholding approach can enable analysis of mIF-stained clinical trial TMA datasets by digital pathology at scale for precision immunoprofiling.

3D pose estimation and localization of construction equipment from single camera images by virtual model integration

Computer vision technologies are receiving much attention for automatic site monitoring. However, previous approaches mainly focused on two-dimensional (2D) pixel information, and there was a limit to providing detailed three-dimensional (3D) pose information. Some studies have built a training database only using virtual models, but there were still limitations in representing the actual equipment. To address these limitations, the authors propose a method for estimating 3D poses of equipment from single camera images by integrating virtual models. The proposed method consists of three main processes: (1) 2D – 3D annotation using a virtual model; (2) preprocessing for pose estimation in two ways (i.e., image matching or key points detection); and (3) 3D pose localization. As a result, the root mean square error for the 3D pose of construction equipment was 1.49 m (using image matching) and 1.56 m (using key points detection). The results imply that the proposed method successfully estimated the 3D pose of the equipment with a single camera. The findings of this study can contribute to detailed productivity or safety analysis based on more diverse 3D information than 2D pixel-based analysis.

Implications of future land-use/cover pattern change on landslide susceptibility at a national level: A scenario-based analysis in Romania

Land-use/cover (LUC) change is a global change forcing with important impacts on landslides, which highlights the need to evaluate the potential future evolution of these hazards under future LUC scenarios. Accordingly, the current paper provides a first national-level exploration on the possible effects of LUC pattern change on landslide susceptibility (LS) in Romania, shedding light on the evolution of one of Europe’s landslide hotspot countries as well as one of the territories significantly affected by recent LUC change, especially after the fall of the communist regime. Two countrywide future LS maps were modelled for 2075, by applying a previously developed national scale-adapted LS empirical procedure, integrating therein two existing long-term LUC pattern forecasts and assuming no change for the other landslide controlling factors. The estimated future LS maps were then compared to the current LS zonation using a pixel-based analysis to estimate the contribution of LUC class transitions to the LS variation in time. The quantitative examination of LS changes suggests an overall net slight decrease at the national level under both scenarios (on an average of 7.3% of the LUC change area), with more significant regional variations. The location and extent of the estimated potential LUC transitions reflect themselves into a clear influence on the future LS level manifesting itself, on average, over 7.1% of the country’s area. The analysis demonstrates that the expected increase in the forest-covered area brings the most important contribution to the decrease of the LS level. An average of 44.2% of the total LS decrease area was mainly predicted by the transitions from pastures and natural grasslands/scrubs to forests, and 11.4% by that from heterogeneous agricultural areas to forests. Conversely, transitions to pastures and natural grasslands resulted in the most important contribution to the increase of the LS level (52.2%). Transitions among other agricultural lands resulted in slight susceptibility variations or in no change. Through the analysed LUC scenarios, differing in terms of the implementation of environmental management regulations inside protected areas, the study proves that a more appropriate land management could have an important influence on decreasing LS, a finding of particular relevance for encouraging and extending a conscious application of protective land measures, especially on slopes which are degraded or prone to instability, in a country otherwise characterised by an overall relative implementation of environmental policies. In addition, the analysis reveals that a significant proportion (24.2%) of the projected new built-up areas is expected to spread by 2075 over lands with moderate, high and very high LS, resulting in potentially new landslide exposure hotspots in the future in both rural and urban areas. The obtained results could be used in directing landslide risk reduction actions and decisions primarily towards the management of LUC and towards prioritizing areas for such measures, while, from a theoretical perspective, this approach could be extended to future investigations related to LUC change implications for landslide hazard and risk in Romania, but could also be followed in case of other national territories.

The Significance of Metadata and Video Compression for Investigating Video Files on Social Media Forensic

Digital forensics is an essential aspect of cyber security and the investigation of digital crimes. Digital recordings are routinely used as important evidence sources in the identification, analysis, presentation, and reporting of evidence. There has recently been concern that images and videos cannot be used as solid evidence since they may be altered very quickly due to the abundance of technologies available for the gathering and processing of multimedia data. The main goal of this endeavour is to comprehend advanced forensic video analysis methods to assist in criminal investigations. We first propose the acquisition extraction analysis in a forensic video analysis framework that employs efficient video and image enhancement techniques for low-quality video that would be transferred through social media applications and for CCTV footage analysis. The reliability of digital video recordings is essential in forensic science and other criminal investigation fields. Digital video forensic analysis is a technique that constantly faces new challenges. Currently, videos are authenticated using a variety of parameters, including pixel-based analysis, frame rate analysis, bit rate analysis, hash value analysis, and, most importantly, metadata analysis. It was believed that the development of technology required the development of a new method for the verification of digital video recordings. In this review study, we made a novel attempt by reviewing the media. Information and structural analysis of video containers in the MP4 file format have been used to distinguish between real and altered videos.

Risk Assessment of Rising Temperatures Using Landsat 4–9 LST Time Series and Meta® Population Dataset: An Application in Aosta Valley, NW Italy

Earth observation data have assumed a key role in environmental monitoring, as well as in risk assessment. Rising temperatures and consequently heat waves due to ongoing climate change represent an important risk considering the population, as well as animals, exposed. This study was focused on the Aosta Valley Region in NW Italy. To assess population exposure to these patterns, the following datasets have been considered: (1) HDX Meta population dataset refined and updated in order to map population distribution and its features; (2) Landsat collection (missions 4 to 9) from 1984 to 2022 obtained and calibrated in Google Earth Engine to model LST trends. A pixel-based analysis was performed considering Aosta Valley settlements and relative population distribution according to the Meta population dataset. From Landsat data, LST trends were modelled. The LST gains computed were used to produce risk exposure maps considering the population distribution and structure (such as ages, gender, etc.). To check the consistency and quality of the HDX population dataset, MAE was computed considering the ISTAT population dataset at the municipality level. Exposure-risk maps were finally realized adopting two different approaches. The first one considers only LST gain maximum by performing an ISODATA unsupervised classification clustering in which the separability of each class obtained and was checked by computing the Jeffries–Matusita (J-M) distances. The second one was to map the rising temperature exposure by developing and performing a risk geo-analysis. In this last case the input parameters considered were defined after performing a multivariate regression in which LST maximum was correlated and tested considering (a) Fractional Vegetation Cover (FVC), (b) Quote, (c) Slope, (d) Aspect, (e) Potential Incoming Solar Radiation (mean sunlight duration in the meteorological summer season), and (f) LST gain mean. Results show a steeper increase in LST maximum trend, especially in the bottom valley municipalities, and especially in new built-up areas, where more than 60% of the Aosta Valley population and domestic animals live and where a high exposure has been detected and mapped with both approaches performed. Maps produced may help the local planners and the civil protection services to face global warming from a One Health perspective.

MP68-03 USING A COMPUTER VISION-MEDIATED ANALYSIS TO DISTINGUISH SURGEON EXPERIENCE DURING ENDOSCOPIC STONE SURGERY

You have accessJournal of UrologyCME1 Apr 2023MP68-03 USING A COMPUTER VISION-MEDIATED ANALYSIS TO DISTINGUISH SURGEON EXPERIENCE DURING ENDOSCOPIC STONE SURGERY Jackson Cabo, Daiwei Lu, Zach Stoebner, Ipek Oguz, and Nick Kavoussi Jackson CaboJackson Cabo More articles by this author , Daiwei LuDaiwei Lu More articles by this author , Zach StoebnerZach Stoebner More articles by this author , Ipek OguzIpek Oguz More articles by this author , and Nick KavoussiNick Kavoussi More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003331.03AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: The assessment of surgical competency is essential for clinical training and safety. Currently, there are no validated, objective, real-time tools for evaluating expertise during endoscopic surgery. We sought to apply validated, automated kidney stone segmentation software to distinguish expert and trainee surgeons during flexible ureteroscopy (fURS). METHODS: Forty-six separate videos of fURS were prospectively recorded. Surgeons were categorized as ‘expert’ (n=2, fellowship trained endourologist, case volume of >100 fURS per year) or trainee (n=2, resident, <100 fURS per year), performing two defined tasks: stone localization, and holmium laser ablation (dusting). Surgeons were randomly assigned to one of these tasks per case. Stone localization was standardized as evaluation of the entire collecting system. For the localization task, we analyzed first 5 seconds of video after stone identification. For the laser ablation task, we analyzed the first 20 seconds of stone ablation. All videos were visually validated for quality and frames extracted at 30fps. Frames were analyzed by previously validated automated stone segmentation models. We performed a pixel-based analysis, evaluating percent occupancy of stone for each frame and compared differences between trainees and experts for each task. RESULTS: Of 46 videos (N=14299 frames), 28 were evaluated for the localization task (14 trainee, 14 expert) and 18 for the laser ablation task (9 novice, 9 expert). The percentage of frames without stones identified was higher in trainees compared to experts for both localization (25% vs. 5%, p<0.01) and laser ablation (16% vs. 8%, p<0.01), indicating more frequent loss of stone visualization by trainees. Stones occupied more of each frame for trainees compared to experts for both localization (18% vs. 11% , p<0.01) and laser ablation (20% vs. 16%, p<0.01), suggesting closer stone visualization during stone treatment by trainees (Fig. 1). There was greater variation in frame-to-frame stone occupancy for trainees compared to experts during stone localization (2.9% vs 1.5%, p<0.01). CONCLUSIONS: Objective and automated computer vision-mediated analysis can distinguish surgical experience between experts and trainee surgeons performing fURS for kidney stones. Source of Funding: Endourologic Society Grant Funding © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e953 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Jackson Cabo More articles by this author Daiwei Lu More articles by this author Zach Stoebner More articles by this author Ipek Oguz More articles by this author Nick Kavoussi More articles by this author Expand All Advertisement PDF downloadLoading ...