Open-source Image Research Articles

Closed-loop vasculature network design for bioprinting large, solid tissue scaffolds

Vascularization is an indispensable requirement for fabricating large solid tissues and organs. The natural vasculature derived from medical imaging modalities for large tissues and organs are highly complex and convoluted. However, the present bioprinting capabilities limit the fabrication of such complex natural vascular networks. Simplified bioprinted vascular networks, on the other hand, lack the capability to sustain large solid tissues. This work proposes a generalized and adaptable numerical model to design the vasculature by utilizing the tissue/organ anatomy. Starting with processing the patient’s medical images, organ structure, tissue-specific cues, and key vasculature tethers are determined. An open-source abdomen magnetic resonance image dataset was used in this work. The extracted properties and cues are then used in a mathematical model for guiding the vascular network formation comprising arterial and venous networks. Next, the generated three-dimensional networks are used to simulate the nutrient transport and consumption within the organ over time and the regions deprived of the nutrients are identified. These regions provide cues to evolve and optimize the vasculature in an iterative manner to ensure the availability of the nutrient transport throughout the bioprinted scaffolds. The mass transport of six components of cell culture media—glucose, glycine, glutamine, riboflavin, human serum albumin, and oxygen was studied within the organ with designed vasculature. As the vascular structure underwent iterations, the organ regions deprived of these key components decreased significantly highlighting the increase in structural complexity and efficacy of the designed vasculature. The numerical method presented in this work offers a valuable tool for designing vascular scaffolds to guide the cell growth and maturation of the bioprinted tissues for faster regeneration post bioprinting.

Hardware-software co-design of an open-source automatic multimodal whole slide histopathology imaging system.

Advanced digital control of microscopes and programmable data acquisition workflows have become increasingly important for improving the throughput and reproducibility of optical imaging experiments. Combinations of imaging modalities have enabled a more comprehensive understanding of tissue biology and tumor microenvironments in histopathological studies. However, insufficient imaging throughput and complicated workflows still limit the scalability of multimodal histopathology imaging. We present a hardware-software co-design of a whole slide scanning system for high-throughput multimodal tissue imaging, including brightfield (BF) and laser scanning microscopy. The system can automatically detect regions of interest using deep neural networks in a low-magnification rapid BF scan of the tissue slide and then conduct high-resolution BF scanning and laser scanning imaging on targeted regions with deep learning-based run-time denoising and resolution enhancement. The acquisition workflow is built using Pycro-Manager, a Python package that bridges hardware control libraries of the Java-based open-source microscopy software Micro-Manager in a Python environment. The system can achieve optimized imaging settings for both modalities with minimized human intervention and speed up the laser scanning by an order of magnitude with run-time image processing. The system integrates the acquisition pipeline and data analysis pipeline into a single workflow that improves the throughput and reproducibility of multimodal histopathological imaging.

The area-reconstruction h-dome technique and its efficient Python implementation for improved particle size image analysis.

The traditional watershed segmentation methods usually suffer from over segmentation for irregularly shaped particles. This is because the distance map of an irregularly shaped particle contains multiple local maxima, and over segmentation would happen if these local maxima were used as seeds for watershed segmentation. In this work, several methods based on morphological reconstruction, including h-dome transform, h-maxima, and area-reconstruction h-dome transform, are introduced to merge, or erase redundant local maxima, and the performance of these methods in avoiding over segmentation is compared. The results show that the area-reconstruction h-dome transform is the most effective method in controlling over segmentation among the evaluated methods. However, the area-reconstruction h-dome transform is achieved by superposition of binary reconstructions at each grayscale level, which is extremely time-consuming and impractical for batch processing. A hybrid pixel-queue algorithm is applied to accelerate the area-reconstruction h-dome transform, and the algorithm is implemented in Cython to further improve the computational efficiency. For a 2592 × 1944 pixel image, on a PC with an Intel Core i5 2.4GHz processor and 8GB RAM, the processing time of the area-reconstruction h-dome transform after acceleration is about 549 ms, which is 249 times faster than the unaccelerated algorithm and 4 times faster than the reconstruction function in the Scikit-image library (an open-source image processing library for the Python programming language) which performs reconstruction by dilation. The accelerated area-reconstruction h-dome transform algorithm was successfully applied to the segmentation of rubber particles in a thermoplastic polyolefin (TPO) compound. RESEARCH HIGHLIGHTS: Techniques for segmenting particles with irregular shapes based on morphological reconstruction are reviewed. A fast algorithm for area-reconstruction h-dome transform is introduced based on Vincent's first approach combined with the pixel queue algorithm and Cython acceleration. The accelerated reconstruction algorithm is 249 times faster than the unaccelerated algorithm. The fast area-reconstruction h-dome transform algorithm is successfully applied to rubber particle segmentation of a thermal plastic polyolefin.

DHM-viewer: Comprehensive, open-source image processing suite for digital holographic microscopy.

In recent years, digital holographic microscopy (DHM) has witnessed extensive use in quantitative phase imaging (QPI) applications such as live cell dynamics imaging, and 3D particle tracking owing to its ability to reconstruct optical fields based on a single exposure. As a computational imaging technique, DHM is highly specific to the processing pipeline depending on optical configuration and processing requirements. Commercial DHM systems additionally rely on proprietary user interfaces and specific workflows that prohibit user modification, limiting their appeal for users who wish to explore different optical configurations or implement the latest processing algorithms. To address these demands, we present a comprehensive open-source software suite (DHM-viewer) compatible with most commercially available and bespoke DHM systems. DHM-viewer provides user-facing functionalities such as optical alignment assistance, stage actuation control, real-time live image acquisition, and post-processing visualization with a Graphical User Interface (GUI) system. Notably, the GUI is driven by a Standard Operation Procedure (SOP) processing layout that assist users and can accommodate various optical configurations and operating modes. From a system control perspective, DHM-viewer exploits the Micro-Manager framework to ensure compatibility with many existing microscope automation toolkits. From a post-processing perspective, the core software supports a variety of state-of-the-art phase aberration correction and Fourier filtering algorithms as well as deep learning-accelerated phase unwrapping and 3D tracking functionality leveraging the computational power of GPU-based processing. Finally, the DHM-viewer backend supports large batch offline processing through dedicated server-side distribution. The functionality and usability of the software were verified on three different DHM platforms, including one hybrid DHM-fluorescence system.

Flexible and open-source programs for quantitative image analysis in microbial ecology

Epifluorescence microscopy is an essential tool for obtaining reliable estimates of the abundance of marine microorganisms including viruses. However, computational analysis is required to gain consistent and quantitative data from digital microscopy images. Many imaging programs are proprietary and cost-prohibitive. The currently available free imaging programs are often platform specific and/or lack the flexibility to analyze microscopy images from natural samples, such as the planktonic environment, which can contain challenges such as debris and high background signals. Here we describe two MATLAB-based open-source image analysis programs that work across computer platforms and provide the tools to analyze a range of image types and cell sizes with a user-friendly interface. The Microbial Image Analysis (MiA) program aims to provide flexibility for the selection, identification, and quantification of cells that vary in size and fluorescence intensity within natural microbial communities. The Viral Image Analysis (ViA) program aims to provide an effective means for quantifying viral abundances from epifluorescence images as well as enumerating the intensity of a primary and secondary stain. In this paper, we provide an overview of the functionality of the MiA and ViA programs and highlight specific program features through several microbial image case studies.

Dual-branche attention network for super-resolution of remote sensing images

ABSTRACT Remote sensing (RS) images are considered to be reflections of the real world. However, RS images often suffered from low resolution, making further research difficult to follow. Although super resolution (SR) techniques based on deep learning have achieved considerable breakthroughs, they show limited performance when dealing with low-quality RS images with complicated backgrounds; for instance, the SR results tend to loss details and have undesired structural distortion. Thus, this paper proposes an innovative dual-branch attention network (DBAN) to produce sufficient details and preserve clear structural information for SR results of RS images. It consists of two components: a feature extraction branch and a high-frequency information learning branch. The features extraction branch, formed as a densely residual structure, combines a series of dual attention blocks that are designed to exploit valid features from different dimensions, and then all these multi-scale features are reused through a global concatenation. The high-frequency information extraction branch, incorporating noise removing units (NRU) and high-frequency attention units (HFU), is responsible for producing the high-frequency features without noise, which enables DBAN to handle the problem of structural distortion. Meanwhile, a composite loss function based on a Laplacian pyramid is proposed to maximize the structural similarity between reconstruction results and real high-resolution RS images. The proposed network is efficient and lightweight because of its strong and effective attention to feature learning. Experimental results on three open-source RS image datasets and the JiLin-01 dataset demonstrate the effectiveness of our DBAN where higher accuracy over state-of-the-art methods in super-resolving complicated images was achieved.

A novel approach for yoga pose estimation based on in-depth analysis of human body joint detection accuracy.

Virtual motion and pose from images and video can be estimated by detecting body joints and their interconnection. The human body has diverse and complicated poses in yoga, making its classification challenging. This study estimates yoga poses from the images using a neural network. Five different yoga poses, viz. downdog, tree, plank, warrior2, and goddess in the form of RGB images are used as the target inputs. The BlazePose model was used to localize the body joints of the yoga poses. It detected a maximum of 33 body joints, referred to as keypoints, covering almost all the body parts. Keypoints achieved from the model are considered as predicted joint locations. True keypoints, as the ground truth body joint for individual yoga poses, are identified manually using the open source image annotation tool named Makesense AI. A detailed analysis of the body joint detection accuracy is proposed in the form of percentage of corrected keypoints (PCK) and percentage of detected joints (PDJ) for individual body parts and individual body joints, respectively. An algorithm is designed to measure PCK and PDJ in which the distance between the predicted joint location and true joint location is calculated. The experiment evaluation suggests that the adopted model obtained 93.9% PCK for the goddess pose. The maximum PCK achieved for the goddess pose, i.e., 93.9%, PDJ evaluation was carried out in the staggering mode where maximum PDJ is obtained as 90% to 100% for almost all the body joints.

Perspective: Disentangling the effects of tES on neurovascular unit.

Transcranial electrical stimulation (tES) can modulate the neurovascular unit, including the perivascular space morphology, but the mechanisms are unclear. In this perspective article, we used an open-source "rsHRF toolbox" and an open-source functional magnetic resonance imaging (fMRI) transcranial direct current stimulation (tDCS) data set to show the effects of tDCS on the temporal profile of the haemodynamic response function (HRF). We investigated the effects of tDCS in the gray matter and at three regions of interest in the gray matter, namely, the anodal electrode (FC5), cathodal electrode (FP2), and an independent site remote from the electrodes (PZ). A "canonical HRF" with time and dispersion derivatives and a finite impulse response (FIR) model with three parameters captured the effects of anodal tDCS on the temporal profile of the HRF. The FIR model showed tDCS onset effects on the temporal profile of HRF for verum and sham tDCS conditions that were different from the no tDCS condition, which questions the validity of the sham tDCS (placebo). Here, we postulated that the effects of tDCS onset on the temporal profile of HRF are subserved by the effects on neurovascular coupling. We provide our perspective based on previous work on tES effects on the neurovascular unit, including mechanistic grey-box modeling of the effects of tES on the vasculature that can facilitate model predictive control (MPC). Future studies need to investigate grey-box modeling of online effects of tES on the neurovascular unit, including perivascular space, neurometabolic coupling, and neurovascular coupling, that can facilitate MPC of the tES dose-response to address the momentary ("state") and phenotypic ("trait") factors.

Intraobserver repeatability for a standardized protocol to quantify keel bone damage in laying hens using discrete and continuous radiographic measures.

Sternal carina damage (keel bone damage, KBD) is an important welfare concern for laying hen producers and backyard flock owners. Quantitative radiographic measures of KBD severity are helpful for researchers who study causes for this problem and the effects of novel interventions. The objectives of the current retrospective secondary analysis study were to develop and test intraobserver repeatability for a standardized protocol to quantify three categories of radiographic KBD using open-source image analysis software and discrete and continuous variables. The standardized protocol was developed and evaluated using triplicate measurements of 470 keel bone radiographs that had been previously acquired for a prospective study. Intraobserver repeatability was assessed using intra-class correlation coefficient (ICC) and control chart methods. Based on control chart analyses, measures within the acceptable range of intra-observer variation using the standardized protocol were the number of complete cranial fractures (97.02%), incomplete cranial fractures (96.38%), complete caudal fractures (95.32%), incomplete caudal fractures (98.09%), cranial calluses (99.79%), and caudal calluses (98.09%); proportion of deviation (POD) measurements (97.87%); and angle of displacement (AOD) measurements (93.60%). Findings can be used as background to support future research studies on KBD in laying hens.

A Feasibility Study of In-Situ Damage Visualization in Basalt-Fiber Reinforced Polymers with Open-Source Digital Volume Correlation.

This work analyses damage formation within the bulk of basalt fiber-reinforced polymers (BFRP) by means of open-source Digital Volume Correlation (DVC). Volumetric image data were obtained from conventional in-situ X-Ray computed micro-tomography (µCT) of samples loaded in tension. The open-source image registration toolkit Elastix was employed to obtain full 3D displacement fields from the image data. We assessed the accuracy of the DVC results using the method of manufactured solution and showed that the approach followed here can detect deformation with a magnitude in the order of a fiber diameter which in the present case is 17 µm. The beneficial influence of regularization on DVC results is presented on the manufactured solution as well as on real in-situ tensile testing CT data of a BFRP sample. Results of the correlation showed that conventional µCT equipment in combination with DVC can be used to detect defects which could previously only be visualized using synchrotron facilities or destructive methods.

Accelerating pharmaceutical R&D with a user-friendly AI system for histopathology image analysis

A system for analysis of histopathology data within a pharmaceutical R&D environment has been developed with the intention of enabling interdisciplinary collaboration. State-of-the-art AI tools have been deployed as easy-to-use self-service modules within an open-source whole slide image viewing platform, so that non-data scientist users (e.g., clinicians) can utilize and evaluate pre-trained algorithms and retrieve quantitative results. The outputs of analysis are automatically cataloged in the database to track data provenance and can be viewed interactively on the slide as annotations or heatmaps. Commonly used models for analysis of whole slide images including segmentation, extraction of hand-engineered features for segmented regions, and slide-level classification using multi-instance learning are included and new models can be added as needed. The source code that supports running inference with these models internally is backed up by a robust CI/CD pipeline to ensure model versioning, robust testing, and seamless deployment of the latest models. Examples of the use of this system in a pharmaceutical development workflow include glomeruli segmentation, enumeration of podocyte count from WT-1 immuno-histochemistry, measurement of beta-1 integrin target engagement from immunofluorescence, digital glomerular phenotyping from periodic acid-Schiff histology, PD-L1 score prediction using multi-instance learning, and the deployment of the open-source Segment Anything model to speed up annotation.

A novel smartphone HPTLC assaying platform versus traditional densitometric method for simultaneous quantification of alfuzosin and solifenacin in their dosage forms as well as monitoring content uniformity and drug residues on the manufacturing equipment.

The large popularity and rapid technology of smartphones have opened new avenues for their integration into different analytical methodologies and drug quality monitoring as a portable, easily accessible, and user-friendly detector. Herein, a novel and portable smartphone-based high-performance thin layer chromatographic (HPTLC) approach is proposed for the simultaneous analysis of two urological drugs, alfuzosin and solifenacin, which treat benign prostatic hyperplasia accompanied by overactive bladder syndrome. First, chromatographic separation was accomplished using an ecofriendly mobile phase, then the developed plates were visualized using Dragendorff's reagent and photographed via a smartphone's rear-facing camera fixed on a fabricated two-illumination-source chamber. The intensities of the drug spots were quantified using open-source image analysis software ImageJ over the concentration ranges of 2.0 to 30.0 μg per band for both drugs with acceptable results in ICH validation parameters. To improve the method's accuracy and reproducibility, various construction and shooting key parameters were investigated and optimized. Moreover, the study was extended to compare the obtained results with those of a benchtop densitometric method using a Camag TLC Scanner 3 at 215.0 nm; the densitometric method provided an additional assessment tool for peak purity and was capable of assaying lower drug concentrations over a linearity range of 0.2-8.0 μg per band for alfuzosin and 0.1-6.0 μg per band for solifenacin. The fast, simple, reliable, green merits of the proposed HPTLC/smartphone method suggest that it is an excellent platform for assaying marketed combined capsules and assuring their content uniformity. Moreover, the high sensitivity of the densitometric method was used, for the first time, to determine the residual content of the cited drugs on manufacturing equipment surfaces for cleaning validation. Finally, the environmental impact of the developed methods was evaluated based on green analytical chemistry principles.

Using Open-Source Image Datasets for Research

The application of AI-based Facial Recognition (FRT) raises various legal and ethical questions. A comprehensive analysis must take into consideration not only the creation and future use of facial recognition but has to start even earlier: namely with the selection of the underlying data sets. While it may seem practical to utilize open-source image datasets for the creation, training and testing of models, the compliant use of such data sets still represents a major legal hurdle, especially with regard to international aspects. This is mainly due the fact, that harmonization in the area of research and data protection is very limited and still largely left to national legislators, even within the European Union. In this article, the authors analyse key problems of the use of open-source datasets for research in the area of FRT based on AI.

Forensic Document Examination Using Image Processing Tools

Paper documents have always been our primary means of transmitting and storing information. However, for better storage and processing, the majority of records are currently only available in electronic format. As a result of digitization, the field of forensic document analysis has improved and evolved to keep up with the times, as it has become relatively easy for anyone to change any type of document using various image modification tools. The current study focuses to examine documents that have been manipulated by computers using open-source image processing tools. Forensically beta, FotoForensics, and Adobe Photoshop tools were used to analyze the sample documents that had been altered. The results suggest that these image processing tools can be used to analyze digitally modified documents to detect manipulations and thus can help Forensic document examiners. The present work reveals that some minute characteristics are present in the transplanted documents which can be identified by careful analysis of soft copies of documents with tools present in Forensically beta, FotoForensics, and Adobe Photoshop.

SoC 환경에서 TIDL NPU를 활용한 딥러닝 기반 도로 영상 인식 기술

Deep learning-based image processing is essential for autonomous vehicles. To process road images in real-time in a System-on-Chip (SoC) environment, we need to execute deep learning models on a NPU (Neural Procesing Units) specialized for deep learning operations. In this study, we imported seven open-source image processing deep learning models, that were developed on GPU servers, to Texas Instrument Deep Learning (TIDL) NPU environment. We confirmed that the models imported in this study operate normally in the SoC virtual environment through performance evaluation and visualization. This paper introduces the problems that occurred during the migration process due to the limitations of NPU environment and how to solve them, and thereby, presents a reference case worth referring to for developers and researchers who want to port deep learning models to SoC environments.

Calibrating the Dice Loss to Handle Neural Network Overconfidence for Biomedical Image Segmentation

The Dice similarity coefficient (DSC) is both a widely used metric and loss function for biomedical image segmentation due to its robustness to class imbalance. However, it is well known that the DSC loss is poorly calibrated, resulting in overconfident predictions that cannot be usefully interpreted in biomedical and clinical practice. Performance is often the only metric used to evaluate segmentations produced by deep neural networks, and calibration is often neglected. However, calibration is important for translation into biomedical and clinical practice, providing crucial contextual information to model predictions for interpretation by scientists and clinicians. In this study, we provide a simple yet effective extension of the DSC loss, named the DSC++ loss, that selectively modulates the penalty associated with overconfident, incorrect predictions. As a standalone loss function, the DSC++ loss achieves significantly improved calibration over the conventional DSC loss across six well-validated open-source biomedical imaging datasets, including both 2D binary and 3D multi-class segmentation tasks. Similarly, we observe significantly improved calibration when integrating the DSC++ loss into four DSC-based loss functions. Finally, we use softmax thresholding to illustrate that well calibrated outputs enable tailoring of recall-precision bias, which is an important post-processing technique to adapt the model predictions to suit the biomedical or clinical task. The DSC++ loss overcomes the major limitation of the DSC loss, providing a suitable loss function for training deep learning segmentation models for use in biomedical and clinical practice. Source code is available at https://github.com/mlyg/DicePlusPlus.

Role of casein kinase 1 in the amoeboid migration of B-cell leukemic and lymphoma cells: A quantitative live imaging in the confined environment.

The migratory properties of leukemic cells are commonly associated with their pathological potential and can significantly affect the disease progression. While the research in immunopathology mostly employed powerful indirect methods such as flow cytometry, these cells were rarely observed directly using live imaging microscopy. This is especially true for the malignant cells of the B-cell lineage, such as those originating from chronic lymphocytic leukemia (CLL) and mantle cell lymphoma (MCL). In this study, we employed open-source image analysis tools to automatically and quantitatively describe the amoeboid migration of four B-cell leukemic and lymphoma cell lines and primary CLL cells. To avoid the effect of the shear stress of the medium on these usually non-adherent cells, we have confined the cells using a modified under-agarose assay. Surprisingly, the behavior of tested cell lines differed substantially in terms of basal motility or response to chemokines and VCAM1 stimulation. Since casein kinase 1 (CK1) was reported as a regulator of B-cell migration and a promoter of CLL, we looked at the effects of CK1 inhibition in more detail. Migration analysis revealed that CK1 inhibition induced rapid negative effects on the migratory polarity of these cells, which was quantitatively and morphologically distinct from the effect of ROCK inhibition. We have set up an assay that visualizes endocytic vesicles in the uropod and facilitates morphological analysis. This assay hints that the effect of CK1 inhibition might be connected to defects in polarized intracellular transport. In summary, 1) we introduce and validate a pipeline for the imaging and quantitative assessment of the amoeboid migration of CLL/MCL cells, 2) we provide evidence that the assay is sensitive enough to mechanistically study migration defects identified by the transwell assay, and 3) we describe the polarity defects induced by inhibition or deletion of CK1ε.

Open-source image analysis tool for the identification and quantification of cortical interruptions and bone erosions in high-resolution peripheral quantitative computed tomography images of patients with rheumatoid arthritis.

Identification of bone erosions and quantification of erosion volume is important for rheumatoid arthritis diagnosis, and can add important information to evaluate disease progression and treatment effects. High-resolution peripheral quantitative computed tomography (HR-pQCT) is well suited for this purpose, however analysis methods are not widely available. The purpose of this study was to develop an open-source software tool for the identification and quantification of bone erosions using images acquired by HR-pQCT. The collection of modules, Bone Analysis Modules (BAM) - Erosion, implements previously published erosion analysis techniques as modules in 3D Slicer, an open-source image processing and visualization tool. BAM includes a module to automatically identify cortical interruptions, from which erosions are manually selected, and a hybrid module that combines morphological and level set operations to quantify the volume of bone erosions. HR-pQCT images of the second and third metacarpophalangeal (MCP) joints were acquired in patients with RA (XtremeCT, n=14, XtremeCTII, n=22). The number of cortical interruptions detected by BAM-Erosion agreed strongly with the previously published cortical interruption detection algorithm for both XtremeCT (r2=0.85) and XtremeCTII (r2=0.87). Erosion volume assessment by BAM-Erosion agreed strongly (r2=0.95) with the Medical Image Analysis Framework. BAM-Erosion provides an open-source erosion analysis tool that produces comparable results to previously published algorithms, with improved options for visualization. The strength of the tool is that it implements multiple image processing algorithms for erosion analysis on a single, widely available, open-source platform that can accommodate future updates.

Sodium fluorescein in pediatric oncological neurosurgery: a pilot study on 50 children.

Sodium fluorescein (SF) is currently considered a valid intraoperative adjunct in the resection of high-grade brain lesions in adults. Experiences in pediatric groups and in low-grade gliomas and other low-grade lesions are still limited in literature, and subjective evaluation of fluorescence is still a limitation. This study retrospectively reviewed all patients with brain or spine lesions operated on from September 2021 to July 2022 in the Pediatric Neurosurgery Unit of Hôpital Femme Mère Enfant, Lyon, who had received 5mg/kg of 10%. Surgery was performed using a YELLOW560 filter at crucial times. At the end of surgery, the first operator completed a questionnaire, including his opinion on whether SF had been useful in tumor resection, recorded as a binary variable. Post hoc, surgical images were reviewed using ImageJ, an open-source Java image processing platform. In order to compare independent discrete variables, we applied the Student's t test, and we applied the Chi-square or Fisher exact test for binary variables. A threshold of p < 0.05 was set for statistical significance. We included 50 pediatric patients (0.2-17.6years old). Forty/50 lesions showed SF uptake (80%). The differentiation between healthy and affected tissue, thanks to SF, subjectively evaluated by the surgeon, had as objective counterpart the statistically significant higher brightness of green in lesions, registered by the software (p < 0.001). SF overall allowed a good differentiation in 33/50 lesions, and overall utility of SF has been noted in 67% of them. When specifically considering gliomas, overall utility reached 75%. SF is a feasible, safe, and useful intraoperative adjunct in pediatric neurosurgery. In particular, it seems to have a promising role in some low-grade infiltrating glial tumors. The subjective evaluation of fluorescence seems to be reliable with respect to image analyses software.

Large-scale individual building extraction from open-source satellite imagery via super-resolution-based instance segmentation approach

Building footprint is a primary dataset of an urban geographic information system (GIS) database. Therefore, it is essential to establish a robust and automated framework for large-scale building extraction. However, the characteristic of remote sensing images complicates the application of the instance segmentation method based on the Mask R-CNN model, which ought to be improved toward extracting and fusing multi-scale features. Moreover, open-source satellite image datasets with wider spatial coverage and temporal resolution than high-resolution images may exhibit different coloration and resolution. This study proposes a large-scale building extraction framework based on super-resolution (SR) and instance segmentation using a relatively lower-resolution (>0.6 m) open-sourced dataset. The framework comprises four steps: color normalization and image super-resolution, scene classification, building extraction, and scene mosaicking. We took Hyogo Prefecture, Japan (19,187 km2) as a test area and extracted 1,726,006 (29.12 km2) of the 3,301,488 buildings (32.46 km2), where the number of buildings and footprint area increased by 3.0 % and 5.0 % respectively. The result indicated that the color normalization and image super-resolution could improve the visual quality of open-source satellite images and contribute to building extraction accuracy. Moreover, the improved Mask R-CNN based on Multi-Path Vision Transformer (MPViT) backbone achieved F1 scores of 0.71, 0.70, 0.81, and 0.67 for non-built-up, rural, suburban, and urban areas, respectively, which is better than those of the baseline model and other mainstream instance segmentation approaches. This study demonstrates the potential of acquiring acceptable building footprint maps from open-source satellite images, which has significant practical implications.