Ground Truth Generation Research Articles

GNN-based reverse design for mechanical systems: Bridging trajectory and mechanical design

In today’s rapidly evolving landscape, conventional construction techniques like masonry and concrete pouring are falling short in meeting the evolving needs of contemporary designs that demand versatile and adaptable mechanical solutions. However, the established methods for crafting such mechanisms are time-intensive and frequently fall short in achieving the desired trajectory outcomes. In light of this challenge, this paper takes up the mantle of presenting a streamlined and accurate approach for devising trajectory-mechanism designs. This is achieved through the synergistic integration of multiple deep-learning prediction models, which serve to unveil the intricate interplay between the components of a linkage mechanism and the intended trajectory. To establish a robust foundation, a ground truth generation system is crafted using Rhino 3D modeling software. This system lays the groundwork for producing essential data to be harnessed in the training and testing of the models. A comprehensive series of experiments is then conducted to unearth solutions that can generate predictive trajectories aligned with stringent design requisites. The efficacy of the proposed framework is tested on the intricate StrandBeest structure to gauge its adaptability. The ensuing quantitative and qualitative analyses offer both empirical evidence and valuable insights into the potency of the proposed methodology. In short, the proposed method introduces an innovative paradigm for fashioning trajectory-mechanism designs in a more resourceful and swift manner when juxtaposed with prior methodologies. This endeavor not only addresses current limitations but also represents a step forward in ushering efficiency and effectiveness into this realm of research and application.

PhyberSIM: a tool for the generation of ground truth to evaluate brain fiber clustering algorithms.

Diffusion Magnetic Resonance Imaging tractography is a non-invasive technique that produces a collection of streamlines representing the main white matter bundle trajectories. Methods, such as fiber clustering algorithms, are important in computational neuroscience and have been the basis of several white matter analysis methods and studies. Nevertheless, these clustering methods face the challenge of the absence of ground truth of white matter fibers, making their evaluation difficult. As an alternative solution, we present an innovative brain fiber bundle simulator that uses spline curves for fiber representation. The methodology uses a tubular model for the bundle simulation based on a bundle centroid and five radii along the bundle. The algorithm was tested by simulating 28 Deep White Matter atlas bundles, leading to low inter-bundle distances and high intersection percentages between the original and simulated bundles. To prove the utility of the simulator, we created three whole-brain datasets containing different numbers of fiber bundles to assess the quality performance of QuickBundles and Fast Fiber Clustering algorithms using five clustering metrics. Our results indicate that QuickBundles tends to split less and Fast Fiber Clustering tends to merge less, which is consistent with their expected behavior. The performance of both algorithms decreases when the number of bundles is increased due to higher bundle crossings. Additionally, the two algorithms exhibit robust behavior with input data permutation. To our knowledge, this is the first whole-brain fiber bundle simulator capable of assessing fiber clustering algorithms with realistic data.

Integrating Point Spread Function into Taxel-based Tactile Pattern Super Resolution.

The past decade has witnessed the development of tactile sensors, which have been increasingly considered as an essential equipment in robotics, especially the dexterous manipulation and collaborative human-robot interactions. There are two major types of tactile sensors, i.e., the vision-based and taxel-based sensors. The latter is capable of achieving lower integration complexity with existing robotic systems, but unable to provide high-resolution (HR) tactile information as that of the vision-based counterpart due to the manufacturing limitations. Therefore, we propose a novel tactile pattern super-resolution (SR) scheme for taxel-based sensors, which is a data-driven scheme enabling customized selection on the number of applied "tapping" actions to achieve improvable performance from single tapping SR (STSR) to the multi-tapping SR (MTSR). In addition, we develop a new dataset for the proposed tactile SR scheme. In order to obtain scalable resolutions (e.g. ×4, ×10, ×20, etc.) of ground-truth HR tactile patterns, we propose a novel tactile point spread function (PSF) scheme to generate HR tactile patterns by leveraging the low-resolution (LR) data gathered directly from the taxel-based sensor and the depth information of contact surfaces. This is in strong contrast to the conventional ground-truth generation approach with overlapped multi-sampling and registration strategy, which can only provide a fixed resolution. Experimental results confirm the efficiency of the proposed scheme.

CID-SIMS: Complex indoor dataset with semantic information and multi-sensor data from a ground wheeled robot viewpoint

Simultaneous localization and mapping (SLAM) and 3D reconstruction have numerous applications for indoor ground wheeled robots such as floor sweeping and food delivery. To advance research in leveraging semantic information and multi-sensor data to enhance the performances of SLAM and 3D reconstruction in complex indoor scenes, we propose a novel and complex indoor dataset named CID-SIMS, where semantic annotated RGBD images, inertial measurement unit (IMU) measurements, and wheel odometer data are provided from a ground wheeled robot viewpoint. The dataset consists of 22 challenging sequences captured in nine different scenes including office building and apartment environments. Notably, our dataset achieves two significant breakthroughs. Firstly, semantic information and multi-sensor data are provided meanwhile for the first time. Secondly, GeoSLAM is utilized for the first time to generate ground truth trajectories and 3D point clouds within two-centimeter accuracy. With spatial-temporal synchronous ground truth trajectories and 3D point clouds, our dataset is capable of evaluating SLAM and 3D reconstruction algorithms in a unified global coordinate system. We evaluate state-of-the-art SLAM and 3D reconstruction approaches on our dataset, demonstrating that our benchmark is applicable. The dataset is publicly available on https://cid-sims.github.io.

Unsupervised CNN-based DIC method for 2D displacement measurement

Digital image correlation (DIC) is a widely used technique for non-contact measurement of deformation. However, traditional DIC methods face challenges in balancing calculation efficiency and the quantity of seed points. Deep learning approaches, particularly supervised learning methods, have shown promise in improving DIC efficiency. However, these methods require high-quality training data, which can be time-consuming to generate ground truth annotations. To address these challenges, we propose an unsupervised convolutional neural network (CNN) based DIC method for 2D displacement measurement. Our approach leverages an encoder-decoder architecture with multi-level feature extraction, a dual-path correlation block, and an attention block to extract informative features from speckle images with varying characteristics. We utilize a speckle image warp model to transform the deformed speckle image to the predicted reference speckle image based on the predicted 2D displacement map. The unsupervised training is achieved by comparing the predicted and original reference speckle images. To optimize the network's parameters, we employ a composite loss function that takes into account both the Mean Squared Error (MSE) and Pearson correlation coefficient. By using unsupervised convolutional neural network (CNN) based DIC method, we eliminate the need for extensive training data annotation, which is a time-consuming process in supervised learning DIC methods. We have conducted several experiments to demonstrate the validity and robustness of our proposed method. The results show a significant reduction in Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) compared to a method proposed by Zhao et al. This indicates that our unsupervised CNN-based DIC approach can achieve accuracy comparable to supervised CNN-based DIC methods. For implementation and evaluation, we provide PyTorch code and datasets, which will be released at the following URL :https://github.com/fead1/DICNet-corr-unsupervised-learning-.

Population data-based federated machine learning improves automated echocardiographic quantification of cardiac structure and function: the Automatisierte Vermessung der Echokardiographie project.

Machine-learning (ML)-based automated measurement of echocardiography images emerges as an option to reduce observer variability. The objective of the study is to improve the accuracy of a pre-existing automated reading tool ('original detector') by federated ML-based re-training. Automatisierte Vermessung der Echokardiographie was based on the echocardiography images of n = 4965 participants of the population-based Characteristics and Course of Heart Failure Stages A-B and Determinants of Progression Cohort Study. We implemented federated ML: echocardiography images were read by the Academic Core Lab Ultrasound-based Cardiovascular Imaging at the University Hospital Würzburg (UKW). A random algorithm selected 3226 participants for re-training of the original detector. According to data protection rules, the generation of ground truth and ML training cycles took place within the UKW network. Only non-personal training weights were exchanged with the external cooperation partner for the refinement of ML algorithms. Both the original detectors as the re-trained detector were then applied to the echocardiograms of n = 563 participants not used for training. With regard to the human referent, the re-trained detector revealed (i) superior accuracy when contrasted with the original detector's performance as it arrived at significantly smaller mean differences in all but one parameter, and a (ii) smaller absolute difference between measurements when compared with a group of different human observers. Population data-based ML in a federated ML set-up was feasible. The re-trained detector exhibited a much lower measurement variability than human readers. This gain in accuracy and precision strengthens the confidence in automated echocardiographic readings, which carries large potential for applications in various settings.

NIMG-45. EXTERNAL EVALUATION OF A MACHINE LEARNING MODEL EMPLOYING RADIOMICS TO IDENTIFY REGIONS OF LOCAL RECURRENCE IN GLIOBLASTOMA FROM POSTOPERATIVE MRI

Abstract The standard surgical goal for operable glioblastomas worldwide is the complete excision of the enhancing tumor. Nevertheless, the peritumoral region harbors infiltrating cells leading to recurrence. Our study aims to evaluate a predictive model designed to predict potential regions of recurrence using voxel-based radiomics analysis on postoperative magnetic resonance imaging (MRI). This retrospective, multi-institutional study involved glioblastoma patients who had undergone complete resection. The training cohort comprised 49 patients from two Spanish institutions, with model evaluation using an external dataset of 27 patients from a Norwegian institution and the public Ivy-Gap dataset. We used follow-up MRI scans for ground truth definition of recurrence, while postoperative multiparametric MRI scans provided the data for extracting voxel-based radiomic features. Our method employed an unsupervised, hybrid approach, which combined a deep neural network and instance optimization, to align the MRI sequences for each patient, registering the follow-up with the postoperative scans. To generate ground truth labels, we segmented the peritumoral region in the postoperative scans, identified as the area showing T2/FLAIR signal alterations, in addition to the enhancing tumor visible in the registered follow-up scan. Overlapping voxels between the peritumoral and enhancing tumor regions were classified as recurrence, while non-overlapping voxels were labeled as nonrecurrence. Radiomic features were subsequently extracted from the postoperative peritumoral region. We trained four machine learning classifiers to predict recurrence. Among these, the Categorical Boosting (CatBoost) classifier demonstrated excellent performance on the test dataset, achieving an average area under the curve (AUC) of 0.83 ± 0.07, and an accuracy of 0.80 ± 0.12, based on voxel-wise comparison with the ground truth. Our study resulted in a method that accurately predicts the area of future tumor recurrence in glioblastoma patients' MRI scans. This innovation could potentially tailor surgical and radiotherapy treatment strategies to these specific areas, possibly extending patient survival.

Multi-branch Segmentation-guided Attention Network for crowd counting

Crowd counting has gained increasing attention recently owing to its importance in public safety. However, it remains a challenging task due to background complexities and high annotation costs. To address these challenges, we propose the Multi-branch Segmentation-guided Attention Network (MSGANet). To deal with the complex background, we incorporate segmentation-guided attention branches into both shallow and deep layers of the baseline model, allowing simultaneous consideration of spatial and semantic information. Multi-level attention maps enable the network to minimize the influence of complex backgrounds while focusing on the regions containing crowds. To tackle the issue of costly annotations, MSGANet utilizes only point annotations to generate ground truth density and segmentation maps, eliminating additional expenses. Our results demonstrate that our approach achieves highly competitive performance compared to state-of-the-art crowd counting methods.

Canine Vertebral Column Segmentation with UNet Models: A Positive Approach to Radiographic Image Analysis

Degenerative diseases of the vertebral column, such as spondylosis, are not uncommon in dogs. Detecting these diseases can be challenging and is often overlooked due to lack of effective treatments. The intricacy of medical images poses difficulties in disease identification. Our approach focuses on segmenting the vertebral column from X-ray images of dogs, facilitating targeted disease detection and improving accuracy. Conventional segmentation methods are ineffective for medical images due to presence of occlusions, and pixel classification methods require the laborious creation of masks. By training our UNet models, we successfully achieved precise segmentation of the vertebral column from radiographic images, significantly reducing the time required for generating ground truth masks. Our UNet models effectively enhanced the isolation of the vertebral column from X-ray images, thereby optimizing the accuracy of degenerative disease detection methods for the vertebral column. Key Words: segmentation, medical imaging, UNet, radiographs, computer vision, canine

Automating Ground Truth Annotations for Gland Segmentation Through Immunohistochemistry

Microscopic evaluation of glands in the colon is of utmost importance in the diagnosis of inflammatory bowel disease and cancer. When properly trained, deep learning pipelines can provide a systematic, reproducible, and quantitative assessment of disease-related changes in glandular tissue architecture. The training and testing of deep learning models require large amounts of manual annotations, which are difficult, time-consuming, and expensive to obtain. Here, we propose a method for automated generation of ground truth in digital hematoxylin and eosin (H&E)–stained slides using immunohistochemistry (IHC) labels. The image processing pipeline generates annotations of glands in H&E histopathology images from colon biopsy specimens by transfer of gland masks from KRT8/18, CDX2, or EPCAM IHC. The IHC gland outlines are transferred to coregistered H&E images for training of deep learning models. We compared the performance of the deep learning models to that of manual annotations using an internal held-out set of biopsy specimens as well as 2 public data sets. Our results show that EPCAM IHC provides gland outlines that closely match manual gland annotations (Dice = 0.89) and are resilient to damage by inflammation. In addition, we propose a simple data sampling technique that allows models trained on data from several sources to be adapted to a new data source using just a few newly annotated samples. The best performing models achieved average Dice scores of 0.902 and 0.89 on Gland Segmentation and Colorectal Adenocarcinoma Gland colon cancer public data sets, respectively, when trained with only 10% of annotated cases from either public cohort. Altogether, the performances of our models indicate that automated annotations using cell type–specific IHC markers can safely replace manual annotations. Automated IHC labels from single-institution cohorts can be combined with small numbers of hand-annotated cases from multi-institutional cohorts to train models that generalize well to diverse data sources.

Semi-supervised labelling of chest x-ray images using unsupervised clustering for ground-truth generation

Supervised classifiers require a lot of data with accurate labels to learn to recognize chest X-ray images (CXR). However, manually labeling an extensive collection of CXR images is time-consuming and costly. To address this issue, a method for the semi-supervised labelling of extensive collections of CXR images is proposed leveraging unsupervised clustering with minimum expert knowledge to generate ground truth images. The proposed methodology entails: using unsupervised clustering techniques such as K-Means and Self-Organizing Maps. Second, the images are fed to five different feature vectors to utilize the potential differences between features to their full advantage. Third, each data point gets the label of the cluster’s center to which it belongs. Finally, a majority vote is used to decide the ground truth image. The number of clusters created by the method chosen strictly limits the amount of human involvement. To evaluate the effectiveness of the proposed method, experiments were conducted on two publicly available CXR datasets, namely VinDR-CXR and Montgomery datasets. The experiments showed that, for a KNN classifier, manually labeling only 1% (VinDr-CXR), or 10% (Montgomery) of the training data, gives a similar performance as labeling the whole dataset. The proposed methodology efficiently generates ground-truth images from publicly available CXR datasets. To our knowledge, this is the first study to use the VinDr-CXR and Montgomery datasets for ground truth image generation. Extensive experimental analysis using machine learning and statistical techniques shows that the proposed methodology efficiently generates ground truth images from CXR datasets.

P13.05.B INCORPORATION OF AI-BASED AUTOSEGMENTATION AND CLASSIFICATION INTO NEURORADIOLOGY WORKFLOW: PACS-BASED AI TO BUILD YALE GLIOMA DATASET

Abstract BACKGROUND Translation of AI algorithms into clinical practice is significantly limited by lack of large individual hospital-based datasets with expert annotations. Current methods for generation of annotated imaging data are significantly limited due to inefficient imaging data transfer, complicated annotation software, and time required for experts to generate ground truth information. We incorporated AI tools for auto-segmentation of gliomas into PACS that is used at our institution for reading clinical studies and developed a workflow for annotation of images and development of volumetric segmentations in neuroradiology clinical workflow. MATERIAL AND METHODS 1990 patients from Yale Radiation Oncology Registry (2012-2019) were screened. Segmentations were performed using a UNETR algorithm trained on BRaTS 2021 and an internal dataset of manually segmented tumors. AI generated segmentation can be revised and confirmed in PACS. Segmentations were validated by a board-certified neuro-radiologist, after which natively embedded PyRadiomics in PACS was used for direct feature extraction. RESULTS In 7 Months (05/2021 - 08/2021, 03/2022 - 05/2022) segmentations and annotations were performed in 1033 patients (429 female, 604 male, mean age 53 yrs). Dataset includes 595 Grade 4 Gliomas (96 Grade 3, 105 Grade 2, 45 Grade 1, 192 unknown). Molecular subtypes include IDH (129 mutated, 651 wildtype, 253 unknown), 1p/19q (94 deleted or co-deleted, 135 intact, 804 unknown), MGMT promotor (216 methylated, 110 partially methylated, 321 unmethylated, 383 unknown), EGFR (125 amplified, 248 not amplified, 660 unknown), ATRX (41 mutated, 238 retained, 754 unknown), Ki-67 (726 known, 307 unknown) and p53 (639 known, 394 unknown). Classification of gliomas between grade 3/4 and grade 1/2, yielded AUC of 0.85. CONCLUSION We developed a method for incorporation of volumetric segmentation, feature extraction, and classification that is easily incorporated into neuroradiology workflow. These tools allowed us to annotate over 100 gliomas per month, thus establishing a proof of concept for rapid development of annotated imaging database for AI applications.

Refined feature fusion for in-field high-density and multi-scale rice panicle counting in UAV images

The yield of rice crops is strongly correlated to the number of panicles per unit area. Computer vision techniques have been utilized in previous studies to count the number of panicles automatically. However, when using images captured by Unmanned Aerial Vehicles (UAVs), the height and coverage area of the UAVs can cause shrinkage of the rice panicle objects, leading to errors in feature learning and reduced counting efficacy. In addition, there is a considerable relative size difference in rice ears, which can further affect counting accuracy. To address these issues, this paper proposes an algorithm named Refined Feature Fusion for Panicle Counting (RFF-PC). This algorithm utilizes a more refined scale division by extracting and fusing optimal features according to the object size distribution. Firstly, the number of rice ears with different sizes is quantified, and a fine division of the scale is made to calculate the receptive field size of different features that need to be fused. Secondly, multi-scale convolution generates ]features at each layer and feature pyramid fusion fuses more appropriate features at different layers to improve the ability to represent objects in complex multi-scale. Redundant information is removed through channel attention. Additionally, a refined Gaussian is used to generate ground truth close to the shape of real rice panicles. Over the UFPC2019 dataset captured at the height of 5 meters, the average counting accuracy of RFF-PC is 89.80%, and the counting accuracy only decreases from 94.33% to 90.58% over the images with an increased number of rice ears from 180 to 260, which outperforms several state-of-the-art algorithms, including MCNN, CSRNet, TasselNetV2+, DSNet, and DMCount.

The Subgroup Imperative: Chest Radiograph Classifier Generalization Gaps in Patient, Setting, and Pathology Subgroups.

To externally test four chest radiograph classifiers on a large, diverse, real-world dataset with robust subgroup analysis. In this retrospective study, adult posteroanterior chest radiographs (January 2016-December 2020) and associated radiology reports from Trillium Health Partners in Ontario, Canada, were extracted and de-identified. An open-source natural language processing tool was locally validated and used to generate ground truth labels for the 197 540-image dataset based on the associated radiology report. Four classifiers generated predictions on each chest radiograph. Performance was evaluated using accuracy, positive predictive value, negative predictive value, sensitivity, specificity, F1 score, and Matthews correlation coefficient for the overall dataset and for patient, setting, and pathology subgroups. Classifiers demonstrated 68%-77% accuracy, 64%-75% sensitivity, and 82%-94% specificity on the external testing dataset. Algorithms showed decreased sensitivity for solitary findings (43%-65%), patients younger than 40 years (27%-39%), and patients in the emergency department (38%-60%) and decreased specificity on normal chest radiographs with support devices (59%-85%). Differences in sex and ancestry represented movements along an algorithm's receiver operating characteristic curve. Performance of deep learning chest radiograph classifiers was subject to patient, setting, and pathology factors, demonstrating that subgroup analysis is necessary to inform implementation and monitor ongoing performance to ensure optimal quality, safety, and equity.Keywords: Conventional Radiography, Thorax, Ethics, Supervised Learning, Convolutional Neural Network (CNN), Machine Learning Algorithms Supplemental material is available for this article. © RSNA, 2023See also the commentary by Huisman and Hannink in this issue.

A Benchmark for Multi-Modal LiDAR SLAM with Ground Truth in GNSS-Denied Environments

LiDAR-based simultaneous localization and mapping (SLAM) approaches have obtained considerable success in autonomous robotic systems. This is in part owing to the high accuracy of robust SLAM algorithms and the emergence of new and lower-cost LiDAR products. This study benchmarks the current state-of-the-art LiDAR SLAM algorithms with a multi-modal LiDAR sensor setup, showcasing diverse scanning modalities (spinning and solid state) and sensing technologies, and LiDAR cameras, mounted on a mobile sensing and computing platform. We extend our previous multi-modal multi-LiDAR dataset with additional sequences and new sources of ground truth data. Specifically, we propose a new multi-modal multi-LiDAR SLAM-assisted and ICP-based sensor fusion method for generating ground truth maps. With these maps, we then match real-time point cloud data using a normal distributions transform (NDT) method to obtain the ground truth with a full six-degrees-of-freedom (DOF) pose estimation. These novel ground truth data leverage high-resolution spinning and solid-state LiDARs. We also include new open road sequences with GNSS-RTK data and additional indoor sequences with motion capture (MOCAP) ground truth, complementing the previous forest sequences with MOCAP data. We perform an analysis of the positioning accuracy achieved, comprising ten unique configurations generated by pairing five distinct LiDAR sensors with five SLAM algorithms, to critically compare and assess their respective performance characteristics. We also report the resource utilization in four different computational platforms and a total of five settings (Intel and Jetson ARM CPUs). Our experimental results show that the current state-of-the-art LiDAR SLAM algorithms perform very differently for different types of sensors. More results, code, and the dataset can be found at GitHub.

A deep learning approach for accurate and robust T1 estimation from cardiac magnetic resonance MOLLI sequences

Abstract Funding Acknowledgements Type of funding sources: None. Background T1 parametric maps of the myocardium are a recent development in cardiac Magnetic Resonance Imaging (MRI). They can be used to characterize the composition of the heart and identify alterations that can be indicative of a pathology. A common method of T1 estimation involves pixel-wise non-linear curve fitting from Modified Look-Locker Inversion Recovery (MOLLI) series [1]. Whilst relatively accurate, this method presents several drawbacks. Firstly, it assumes magnetization recovery to be a purely exponential process dependent on T1, thus neglecting other parameters such as T2 and B1 that can interfere in the process. Secondly, it does not consider that each acquisition during an inversion recovery experiment inverts the magnetization vector by a given angle and thus interferes with the recovery leading to the measurement of a shorter T1* rather than the actual T1 [2]. Additionally, since it considers each pixel individually, neglecting spatial information, it is sensitive to image quality [3]. Finally, since non-linear curve fitting is an iterative algorithm, it is computationally expensive and time-consuming. Methods To address these issues a convolutional neural network, DeepBLESS [4], and two spatially aware variants of it, were trained to predict T1, T2, and B1 with a dataset of MOLLI series simulated from experimental data. The simulation was performed using Bloch simulations and parameters extracted from a dataset composed of 27 single-slice MOLLI series acquired with the 5(3s)3 protocol using a b-SSFP readout at a main magnetic field strength of 3 Tesla. Thus, a dataset could be generated without relying on exponential fitting for ground truth generation. Results The exponential fitting of T1 was shown to consistently underestimate true T1 values by 138 +- 90 ms. In comparison, the spatially naïve DeepBLESS approach showed a mean absolute error of 14 +- 9 ms and the spatially aware DeepBLESS approach (Median-DeepBLESS) showed a mean absolute error of 13 +- 9 ms. As can be seen in Figure 1, Median-DeepBLESS also showed considerably more resilience to noise than exponential fitting and the spatially naïve version of DeepBLESS. Additionally, the deep learning approaches showcased promising results with regards to the estimation of T2 and B1 from MOLLI sequences (See Figure 2). Conclusions It was shown that the Deep Learning approaches eliminated the underestimation bias of exponential fitting with the spatially aware network Median-DeepBLESS being more robust to noise than both exponential fitting and the standard DeepBLESS network for myocardial T1 estimation. Relatively accurate estimation of T2 and B1 from MOLLI sequences was also accomplished through the Deep Learning approach, which may lead to simultaneous T1, T2, and B1 estimation in common clinical practice in the future.

MEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops.

Connectomics is fundamental in propelling our understanding of the nervous system's organization, unearthing cells and wiring diagrams reconstructed from volume electron microscopy (EM) datasets. Such reconstructions, on the one hand, have benefited from ever more precise automatic segmentation methods, which leverage sophisticated deep learning architectures and advanced machine learning algorithms. On the other hand, the field of neuroscience at large, and of image processing in particular, has manifested a need for user-friendly and open source tools which enable the community to carry out advanced analyses. In line with this second vein, here we propose mEMbrain, an interactive MATLAB-based software which wraps algorithms and functions that enable labeling and segmentation of electron microscopy datasets in a user-friendly user interface compatible with Linux and Windows. Through its integration as an API to the volume annotation and segmentation tool VAST, mEMbrain encompasses functions for ground truth generation, image preprocessing, training of deep neural networks, and on-the-fly predictions for proofreading and evaluation. The final goals of our tool are to expedite manual labeling efforts and to harness MATLAB users with an array of semi-automatic approaches for instance segmentation. We tested our tool on a variety of datasets that span different species at various scales, regions of the nervous system and developmental stages. To further expedite research in connectomics, we provide an EM resource of ground truth annotation from four different animals and five datasets, amounting to around 180 h of expert annotations, yielding more than 1.2 GB of annotated EM images. In addition, we provide a set of four pre-trained networks for said datasets. All tools are available from https://lichtman.rc.fas.harvard.edu/mEMbrain/. With our software, our hope is to provide a solution for lab-based neural reconstructions which does not require coding by the user, thus paving the way to affordable connectomics.

Deep learning to reconstruct quasi-steady-state chemical exchange saturation transfer from a non-steady-state experiment.

The insufficiently long RF saturation duration and relaxation delay in chemical exchange saturation transfer (CEST)-MRI experiments may result in underestimation of CEST measurements. To maintain the CEST effect without prolonging the saturation duration and reach quasi-steady state (QUASS), a deep learning method was developed to reconstruct a QUASS CEST image pixel by pixel from non-steady-state CEST acquired in experiments. In this work, we established a tumor-bearing rat model on a 7 T horizontal bore small-animal MRI scanner, allowing ground-truth generation, after which a bidirectional long short-term memory network was formulated and trained on simulated CEST Z-spectra to reconstruct the QUASS CEST; finally, the ground truth yielded by experiments was used to evaluate the performance of the reconstruction model by comparing the estimates with the ground truth. For quantitation evaluation, linear regression analysis, structural similarity index (SSIM) and peak signal-to-noise ratio (peak SNR) were used to assess the proposed model in the QUASS CEST reconstruction. In the linear regression analysis of in vivo data, the coefficient of determination for six different representative frequency offsets was at least R2 = 0.9521. Using the SSIM and peak SNR as evaluation metrics, the reconstruction accuracies of in vivo QUASS CEST were found to be 0.9991 and 46.7076, respectively. Experimental results demonstrate that the proposed model provides a robust and accurate solution for QUASS CEST reconstruction using a deep learning mechanism.

Dorsal Hand Vein Pattern Recognition: A Comparison between Manual and Automatic Segmentation Methods.

Various techniques for dorsal hand vein (DHV) pattern extraction have been introduced using small datasets with poor and inconsistent segmentation. This work compared manual segmentation with our proposed hybrid automatic segmentation method (HHM) for this classification problem. Manual segmentation involved selecting a region-of-interest (ROI) in images from the Bosphorus dataset to generate ground truth data. The HHM combined histogram equalization and morphological and thresholding-based algorithms to localize veins from hand images. The data were divided into training, validation, and testing sets with an 8:1:1 ratio before training AlexNet. We considered three image augmentation strategies to enlarge our training sets. The best training hyperparameters were found using the manually segmented dataset. We obtained a good test accuracy (91.5%) using the model trained with manually segmented images. The HHM method showed slightly inferior performance (76.5%). Considerable improvement was observed in the test accuracy of the model trained with the inclusion of automatically segmented and augmented images (84%), with low false acceptance and false rejection rates (0.00035% and 0.095%, respectively). A comparison with past studies further demonstrated the competitiveness of our technique. Our technique can be feasible for extracting the ROI in DHV images. This strategy provides higher consistency and greater efficiency than the manual approach.

Traffic Data on-the-Fly: Developing a Statewide Crosswalk Inventory Using Artificial Intelligence and Aerial Images (AI2) for Pedestrian Safety Policy Improvements in Florida

Pedestrian fatalities have been rising sharply in the U.S. over the last decade. To reverse this trend, several pedestrian safety countermeasures are developed by the authorities, particularly for the uncontrolled crosswalks where the majority of those fatalities occurred. Although geocoded and categorized crosswalk inventories are essential to determine where to implement the most appropriate countermeasure, building such an inventory database may take years with manual observations. Thus, this study proposes a framework using Aerial Imagery and Artificial Intelligence (AI2) to map all crosswalks on Florida public roadways according to their control strategy. A transfer learning approach with anchor box modifications is applied to the You-Only-Look-Once (YOLOv2) object detection algorithm. Crosswalks with various pavement markings and width/height ratios are identified by running YOLOv2 crosswalk detector on statewide high-resolution aerial images. Additionally, Geographic Information System-GIS techniques are used to reduce the input data, establish a geocoded connection for each detected crosswalk, and conduct a sensitivity analysis to identify signal, midblock, and driveway crosswalks. According to the performance evaluations conducted with a manually generated ground truth dataset, the proposed approach can inventory the crosswalks with 85.9% recall and 88.7% precision, while accuracy is significantly higher for signal crosswalks (94%) and major road crossings (96.6%) regardless of zebra or non-zebra pavement markings. The results revealed that there are 861 midblock, 30,784 signal, and 29,307 driveway crosswalks on Florida State Roads. The proposed framework and findings of this study can contribute to the wider use of machine learning in transport policy.