Ground Truth Generation Research Articles

EuroCity Persons 2.0: A Large and Diverse Dataset of Persons in Traffic.

We present the EuroCity Persons (ECP) 2.0 dataset, a novel image dataset for person detection, tracking and prediction in traffic. The dataset was collected on-board a vehicle driving through 29 cities in 11 European countries. It contains more than 250K unique person trajectories, in more than 2.0M images and comes with a size of 11 TB. ECP2.0 is about one order of magnitude larger than previous state-of-the-art person datasets in automotive context. It offers remarkable diversity in terms of geographical coverage, time of day, weather and seasons. We discuss the novel semi-supervised approach that was used to generate the temporally dense pseudo ground-truth (i.e., 2D bounding boxes, 3D person locations) from sparse, manual annotations at keyframes. Our approach leverages auxiliary LiDAR data for 3D uplifting and vehicle inertial sensing for ego-motion compensation. It incorporates keyframe information in a three-stage approach (tracklet generation, tracklet merging into tracks, track smoothing) for obtaining accurate person trajectories. We validate our pseudo ground-truth generation approach in ablation studies, and show that it significantly outperforms existing methods. Furthermore, we demonstrate its benefits for training and testing of state-of-the-art tracking methods. Our approach provides a speed-up factor of about 34 compared to frame-wise manual annotation. The ECP2.0 dataset is made freely available for non-commercial research use.

An artificial intelligence-based recognition model of colorectal liver metastases in intraoperative ultrasonography with improved accuracy through algorithm integration.

Contrast-enhanced intraoperative ultrasonography (CE-IOUS) is crucial for detecting colorectal liver metastases (CLM) during surgery. Although artificial intelligence shows potential in diagnostic systems, its application in CE-IOUS is limited. This study aimed to develop an automatic tumor detection model using Mask region-based convolutional neural network (Mask R-CNN) for CE-IOUS images. CE-IOUS videos of the CLM from 121 patients were collected, generating ground truth data. A total of 2659 images were obtained. Two models were developed: the basic recognition model (BRM), which was trained on CE-mode images, and the subtraction model (SM), which used images created by a subtraction algorithm that highlighted the differences in pixel values between the basic-mode and CE-mode images. The subtraction algorithm focuses on echogenicity differences. These two models were combined into a combination model (CM), which assessed outcomes using the prediction probabilities from both models. The optimal epochs were determined by the maximum area under the curve (AUC), and the thresholds were calculated accordingly. BRM, SM, and CM achieved 89.4%, 86.6%, and 96.5% accuracy, respectively. CM outperformed the individual models, achieving an AUC of 0.99. A novel automated recognition model was developed for accurate CLM detection in CE-IOUS by integrating image- and algorithm-based models.

Using a Device-Free Wi-Fi Sensing System to Assess Daily Activities and Mobility in Low-Income Older Adults: Protocol for a Feasibility Study.

Older adults belonging to racial or ethnic minorities with low socioeconomic status are at an elevated risk of developing dementia, but resources for assessing functional decline and detecting cognitive impairment are limited. Cognitive impairment affects the ability to perform daily activities and mobility behaviors. Traditional assessment methods have drawbacks, so smart home technologies (SmHT) have emerged to offer objective, high-frequency, and remote monitoring. However, these technologies usually rely on motion sensors that cannot identify specific activity types. This group often lacks access to these technologies due to limited resources and technology experience. There is a need to develop new sensing technology that is discreet, affordable, and requires minimal user engagement to characterize and quantify various in-home activities. Furthermore, it is essential to explore the feasibility of developing machine learning (ML) algorithms for SmHT through collaborations between clinical researchers and engineers and involving minority, low-income older adults for novel sensor development. This study aims to examine the feasibility of developing a novel channel state information-based device-free, low-cost Wi-Fi sensing system, and associated ML algorithms for localizing and recognizing different patterns of in-home activities and mobility in residents of low-income senior housing with and without mild cognitive impairment. This feasibility study was conducted in collaboration with a wellness care group, which serves the healthy aging needs of low-income housing residents. Prior to this feasibility study, we conducted a pilot study to collect channel state information data from several activity scenarios (eg, sitting, walking, and preparing meals) using the proposed Wi-Fi sensing system continuously over a week in apartments of low-income housing residents. These activities were videotaped to generate ground truth annotations to test the accuracy of the ML algorithms derived from the proposed system. Using qualitative individual interviews, we explored the acceptability of the Wi-Fi sensing system and implementation barriers in the low-income housing setting. We use the same study protocol for the proposed feasibility study. The Wi-Fi sensing system deployment began in November 2022, with participant recruitment starting in July 2023. Preliminary results will be available in the summer of 2025. Preliminary results are focused on the feasibility of developing ML models for Wi-Fi sensing-based activity and mobility assessment, community-based recruitment and data collection, ground truth, and older adults' Wi-Fi sensing technology acceptance. This feasibility study can make a contribution to SmHT science and ML capabilities for early detection of cognitive decline among socially vulnerable older adults. Currently, sensing devices are not readily available to this population due to cost and information barriers. Our sensing device has the potential to identify individuals at risk for cognitive decline by assessing their level of physical function by tracking their in-home activities and mobility behaviors, at a low cost. DERR1-10.2196/53447.

Predicting creep behavior in composites from microstructural features using deep learning

This study uses a multilayer perceptron deep learning model to predict creep strain vs time curves in composite materials based on their microstructural features. Finite element simulations generate ground truth data for model training and validation. The multilayer perceptron model, trained on this comprehensive dataset, effectively captures the complex relationships between the microstructure and creep behavior, achieving high accuracy. Comparative analysis with traditional models shows the multilayer perceptron model’s superior performance. This demonstrates the model’s potential for reliable application in various engineering fields, offering improved predictions of composite material behavior under creep conditions.

People make mistakes: Obtaining accurate ground truth from continuous annotations of subjective constructs.

Accurately representing changes in mental states over time is crucial for understanding their complex dynamics. However, there is little methodological research on the validity and reliability of human-produced continuous-time annotation of these states. We present a psychometric perspective on valid and reliable construct assessment, examine the robustness of interval-scale (e.g., values between zero and one) continuous-time annotation, and identify three major threats to validity and reliability in current approaches. We then propose a novel ground truth generation pipeline that combines emerging techniques for improving validity and robustness. We demonstrate its effectiveness in a case study involving crowd-sourced annotation of perceived violence in movies, where our pipeline achieves a .95 Spearman correlation in summarized ratings compared to a .15 baseline. These results suggest that highly accurate ground truth signals can be produced from continuous annotations using additional comparative annotation (e.g., a versus b) to correct structured errors, highlighting the need for a paradigm shift in robust construct measurement over time.

Ultrack: pushing the limits of cell tracking across biological scales.

Tracking live cells across 2D, 3D, and multi-channel time-lapse recordings is crucial for understanding tissue-scale biological processes. Despite advancements in imaging technology, achieving accurate cell tracking remains challenging, particularly in complex and crowded tissues where cell segmentation is often ambiguous. We present Ultrack, a versatile and scalable cell-tracking method that tackles this challenge by considering candidate segmentations derived from multiple algorithms and parameter sets. Ultrack employs temporal consistency to select optimal segments, ensuring robust performance even under segmentation uncertainty. We validate our method on diverse datasets, including terabyte-scale developmental time-lapses of zebrafish, fruit fly, and nematode embryos, as well as multi-color and label-free cellular imaging. We show that Ultrack achieves state-of-the-art performance on the Cell Tracking Challenge and demonstrates superior accuracy in tracking densely packed embryonic cells over extended periods. Moreover, we propose an approach to tracking validation via dual-channel sparse labeling that enables high-fidelity ground truth generation, pushing the boundaries of long-term cell tracking assessment. Our method is freely available as a Python package with Fiji and napari plugins and can be deployed in a high-performance computing environment, facilitating widespread adoption by the research community.

Multi-label deep learning for comprehensive optic nerve head segmentation through data of fundus images

Early diagnosis and continuous monitoring of patients with eye diseases are critical in computer-aided detection (CAD) techniques. Semantic segmentation, a key component in computer vision, enables pixel-level classification and provides detailed information about objects within images. In this study, we present three U-Net models designed for multi-class semantic segmentation, leveraging the U-Net architecture with transfer learning. To generate ground truth for the HRF dataset, we combine two U-Net models, namely MSU-Net and BU-Net, to predict probability maps for the optic disc and cup regions. Binary masks are then derived from these probability maps to extract the optic disc and cup regions from retinal images. The dataset used in this study includes pre-existing blood vessels and manually annotated peripapillary atrophy zones (alpha and beta) provided by expert ophthalmologists. This comprehensive dataset, integrating existing blood vessels and expert-marked peripapillary atrophy zones, fulfills the study's objectives. The effectiveness of the proposed approach is validated by training nine pre-trained models on the HRF dataset comprising 45 retinal images, successfully segmenting the optic disc, cup, blood vessels, and peripapillary atrophy zones (alpha and beta). The results demonstrate 87.7 % pixel accuracy, 87 % Intersection over Union (IoU), 86.9 % F1 Score, 85 % mean IoU (mIoU), and 15 % model loss, significantly contributing to the early diagnosis and monitoring of glaucoma and optic nerve disorders.

Ground-truth generation through crowdsourcing with probabilistic indexes

Automatic transcription of large series of historical handwritten documents generally aims at allowing to search for textual information in these documents. However, automatic transcripts often lack the level of accuracy needed for reliable text indexing and search purposes. Probabilistic Indexing (PrIx) offers a unique alternative to raw transcripts. Since it needs training data to achieve good search performance, PrIx-based crowdsourcing techniques are introduced in this paper to gather the required data. In the proposed approach, PrIx confidence measures are used to drive a correction process in which users can amend errors and possibly add missing text. In a further step, corrected data are used to retrain the PrIx models. Results on five large series are reported which show consistent improvements after retraining. However, it can be argued whether the overall costs of the crowdsourcing operation pay off for the improvements, or perhaps it would have been more cost-effective to just start with a larger and cleaner amount of professionally produced training transcripts.

Universal Facial Encoding of Codec Avatars from VR Headsets

Faithful real-time facial animation is essential for avatar-mediated telepresence in Virtual Reality (VR). To emulate authentic communication, avatar animation needs to be efficient and accurate: able to capture both extreme and subtle expressions within a few milliseconds to sustain the rhythm of natural conversations. The oblique and incomplete views of the face, variability in the donning of headsets, and illumination variation due to the environment are some of the unique challenges in generalization to unseen faces. In this paper, we present a method that can animate a photorealistic avatar in realtime from head-mounted cameras (HMCs) on a consumer VR headset. We present a self-supervised learning approach, based on a cross-view reconstruction objective, that enables generalization to unseen users. We present a lightweight expression calibration mechanism that increases accuracy with minimal additional cost to run-time efficiency. We present an improved parameterization for precise ground-truth generation that provides robustness to environmental variation. The resulting system produces accurate facial animation for unseen users wearing VR headsets in realtime. We compare our approach to prior face-encoding methods demonstrating significant improvements in both quantitative metrics and qualitative results.

A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context.

We present a new set of computational tools that enable accurate and widely applicable 3D segmentation of nuclei in various 3D digital organs. We have developed an approach for ground truth generation and iterative training of 3D nuclear segmentation models, which we applied to popular CellPose, PlantSeg and StarDist algorithms. We provide two high-quality models trained on plant nuclei that enable 3D segmentation of nuclei in datasets obtained from fixed or live samples, acquired from different plant and animal tissues, and stained with various nuclear stains or fluorescent protein-based nuclear reporters. We also share a diverse high-quality training dataset of about 10,000 nuclei. Furthermore, we advanced the MorphoGraphX analysis and visualization software by, among other things, providing a method for linking 3D segmented nuclei to their surrounding cells in 3D digital organs. We found that the nuclear-to-cell volume ratio varies between different ovule tissues and during the development of a tissue. Finally, we extended the PlantSeg 3D segmentation pipeline with a proofreading tool that uses 3D segmented nuclei as seeds to correct cell segmentation errors in difficult-to-segment tissues.

Probabilistic volumetric speckle suppression in OCT using deep learning.

We present a deep learning framework for volumetric speckle reduction in optical coherence tomography (OCT) based on a conditional generative adversarial network (cGAN) that leverages the volumetric nature of OCT data. In order to utilize the volumetric nature of OCT data, our network takes partial OCT volumes as input, resulting in artifact-free despeckled volumes that exhibit excellent speckle reduction and resolution preservation in all three dimensions. Furthermore, we address the ongoing challenge of generating ground truth data for supervised speckle suppression deep learning frameworks by using volumetric non-local means despeckling-TNode- to generate training data. We show that, while TNode processing is computationally demanding, it serves as a convenient, accessible gold-standard source for training data; our cGAN replicates efficient suppression of speckle while preserving tissue structures with dimensions approaching the system resolution of non-local means despeckling while being two orders of magnitude faster than TNode. We demonstrate fast, effective, and high-quality despeckling of the proposed network in different tissue types that are not part of the training. This was achieved with training data composed of just three OCT volumes and demonstrated in three different OCT systems. The open-source nature of our work facilitates re-training and deployment in any OCT system with an all-software implementation, working around the challenge of generating high-quality, speckle-free training data.

A scale-similarity based workflow for self-supervised reconstruction of small-scales in turbulence using deep learning

Deep learning has been extensively utilized for modeling and analysis of fluid turbulence. One such application is the use of super-resolution (SR) algorithms to reconstruct small-scale structures from their large-scale counterparts for turbulent flows. To date, all SR algorithms have been supervised or require unpaired reference data at a high resolution for training. This renders the model inapplicable to practical fluid flow scenarios, in which the generation of a high-resolution ground truth by resolving all scales down to the Kolmogorov scale becomes prohibitive. Hence, it is imperative to develop physics-guided models that exploit the multiscale nature of turbulence. Considering SR as a state-estimation problem, we present a self-supervised workflow based on deep neural networks to reconstruct small-scale structures that are relevant to homogeneous isotropic turbulence. In addition to visual similarity, we assessed the quality of the obtained reconstruction using spectra, structure functions, and probability density functions of the gradients of velocity and a passive scalar. From the analysis, we infer that the outputs of the workflow are in statistical agreement with the ground truth, for which the training pipeline is agnostic. Insights into learnability, interpretability, and generality of the trained networks have been provided as well. The results of this study can be leveraged to devise techniques for the reconstruction of small-scale structures using large-eddy simulation data.

Beyond supervision: An unsupervised spatio-temporal point cloud noise modeling for event vision sensor

Noise modeling is a fundamental unexplored problem in many event camera applications. The noise modeling methods of conventional cameras are inadequate for capturing the intricate noise patterns unique to event cameras. To address this gap, we propose a novel event camera-driven noise modeling approach specifically designed to account for the unique characteristics of event camera data. Given the challenges in generating ground truth data, the proposed method is designed using unsupervised statistical techniques. The proposed noise model is camera-aware, i.e. it allows adaptability to the unique noise patterns of each sensor. It also facilitates asynchronous event processing as applied to downstream computer vision tasks. To validate the proposed noise modeling approach, we conducted extensive experiments in the context of denoising tasks. The results indicate that our noise model accurately represents the noise, leading to state-of-the-art performance across various datasets.

Approximate ground truth generation for semantic labeling of historical documents with minimal human effort

Deep learning approaches have shown high performance for layout analysis of historical documents, provided that enough labeled data is available. This is not an issue for generic tasks such as image binarization, text graphics separation, or text line and text block detection but can become an impediment for more specialized tasks specific to one or a few books only. This paper addresses layout analysis of medieval books with rich and complex layouts, for which no labeled data is initially available. The proposed strategy consists of training an initial model with artificial data created to reflect the rules a deep neural network should learn. Then, the model is iteratively fine-tuned by mixing the artificial data with real data obtained by previous predictions, post-processed, and manually selected by an expert user. Such a strategy needs less human effort than manual ground truthing. The approach is qualitatively and quantitatively assessed and shows that the system converges to an accurate model that finally produces approximate ground truth stable and good enough to train a final model to solve the targeted task with high accuracy.

Synergizing Deep Learning-Enabled Preprocessing and Human-AI Integration for Efficient Automatic Ground Truth Generation.

The progress of incorporating deep learning in the field of medical image interpretation has been greatly hindered due to the tremendous cost and time associated with generating ground truth for supervised machine learning, alongside concerns about the inconsistent quality of images acquired. Active learning offers a potential solution to these problems of expanding dataset ground truth by algorithmically choosing the most informative samples for ground truth labeling. Still, this effort incurs the costs of human labeling, which needs minimization. Furthermore, automatic labeling approaches employing active learning often exhibit overfitting tendencies while selecting samples closely aligned with the training set distribution and excluding out-of-distribution samples, which could potentially improve the model's effectiveness. We propose that the majority of out-of-distribution instances can be attributed to inconsistent cross images. Since the FDA approved the first whole-slide image system for medical diagnosis in 2017, whole-slide images have provided enriched critical information to advance the field of automated histopathology. Here, we exemplify the benefits of a novel deep learning strategy that utilizes high-resolution whole-slide microscopic images. We quantitatively assess and visually highlight the inconsistencies within the whole-slide image dataset employed in this study. Accordingly, we introduce a deep learning-based preprocessing algorithm designed to normalize unknown samples to the training set distribution, effectively mitigating the overfitting issue. Consequently, our approach significantly increases the amount of automatic region-of-interest ground truth labeling on high-resolution whole-slide images using active deep learning. We accept 92% of the automatic labels generated for our unlabeled data cohort, expanding the labeled dataset by 845%. Additionally, we demonstrate expert time savings of 96% relative to manual expert ground-truth labeling.

Tomato pose estimation using the association of tomato body and sepal

In facility horticulture smart farms, harvesting robotic systems to automate harvesting tasks are challenging due to the complex environment, irregular growth and fruits pose. To harvest a fruits, the vision information required by a harvesting robot is accurate position and pose. Especially, fruit pose information is essential for planning paths to avoid damaging stems, leaves, branches, and obstacles, preventing damage to the fruit and harvesting robots, and planning efficient harvest sequences. This paper presents a method that uses information from tomato sepals to estimate the orientation of tomatoes, which are a commonly grown crop in horticulture. First, we train a YOLOv8 model to detect and segment bodies and sepals of tomatoes. For robust training of the model, the training data is constructed using effective data augmentation, which synthesizes segmented foreground objects and inserting them into the background. Then, for finding association between the body and sepal of a tomato, we apply IoU score based matching and the Hungarian algorithm. Consequently, we obtain point clouds for both parts using RGB-D data. Finally, we compute the center point of each object by using spherical fitting and the statistics of the point cloud, respectively. Then, we estimate the pose of the tomato as a vector between two center points of the body and sepal. To accurately and practically evaluate the proposed method, we generated ground truth data using calibration patterns in tomato greenhouse. As the experimental results show, The segmentation results show that AP50,sepal is 94.7, AP50,tomato is 96.3, and mAP is 61.5. The result of the pose estimation for the pose validation dataset is a mean error angle of 6.79 ± 3.18 and angle errors of less than 10 degrees account for 87.2%. The total algorithm proposed in this paper requires about 0.038s for each tomato greenhouse image. As a result, our proposed pose estimation can be practically utilized in robot systems for tomato harvesting.

Development of a Statistical Model for Automated Ground Truth Generation in Low-Resource Languages

Development of a Statistical Model for Automated Ground Truth Generation in Low-Resource Languages

DeepX-Ray: A Comparative Study of Deep Learning-Based Classification and Segmentation Techniques for Automated Detection and Diagnosis of COVID-19 from Chest X-ray Images

The relentless spread of the SARS-CoV-2 virus, causing COVID-19, has underscored the urgent need for efficient early detection and diagnosis methods to mitigate its impact. While traditional techniques like RT-PCR are valuable, they often suffer from time-consuming processes. In this study, DeepX-Ray is presented, a comprehensive investigation into deep learning-based classification and segmentation approaches for the automated detection of COVID-19 from chest X-ray images. Specifically, the authors focus on constructing a custom convolutional neural network (CNN) model to distinguish between COVID-19 and standard X-ray images and benchmark its performance against established models. For segmentation tasks, the effectiveness of various backbone architectures, including ResNet34, ResNet101, DenseNet201, and ResNet50 within the UNet model framework, is explored, with ResNet50 exhibiting superior performance. Furthermore, a novel dataset comprising 7657 images sourced from three publicly available authentic datasets is introduced, and the labelMe tool is employed by the authors to generate ground truth mask datasets for 4137 images to facilitate segmentation of infected lung areas. The authors' custom CNN model achieves an outstanding classification accuracy of 100%, while the segmentation approach attains a mean Intersection over Union (IoU) score of 96.19%. These results underscore the efficacy of the proposed model in enabling early automatic detection and diagnosis of COVID-19, particularly in resource-constrained and remote settings where establishing traditional laboratories may be impractical. This research significantly advances medical imaging techniques for combating the COVID-19 pandemic.

Hue Channel Mesoscale Convective System Identification Techniques using Kalpana Geostationary Satellite Observations

Climate change is leading to sudden extreme weather events like floods and heavy rain to occur. One of the most significant rain-bearing systems, Mesoscale Convective Systems (MCSs), is in charge of catastrophic rainfall and flood events that may cause loss of life and property. MCSs are one of the vital components of the climate system on Earth. Many studies contributed to two significant steps, MCS identification and tracking. There are a variety of algorithms focused on tracking whereas for MCS identification only limited methods are present. They contribute to global as well as regional climate patterns by transporting heat and moisture. It is necessary to identify MCS properly to track MCS occurrences over time and comprehend their typical lifecycle to get early warnings of their existence. Three MCS identification techniques based on the Hue channel of the Hue Saturation Value (HSV) color model are implemented in this research, and their effectiveness is assessed. These techniques execute segmentation based on Hue channel Thresholding (HT), K means clustering combined with Hue channel Thresholding (KMCHT) and the modified Source Apportionment Technique combined with Hue channel Thresholding (SATHT). Image pixel values are used to depict infrared brightness temperature data obtained from the Indian geostationary satellite Kalpana-1. The generated ground truth images and performance measurements are used to assess the effectiveness of the methods. The proposed SATHT method for multiple cloud segmentation results in superior performance metrics than the HT and KMCHT approaches.

A Semi-supervised Pipeline for Accurate Neuron Segmentation with Fewer Ground Truth Labels.

Recent advancements in two-photon calcium imaging have enabled scientists to record the activity of thousands of neurons with cellular resolution. This scope of data collection is crucial to understanding the next generation of neuroscience questions, but analyzing these large recordings requires automated methods for neuron segmentation. Supervised methods for neuron segmentation achieve state of-the-art accuracy and speed but currently require large amounts of manually generated ground truth training labels. We reduced the required number of training labels by designing a semi-supervised pipeline. Our pipeline used neural network ensembling to generate pseudolabels to train a single shallow U-Net. We tested our method on three publicly available datasets and compared our performance to three widely used segmentation methods. Our method outperformed other methods when trained on a small number of ground truth labels and could achieve state-of-the-art accuracy after training on approximately a quarter of the number of ground truth labels as supervised methods. When trained on many ground truth labels, our pipeline attained higher accuracy than that of state-of-the-art methods. Overall, our work will help researchers accurately process large neural recordings while minimizing the time and effort needed to generate manual labels.