Ground Truth Annotations Research Articles

Event Classification on Subsea Pipeline Inspection Data Using an Ensemble of Deep Learning Classifiers

Subsea pipelines are the backbone of the modern oil and gas industry, transporting a total of 28% of global oil production. Due to several factors, such as corrosion or deformations, the pipelines might degrade over time, which might lead to serious economic and environmental damages if not addressed promptly. Therefore, it is crucial to detect any serious damage to subsea pipelines before they cause dangerous catastrophes. Inspections of subsea pipelines are usually made using a Remote Operating Vehicle and the inspection data is usually processed manually, which is subject to human errors, and requires experienced Remote Operating Vehicle operators. It is thus necessary to automate the inspection process to enable more efficiency as well as reduce costs. Besides, it is recognised that specific challenges of noisy and low-quality inspection data arising from the underwater environment prevent the industry from taking full advantage of the recent development in the Artificial Intelligence field to the problem of subsea pipeline inspection. In this paper, we developed an ensemble of deep learning classifiers to further improve the performance of single deep learning models in classifying anomalous events on the subsea pipeline inspection data. The output of the proposed ensemble was combined based on a weighted combining method. The weights of base classifiers were found by minimising the difference between the weighted combining result and the given associated ground truth annotation information. Three inspection datasets, gathered from different oil and gas companies in the United Kingdom, were analysed. These datasets were recorded under varying conditions and include a range of anomalies. The results showed that the proposed ensemble achieves around 78% accuracy on two datasets and more than 99% accuracy on one dataset, which is better compared to base classifiers and two popular ensembles.

Cross-Domain HAR: Few Shot Transfer Learning for Human Activity Recognition

The ubiquitous availability of smartphones and smartwatches with integrated inertial measurement units (IMUs) enables straightforward capturing of human activities through collecting movement data. For specific applications of sensor based human activity recognition (HAR), however, logistical challenges and burgeoning costs render especially the ground truth annotation of such data a difficult endeavor, resulting in limited scale and diversity of datasets available for deriving effective HAR systems and less than ideal recognition capabilities. Transfer learning, i.e., leveraging publicly available labeled datasets to first learn useful representations that can then be fine-tuned using limited amounts of labeled data from a target domain, can alleviate some of the performance issues of contemporary HAR systems. Yet they can fail when the differences between source and target conditions are too large and / or only few samples from a target application domain are available – each of which are typical challenges in real-world human activity recognition scenarios. In this paper, we present an approach for economic use of publicly available labeled HAR datasets for effective transfer learning. We introduce a novel transfer learning framework–Cross-Domain HAR–which follows the teacher-student self-training paradigm to more effectively recognize activities with very limited label information. It bridges conceptual gaps between source and target domains, including sensor locations and type of activities. Cross-Domain HAR enables substantial performance improvements over the state-of-the-art in sensor-based HAR scenarios. Through our extensive experimental evaluation on a range of benchmark datasets we specifically demonstrate the effectiveness of our approach for practically relevant few shot activity recognition scenarios. We also present a detailed analysis into how the individual components of our framework affect downstream performance and provide practical suggestions for using the framework in real-world applications.

Sparse Annotation is Sufficient for Bootstrapping Dense Neuron Segmentation.

Producing dense 3D reconstructions from biological imaging data is a challenging instance segmentation task that requires significant ground-truth training data for effective and accurate deep learning-based models. Generating training data requires intense human effort to annotate each instance of an object across serial section images. Our focus is on the especially complicated brain neuropil, comprising an extensive interdigitation of dendritic, axonal, and glial processes visualized through serial section electron microscopy. We developed a novel deep learning-based method to generate dense 3D segmentations rapidly from sparse 2D annotations of a few objects on single sections. Models trained on the rapidly generated segmentations achieved similar accuracy as those trained on expert dense ground-truth annotations. Human time to generate annotations was reduced by three orders of magnitude and could be produced by non-expert annotators. This capability will democratize generation of training data for large image volumes needed to achieve brain circuits and measures of circuit strengths.

Using a Device-Free Wi-Fi Sensing System to Assess Daily Activities and Mobility in Low-Income Older Adults: Protocol for a Feasibility Study.

Older adults belonging to racial or ethnic minorities with low socioeconomic status are at an elevated risk of developing dementia, but resources for assessing functional decline and detecting cognitive impairment are limited. Cognitive impairment affects the ability to perform daily activities and mobility behaviors. Traditional assessment methods have drawbacks, so smart home technologies (SmHT) have emerged to offer objective, high-frequency, and remote monitoring. However, these technologies usually rely on motion sensors that cannot identify specific activity types. This group often lacks access to these technologies due to limited resources and technology experience. There is a need to develop new sensing technology that is discreet, affordable, and requires minimal user engagement to characterize and quantify various in-home activities. Furthermore, it is essential to explore the feasibility of developing machine learning (ML) algorithms for SmHT through collaborations between clinical researchers and engineers and involving minority, low-income older adults for novel sensor development. This study aims to examine the feasibility of developing a novel channel state information-based device-free, low-cost Wi-Fi sensing system, and associated ML algorithms for localizing and recognizing different patterns of in-home activities and mobility in residents of low-income senior housing with and without mild cognitive impairment. This feasibility study was conducted in collaboration with a wellness care group, which serves the healthy aging needs of low-income housing residents. Prior to this feasibility study, we conducted a pilot study to collect channel state information data from several activity scenarios (eg, sitting, walking, and preparing meals) using the proposed Wi-Fi sensing system continuously over a week in apartments of low-income housing residents. These activities were videotaped to generate ground truth annotations to test the accuracy of the ML algorithms derived from the proposed system. Using qualitative individual interviews, we explored the acceptability of the Wi-Fi sensing system and implementation barriers in the low-income housing setting. We use the same study protocol for the proposed feasibility study. The Wi-Fi sensing system deployment began in November 2022, with participant recruitment starting in July 2023. Preliminary results will be available in the summer of 2025. Preliminary results are focused on the feasibility of developing ML models for Wi-Fi sensing-based activity and mobility assessment, community-based recruitment and data collection, ground truth, and older adults' Wi-Fi sensing technology acceptance. This feasibility study can make a contribution to SmHT science and ML capabilities for early detection of cognitive decline among socially vulnerable older adults. Currently, sensing devices are not readily available to this population due to cost and information barriers. Our sensing device has the potential to identify individuals at risk for cognitive decline by assessing their level of physical function by tracking their in-home activities and mobility behaviors, at a low cost. DERR1-10.2196/53447.

Automated Medical Records Review for Mild Cognitive Impairment and Dementia.

Objectives : Unstructured and structured data in electronic health records (EHR) are a rich source of information for research and quality improvement studies. However, extracting accurate information from EHR is labor-intensive. Here we introduce an automated EHR phenotyping model to identify patients with Alzheimer's Disease, related dementias (ADRD), or mild cognitive impairment (MCI). Methods : We assembled medical notes and associated International Classification of Diseases (ICD) codes and medication prescriptions from 3,626 outpatient adults from two hospitals seen between February 2015 and June 2022. Ground truth annotations regarding the presence vs. absence of a diagnosis of MCI or ADRD were determined through manual chart review. Indicators extracted from notes included the presence of keywords and phrases in unstructured clinical notes, prescriptions of medications associated with MCI/ADRD, and ICD codes associated with MCI/ADRD. We trained a regularized logistic regression model to predict the ground truth annotations. Model performance was evaluated using area under the receiver operating curve (AUROC), area under the precision-recall curve (AUPRC), accuracy, specificity, precision/positive predictive value, recall/sensitivity, and F1 score (harmonic mean of precision and recall). Results : Thirty percent of patients in the cohort carried diagnoses of MCI/ADRD based on manual review. When evaluated on a held-out test set, the best model using clinical notes, ICDs, and medications, achieved an AUROC of 0.98, an AUPRC of 0.98, an accuracy of 0.93, a sensitivity (recall) of 0.91, a specificity of 0.96, a precision of 0.96, and an F1 score of 0.93 The estimated overall accuracy for patients randomly selected from EHRs was 99.88%. Conclusion : Automated EHR phenotyping accurately identifies patients with MCI/ADRD based on clinical notes, ICD codes, and medication records. This approach holds potential for large-scale MCI/ADRD research utilizing EHR databases.

AL-SAR: Active Learning for Skeleton-Based Action Recognition.

Action recognition from temporal multivariate sequences of features, such as identifying human actions, is typically approached by supervised training as it requires many ground truth annotations to reach high recognition accuracy. Unsupervised methods for the organization of sequences into clusters have been introduced, however, such methods continue to require annotations to associate clusters with actions. The challenges in annotation necessitate an effective classification methodology that minimizes the required number of labels. Active learning (AL) approaches have been proposed to address these challenges and were able to establish robust results on image classification. Such approaches are not directly applicable to sequences, since for sequences, the variations are in both spatial and temporal domains. In this brief, we introduce a novel method for AL for sequences, called "AL-SAR," which combines unsupervised training with sparsely supervised annotation. In particular, AL-SAR employs a multi-head mechanism for robust uncertainty evaluation of the latent space learned by an encoder-decoder framework. It aims to iteratively select a sparse set of samples, which annotation contributes the most to the disentanglement of the latent space. We evaluate our system on common benchmark datasets with multiple sequences and actions, such as NW-UCLA, NTU RGB+D 60, and UWA3D. Our results indicate that AL-SAR coupled with encoder-decoder network outperforms other AL methods coupled with the same network structure.

Ki-67 evaluation using deep-learning model-assisted digital image analysis in breast cancer.

To test the efficacy of artificial intelligence (AI)-assisted Ki-67 digital image analysis in invasive breast carcinoma (IBC) with quantitative assessment of AI model performance. This study used 494 cases of Ki-67 slide images of IBC core needle biopsies. The methods were divided into two steps: (i) construction of a deep-learning model (DL); and (ii) DL implementation for Ki-67 analysis. First, a DL tissue classifier model (DL-TC) and a DL nuclear detection model (DL-ND) were constructed using HALO AI DenseNet V2 algorithm with 31,924 annotations in 300 Ki-67 digital slide images. Whether the class predicted by DL-TC in the test set was correct compared with the annotation of ground truth at the pixel level was evaluated. Second, DL-TC- and DL-ND-assisted digital image analysis (DL-DIA) was performed in the other 194 luminal-type cases and correlations with manual counting and clinical outcome were investigated to confirm the accuracy and prognostic potential of DL-DIA. The performance of DL-TC was excellent and invasive carcinoma nests were well segmented from other elements (average precision: 0.851; recall: 0.878; F1-score: 0.858). Ki-67 index data and the number of nuclei from DL-DIA were positively correlated with data from manual counting (ρ = 0.961, and 0.928, respectively). High Ki-67 index (cutoff 20%) cases showed significantly worse recurrence-free survival and breast cancer-specific survival (P = 0.024, and 0.032, respectively). The performances of DL-TC and DL-ND were excellent. DL-DIA demonstrated a high degree of concordance with manual counting of Ki-67 and the results of this approach have prognostic potential.

Sparse Annotation is Sufficient for Bootstrapping Dense Segmentation.

Producing dense 3D reconstructions from biological imaging data is a challenging instance segmentation task that requires significant ground-truth training data for effective and accurate deep learning-based models. Generating training data requires intense human effort to annotate each instance of an object across serial section images. Our focus is on the especially complicated brain neuropil, comprising an extensive interdigitation of dendritic, axonal, and glial processes visualized through serial section electron microscopy. We developed a novel deep learning-based method to generate dense 3D segmentations rapidly from sparse 2D annotations of a few objects on single sections. Models trained on the rapidly generated segmentations achieved similar accuracy as those trained on expert dense ground-truth annotations. Human time to generate annotations was reduced by three orders of magnitude and could be produced by non-expert annotators. This capability will democratize generation of training data for large image volumes needed to achieve brain circuits and measures of circuit strengths.

Semi-Supervised Online Continual Learning for 3D Object Detection in Mobile Robotics

Continual learning addresses the challenge of acquiring and retaining knowledge over time across multiple tasks and environments. Previous research primarily focuses on offline settings where models learn through increasing tasks from samples paired with ground truth annotations. In this work, we focus on an unsolved, challenging, yet practical scenario, specifically, the semi-supervised online continual learning in autonomous driving and mobile robotics. In our settings, models are tasked with learning new distributions from streaming unlabeled samples and performing 3D object detection as soon as the LiDAR point cloud arrives. Additionally, we conducted experiments on both the KITTI dataset, our newly built IUSL dataset and Canadian Adverse Driving Conditions (CADC) Dataset. The results indicate that our method achieves a balance between rapid adaptation and knowledge retention, showcasing its effectiveness in the dynamic and complex environment of autonomous driving and mobile robotics. The developed ROS packages and IUSL dataset will be publicly available at: https://github.com/npu-ius-lab/OCL3D.

Improved ground truth annotation by multimodal image registration from 3D ultrasound to histopathology for resected tongue carcinoma.

This study's objectives are (1) to investigate the registration accuracy from intraoperative ultrasound (US) to histopathological images, (2) to assess the agreement and correlation between measurements in registered 3D US and histopathology, and (3) to train a nnUNet model for automatic segmentation of 3D US volumes of resected tongue specimens. Ten 3D US volumes were acquired, including the corresponding digitalized histopathological images (n = 29). Based on corresponding landmarks, the registrations between 3D US and histopathology images were calculated and evaluated using the target registration error (TRE). Tumor thickness and resection margins were measured based on three annotations: (1) manual histopathological tumor annotation (HTA), manual 3D US tumor annotation, and (2) the HTA registered in the 3D US. The agreement and correlation were computed between the measurements based on the HTA and those based on the manual US and registered HTA in US. A deep-learning model with nnUNet was trained on 151 3D US volumes. Segmentation metrics quantified the model's performance. The median TRE was 0.42mm. The smallest mean difference was between registered HTA in US and histopathology with 2.16mm (95% CI - 1.31; 5.63) and a correlation of 0.924 (p < 0.001). The nnUNet predicted the tumor with a Dice similarity coefficient of 0.621, an average surface distance of 1.15mm, and a Hausdorff distance of 3.70mm. Multimodal image registration enabled the HTA's registration in the US images and improved the agreement and correlation between the modalities. In the future, this could be used to annotate ground truth labels accurately.

Weakly‐supervised structural component segmentation via scribble annotations

AbstractSegmentation of structural components in infrastructure inspection images is crucial for automated and accurate condition assessment. While deep neural networks hold great potential for this task, existing methods typically require fully annotated ground truth masks, which are time‐consuming and labor‐intensive to create. This paper introduces Scribble‐supervised Structural Component Segmentation Network (ScribCompNet), the first weakly‐supervised method requiring only scribble annotations for multiclass structural component segmentation. ScribCompNet features a dual‐branch architecture with higher‐resolution refinement to enhance fine detail detection. It extends supervision from labeled to unlabeled pixels through a combined objective function, incorporating scribble annotation, dynamic pseudo label, semantic context enhancement, and scale‐adaptive harmony losses. Experimental results show that ScribCompNet outperforms other scribble‐supervised methods and most fully‐supervised counterparts, achieving 90.19% mean intersection over union (mIoU) with an 80% reduction in labeling time. Further evaluations confirm the effectiveness of the novel designs and robust performance, even with lower‐quality scribble annotations.

ESCAPE: Energy-based Selective Adaptive Correction for Out-of-distribution 3D Human Pose Estimation

Despite recent advances in human pose estimation (HPE), poor generalization to out-of-distribution (OOD) data remains a difficult problem. While previous works have proposed Test-Time Adaptation (TTA) to bridge the train-test domain gap by refining network parameters at inference, the absence of ground-truth annotations makes it highly challenging and existing methods typically increase inference times by one or more orders of magnitude. We observe that (1) not every test time sample is OOD, and (2) HPE errors are significantly larger on distal keypoints (wrist, ankle). To this end, we propose ESCAPE: a lightweight correction and selective adaptation framework which applies a fast, forward-pass correction on most data while reserving costly TTA for OOD data. The free energy function is introduced to separate OOD samples from incoming data and a correction network is trained to estimate the errors of pretrained backbone HPE predictions on the distal keypoints. For OOD samples, we propose a novel self-consistency adaptation loss to update the correction network by leveraging the constraining relationship between distal keypoints and proximal keypoints (shoulders, hips), via a second “reverse” network. ESCAPE improves the distal MPJPE of five popular HPE models by up to 7% on unseen data, achieves state-of-the-art results on two popular HPE benchmarks, and is significantly faster than existing adaptation methods.

Vocal Call Locator Benchmark (VCL) for localizing rodent vocalizations from multi-channel audio.

Understanding the behavioral and neural dynamics of social interactions is a goal of contemporary neuroscience. Many machine learning methods have emerged in recent years to make sense of complex video and neurophysiological data that result from these experiments. Less focus has been placed on understanding how animals process acoustic information, including social vocalizations. A critical step to bridge this gap is determining the senders and receivers of acoustic information in social interactions. While sound source localization (SSL) is a classic problem in signal processing, existing approaches are limited in their ability to localize animal-generated sounds in standard laboratory environments. Advances in deep learning methods for SSL are likely to help address these limitations, however there are currently no publicly available models, datasets, or benchmarks to systematically evaluate SSL algorithms in the domain of bioacoustics. Here, we present the VCL Benchmark: the first large-scale dataset for benchmarking SSL algorithms in rodents. We acquired synchronized video and multi-channel audio recordings of 767,295 sounds with annotated ground truth sources across 9 conditions. The dataset provides benchmarks which evaluate SSL performance on real data, simulated acoustic data, and a mixture of real and simulated data. We intend for this benchmark to facilitate knowledge transfer between the neuroscience and acoustic machine learning communities, which have had limited overlap.

Automated Deep Learning-Based Finger Joint Segmentation in 3-D Ultrasound Images With Limited Dataset.

Ultrasound imaging has shown promise in assessing synovium inflammation associated early stages of rheumatoid arthritis (RA). The precise identification of the synovium and the quantification of inflammation-specific imaging biomarkers is a crucial aspect of accurately quantifying and grading RA. In this study, a deep learning-based approach is presented that automates the segmentation of the synovium in ultrasound images of finger joints affected by RA. Two convolutional neural network architectures for image segmentation were trained and validated in a limited number of 2-D images, extracted from N = 18 3-D ultrasound volumes acquired from N = 9 RA patients, with sparse ground truth annotations of the synovium. Various augmentation strategies were employed to enhance the diversity and size of the training dataset. The utilization of geometric and noise augmentation transforms resulted in the highest dice score (0.768 0.040, N = 6), as determined via six-fold cross-validation. In addition, the segmentation model is used to generate dense 3-D segmentation maps in the ultrasound volumes, based on the available sparse annotations. The developed technique shows promise in facilitating more efficient and standardized workflow for RA screening using ultrasound imaging.

Open-world object detection: A solution based on reselection mechanism and feature disentanglement

Traditional object detection algorithms operate within a closed set, where the training data may not cover all real-world objects. Therefore, the issue of open-world object detection has attracted significant attention. Open-world object detection faces two major challenges: “neglecting unknown objects” and “misclassifying unknown objects as known ones.” In our study, we address these challenges by utilizing the Region Proposal Network (RPN) outputs to identify potential unknown objects with high object scores that do not overlap with ground truth annotations. We introduce the reselection mechanism, which separates unknown objects from the background. Subsequently, we employ the simulated annealing algorithm to disentangle features of unknown and known classes, guiding the detector’s learning process. Our method has improved on multiple evaluation metrics such as U-mAP, U-recall, and UDP, greatly alleviating the challenges faced by open world object detection.

A robust image segmentation and synthesis pipeline for histopathology

Significant diagnostic variability between and within observers persists in pathology, despite the fact that digital slide images provide the ability to measure and quantify features much more precisely compared to conventional methods. Automated and accurate segmentation of cancerous cell and tissue regions can streamline the diagnostic process, providing insights into the cancer progression, and helping experts decide on the most effective treatment. Here, we evaluate the performance of the proposed PathoSeg model, with an architecture comprising of a modified HRNet encoder and a UNet++ decoder integrated with a CBAM block to utilize attention mechanism for an improved segmentation capability. We demonstrate that PathoSeg outperforms the current state-of-the-art (SOTA) networks in both quantitative and qualitative assessment of instance and semantic segmentation. Notably, we leverage the use of synthetic data generated by PathopixGAN, which effectively addresses the data imbalance problem commonly encountered in histopathology datasets, further improving the performance of PathoSeg. It utilizes spatially adaptive normalization within a generative and discriminative mechanism to synthesize diverse histopathological environments dictated through semantic information passed through pixel-level annotated Ground Truth semantic masks.Besides, we contribute to the research community by providing an in-house dataset that includes semantically segmented masks for breast carcinoma tubules (BCT), micro/macrovesicular steatosis of the liver (MSL), and prostate carcinoma glands (PCG). In the first part of the dataset, we have a total of 14 whole slide images from 13 patients’ liver, with fat cell segmented masks, totaling 951 masks of size 512 × 512 pixels. In the second part, it includes 17 whole slide images from 13 patients with prostate carcinoma gland segmentation masks, amounting to 30,000 masks of size 512 × 512 pixels. In the third part, the dataset contains 51 whole slides from 36 patients, with breast carcinoma tubule masks totaling 30,000 masks of size 512 × 512 pixels. To ensure transparency and encourage further research, we will make this dataset publicly available for non-commercial and academic purposes. To facilitate reproducibility and encourage further research, we will also make our code and pre-trained models publicly available at https://github.com/DeepMIALab/PathoSeg.

Improving deep learning-based automatic cranial defect reconstruction by heavy data augmentation: From image registration to latent diffusion models

Modeling and manufacturing of personalized cranial implants are important research areas that may decrease the waiting time for patients suffering from cranial damage. The modeling of personalized implants may be partially automated by the use of deep learning-based methods. However, this task suffers from difficulties with generalizability into data from previously unseen distributions that make it difficult to use the research outcomes in real clinical settings. Due to difficulties with acquiring ground-truth annotations, different techniques to improve the heterogeneity of datasets used for training the deep networks have to be considered and introduced. In this work, we present a large-scale study of several augmentation techniques, varying from classical geometric transformations, image registration, variational autoencoders, and generative adversarial networks, to the most recent advances in latent diffusion models. We show that the use of heavy data augmentation significantly increases both the quantitative and qualitative outcomes, resulting in an average Dice Score above 0.94 for the SkullBreak and above 0.96 for the SkullFix datasets. The results show that latent diffusion models combined with vector quantized variational autoencoder outperform other generative augmentation strategies. Moreover, we show that the synthetically augmented network successfully reconstructs real clinical defects, without the need to acquire costly and time-consuming annotations. The findings of the work will lead to easier, faster, and less expensive modeling of personalized cranial implants. This is beneficial to numerous people suffering from cranial injuries. The work constitutes a considerable contribution to the field of artificial intelligence in the automatic modeling of personalized cranial implants.

Jazz Trio Database: Automated Annotation of Jazz Piano Trio Recordings Processed Using Audio Source Separation

Recent advances in automatic music transcription have facilitated the creation of large databases of symbolic transcriptions of improvised music forms including jazz, where traditional notated scores are not normally available. In conjunction with music source separation models that enable audio to be “demixed” into separate signals for multiple instrument classes, these algorithms can also be applied to generate annotations for every musician in a performance. This can enable the analysis of interesting performer-level and ensemble-level features that have often been difficult to explore. To this end, we introduce Jazz Trio Database (JTD), a dataset of 44.5 h of jazz piano solos accompanied by bass and drums, with automatically generated annotations for each performer. These annotations consist of onset, beat, and downbeat timestamps, alongside MIDI for the piano soloist. Suitable recordings, broadly representative of the “straight-ahead” jazz style, were identified by scraping user-based listening and discographic data; source separation models were applied to isolate audio for each performer in the trio; annotations were generated by applying appropriate algorithms to both the separated and the mixed audio sources. Onset annotations generated by the pipeline achieved a mean F-measure of 0.94 when compared with ground truth annotations. We conduct several analyses of JTD, including with relation to swing and inter-performer synchronization. We anticipate that JTD will be useful in a variety of music information–retrieval tasks, including artist identification and expressive performance modeling. We have made JTD, including the annotations and associated source code, available at https://github.com/HuwCheston/Jazz-Trio-Database

Extent and Topography of Subretinal Drusenoid Deposits Associate With Rod-Mediated Vision in Aging and AMD: ALSTAR2 Baseline.

In AMD, rod-mediated dark adaptation (RMDA) at 5° eccentricity is slower in eyes with subretinal drusenoid deposits (SDDs) than in eyes without. Here we quantified SDD burden using supervised deep learning for comparison to vision and photoreceptor topography. In persons ≥60 years from the Alabama Study on Early Age-Related Macular Degeneration 2, normal, early AMD, and intermediate AMD eyes were classified by the AREDS nine-step system. A convolutional neural network was trained on 55°-wide near-infrared reflectance images for SDD segmentation. Trained graders annotated ground truth (SDD yes/no). Predicted and true datasets agreed (Dice coefficient, 0.92). Inference was manually proofread using optical coherence tomography. The mean SDD area (mm2) was compared among diagnostic groups (linear regression) and to vision (age-adjusted Spearman correlations). Fundus autofluorescence images were used to mask large vessels in SDD maps. In 428 eyes of 428 persons (normal, 218; early AMD, 120; intermediate AMD, 90), the mean SDD area differed by AMD severity (P < 0.0001): 0.16 ± 0.87 (normal), 2.48 ± 11.23 (early AMD), 11.97 ± 13.33 (intermediate AMD). Greater SDD area was associated with worse RMDA (r = 0.27; P < 0.0001), mesopic (r = -0.13; P = 0.02) and scotopic sensitivity (r = -0.17; P < 0.001). SDD topography peaked at 5° superior, extended beyond the Early Treatment of Diabetic Retinopathy Study grid and optic nerve, then decreased. SDD area is associated with degraded rod-mediated vision. RMDA 5° (superior retina) probes where SDD is maximal, closer to the foveal center than the rod peak at 3 to 6 mm (10.4°-20.8°) superior and the further eccentric peak of rod:cone ratio. Topographic data imply that factors in addition to rod density influence SDD formation.

Human–object interaction detection via recycling of ground-truth annotations

Human–object interaction (HOI) detection is important to understand human-centric scenes, the hard core of which lies in learning the structure information from various types of relations. Existing works tackle it by introducing spatial context, extra knowledge or graph-based propagation based on the original hard labels. However, they still face challenges in dealing with action co-occurrence and complex HOIs. In this paper, we creatively propose to recycle the ground-truth annotations to get implied information for structure representations in HOIs. The action-aware closeness labeling (ACL) task is designed to capture the contextual information based on the statistic of action co-occurrence from the data source. Furthermore, we present a H-O relation graph supervision (RGS) to get more reliable relations in complicated scenes by constraining the attention weights of the H-O relation graph. Such a direct supervision on mutual relations is ignored in existing works. Experiments for HOI detection on the V-COCO and HICO-DET datasets indicate the superiority of the proposed method even without any extra knowledge.