High-quality Manual Annotation Research Articles

HAIC-NET: Semi-supervised OCTA vessel segmentation with self-supervised pretext task and dual consistency training

Optical Coherence Tomography Angiography(OCTA) vessel segmentation is a challenging task. On the one hand, the complex structure of the capillary networks presents significant obstacles to achieving accurate vessel segmentation. On the other hand, current research on OCTA vessel segmentation heavily relies on high-quality manual annotations, especially in fully-supervised approaches. In contrast, pixel-level annotation of OCTA vessels is time-consuming and labor-intensive. To address these issues, we propose a semi-supervised method called HAIC-Net, which integrates self-supervised learning with a homologous augmented image classification pretext task and dual consistency training with data perturbation consistency and topological connectivity consistency. Firstly, we design a self-supervised homologous augmented image classification pretext task that directs the model’s attention to similar vascular features in homologous augmented images, thereby extracting rich vessel information from unlabeled images and reducing the dependence on manual annotations. Secondly, we introduce a dual-consistency structure with topological connectivity consistency to provide constraint from a topological perspective, which is consistent with the topological characteristics of the vascular network, to enhance the segmentation network’s sensitivity to vessel connectivity and decrease the topological errors in segmentation results. We conduct experiments on two publicly available datasets and one private dataset and validate the state-of-the-art performance of the proposed method. On the ROSE-1 dataset, our method achieves 0.9143 accuracy and 0.7658 dice coefficient, surpassing other current semi-supervised methods and approaching the performance of state-of-the-art fully supervised methods. The same result can also be observed on OCTA500 and our private dataset, demonstrating the effectiveness and superiority of our approach.

A weakly-supervised deep learning model for end-to-end detection of airfield pavement distress

Accurate and timely surveying of airfield pavement distress is crucial for cost-effective airport maintenance management. Deep learning (DL) approaches, leveraging advancements in computer science and image acquisition techniques, have become the mainstream for automated airfield pavement distress detection. However, fully-supervised DL methods require a large number of manually annotated ground truth labels to achieve high accuracy. To address the challenge of limited high-quality manual annotations, we propose a novel end-to-end distress detection model called WSDD-CAM (Class Activation Map informed Weakly-Supervised Distress Detection). Based on YOLOv5, WSDD-CAM consists of an efficient backbone, a classification branch, and a localization network. By utilizing CAM (Class Activation Map) information, our model significantly reduces the need for manual annotations, automatically generating pseudo bounding boxes with a 71% overlap with the ground truth. To evaluate WSDD-CAM, we tested it on a self-made dataset and compared it with other weakly-supervised and fully-supervised models. The results show that our model achieves 49.2% mean average precision (mAP), outperforming other weakly-supervised methods and even approaching state-of-the-art fully-supervised methods. Additionally, ablation experiments confirm the effectiveness of our architecture design. In conclusion, our WSDD-CAM model offers a promising solution for airfield pavement distress detection, reducing manual annotation time while maintaining high accuracy. This efficient and effective approach can significantly contribute to cost-effective airport maintenance management.

Learning to Answer Visual Questions from Web Videos.

Recent methods for visual question answering rely on large-scale annotated datasets. Manual annotation of questions and answers for videos, however, is tedious, expensive and prevents scalability. In this work, we propose to avoid manual annotation and generate a large-scale training dataset for video question answering making use of automatic cross-modal supervision. We leverage a question generation transformer trained on text data and use it to generate question-answer pairs from transcribed video narrations. Given narrated videos, we then automatically generate the HowToVQA69M dataset with 69M video-question-answer triplets. To handle the open vocabulary of diverse answers in this dataset, we propose a training procedure based on a contrastive loss between a video-question multi-modal transformer and an answer transformer. We introduce the zero-shot VideoQA task and the VideoQA feature probe evaluation setting and show excellent results. Furthermore, our method achieves competitive results on MSRVTT-QA, ActivityNet-QA, MSVD-QA and How2QA datasets. We also show that our approach generalizes to another source of web video and text data. We generate the WebVidVQA3M dataset from videos with alt-text annotations, and show its benefits for training VideoQA models. Finally, for a detailed evaluation we introduce iVQA, a new VideoQA dataset with reduced language bias and high-quality manual annotations.

A distributable German clinical corpus containing cardiovascular clinical routine doctor’s letters

We present CARDIO:DE, the first freely available and distributable large German clinical corpus from the cardiovascular domain. CARDIO:DE encompasses 500 clinical routine German doctor’s letters from Heidelberg University Hospital, which were manually annotated. Our prospective study design complies well with current data protection regulations and allows us to keep the original structure of clinical documents consistent. In order to ease access to our corpus, we manually de-identified all letters. To enable various information extraction tasks the temporal information in the documents was preserved. We added two high-quality manual annotation layers to CARDIO:DE, (1) medication information and (2) CDA-compliant section classes. To the best of our knowledge, CARDIO:DE is the first freely available and distributable German clinical corpus in the cardiovascular domain. In summary, our corpus offers unique opportunities for collaborative and reproducible research on natural language processing models for German clinical texts.

Self-Supervised Graphs for Audio Representation Learning With Limited Labeled Data

Large scale databases with high-quality manual annotations are scarce in audio domain. We thus explore a self-supervised graph approach to learning audio representations from highly limited labelled data. Considering each audio sample as a graph node, we propose a subgraph-based framework with novel self-supervision tasks that can learn effective audio representations. During training, subgraphs are constructed by sampling the entire pool of available training data to exploit the relationship between the labelled and unlabeled audio samples. During inference, we use random edges to alleviate the overhead of graph construction. We evaluate our model on three benchmark audio databases, and two tasks: acoustic event detection and speech emotion recognition. Our semi-supervised model performs better or on par with fully supervised models and outperforms several competitive existing models. Our model is compact (240k parameters), and can produce generalized audio representations that are robust to different types of signal noise.

Distantly Supervised Named Entity Recognition with Self-Adaptive Label Correction

Named entity recognition has achieved remarkable success on benchmarks with high-quality manual annotations. Such annotations are labor-intensive and time-consuming, thus unavailable in real-world scenarios. An emerging interest is to generate low-cost but noisy labels via distant supervision, hence noisy label learning algorithms are in demand. In this paper, a unified self-adaptive learning framework termed Self-Adaptive Label cOrrection (SALO) is proposed. SALO adaptively performs a label correction process, both in an implicit and an explicit manners, turning noisy labels into correct ones, thus benefiting model training. The experimental results on four benchmark datasets demonstrated the superiority of SALO over the state-of-the-art distantly supervised methods. Moreover, a better version of noisy labels by ensembling several semantic matching methods was built. Experiments were carried out and consistent improvements were observed, validating the generalization of the proposed SALO.

Annotation-efficient deep learning for automatic medical image segmentation

Automatic medical image segmentation plays a critical role in scientific research and medical care. Existing high-performance deep learning methods typically rely on large training datasets with high-quality manual annotations, which are difficult to obtain in many clinical applications. Here, we introduce Annotation-effIcient Deep lEarning (AIDE), an open-source framework to handle imperfect training datasets. Methodological analyses and empirical evaluations are conducted, and we demonstrate that AIDE surpasses conventional fully-supervised models by presenting better performance on open datasets possessing scarce or noisy annotations. We further test AIDE in a real-life case study for breast tumor segmentation. Three datasets containing 11,852 breast images from three medical centers are employed, and AIDE, utilizing 10% training annotations, consistently produces segmentation maps comparable to those generated by fully-supervised counterparts or provided by independent radiologists. The 10-fold enhanced efficiency in utilizing expert labels has the potential to promote a wide range of biomedical applications.

Linked annotations: a middle ground for manual curation of biomedical databases and text corpora

Summary Annotators of text corpora and biomedical databases carry out the same labor-intensive task to manually extract structured data from unstructured text. Tasks are needlessly repeated because text corpora are widely scattered. We envision that a linked annotation resource unifying many corpora could be a game changer. Such an open forum will help focus on novel annotations and on optimally benefiting from the energy of many experts. As proof-of-concept, we annotated protein subcellular localization in 100 abstracts cited by UniProtKB. The detailed comparison between our new corpus and the original UniProtKB annotations revealed sustained novel annotations for 42% of the entries (proteins). In a unified linked annotation resource these could immediately extend the utility of text corpora beyond the textmining community. Our example motivates the central idea that linked annotations from text corpora can complement database annotations. Background The natural language processing (NLP) and biomedical research communities have in common that they invest great effort into making high-quality manual annotation of biomedical literature. The focus and the annotation strategies of the two communities have, however, differed so much that collaborations remained stunningly limited. Most text corpora contain detailed markup of only a few types of entities and relationships

From manual curation to visualization of gene families and networks across Solanaceae plant species

High-quality manual annotation methods and practices need to be scaled to the increased rate of genomic data production. Curation based on gene families and gene networks is one approach that can significantly increase both curation efficiency and quality. The Sol Genomics Network (SGN; http://solgenomics.net) is a comparative genomics platform, with genetic, genomic and phenotypic information of the Solanaceae family and its closely related species that incorporates a community-based gene and phenotype curation system. In this article, we describe a manual curation system for gene families aimed at facilitating curation, querying and visualization of gene interaction patterns underlying complex biological processes, including an interface for efficiently capturing information from experiments with large data sets reported in the literature. Well-annotated multigene families are useful for further exploration of genome organization and gene evolution across species. As an example, we illustrate the system with the multigene transcription factor families, WRKY and Small Auxin Up-regulated RNA (SAUR), which both play important roles in responding to abiotic stresses in plants.Database URL: http://solgenomics.net/

GoGene: gene annotation in the fast lane

High-throughput screens such as microarrays and RNAi screens produce huge amounts of data. They typically result in hundreds of genes, which are often further explored and clustered via enriched GeneOntology terms. The strength of such analyses is that they build on high-quality manual annotations provided with the GeneOntology. However, the weakness is that annotations are restricted to process, function and location and that they do not cover all known genes in model organisms. GoGene addresses this weakness by complementing high-quality manual annotation with high-throughput text mining extracting co-occurrences of genes and ontology terms from literature. GoGene contains over 4 000 000 associations between genes and gene-related terms for 10 model organisms extracted from more than 18 000 000 PubMed entries. It does not cover only process, function and location of genes, but also biomedical categories such as diseases, compounds, techniques and mutations. By bringing it all together, GoGene provides the most recent and most complete facts about genes and can rank them according to novelty and importance. GoGene accepts keywords, gene lists, gene sequences and protein sequences as input and supports search for genes in PubMed, EntrezGene and via BLAST. Since all associations of genes to terms are supported by evidence in the literature, the results are transparent and can be verified by the user. GoGene is available at http://gopubmed.org/gogene.