Annotation Process Research Articles

Mangrove species detection using YOLOv5 with RGB imagery from consumer unmanned aerial vehicles (UAVs)

Despite comprising only one per cent of global forests, mangroves provide vital ecological and economic benefits to their ecosystems. Due to its decreasing extent over the past decade, there is a rise in research innovations supporting mangrove conservation. Specifically, consumer-grade Unmanned Aerial Vehicles (UAV) were proven effective as potential remote sensing alternatives to support mangrove research and monitoring in recent studies. As most studies use custom UAV-mounted sensors for mangrove species classification, similar studies using a UAV’s default red–green–blue (RGB) cameras were scarce. This study explores the potential of high-resolution RGB aerial images through state-of-the-art object detection algorithm, YOLOv5 to detect the dominant Rhizophora mangroves in Sarawak, Malaysia. A total of 400 RGB images were equally selected from two study areas and allocated into three datasets, two corresponding to each study area and one combining all images. The annotation process was performed using a previously proposed novel method, assisted by YOLOv5 for a semi-automated annotation process with expert verification. Systematic training experiments were conducted to select an optimal epoch size across models trained with each dataset. The final models produced an average true positive rate of 73.8% and 71.7% for each study site, while the combined dataset model produced an average true positive rate of 73.7%. Overall, this study demonstrated the potential of UAV-based RGB images and deep learning object detection architectures to identify specific mangrove objects, while also highlighting key considerations for similar future research.

Masked autoencoder: influence of self-supervised pretraining on object segmentation in industrial images

The amount of labelled data in industrial use cases is limited because the annotation process is time-consuming and costly. As in research, self-supervised pretraining such as MAE resulted in training segmentation models with fewer labels, this is also an interesting direction for industry. The reduction of required labels is achieved with large amounts of unlabelled images for the pretraining that aims to learn image features. This paper analyses the influence of MAE pretraining on the efficiency of label usage for semantic segmentation with UNETR. This is investigated for the use case of log-yard cranes. Additionally, two transfer learning cases with respect to crane type and perspective are considered in the context of label-efficiency. The results show that MAE is successfully applicable to the use case. With respect to the segmentation, an IoU improvement of 3.26% is reached while using 2000 labels. The strongest positive influence is found for all experiments in the lower label amounts. The highest effect is achieved with transfer learning regarding cranes, where IoU and Recall increase about 4.31% and 8.58%, respectively. Further analyses show that improvements result from a better distinction between the background and the segmented crane objects.

Accelerating Clinical Text Annotation in Underrepresented Languages: A Case Study on Text De-Identification.

Clinical notes contain valuable information for research and monitoring quality of care. Named Entity Recognition (NER) is the process for identifying relevant pieces of information such as diagnoses, treatments, side effects, etc., and bring them to a more structured form. Although recent advancements in deep learning have facilitated automated recognition, particularly in English, NER can still be challenging due to limited specialized training data. This exacerbated in hospital settings where annotations are costly to obtain without appropriate incentives and often dependent on local specificities. In this work, we study whether this annotation process can be effectively accelerated by combining two practical strategies. First, we convert usually passive annotation tasks into a proactive contest to motivate human annotators in performing a task often considered tedious and time-consuming. Second, we provide pre-annotations for the participants to evaluate how recall and precision of the pre-annotations can boost or deteriorate annotation performance. We applied both strategies to a text de-identification task on French clinical notes and discharge summaries at a large Swiss university hospital. Our results show that proactive contest and average quality pre-annotations can significantly speed up annotation time and increase annotation quality, enabling us to develop a text de-identification model for French clinical notes with high performance (F1 score 0.94).

Towards Reducing Continuous Emotion Annotation Effort During Video Consumption: A Physiological Response Profiling Approach

Emotion-aware video applications (e.g., gaming, online meetings, online tutoring) strive to moderate the content presentations for a more engaging and improved user experience. These services typically deploy a machine-learning model that continuously infers the user's emotion (based on different physiological signals, facial expressions, etc.) to adapt the content delivery. Therefore, to train such models, the emotion ground truth labels also need to be collected continuously. Typically, those are collated as emotion self-reports from users in a continuous manner (using an auxiliary device such as a joystick) when they watch some videos. This process of continuous emotion annotation not only increases the cognitive load and survey fatigue but also significantly deteriorates the viewing experience. To address this problem, we propose a framework, PResUP that probes a user for emotion self-reports opportunistically based on the physiological response variations of the user. Specifically, the framework implements a sequence of phases - (a) user profile construction based on physiological responses, (b) similar user clustering based on the user profile, and (c) training a parameterized activation-guided LSTM (Long short-term memory) model by sharing data among similar users to detect the opportune self-report collection moments. All these steps together help to reduce the continuous emotion annotation overhead by probing at the opportune moments without compromising the annotation quality. We evaluated PResUP by conducting a user study (N=36) during which the participants watched eight videos, and their physiological responses and continuous emotion self-reports were recorded. The key results from this evaluation reveal that PResUP reduces the annotation overhead by reducing the probing rate by an average of 34.80% and detects the opportune probing moments with an average TPR of 80.07% without compromising the annotation quality. Motivated by these findings, we deployed PResUP by performing a follow-up user study (N=18). In this deployment scenario, we also obtained similar performance in terms of the probing rate reduction (average reduction of 38.05%), and opportune moment detection performance (average TPR of 82.26%). These findings underscore the utility of PResUP in reducing the continuous emotion annotation effort during video consumption.

Large language models: a new approach for privacy policy analysis at scale

The number and dynamic nature of web sites and mobile applications present regulators and app store operators with significant challenges when it comes to enforcing compliance with applicable privacy and data protection laws. Over the past several years, people have turned to Natural Language Processing (NLP) techniques to automate privacy compliance analysis (e.g., comparing statements in privacy policies with analysis of the code and behavior of mobile apps) and to answer people’s privacy questions. Traditionally, these NLP techniques have relied on labor-intensive and potentially error-prone manual annotation processes to build the corpora necessary to train them. This article explores and evaluates the use of Large Language Models (LLMs) as an alternative for effectively and efficiently identifying and categorizing a variety of data practice disclosures found in the text of privacy policies. Specifically, we report on the performance of ChatGPT and Llama 2, two particularly popular LLM-based tools. This includes engineering prompts and evaluating different configurations of these LLM techniques. Evaluation of the resulting techniques on well-known corpora of privacy policy annotations yields an F1 score exceeding 93%. This score is higher than scores reported earlier in the literature on these benchmarks. This performance is obtained at minimal marginal cost (excluding the cost required to train the foundational models themselves). These results, which are consistent with those reported in other domains, suggest that LLMs offer a particularly promising approach to automated privacy policy analysis at scale.

The biomedical relationship corpus of the BioRED track at the BioCreative VIII challenge and workshop.

The automatic recognition of biomedical relationships is an important step in the semantic understanding of the information contained in the unstructured text of the published literature. The BioRED track at BioCreative VIII aimed to foster the development of such methods by providing the participants the BioRED-BC8 corpus, a collection of 1000 PubMed documents manually curated for diseases, gene/proteins, chemicals, cell lines, gene variants, and species, as well as pairwise relationships between them which are disease-gene, chemical-gene, disease-variant, gene-gene, chemical-disease, chemical-chemical, chemical-variant, and variant-variant. Furthermore, relationships are categorized into the following semantic categories: positive correlation, negative correlation, binding, conversion, drug interaction, comparison, cotreatment, and association. Unlike most of the previous publicly available corpora, all relationships are expressed at the document level as opposed to the sentence level, and as such, the entities are normalized to the corresponding concept identifiers of the standardized vocabularies, namely, diseases and chemicals are normalized to MeSH, genes (and proteins) to National Center for Biotechnology Information (NCBI) Gene, species to NCBI Taxonomy, cell lines to Cellosaurus, and gene/protein variants to Single Nucleotide Polymorphism Database. Finally, each annotated relationship is categorized as 'novel' depending on whether it is a novel finding or experimental verification in the publication it is expressed in. This distinction helps differentiate novel findings from other relationships in the same text that provides known facts and/or background knowledge. The BioRED-BC8 corpus uses the previous BioRED corpus of 600 PubMed articles as the training dataset and includes a set of newly published 400 articles to serve as the test data for the challenge. All test articles were manually annotated for the BioCreative VIII challenge by expert biocurators at the National Library of Medicine, using the original annotation guidelines, where each article is doubly annotated in a three-round annotation process until full agreement is reached between all curators. This manuscript details the characteristics of the BioRED-BC8 corpus as a critical resource for biomedical named entity recognition and relation extraction. Using this new resource, we have demonstrated advancements in biomedical text-mining algorithm development. Database URL: https://codalab.lisn.upsaclay.fr/competitions/16381.

Robust ROI Detection in Whole Slide Images Guided by Pathologists' Viewing Patterns.

Deep learning techniques offer improvements in computer-aided diagnosis systems. However, acquiring image domain annotations is challenging due to the knowledge and commitment required of expert pathologists. Pathologists often identify regions in whole slide images with diagnostic relevance rather than examining the entire slide, with a positive correlation between the time spent on these critical image regions and diagnostic accuracy. In this paper, a heatmap is generated to represent pathologists' viewing patterns during diagnosis and used to guide a deep learning architecture during training. The proposed system outperforms traditional approaches based on color and texture image characteristics, integrating pathologists' domain expertise to enhance region of interest detection without needing individual case annotations. Evaluating our best model, a U-Net model with a pre-trained ResNet-18 encoder, on a skin biopsy whole slide image dataset for melanoma diagnosis, shows its potential in detecting regions of interest, surpassing conventional methods with an increase of 20%, 11%, 22%, and 12% in precision, recall, F1-score, and Intersection over Union, respectively. In a clinical evaluation, three dermatopathologists agreed on the model's effectiveness in replicating pathologists' diagnostic viewing behavior and accurately identifying critical regions. Finally, our study demonstrates that incorporating heatmaps as supplementary signals can enhance the performance of computer-aided diagnosis systems. Without the availability of eye tracking data, identifying precise focus areas is challenging, but our approach shows promise in assisting pathologists in improving diagnostic accuracy and efficiency, streamlining annotation processes, and aiding the training of new pathologists.

Aproximación estadística al reparto de <b> y <v> en manuscritos latinos de autores hispanos

Abstract This paper studies the graphemes <b> and <u, v> in a corpus of medieval Latin manuscripts from the 8th to the 12th centuries, containing texts by three Hispanic writers: Isidore of Seville († 636), Braulio of Zaragoza († 651), and Ildefonso of Toledo († 667). The manuscripts written in Hispania are compared with those produced outside the Iberian Peninsula to extract data about the graphic distribution of <b> and <u/v> in the textual transmission. After data annotation and statistical processing, the results are analyzed with two complementary objectives. Firstly, the mentioned graphemes are studied as evidence of the geographical distribution of the geographical distribution of the velar semiconsonant and the sonorous bilabial stop, and their Romance outcomes. Secondly, the results contribute to the phonetic and graphic characterization of Hispanic medieval Latin in the context of the emergence of Romance languages.

Weakly Supervised Lymph Nodes Segmentation Based on Partial Instance Annotations with Pre-trained Dual-branch Network and Pseudo Label Learning

Assessing the presence of potentially malignant lymph nodes aids in estimating cancer progression, and identifying surrounding benign lymph nodes can assist in determining potential metastatic pathways for cancer. For quantitative analysis, automatic segmentation of lymph nodes is crucial. However, due to the labor-intensive and time-consuming manual annotation process required for a large number of lymph nodes, it is more practical to annotate only a subset of the lymph node instances to reduce annotation costs. In this study, we propose a pre-trained Dual-Branch network with Dynamically Mixed Pseudo label (DBDMP) to learn from partial instance annotations for lymph nodes segmentation. To obtain reliable pseudo labels for lymph nodes that are not annotated, we employ a dual-decoder network to generate different outputs that are then dynamically mixed. We integrate the original weak partial annotations with the mixed pseudo labels to supervise the network. To further leverage the extensive amount of unannotated voxels, we apply a self-supervised pre-training strategy to enhance the model’s feature extraction capability. Experiments on the mediastinal Lymph Node Quantification (LNQ) dataset demonstrate that our method, compared to directly learning from partial instance annotations, significantly improves the Dice Similarity Coefficient (DSC) from 11.04% to 54.10% and reduces the Average Symmetric Surface Distance (ASSD) from 20.83 mm to 8.72 mm. The code is available at <a href='https://github.com/WltyBY/LNQ2023_training_code'>https://github.com/WltyBY/LNQ2023_training_code</a>

The development of a research intelligence tool for rare disease research in the Netherlands

Abstract This article describes the process of the development of a research intelligence tool to analyze rare disease research in the Netherlands. To the best of our knowledge, this is the first tool that can surface and organize scientific output on rare diseases using established annotation and natural language processing mechanisms. We focus on the track leading up to the development, including strategic motivation and user needs, of a proof-of-concept tool, upscaling the idea to a national collaboration project, the development of the final tool and a usability evaluation and subsequent fine-tuning. The tool is a unique visualization that allows users to benefit with a few clicks from getting the information they require for their needs, and offers novel scientific indicators to characterize (relatively) rare disease research activity. We discuss the applications of insights derived from this tool for science policy and to support decision-making, and to identify opportunities and potential collaborations and make recommendations for future developments, including a broadening of the scope, and discuss potential novel applications.

CORAL: Collaborative Automatic Labeling System Based on Large Language Models

In the era of big data, data annotation is integral to numerous applications. However, it is widely acknowledged as a laborious and time-consuming process, significantly impeding the scalability and efficiency of data-driven applications. To reduce the human cost, we demonstrate CORAL, a collaborative automatic labeling system driven by large language models (LLMs), which achieves high-quality annotation with the least human effort. Firstly, CORAL employs LLM to automatically annotate vast datasets, generating coarse-grained labels. Subsequently, a weakly-supervised learning module trains small language models (SLMs) using noisy label learning techniques to distill accurate labels from LLM's annotations. It also allows statistical analysis of model outcomes to identify potentially erroneous labels, reducing the human cost of error detection. Furthermore, CORAL supports iterative refinement by LLMs and SLMs using manually corrected labels, thereby ensuring continual enhancement in annotation quality and model performance. A visual interface enables annotation process monitoring and result analysis.

Towards a cyberbullying detection approach: fine-tuned contrastive self-supervised learning for data augmentation

AbstractCyberbullying on social media platforms is pervasive and challenging to detect due to linguistic subtleties and the need for extensive data annotation. We introduce a Deep Contrastive Self-Supervised Learning (DCSSL) model that integrates a Natural Language Inference (NLI) dataset, a fine-tuned sentence encoder, and data augmentation to enhance the understanding of cyberbullying's nuanced semantics and offensiveness. The DCSSL model effectively captures contextual dependencies and the varied semantic implications inherent in cyberbullying instances, addressing the limitations of manual data annotation processes when compared against established models such as BERT and Bi-LSTM. Our proposed model registers a significant improvement, achieving a macro average F1 score of 0.9231 on cyberbullying datasets, highlighting its applicability in environments where manual annotation is impractical or unavailable.

Building RadiologyNET: an unsupervised approach to annotating a large-scale multimodal medical database

BackgroundThe use of machine learning in medical diagnosis and treatment has grown significantly in recent years with the development of computer-aided diagnosis systems, often based on annotated medical radiology images. However, the lack of large annotated image datasets remains a major obstacle, as the annotation process is time-consuming and costly. This study aims to overcome this challenge by proposing an automated method for annotating a large database of medical radiology images based on their semantic similarity.ResultsAn automated, unsupervised approach is used to create a large annotated dataset of medical radiology images originating from the Clinical Hospital Centre Rijeka, Croatia. The pipeline is built by data-mining three different types of medical data: images, DICOM metadata and narrative diagnoses. The optimal feature extractors are then integrated into a multimodal representation, which is then clustered to create an automated pipeline for labelling a precursor dataset of 1,337,926 medical images into 50 clusters of visually similar images. The quality of the clusters is assessed by examining their homogeneity and mutual information, taking into account the anatomical region and modality representation.ConclusionsThe results indicate that fusing the embeddings of all three data sources together provides the best results for the task of unsupervised clustering of large-scale medical data and leads to the most concise clusters. Hence, this work marks the initial step towards building a much larger and more fine-grained annotated dataset of medical radiology images.

Efficient musculoskeletal annotation using free-form deformation

Traditionally, constructing training datasets for automatic muscle segmentation from medical images involved skilled operators, leading to high labor costs and limited scalability. To address this issue, we developed a tool that enables efficient annotation by non-experts and assessed its effectiveness for training an automatic segmentation network. Our system allows users to deform a template three-dimensional (3D) anatomical model to fit a target magnetic-resonance image using free-form deformation with independent control points for axial, sagittal, and coronal directions. This method simplifies the annotation process by allowing non-experts to intuitively adjust the model, enabling simultaneous annotation of all muscles in the template. We evaluated the quality of the tool-assisted segmentation performed by non-experts, which achieved a Dice coefficient greater than 0.75 compared to expert segmentation, without significant errors such as mislabeling adjacent muscles or omitting musculature. An automatic segmentation network trained with datasets created using this tool demonstrated performance comparable to or superior to that of networks trained with expert-generated datasets. This innovative tool significantly reduces the time and labor costs associated with dataset creation for automatic muscle segmentation, potentially revolutionizing medical image annotation and accelerating the development of deep learning-based segmentation networks in various clinical applications.

Leveraging transfer learning and active learning for data annotation in passive acoustic monitoring of wildlife

Passive Acoustic Monitoring (PAM) has emerged as a pivotal technology for wildlife monitoring, generating vast amounts of acoustic data. However, the successful application of machine learning methods for sound event detection in PAM datasets heavily relies on the availability of annotated data, which can be laborious to acquire. In this study, we investigate the effectiveness of transfer learning and active learning techniques to address the data annotation challenge in PAM. Transfer learning allows us to use pre-trained models from related tasks or datasets to bootstrap the learning process for sound event detection. Furthermore, active learning promises strategic selection of the most informative samples for annotation, effectively reducing the annotation cost and improving model performance. We evaluate an approach that combines transfer learning and active learning to efficiently exploit existing annotated data and optimize the annotation process for PAM datasets. Our transfer learning observations show that embeddings produced by BirdNet, a model trained on high signal-to-noise recordings of bird vocalisations, can be effectively used for predicting anurans in PAM data: a linear classifier constructed using these embeddings outperforms the benchmark by 21.7%. Our results indicate that active learning is superior to random sampling, although no clear winner emerges among the strategies employed. The proposed method holds promise for facilitating broader adoption of machine learning techniques in PAM and advancing our understanding of biodiversity dynamics through acoustic data analysis.

SPA: Annotating Small Object with a Single Point in Remote Sensing Images

Detecting oriented small objects is a critical task in remote sensing, but the development of high-performance deep learning-based detectors is hindered by the need for large-scale and well-annotated datasets. The high cost of creating these datasets, due to the dense and numerous distribution of small objects, significantly limits the application and development of such detectors. To address this problem, we propose a single-point-based annotation approach (SPA) based on the graph cut method. In this framework, user annotations act as the origin of positive sample points, and a similarity matrix, computed from feature maps extracted by deep learning networks, facilitates an intuitive and efficient annotation process for building graph elements. Utilizing the Maximum Flow algorithm, SPA derives positive sample regions from these points and generates oriented bounding boxes (OBBOXs). Experimental results demonstrate the effectiveness of SPA, with at least a 50% improvement in annotation efficiency. Furthermore, the intersection-over-union (IoU) metric of our OBBOX is 3.6% higher than existing methods such as the “Segment Anything Model”. When applied in training, the model annotated with SPA shows a 4.7% higher mean average precision (mAP) compared to models using traditional annotation methods. These results confirm the technical advantages and practical impact of SPA in advancing small object detection in remote sensing.

A 2.5D Self-Training Strategy for Carotid Artery Segmentation in T1-Weighted Brain Magnetic Resonance Images.

Precise annotations for large medical image datasets can be time-consuming. Additionally, when dealing with volumetric regions of interest, it is typical to apply segmentation techniques on 2D slices, compromising important information for accurately segmenting 3D structures. This study presents a deep learning pipeline that simultaneously tackles both challenges. Firstly, to streamline the annotation process, we employ a semi-automatic segmentation approach using bounding boxes as masks, which is less time-consuming than pixel-level delineation. Subsequently, recursive self-training is utilized to enhance annotation quality. Finally, a 2.5D segmentation technique is adopted, wherein a slice of a volumetric image is segmented using a pseudo-RGB image. The pipeline was applied to segment the carotid artery tree in T1-weighted brain magnetic resonance images. Utilizing 42 volumetric non-contrast T1-weighted brain scans from four datasets, we delineated bounding boxes around the carotid arteries in the axial slices. Pseudo-RGB images were generated from these slices, and recursive segmentation was conducted using a Res-Unet-based neural network architecture. The model's performance was tested on a separate dataset, with ground truth annotations provided by a radiologist. After recursive training, we achieved an Intersection over Union (IoU) score of (0.68 ± 0.08) on the unseen dataset, demonstrating commendable qualitative results.

Is Boundary Annotation Necessary? Evaluating Boundary-Free Approaches to Improve Clinical Named Entity Annotation Efficiency: Case Study.

Named entity recognition (NER) is a fundamental task in natural language processing. However, it is typically preceded by named entity annotation, which poses several challenges, especially in the clinical domain. For instance, determining entity boundaries is one of the most common sources of disagreements between annotators due to questions such as whether modifiers or peripheral words should be annotated. If unresolved, these can induce inconsistency in the produced corpora, yet, on the other hand, strict guidelines or adjudication sessions can further prolong an already slow and convoluted process. The aim of this study is to address these challenges by evaluating 2 novel annotation methodologies, lenient span and point annotation, aiming to mitigate the difficulty of precisely determining entity boundaries. We evaluate their effects through an annotation case study on a Japanese medical case report data set. We compare annotation time, annotator agreement, and the quality of the produced labeling and assess the impact on the performance of an NER system trained on the annotated corpus. We saw significant improvements in the labeling process efficiency, with up to a 25% reduction in overall annotation time and even a 10% improvement in annotator agreement compared to the traditional boundary-strict approach. However, even the best-achieved NER model presented some drop in performance compared to the traditional annotation methodology. Our findings demonstrate a balance between annotation speed and model performance. Although disregarding boundary information affects model performance to some extent, this is counterbalanced by significant reductions in the annotator's workload and notable improvements in the speed of the annotation process. These benefits may prove valuable in various applications, offering an attractive compromise for developers and researchers.

The complete plastid genome of Abrus pulchellus subsp. mollis (Leguminosae): a medicinal plant in Southern China

The subspecies Abrus pulchellus subsp. mollis exhibits pharmacological properties akin to the traditional Chinese medicinal plant Abri Herba (A. pulchellus subsp. cantoniensis (Hance) Verdc.). In this report, we unveil the plastid genome of A. pulchellus subsp. mollis. The genome spans 156,322 base pairs (bp), comprising a large single-copy (LSC) region of 86,633 bp, a small single-copy (SSC) region of 18,219 bp, and two distinct inverted repeat regions (IRs) of 25,735 bp each. Annotation process cataloged a total of 111 genes within this genome, including 77 protein-coding genes, 30 transfer RNA (tRNA) genes, and four ribosomal RNA (rRNA) genes. The overall guanine-cytosine (GC) content of the plastome is 35.5%. Phylogenetic analysis utilizing maximum-likelihood (ML) based on 16 complete plastid genomes reveals a close clustering of three Abrus taxa, namely A. pulchellus subsp. mollis, A. pulchellus subsp. cantoniensis, and A. precatorius. Notably, A. pulchellus subsp. cantoniensis clusters with A. precatorius as a sister group, distinct from A. pulchellus subsp. mollis. These findings highlight significant differences between the plastid genomes of the two subspecies, laying the foundation for future research on the identification of medicinal herbs and germplasm resources related to these subspecies.

FedIOD: Federated Multi-Organ Segmentation From Partial Labels by Exploring Inter-Organ Dependency.

Multi-organ segmentation is a fundamental task and existing approaches usually rely on large-scale fully-labeled images for training. However, data privacy and incomplete/partial labels make those approaches struggle in practice. Federated learning is an emerging tool to address data privacy but federated learning with partial labels is under-explored. In this work, we explore generating full supervision by building and aggregating inter-organ dependency based on partial labels and propose a single-encoder-multi-decoder framework named FedIOD. To simulate the annotation process where each organ is labeled by referring to other closely-related organs, a transformer module is introduced and the learned self-attention matrices modeling pairwise inter-organ dependency are used to build pseudo full labels. By using those pseudo-full labels for regularization in each client, the shared encoder is trained to extract rich and complete organ-related features rather than being biased toward certain organs. Then, each decoder in FedIOD projects the shared organ-related features into a specific space trained by the corresponding partial labels. Experimental results based on five widely-used datasets, including LiTS, KiTS, MSD, BCTV, and ACDC, demonstrate the effectiveness of FedIOD, outperforming the state-of-the-art approaches under in-federation evaluation and achieving the second-best performance under out-of-federation evaluation for multi-organ segmentation from partial labels.