Minimal Annotation Research Articles

Consensus-based iterative meta-pseudo-labeling for deep semi-supervised learning

A known issue that hinders the development of deep learning models is the need for accurate annotation of a large quantity of samples – a time-consuming, labor-intensive, and error-prone task. This limitation is particularly critical in areas where data annotation requires expert knowledge. Semi-supervised learning methods, such as pseudo-labeling, can alleviate the problem by capitalizing on both limited labeled and plentiful unlabeled data; nonetheless, state-of-the-art methods often require pre-trained encoders and validation sets to deliver effective solutions. Herein, we introduce a teacher-student-based iterative meta-pseudo-labeling approach, named consensus Deep Feature Annotation (cons-DeepFA), that enables the training of custom Convolutional Neural Networks (CNNs) from small quantities of labeled samples without reliance on pre-trained encoders and validation sets. cons-DeepFA explores Feature Learning from Image Markers (FLIM) to initialize the filters of a target CNN (student) from minimal data annotation – i.e., user-drawn markers on discriminative regions of a few selected images per class. During each of a few iterations, the latent space of the student's last dense layer is non-linearly projected onto a two-dimensional space for downstream label propagation via an optimum-connectivity-based approach (teacher); afterward, the student is re-trained using pseudo-labeled samples selected by the proposed consensus mechanism, which jointly improves the latent space, its projection, and the student's generalization ability as iterations progress. This strategy was recently introduced with pre-trained encoders by selecting the most confident pseudo-labeled samples to re-train the student. While building on previous methods, cons-DeepFA presents two key contributions. It (i) incorporates FLIM to enable training a custom CNN from scratch with faster convergence, improving its generalization ability, and (ii) introduces a consensus-based procedure over multiple iterations that selects more accurately pseudo-labeled samples for re-training the CNN. Lastly, cons-DeepFA is evaluated on five challenging biological image datasets, demonstrating its effectiveness and competitiveness when compared to seven state-of-the-art methods from four semi-supervised learning paradigms.

VGTS: Visually Guided Text Spotting for novel categories in historical manuscripts

In the field of historical manuscript research, scholars frequently encounter novel symbols in ancient texts, investing considerable effort in their identification and documentation. Although existing object detection methods achieve impressive performance on known categories, they struggle to recognize novel symbols without retraining. To address this limitation, we propose a Visually Guided Text Spotting (VGTS) approach that accurately spots novel characters using just one annotated support sample. The core of VGTS is a spatial alignment module consisting of a Dual Spatial Attention (DSA) block and a Geometric Matching (GM) block. The DSA block aims to identify, focus on, and learn discriminative spatial regions in the support and query images, mimicking the human visual spotting process. It first refines the support image by analyzing inter-channel relationships to identify critical areas, and then refines the query image by focusing on informative key points. The GM block, on the other hand, establishes the spatial correspondence between the two images, enabling accurate localization of the target character in the query image. To tackle the example imbalance problem in low-resource spotting tasks, we develop a novel torus loss function that enhances the discriminative power of the embedding space for distance metric learning. To further validate our approach, we introduce a new dataset featuring ancient Dongba hieroglyphics (DBH) associated with the Naxi minority of China. Extensive experiments on the DBH dataset and other public datasets, including Egyptian Hieroglyph (EGY), Historical Arabic Documents (HAD), Tripitaka Koreana in Han (TKH), and Notary Charters (NC), show that VGTS consistently surpasses state-of-the-art methods. The proposed framework exhibits great potential for application in historical manuscript text spotting, enabling scholars to efficiently identify and document novel symbols with minimal annotation effort.

Enhancing MAUDE Database Utility by GPT-4 and Cause-Effect Visualization.

The MAUDE database is a valuable public resource for understanding malfunctions and adverse events related to medical devices and health IT. However, its extensive data and complex structure pose challenges. To overcome this, we have developed an automated analytical pipeline using GPT-4, a cutting-edge large language model. This pipeline is intended to efficiently extract, categorize, and visualize safety events with minimal human annotation. In our analysis of 4,459 colonoscopy reports from MAUDE (2011-2021), the events were categorized into operational, human factor, and device-related. Ishikawa diagrams visualized a subset stored in a vector database for easy retrieval and comparison through a similarity search. This innovative approach streamlines access to vital safety insights, reducing the workload on human annotators, and holds promise to enhance the utility of the MAUDE database.

Low-resource entity resolution with domain generalization and active learning

Entity Resolution (ER), a fundamental task in data cleaning and integration, is critical in various fields such as healthcare, e-commerce, and social networks. Traditional ER methods are constrained by the need for substantial labeled samples and the challenge of generalization to unseen domains. To address the low-resource challenge in ER, a novel two-phase framework is proposed. Initially, we introduce the Domain Generalization Entity Resolution (DGER) framework, combining domain adversarial learning and simulated target learning to improve the generalization performance on unseen target domains. Subsequently, to further adapt to the target dataset, we present a novel active learning approach called Domain-Aware Uncertainty Active Learning (DUAL), for fine-tuning the DGER model with minimal annotation cost. DUAL manually annotates target domain samples that are highly uncertain and exhibit high divergence from the source, while assigning pseudo-labels to high-confidence samples. Experimental results on multiple real-world datasets demonstrate that our framework outperforms traditional ER methods in generalizing to unseen domains. Specifically, our DGER method outperforms the best-performing ER baseline in each task, achieving an average F1 score improvement of 9.02% across eight different test tasks. Moreover, within a limited annotation budget during the active learning phase, our DUAL fine-tuning strategy for the ER model outperforms uncertainty-based active learning techniques.

Efficient Wheat Head Segmentation with Minimal Annotation: A Generative Approach.

Deep learning models have been used for a variety of image processing tasks. However, most of these models are developed through supervised learning approaches, which rely heavily on the availability of large-scale annotated datasets. Developing such datasets is tedious and expensive. In the absence of an annotated dataset, synthetic data can be used for model development; however, due to the substantial differences between simulated and real data, a phenomenon referred to as domain gap, the resulting models often underperform when applied to real data. In this research, we aim to address this challenge by first computationally simulating a large-scale annotated dataset and then using a generative adversarial network (GAN) to fill the gap between simulated and real images. This approach results in a synthetic dataset that can be effectively utilized to train a deep-learning model. Using this approach, we developed a realistic annotated synthetic dataset for wheat head segmentation. This dataset was then used to develop a deep-learning model for semantic segmentation. The resulting model achieved a Dice score of 83.4% on an internal dataset and Dice scores of 79.6% and 83.6% on two external datasets from the Global Wheat Head Detection datasets. While we proposed this approach in the context of wheat head segmentation, it can be generalized to other crop types or, more broadly, to images with dense, repeated patterns such as those found in cellular imagery.

Semi-Self-Supervised Domain Adaptation: Developing Deep Learning Models with Limited Annotated Data for Wheat Head Segmentation

Precision agriculture involves the application of advanced technologies to improve agricultural productivity, efficiency, and profitability while minimizing waste and environmental impacts. Deep learning approaches enable automated decision-making for many visual tasks. However, in the agricultural domain, variability in growth stages and environmental conditions, such as weather and lighting, presents significant challenges to developing deep-learning-based techniques that generalize across different conditions. The resource-intensive nature of creating extensive annotated datasets that capture these variabilities further hinders the widespread adoption of these approaches. To tackle these issues, we introduce a semi-self-supervised domain adaptation technique based on deep convolutional neural networks with a probabilistic diffusion process, requiring minimal manual data annotation. Using only three manually annotated images and a selection of video clips from wheat fields, we generated a large-scale computationally annotated dataset of image–mask pairs and a large dataset of unannotated images extracted from video frames. We developed a two-branch convolutional encoder–decoder model architecture that uses both synthesized image–mask pairs and unannotated images, enabling effective adaptation to real images. The proposed model achieved a Dice score of 80.7% on an internal test dataset and a Dice score of 64.8% on an external test set composed of images from five countries and spanning 18 domains, indicating its potential to develop generalizable solutions that could encourage the wider adoption of advanced technologies in agriculture.

Active Dynamic Weighting for multi-domain adaptation

Multi-source unsupervised domain adaptation aims to transfer knowledge from multiple labeled source domains to an unlabeled target domain. Existing methods either seek a mixture of distributions across various domains or combine multiple single-source models for weighted fusion in the decision process, with little insight into the distributional discrepancy between different source domains and the target domain. Considering the discrepancies in global and local feature distributions between different domains and the complexity of obtaining category boundaries across domains, this paper proposes a novel Active Dynamic Weighting (ADW) for multi-source domain adaptation. Specifically, to effectively utilize the locally advantageous features in the source domains, ADW designs a multi-source dynamic adjustment mechanism during the training process to dynamically control the degree of feature alignment between each source and target domain in the training batch. In addition, to ensure the cross-domain categories can be distinguished, ADW devises a dynamic boundary loss to guide the model to focus on the hard samples near the decision boundary, which enhances the clarity of the decision boundary and improves the model’s classification ability. Meanwhile, ADW applies active learning to multi-source unsupervised domain adaptation for the first time, guided by dynamic boundary loss, proposes an efficient importance sampling strategy to select target domain hard samples to annotate at a minimal annotation budget, integrates it into the training process, and further refines the domain alignment at the category level. Experiments on various benchmark datasets consistently demonstrate the superiority of our method.

GPT for medical entity recognition in Spanish

AbstractIn recent years, there has been a remarkable surge in the development of Natural Language Processing (NLP) models, particularly in the realm of Named Entity Recognition (NER). Models such as BERT have demonstrated exceptional performance, leveraging annotated corpora for accurate entity identification. However, the question arises: Can newer Large Language Models (LLMs) like GPT be utilized without the need for extensive annotation, thereby enabling direct entity extraction? In this study, we explore this issue, comparing the efficacy of fine-tuning techniques with prompting methods to elucidate the potential of GPT in the identification of medical entities within Spanish electronic health records (EHR). This study utilized a dataset of Spanish EHRs related to breast cancer and implemented both a traditional NER method using BERT, and a contemporary approach that combines few shot learning and integration of external knowledge, driven by LLMs using GPT, to structure the data. The analysis involved a comprehensive pipeline that included these methods. Key performance metrics, such as precision, recall, and F-score, were used to evaluate the effectiveness of each method. This comparative approach aimed to highlight the strengths and limitations of each method in the context of structuring Spanish EHRs efficiently and accurately.The comparative analysis undertaken in this article demonstrates that both the traditional BERT-based NER method and the few-shot LLM-driven approach, augmented with external knowledge, provide comparable levels of precision in metrics such as precision, recall, and F score when applied to Spanish EHR. Contrary to expectations, the LLM-driven approach, which necessitates minimal data annotation, performs on par with BERT’s capability to discern complex medical terminologies and contextual nuances within the EHRs. The results of this study highlight a notable advance in the field of NER for Spanish EHRs, with the few shot approach driven by LLM, enhanced by external knowledge, slightly edging out the traditional BERT-based method in overall effectiveness. GPT’s superiority in F-score and its minimal reliance on extensive data annotation underscore its potential in medical data processing.

Structuring medication signeturs as a language regression task: comparison of zero- and few-shot GPT with fine-tuned models.

Electronic health record textual sources such as medication signeturs (sigs) contain valuable information that is not always available in structured form. Commonly processed through manual annotation, this repetitive and time-consuming task could be fully automated using large language models (LLMs). While most sigs include simple instructions, some include complex patterns. We aimed to compare the performance of GPT-3.5 and GPT-4 with smaller fine-tuned models (ClinicalBERT, BlueBERT) in extracting the average daily dose of 2 immunomodulating medications with frequent complex sigs: hydroxychloroquine, and prednisone. Using manually annotated sigs as the gold standard, we compared the performance of these models in 702 hydroxychloroquine and 22104 prednisone prescriptions. GPT-4 vastly outperformed all other models for this task at any level of in-context learning. With 100 in-context examples, the model correctly annotates 94% of hydroxychloroquine and 95% of prednisone sigs to within 1 significant digit. Error analysis conducted by 2 additional manual annotators on annotator-model disagreements suggests that the vast majority of disagreements are model errors. Many model errors relate to ambiguous sigs on which there was also frequent annotator disagreement. Paired with minimal manual annotation, GPT-4 achieved excellent performance for language regression of complex medication sigs and vastly outperforms GPT-3.5, ClinicalBERT, and BlueBERT. However, the number of in-context examples needed to reach maximum performance was similar to GPT-3.5. LLMs show great potential to rapidly extract structured data from sigs in no-code fashion for clinical and research applications.

Multicentric intelligent cardiotocography signal interpretation using deep semi-supervised domain adaptation via minimax entropy and domain invariance

Background and ObjectiveObstetricians use Cardiotocography (CTG), which is the continuous recording of fetal heart rate and uterine contraction, to assess fetal health status. Deep learning models for intelligent fetal monitoring trained on extensively labeled and identically distributed CTG records have achieved excellent performance. However, creation of these training sets requires excessive time and specialist labor for the collection and annotation of CTG signals. Previous research has demonstrated that multicenter studies can improve model performance. However, models trained on cross-domain data may not generalize well to target domains due to variance in distribution among datasets. Hence, this paper conducted a multicenter study with Deep Semi-Supervised Domain Adaptation (DSSDA) for intelligent interpretation of antenatal CTG signals. This approach helps to align cross-domain distribution and transfer knowledge from a label-rich source domain to a label-scarce target domain. MethodsWe proposed a DSSDA framework that integrated Minimax Entropy and Domain Invariance (DSSDA-MMEDI) to reduce inter-domain gaps and thus achieve domain invariance. The networks were developed using GoogLeNet to extract features from CTG signals, with fully connected, softmax layers for classification. We designed a Dynamic Gradient-driven strategy based on Mutual Information (DGMI) to unify the losses from Minimax Entropy (MME), Domain Invariance (DI), and supervised cross-entropy during iterative learning. ResultsWe validated our DSSDA model on two datasets collected from collaborating healthcare institutions and mobile terminals as the source and target domains, which contained 16,355 and 3,351 CTG signals, respectively. Compared to the results achieved with deep learning networks without DSSDA, DSSDA-MMEDI significantly improved sensitivity and F1-score by over 6%. DSSDA-MMEDI also outperformed other state-of-the-art DSSDA approaches for CTG signal interpretation. Ablation studies were performed to determine the unique contribution of each component in our DSSDA mechanism. ConclusionsThe proposed DSSDA-MMEDI is feasible and effective for alignment of cross-domain data and automated interpretation of multicentric antenatal CTG signals with minimal annotation cost.

227 Spine-tuned Natural Language Models and Bespoke Regular Expression Classifiers for Automated Spinal Surgery Registry Development

INTRODUCTION: Surgical research demands the development of clinical registries, often through time-intensive manual chart review. Natural language processing (NLP) may accelerate registry development, and an ideal automatic registry (autoregistry) algorithm would be highly accurate while requiring minimal manual data annotation. NLP approaches including bespoke Regular Expression (RegEx) classifiers and Large Language Models (LLM) possess distinct strengths and weaknesses and have not been compared in the setting of autoregistry development. METHODS: We used an institutional data lake to retrieve 31,502 neurosurgical operative notes. A standardized set of spinal procedures was chosen for inclusion in the autoregistry. 200 manually annotated notes were used for training and testing purposes. RegEx classifiers were engineered to retrieve procedural info from unprocessed notes. A family of 110-million parameter BERT models, including LLM pre-trained on clinical text, was fine-tuned for the same tasks. We also tested a open-source 7-billion parameter LLM chatbot, Vicuna, without fine-tuning. RESULTS: The RegEx classifiers were able to identify spinal procedures and associated vertebral levels in nearly 99% of operative notes. Fine-tuned LLM identified common procedures (e.g. spinal fusion and laminectomy) with greater than 95% accuracy but performed poorly for rarer procedures (e.g. XLIF, corpectomy) and vertebral body identification. Qualitative evaluation of the Vicuna chatbot showed potential for the same tasks, following iteratively refined prompting. CONCLUSIONS: The goal of autoregistry development is to minimize time- and labor-intensive manual chart review. We found that fine-tuned LLM could not match the accuracy and efficiency of the RegEx classifier. However, LLM may be well-suited to expand existing clinical databases that provide a robust training set. Further work combining NLP approaches will attempt to develop a pipeline for autoregistry development from natural language (plain English) queries.

Magnifying Networks for Histopathological Images with Billions of Pixels.

Amongst the other benefits conferred by the shift from traditional to digital pathology is the potential to use machine learning for diagnosis, prognosis, and personalization. A major challenge in the realization of this potential emerges from the extremely large size of digitized images, which are often in excess of 100,000 × 100,000 pixels. In this paper, we tackle this challenge head-on by diverging from the existing approaches in the literature-which rely on the splitting of the original images into small patches-and introducing magnifying networks (MagNets). By using an attention mechanism, MagNets identify the regions of the gigapixel image that benefit from an analysis on a finer scale. This process is repeated, resulting in an attention-driven coarse-to-fine analysis of only a small portion of the information contained in the original whole-slide images. Importantly, this is achieved using minimal ground truth annotation, namely, using only global, slide-level labels. The results from our tests on the publicly available Camelyon16 and Camelyon17 datasets demonstrate the effectiveness of MagNets-as well as the proposed optimization framework-in the task of whole-slide image classification. Importantly, MagNets process at least five times fewer patches from each whole-slide image than any of the existing end-to-end approaches.

Self-Supervised Interactive Embedding for One-shot Organ Segmentation.

One-shot organ segmentation (OS2) aims at segmenting the desired organ regions from the input medi-cal imaging data with only one pre-annotated example as the reference. By using the minimal annotation data to facilitate organ segmentation, OS2 receives great attention in the medical image analysis community due to its weak re-quirement on human annotation. In OS2, one core issue is to explore the mutual information between the support (ref-erence slice) and the query (test slice). Existing methods rely heavily on the similarity between slices, and additional slice allocation mechanisms need to be designed to reduce the impact of the similarity between slices on the segmen-tation performance. To address this issue, we build a novel support-query interactive embedding (SQIE) module, which is equipped with the channel-wise co-attention, spatial-wise co-attention, and spatial bias transformation blocks to identify "what to look", "where to look", and "how to look" in the input test slice. By combining the three mechanisms, we can mine the interactive information of the intersection area and the disputed area between slices, and establish the feature connection between the target in slices with low similarity. We also propose a self-supervised contrastive learning framework, which transforms knowledge from the physical position to the embedding space to facilitate the self-supervised interactive embedding of the query and support slices. Comprehensive experiments on two large benchmarks demonstrate the superior capacity of the pro-posed approach when compared with the current alterna-tives and baseline models.

Weakly-Interactive-Mixed Learning: Less Labelling Cost for Better Medical Image Segmentation.

Common medical image segmentation tasks require large training datasets with pixel-level annotations which are very expensive and time-consuming to prepare. To overcome such limitation and achieve the desired segmentation accuracy, a novel Weakly-Interactive-Mixed Learning (WIML) framework is proposed by efficiently using weak labels. On one hand, utilize weak labels to reduce annotation time for high-quality strong labels by designing a Weakly-Interactive Annotation (WIA) part of the WIML which prudently introduces interactive learning into the weakly-supervised segmentation strategy. On the other hand, utilize weak labels and very few strong labels to achieve desired segmentation accuracy by designing a Mixed-Supervised Learning (MSL) part of the WIML which can boost the segmentation accuracy by providing strong prior knowledge during training. Besides, a multi-task Full-Parameter-Sharing Network (FPSNet) is proposed to better implement this framework. Specifically, to further reduce annotation time, attention modules (scSE) are integrated into FPSNet to improve the class activation map (CAM) performance for the first time. To further improve segmentation accuracy, a Full-Parameter-Sharing (FPS) strategy is designed in FPSNet to alleviate the overfitting of the segmentation task supervised by very few strong labels. The proposed method is validated on the BraTS 2019 and LiTS 2017 datasets, and experiments demonstrate that the proposed method WIML-FPSNet outperforms several state-of-the-art segmentation methods with minimal annotation efforts.

#6541 GLOMERULONEPHRITIS DIAGNOSIS BY MACHINE LEARNING ON PERIODIC ACID-SCHIFF (PAS) WHOLE SLIDE IMAGES

Abstract Background and Aims Machine learning (ML) holds great promise for improving diagnostics, prognostication and theranostics in nephropathology. So far, applications have not gone much further than segmentation of tissue compartments on whole slide images (WSIs) of paraffin sections. As a proof-of-concept study, we describe the development of a diagnostic classifier for glomerulephritis based on expert-annotated or automatically segmented glomerular transections from periodic-acid Schiff (PAS) paraffin sections only. Method A total of n = 350 biopsies from 5 institutions with 12 classes of glomerulonephritis IgA nephropathy (IgAN), membranous nephropathy (Membranous), anti-glomerular basement membrane antibody GN (ABMGN), infection-associated GN (IAGN), ANCA-associated GN (ANCA-GN), idiopathic membranoproliferative GN (MPGN), SLE GN class IV (SLE-GN-IV), cryglobulinemic GN (CryoGN), C3 GN (C3-GN), dense deposit disease (DDD), fibrillary GN (FibrillaryGN) and proliferative GN with monoclonal immunoglobulin deposits (PGNMID) were included in the study with their respective PAS sections. Glomerular transections were expert-annotated by a nephropathologist and automatically segmented with our own transformer-based segmentation model trained on 100 biopsies with thrombotic microangiopathies and a range of vascular, vasculitic and glomerular diseases closely resembling/mimicking thrombotic microangiopathies. For classification, we divided the cohort into 5 folds for internal cross-validation, performed sample size augmentation with various methods (including shifts in resolution/scale, AutoAugment and others) and trained our proprietary self-attention-based MILx architecture on an EfficientNet backbone with selection of glomerular crop batches by soft Markov chain Monte Carlo sampling in a semi-supervised fashion, with diagnostic class labels for each biopsy. We compared the performance of our proprietary architecture on both expert-annotated and automatically segmented glomerular crops with a recently published benchmark architecture (CLAM) for multiple-instance learning in histopathology. Results Automatic glomerular segmentation performance was excellent with mean AUC and sensitivity (mean average recall) over all classes at 0.904, with near perfect mean average specificity (0.994), as expected best for Membranous, worst for ABMGN. Classification performance of MILx with expert-annotated glomerular crops as inputs had a mean balanced accuracy of 0.84, with AUC metrics in descending order of 0.97 for Membranous, 0.89 for ABMGN, 0.88 for IgAN, 0.86 for Fibrillary, 0.83 for MPGN, 0.80 for ANCA-GN, 0.79 for DDD, 0.78 for PGNMID, 0.75 for IAGN, 0.73 for SLE-GN-IV and CryoGN, 0.67 for C3-GN. Performance with MILx was similar for automatically segmented glomerular crops as input. On this dataset, MILx outperformed CLAM with both entire WSIs as well as expert-annotated glomerular crops as inputs (mean balanced accuracy of 0.72) by a significant margin. Conclusion This proof-of-concept-study indicates that nephropathology-specific architectures like our MILx can be trained for complex tasks on relatively small biopsy cohorts. We should be able to deliver an end-to-end-pipeline for this diagnostic and other tasks based on training sets with case-labels provided by trusted institutions with only minimal expert labeling or annotation required. PAC and HQ contributed equally to this work.

A unified microstructure segmentation approach via human-in-the-loop machine learning

Microstructure segmentation, a technique for extracting structural statistics from microscopy images, is an essential step for establishing quantitative structure–property relationships in a broad range of materials research areas. However, the task is challenging due to the morphological complexity and diversity of structural constituents. While recent breakthroughs in deep learning have led to significant progress in microstructure segmentation, there remain two fundamental challenges to its further diffusion: prohibitive annotation costs and an unreliable decision-making process. To tackle these challenges, we propose a human-in-the-loop machine learning framework for unified microstructure segmentation, which leverages recent advances in both weakly supervised learning and active learning techniques. The key idea behind our approach lies in the integration of human and machine capabilities to make not only precise but also reliable microstructure segmentation at minimal annotation costs. Extensive experiments demonstrate the generality of our framework across different material classes, structural constituents, and microscopic imaging modalities.

Understudied Proteins and Understudied Functions in the Model Bacterium Bacillus subtilis - a Major Challenge in Current Research.

Model organisms such as the Gram-positive bacterium Bacillus subtilis have been studied intensively for decades. However, even for model organisms no function has been identified for about one fourth of all proteins. It has recently been realized that such understudied proteins as well as poorly studied functions set a limitation to our understanding of the requirements for cellular life, and the Understudied Proteins Initiative has been launched. Of poorly studied proteins, those that are strongly expressed are likely to be important to the cell and should therefore be considered high priority in further studies. Since the functional analysis of unknown proteins can be extremely laborious, a minimal knowledge is required prior to targeted functional studies. In this review, we discuss strategies to obtain such a minimal annotation, e.g. from global interaction, expression or localization studies. We present a set of 41 highly expressed and poorly studied proteins of B. subtilis. Several of these proteins are thought or known to bind RNA and/or the ribosome, some may control the metabolism of B. subtilis, and another subset of particularly small proteins may act as regulatory elements to control the expression of downstream genes. Moreover, we discuss the challenges of poorly studied functions with a focus on RNA-binding proteins, amino acid transport and the control of metabolic homeostasis. The identification of the functions of the selected proteins will strongly advance our knowledge on B. subtilis, but also on other organisms since many of the proteins are conserved in many groups of bacteria.

Semantic segmentation of water bodies in very high-resolution satellite and aerial images

This study evaluates the performance of convolutional neural networks for semantic segmentation of water bodies in very high-resolution satellite and aerial images from multiple sensors with particular focus on flood emergency response applications. Different model architectures (U-Net and DeepLab-V3+) are combined with encoder backbones (MobileNet-V3, ResNet-50 and EfficientNet-B4) and tested for their ability to delineate inundated areas under varying environmental conditions and data availability scenarios. An unprecedented reference dataset of 1120 globally sampled images with quality checked binary water masks is introduced and used to train, validate and test the models for water body segmentation. Furthermore, independent test datasets are developed to test the generalization ability of the trained models across regions, sensors (IKONOS, GeoEye-1, WorldView-2, WorldView-3 and four different airborne camera systems) and tasks (normal water and flood water segmentation). Results indicate that across all tested scenarios a U-Net model with Mobilenet-V3 backbone pre-trained on ImageNet performs best. While using R-G-B image bands performs well, adding the near infrared band (if available) slightly improves prediction results. Similarly, adding slope information from an independent digital elevation model increases accuracies. Train-time augmentation and contrast enhancement could improve transferability across sensors and in particular between satellite and aerial images. Moreover, adding noisy training data from freely available online resources could further improve performance with minimal annotation effort.

Self-supervised learning based transformer and convolution hybrid network for one-shot organ segmentation

Few-shot medical segmentation aims at segmenting the desired regions from the input medical imaging data with only a pre-annotated example as the reference. By using the minimal annotation data to facilitate the segmentation, the few-shot manner receives great attention in the medial image analysis community due to its weak requirement on human annotation. For one-shot segmentation methods, one core issue is to learn a feature embedding space where the features of the desired segmentation regions on the unlabeled image and the references are high correlation. Previous works rely on the similarity between image features as a constraint to establish such embedding space, ignoring the correlation between samples mined from image features. To address this issue, we propose a novel transformer and convolution hybrid network for building the global correlation between the reference sample (support) and the desired segmentation sample (query). The convolution network is first to extract the local features of the support and query, then, the transformer further extracts the global features from the local feature space. To build the global correction between the support and query, we proposed a semantic dependency relationship embedding which introduces the channel-wise and spatial-wise co-information of them to the transformer. We employ superpixel-based self-supervised learning to train the proposed network to solve the problem of insufficient training samples in the field of medical image segmentation. Comprehensive experiments on two benchmarks demonstrate the superior capacity of the proposed approach when compared to the current alternatives and baseline models.

Motion-region annotation for complex videos via label propagation across occluders

Motion cue is pivotal in moving object analysis, which is the root for motion segmentation and detection. These preprocessing tasks are building blocks for several applications such as recognition, matching and estimation. To devise a robust algorithm for motion analysis, it is imperative to have a comprehensive dataset to evaluate an algorithm’s performance. The main limitation in making these kind of datasets is the creation of ground-truth annotation of motion, as each moving object might span over multiple frames with changes in size, illumination and angle of view. Besides the optical changes, the object can undergo occlusion by static or moving occluders. The challenge increases when the video is captured by a moving camera. In this paper, we tackle the task of providing ground-truth annotation on motion regions in videos captured from a moving camera. With minimal manual annotation of an object mask, we are able to propagate the label mask in all the frames. Object label correction based on static and moving occluder is also performed by applying occluder mask tracking for a given depth ordering. A motion annotation dataset is also proposed to evaluate algorithm performance. The results show that our cascaded-naive approach provides successful results. All the resources of the annotation tool are publicly available at http://dixie.udg.edu/anntool/.