Annotated Samples Research Articles

Query semantic reconstruction for background in few-shot segmentation

Few-shot segmentation (FSS) aims to segment unseen classes using a few annotated samples. Typically, a prototype representing the foreground class is extracted from annotated support image(s) and is matched to features representing each pixel in the query image. However, models learnt in this way are insufficiently discriminatory, and often produce false positives: misclassifying background pixels as foreground. Some FSS methods try to address this issue by using the background in the support image(s) to help identify the background in the query image. However, the backgrounds of these images are often quite distinct, and hence, the support image background information is uninformative. This article proposes a method, QSR, that extracts the background from the query image itself, and as a result is better able to discriminate between foreground and background features in the query image. This is achieved by modifying the training process to associate prototypes with class labels including known classes from the training data and latent classes representing unknown background objects. This class information is then used to extract a background prototype from the query image. To successfully associate prototypes with class labels and extract a background prototype that is capable of predicting a mask for the background regions of the image, the machinery for extracting and using foreground prototypes is induced to become more discriminative between different classes. Experiments achieves state-of-the-art results for both 1-shot and 5-shot FSS on the PASCAL-5i\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$5^{i}$$\\end{document} and COCO-20i\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$20^{i}$$\\end{document} dataset. As QSR operates only during training, results are produced with no extra computational complexity during testing.

A Demonstrator for Threat Detection in Volumetric CT Scans

The detection of threat items and prohibited items inside closed containers is an active field of research and an even more important real-world application. While conventional systems use X-ray projections for inspection, they lack true three-dimensional information which is crucial for a reliable detection. In this work, we introduce a demonstrator setup for three-dimensional threat detection within a 3D printed miniature container. We created a data processing pipeline that automatically integrates CT scanning, volumetric reconstruction and the actual detection of threat objects by means of a neural network. The achieved results are very promising and could be obtained using very few annotated training samples which is a realistic assumption in this application.

Fault Diagnosis of Rotating Machinery With Limited Expert Interaction: A Multicriteria Active Learning Approach Based on Broad Learning System

Recently, research on the fault diagnosis of rotating machinery, especially for the compound or unknown cases, has drawn increasing attention. Some advanced learning-based approaches have achieved good fault diagnosis performance to some degree. However, in practical applications, how to utilize prior knowledge as fully as possible for fault diagnosis with constraints of limited expert interaction remains an open issue. In this brief, a fault diagnosis methodology of rotating machinery with limited expert interaction is proposed. With related feature extraction techniques, a novel multicriteria active learning (MCAL) query strategy is designed to select the relatively valuable samples for annotation. In addition, the broad learning system (BLS) is exploited to achieve fast incrementally updating or retrain procedures with high diagnostic accuracy in different diagnosis scenarios. Several experiments are conducted on a real-world rotating machinery fault diagnosis (RMFD) experimental platform. Compared with other existing advanced approaches, the diagnosis performance of the proposal shows high stability and flexibility. The annotation cost of experts is also significantly reduced, which makes the proposal more suitable for dealing with practical problems.

Self-Supervised Representation Learning for Video Quality Assessment

No-reference (NR) video quality assessment (VQA) is a challenging problem due to the difficulty in model training caused by insufficient annotation samples. Previous work commonly utilizes transfer learning to directly migrate pre-trained models on the image database, which suffers from domain inadaptation. Recently, self-supervised representation learning has become a hot spot for the independence of large-scale labeled data. However, existing self-supervised representation learning method only considers the distortion types and contents of the video, there needs to investigate the intrinsic properties of videos for the VQA task. To amend this, here we propose a novel multi-task self-supervised representation learning framework to pre-train a video quality assessment model. Specifically, we consider the effects of distortion degrees, distortion types, and frame rates on the perceived quality of videos, and utilize them as guidance to generate self-supervised samples and labels. Then, we optimize the ability of the VQA model in capturing spatio-temporal differences between the original video and the distorted version using three pretext tasks. The resulting framework not only eases the requirements for the quality of the original video but also benefits from the self-supervised labels as well as the Siamese network. In addition, we propose a Transformer-based VQA model, where short-term spatio-temporal dependencies of videos are modeled by 3D-CNN and 2D-CNN, and then the long-term spatio-temporal dependencies are modeled by Transformer because of its excellent long-term modeling capability. We evaluated the proposed method on four public video quality assessment databases and found that it is competitive with all compared VQA algorithms.

Deep stacked pinball transfer matrix machine with its application in roller bearing fault diagnosis

At present, the matrix-based classification methods have played an extremely important role to mechanical fault diagnosis. However, the traditional matrix-based classification methods are mostly shallow classifiers, which are difficult to obtain the deep-seated sensitive features of vibration signals. To learn and extract low-rank structure information, a new deep stacked pinball transfer matrix machine (DSPTMM) is proposed to enhance the performance of traditional shallow matrix-based classifiers, which takes the stacking generalization principle as the main idea. In DSPTMM, a pinball transfer module (PTM) is constructed as the basic module of deep stacking network, in which the pinball loss is used to achieve an enhanced noise robustness. Specifically, PTM performs the function to obtain the weak prediction of the previous module, and the original results of previous weak prediction are modified by random projection and then will be input as a new feature set in the next module. Therefore, DSPTMM has a better classification performance and can modeled without enough annotation samples. Extensive experiments are carried out on two different datasets of roller bearing, and the results of experiment show that the proposed DSPTMM can use limited samples to establish the accurate model.

DeepInsight-3D architecture for anti-cancer drug response prediction with deep-learning on multi-omics

Modern oncology offers a wide range of treatments and therefore choosing the best option for particular patient is very important for optimal outcome. Multi-omics profiling in combination with AI-based predictive models have great potential for streamlining these treatment decisions. However, these encouraging developments continue to be hampered by very high dimensionality of the datasets in combination with insufficiently large numbers of annotated samples. Here we proposed a novel deep learning-based method to predict patient-specific anticancer drug response from three types of multi-omics data. The proposed DeepInsight-3D approach relies on structured data-to-image conversion that then allows use of convolutional neural networks, which are particularly robust to high dimensionality of the inputs while retaining capabilities to model highly complex relationships between variables. Of particular note, we demonstrate that in this formalism additional channels of an image can be effectively used to accommodate data from different omics layers while implicitly encoding the connection between them. DeepInsight-3D was able to outperform other state-of-the-art methods applied to this task. The proposed improvements can facilitate the development of better personalized treatment strategies for different cancers in the future.

Federated-Learning-Empowered Semi-Supervised Active Learning Framework for Intrusion Detection in ZSM

Exponential growth of novel radical applications and services in sixth-generation (6G) networks is expected to increase the complexity of managing existing network infrastructures. In this context, the zero touch network and service management (ZSM) paradigm, which leverages AI, SDN, and NFV techniques, is seen as a promising solution to automatically manage and orchestrate network resources. However, due to the closed-loop operation and automated end-to-end framework in a distributed 6G network, the ZSM architecture, along with its potential benefits, is exposed to various security threats. A recently proposed solution to address privacy concerns is federated learning (FL), whereby distributed training is performed, and the aggregated model parameters, instead of clients' raw data, are forwarded to the global server. However, most of the existing FL and semi-supervised learning (SSL) models for intrusion detection are based on the assumption that fully labeled data are always available at the server and client sides, which is not practical due to the high labeling costs and privacy constraints in the 6G network. In this article, we propose a novel FL-empowered semi-supervised active learning (FL-SSAL) security orchestration framework for the Label-at-Client scenario where, along with unlabeled samples, clients have a small portion of labeled data. The entropy-based active learning selects the most informative samples for data annotation and leverages the unlabeled data using a semi-supervised approach. The results of our experimental evaluations performed on the private, not independent and identically distributed (non-IID) dataset demonstrate that FL-SSAL achieves higher intrusion detection accuracy and has less communication overhead than baseline schemes with less labeled data.

Hierarchical Self-Supervised Learning for 3D Tooth Segmentation in Intra-Oral Mesh Scans.

Accurately delineating individual teeth and the gingiva in the three-dimension (3D) intraoral scanned (IOS) mesh data plays a pivotal role in many digital dental applications, e.g., orthodontics. Recent research shows that deep learning based methods can achieve promising results for 3D tooth segmentation, however, most of them rely on high-quality labeled dataset which is usually of small scales as annotating IOS meshes requires intensive human efforts. In this paper, we propose a novel self-supervised learning framework, named STSNet, to boost the performance of 3D tooth segmentation leveraging on large-scale unlabeled IOS data. The framework follows two-stage training, i.e., pre-training and fine-tuning. In pre-training, three hierarchical-level, i.e., point-level, region-level, cross-level, contrastive losses are proposed for unsupervised representation learning on a set of predefined matched points from different augmented views. The pretrained segmentation backbone is further fine-tuned in a supervised manner with a small number of labeled IOS meshes. With the same amount of annotated samples, our method can achieve an mIoU of 89.88%, significantly outperforming the supervised counterparts. The performance gain becomes more remarkable when only a small amount of labeled samples are available. Furthermore, STSNet can achieve better performance with only 40% of the annotated samples as compared to the fully supervised baselines. To the best of our knowledge, we present the first attempt of unsupervised pre-training for 3D tooth segmentation, demonstrating its strong potential in reducing human efforts for annotation and verification.

A Domain-Specific Evaluation of the Performance of Selected Web-based Sentiment Analysis Platforms

There is now an increasing number of sentiment analysis software-as-a-service (SA-SaaS) offerings in the market. Approaches to sentiment analysis and their implementation as SA-SaaS vary, and there really is no sure way of knowing what SA-SaaS uses which approach. For potential users, SA-SaaS products are black boxes. Black boxes, however, can be evaluated using a set of standard input and a comparison of the output. Using a test data set drawn from human annotated samples in existing studies covering sentiment polarity of news headlines, this study compares the performance of selected popular and free (or at least free-to-try) SA-SaaS in terms of the accuracy, precision, recall and specificity of the sentiment classification using the black box testing methodology. SentiStrength, developed at the University of Wolverhampton in the UK, emerged as consistent performer across all metrics.

Attention Mechanism Trained with Small Datasets for Biomedical Image Segmentation

The understanding of long-range pixel–pixel dependencies plays a vital role in image segmentation. The use of a CNN plus an attention mechanism still has room for improvement, since existing transformer-based architectures require many thousands of annotated training samples to model long-range spatial dependencies. This paper presents a smooth attention branch (SAB), a novel architecture that simplifies the understanding of long-range pixel–pixel dependencies for biomedical image segmentation in small datasets. The SAB is essentially a modified attention operation that implements a subnetwork via reshaped feature maps instead of directly calculating a softmax value over the attention score for each input. The SAB fuses multilayer attentive feature maps to learn visual attention in multilevel features. We also introduce position blurring and inner cropping specifically for small-scale datasets to prevent overfitting. Furthermore, we redesign the skip pathway for the reduction of the semantic gap between every captured feature of the contracting and expansive path. We evaluate the architecture of U-Net with the SAB (SAB-Net) by comparing it with the original U-Net and widely used transformer-based models across multiple biomedical image segmentation tasks related to the Brain MRI, Heart MRI, Liver CT, Spleen CT, and Colonoscopy datasets. Our training set was made of random 100 images of the original training set, since our goal was to adopt attention mechanisms for biomedical image segmentation tasks with small-scale labeled data. An ablation study conducted on the brain MRI test set demonstrated that every proposed method achieved an improvement in biomedical image segmentation. Integrating the proposed methods helped the resulting models consistently achieve outstanding performance on the above five biomedical segmentation tasks. In particular, the proposed method with U-Net improved its segmentation performance over that of the original U-Net by 13.76% on the Brain MRI dataset. We proposed several novel methods to address the need for modeling long-range pixel–pixel dependencies in small-scale biomedical image segmentation. The experimental results illustrated that each method could improve the medical image segmentation accuracy to various degrees. Moreover, SAB-Net, which integrated all proposed methods, consistently achieved outstanding performance on the five biomedical segmentation tasks.

Semantic-aware alignment and label propagation for cross-domain arrhythmia classification

Cross-domain arrhythmia classification (CAC) aims to transfer the model trained on a label-sufficient source domain to a label-scarce target domain. To the best of our knowledge, almost all existing CAC models focus on the unsupervised setting, where no labeled target samples are available. However, in most practical scenes, acquiring limited annotated target samples is feasible, which can provide reliable target semantic information for model learning directly. Consequently, we first propose a more realistic semi-supervised CAC setting, where only the source samples and extremely limited target samples are annotated. Most previous CAC models realize cross-domain learning by aligning the feature distributions of source and target domains coarsely and globally, where the semantic invariance within each class is not taken into consideration during the domain alignment process. Additionally, the semantic information contained in the feature space is not fully utilized for target pseudo label mining. To address the above two issues, a unified framework containing the semantic-aware feature alignment (SAFA) and prototype-based label propagation (PBLP) modules is proposed. In the proposed framework, SAFA and PBLP are complementary to each other. Specifically, SAFA provides more robust prototypes for PBLP by performing semantic-aware feature alignment, and PBLP offers more reliable target pseudo labels for more effective semantic-aware feature alignment learning. Comprehensive qualitative and quantitative experimental results on different benchmarks verify the effectiveness of the proposed method.

Few-shot learning on point clouds for railroad segmentation

Infrastructure maintenance of complex environments like railroads is a very expensive operation. Recent advances in mobile mapping systems to collect 3D point cloud data and in deep learning for detection and segmentation can prove to be very helpful in automating this maintenance and allowing preventive maintenance at certain locations before big failures occur. Some fully-supervised methods have been developed for understanding dynamic railroad environments. These methods often fail to generalize to infrastructure changes or new classes in low-labeled data. To address this issue, we propose a railroad segmentation method that leverages few-shot learning by generating class prototypes for the most relevant infrastructure classes. This method takes advantage of existing embedding networks for point clouds, taking the geometrical and spatial context into account for feature representation of complex connected classes. We evaluate our method on real-world data measured on Belgian railway tracks. Our model achieves promising results on connected classes, exposed to only a few annotated samples at test time.

TaxonPrompt: Taxonomy-aware curriculum prompt learning for few-shot event classification

Event classification (EC) aims to assign the event labels to unlabeled sentences and tends to struggle in real-world applications when only a few annotated samples are available. Previous studies have mainly focused on using meta-learning to overcome the low-resource problem where label data from other tasks are still required for model learning and selection. Accordingly, prompt learning-based approaches are proposed to address the low-resource issue. However, such approaches generally ignore task-specific information and adopt demonstration learning for fine-tuning, which fails to leverage the most informative examples for training and hurts performance. Thus, we propose a taxonomy-aware prompt learning framework TaxonPrompt that trains the language model with samples from easy to hard by imitating the human curricula, which effectively alleviates the classification bottleneck caused by insufficient data. We first design an event prompt generation (EPG) for automatically generating task-specific templates using sentences, labels, and trigger words. Then, we propose a Fisher information-based demonstration filtering (FDF) to dynamically select the most informative support examples for each query to train the model. We have conducted extensive experiments on two EC datasets: FewEvent and RAMS. The experimental results demonstrate the superiority of the proposed model over state-of-the-art baselines. In particular, our approach works well in the scenario of an extremely small number of available task resources and therefore constitutes a solution for few-shot event classification.

A self-supervised contrastive learning framework with the nearest neighbors matching for the fault diagnosis of marine machinery

The problem of limited annotated samples is prevalent in the shipping industry, and it significantly deteriorates the performance of fault diagnosis based on data-driven approaches. In this paper, a self-supervised contrastive learning framework with the nearest neighbors matching (SCLNNM) is proposed to learn discriminative feature representation from large-scale unlabeled datasets for fault diagnosis. Due to the collected 1D signals of machinery different from 2D imagines, in addition to a designed reasonable composition of data augmentation to generate a similar instance for the 1D sequence, our scheme also finds the nearest neighbors in the support set as the positive instance of the input signal to increase the diversity of representations. In this framework, the 1D CNN model combined with contrastive learning is designed to learn robust and general representations from different augmented signals. On this basis, the limited annotated data is finally used to investigate what kind of feature representations are suitable, and train a simple classifier for the fault diagnosis. The collected engine dataset of an operational ship shows that the proposed framework can efficiently extract valuable feature information and improve classification accuracy under the limited annotated dataset.

SDFC dataset: a large-scale benchmark dataset for hyperspectral image classification

Hyperspectral image (HSI) classification plays an important role in a wide range of remote sensing applications in military and civilian fields. During past decades, significant efforts have been made on developing datasets and introducing novel approaches to promote HSI classification, such that promising classification performance has been achieved. However, existing datasets generally pose following issues, including the limited categories and annotated samples, the lack of sample diversity, as well as the low spatial resolution. These limitations severely restrict the development and evaluation of data-driven models, especially deep neural network-based ones. In recent years, advances in imaging spectroscopy provide us the opportunity to obtain the hyperspectral image data with high spectral and spatial resolution, therefore, in this paper, we contribute a large-scale benchmark dataset for conducting hyperspectral image classification to address issues raised by existing datasets, noted as ShanDongFeiCheng (SDFC). The proposed SDFC is characterized by (1) The large-scale annotated samples with diverse categories; (2) The high spatial resolution; and (3) The high intra-class variance yet relatively low inter-class variance, making the HSI classification task much more challenging on it. We evaluated 10 classic traditional and deep neural network-based models on SDFC, of which the results can be regarded as useful baselines for further experiments. Moreover, given the state-of-the-art performance of SpectralNet, we selected it as the representation method, and evaluated it across datasets to analyze the difference effects on the classification model induced by different datasets. The comprehensive review and analysis of the representative classification models on both existing and proposed datasets demonstrate the advantages and challenges of our proposed dataset, and provide promising perspectives for future HSI classification studies.

Uncertainty-aware semi-supervised few shot segmentation

Few shot segmentation (FSS) aims to learn pixel-level classification of a target object in a query image using only a few annotated support samples. This is challenging as it requires modeling appearance variations of target objects and the diverse visual cues between query and support images with limited information. To address this problem, we propose a semi-supervised FSS strategy that leverages additional prototypes from unlabeled images with uncertainty guided pseudo label refinement. To obtain reliable prototypes from unlabeled images, we meta-train a neural network to jointly predict segmentation and estimate the uncertainty of predictions. We employ the uncertainty estimates to exclude predictions with high degrees of uncertainty for pseudo label construction to obtain additional prototypes from the refined pseudo labels. During inference, query segmentation is predicted using prototypes from both support and unlabeled images including low-level features of the query images. Our approach can easily supplement existing approaches without the requirement of additional training when employing unlabeled samples. Extensive experiments on PASCAL-5i and COCO-20i demonstrate that our model can effectively remove unreliable predictions to refine pseudo labels and significantly improve upon baseline performance.

CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation.

Automated cellular instance segmentation is a process utilized for accelerating biological research for the past two decades, and recent advancements have produced higher quality results with less effort from the biologist. Most current endeavors focus on completely cutting the researcher out of the picture by generating highly generalized models. However, these models invariably fail when faced with novel data, distributed differently than the ones used for training. Rather than approaching the problem with methods that presume the availability of large amounts of target data and computing power for retraining, in this work we address the even greater challenge of designing an approach that requires minimal amounts of new annotated data as well as training time. We do so by designing specialized contrastive losses that leverage the few annotated samples very efficiently. A large set of results show that 3 to 5 annotations lead to models with accuracy that: 1) significantly mitigate the covariate shift effects; 2) matches or surpasses other adaptation methods; 3) even approaches methods that have been fully retrained on the target distribution. The adaptation training is only a few minutes, paving a path towards a balance between model performance, computing requirements and expert-level annotation needs.

Annotation Cost Minimization for Ultrasound Image Segmentation using Cross-domain Transfer Learning.

Deep learning techniques can help minimize inter-physician analysis variability and the medical expert workloads, thereby enabling more accurate diagnoses. However, their implementation requires large-scale annotated dataset whose acquisition incurs heavy time and human-expertise costs. Hence, to significantly minimize the annotation cost, this study presents a novel framework that enables the deployment of deep learning methods in ultrasound (US) image segmentation requiring only very limited manually annotated samples. We propose SegMix, a fast and efficient approach that exploits a segment-paste-blend concept to generate large number of annotated samples based on a few manually acquired labels. Besides, a series of US-specific augmentation strategies built upon image enhancement algorithms are introduced to make maximum use of the available limited number of manually delineated images. The feasibility of the proposed framework is validated on the left ventricle (LV) segmentation and fetal head (FH) segmentation tasks, respectively. Experimental results demonstrate that using only 10 manually annotated images, the proposed framework can achieve a Dice and JI of 82.61% and 83.92%, and 88.42% and 89.27% for LV segmentation and FH segmentation, respectively. Compared with training using the entire training set, there is over 98% of annotation cost reduction while achieving comparable segmentation performance. This indicates that the proposed framework enables satisfactory deep leaning performance when very limited number of annotated samples is available. Therefore, we believe that it can be a reliable solution for annotation cost reduction in medical image analysis.

Brain Tumour Segmentation Using S-Net and SA-Net

Image segmentation is an application area of computer vision and digital image processing that partitions a digital image into multiple image regions or segments. This process involves the extraction of a set of contours from the input digital image in such a manner that pixels belonging to a region share some common characteristics or computed properties, such as color, texture, or intensity. The application domain of image segmentation is widespread, and includes video surveillance, object detection, traffic control systems, and medical imaging. The application of image segmentation techniques in the field of medical imaging can be further subcategorized into virtual surgery simulation, diagnosis, study of anatomical structures, measurement of tissue volumes, location of tumors and other pathologies. In this study, we have proposed two new Convolutional Neural Network (CNN)-based models: (a) S-Net and (b) Attention S-Net (SA-Net) to perform image segmentation tasks in the field of medical imaging, especially to generate segmentation masks for brain tumours if present in brain Medical Resonance Imaging (MRI). Both proposed models were developed by considering U-Net as the base architecture. The newly proposed models have leveraged the concept of ‘Merge Block’ to infuse both the local and global context, and ‘Attention Block’ to focus on the region of interest having a specific object. Additionally, it uses techniques, such as data augmentation to utilize the available annotated samples more efficiently. The proposed models achieved a Dice Similarity Coefficient (DSC) measure of 0.78 and 0.80 for the High-Grade Glioma (HGG) and Low-Grade Glioma (LGG) datasets, respectively.

Graph Neural Networks With Triple Attention for Few-Shot Learning

Recent advances in Graph Neural Networks (GNNs) have achieved superior results in many challenging tasks, such as few-shot learning. Despite its capacity to learn and generalize a model from only a few annotated samples, GNN is limited in scalability, as deep GNN models usually suffer from severe over-fitting and over-smoothing. In this work, we propose a novel GNN framework with a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">triple-attention mechanism</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> node self-attention, neighbor attention, and layer memory attention, to tackle these challenges. We provide both theoretical analysis and illustrations to explain why the proposed attentive modules can improve GNN scalability for few-shot learning tasks. Our experiments show that the proposed Attentive GNN model outperforms the state-of-the-art few-shot learning methods using both GNN and non-GNN approaches. The improvement is consistent over the mini-ImageNet, tiered-ImageNet, CUB-200-2011, and Flowers-102 benchmarks, using both ConvNet-4 and ResNet-12 backbones, and under both the inductive and transductive settings. Furthermore, we demonstrate the superiority of our method for few-shot fine-grained and semi-supervised classification tasks with extensive experiments. The code for this work is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/chenghao-ch94/AGNN</uri> .