High Annotation Cost Research Articles

Multi-source weighted source-free domain transfer method for rotating machinery fault diagnosis

The mainstream approach to addressing the issues of insufficient historical data and high annotation costs in the domain of rotating machinery is to build transfer learning models based on labeled multi-source data. However, the practical diagnosis of failure cases often relies on data privacy, thereby limiting the widespread application of current multi-source domain transfer approaches for the ‘data silos’ problem of. In view of the above problem, a multi-source weighted source-free domain transfer approach is designed for rotating machinery fault diagnosis, and the designed scheme can efficiently achieve data privacy and domain transfer. Specifically, the proposed approach achieves knowledge transfer from the source to the target during the training process of the unlabeled target data without accessing the source data. This is accomplished through the utilization of a designed reinforced information maximization strategy and improved self-training mechanism. Additionally, a weighted strategy is devised to automatically apply optimal values to all source domains based on their relevance to the target domain. The proposed framework demonstrates accuracy exceeding 96% across eight cross-domain diagnostic cases in two sets of rotating machinery data, with an average accuracy of 98.26%. These results underscore the exceptional ability of the proposed method to address cross-domain fault diagnosis in rotating machinery while ensuring privacy protection.

Progressive mask‐oriented unsupervised fabric defect detection under background repair

AbstractDetection of defects is an essential quality control method in fabric production. Unsupervised deep learning‐based reconstruction algorithms have recently been deeply concerned owing to scarce fabric defect samples, high annotation cost, and deficient prior knowledge. Most unsupervised reconstruction models are prone to overfitting and poor generalisation performance, resulting in blurred images, residual defects, and uneven textures in the reconstruction results. On this account, an unsupervised fabric surface defect detection method using the Progressive Mask Repair Model (PMRM) has been developed. Specifically, PMRM with transformer architecture gathers detailed feature information. In order to pay closer attention to the textural properties of fabrics, the model incorporates structural similarity as a constraint in the training stage. In the detection stage, we designate the non‐defective area of the fabric image as the background and the defective area as the foreground. Next, a progressive mask is applied to repair the background of the defective area, which avoids defect false detection resulting from the poor reconstruction effect of the traditional reconstruction model in the non‐defective area. Finally, image processing methods such as image difference, frequency‐tuned salient detection, and threshold binarisation are used to segment the defects. Relative to the other six unsupervised defect detection methods, the proposed scheme increases the F1 score and intersection over union (IoU) by at least 9.34% and 8.49%, respectively. According to the earlier results, PMRM is effective and exhibits superiority.

Improve the performance of CT-based pneumonia classification via source data reweighting

Pneumonia is a life-threatening disease. Computer tomography (CT) imaging is broadly used for diagnosing pneumonia. To assist radiologists in accurately and efficiently detecting pneumonia from CT scans, many deep learning methods have been developed. These methods require large amounts of annotated CT scans, which are difficult to obtain due to privacy concerns and high annotation costs. To address this problem, we develop a three-level optimization based method which leverages CT data from a source domain to mitigate the lack of labeled CT scans in a target domain. Our method automatically identifies and downweights low-quality source CT data examples which are noisy or have large domain discrepancy with target data, by minimizing the validation loss of a target model trained on reweighted source data. On a target dataset with 2218 CT scans and a source dataset with 349 CT images, our method achieves an F1 score of 91.8% in detecting pneumonia and an F1 score of 92.4% in detecting other types of pneumonia, which are significantly better than those achieved by state-of-the-art baseline methods.

Chinese Short-Text Sentiment Prediction: A Study of Progressive Prediction Techniques and Attentional Fine-Tuning

With sentiment prediction technology, businesses can quickly look at user reviews to find ways to improve their products and services. We present the BertBilstm Multiple Emotion Judgment (BBMEJ) model for small-sample emotion prediction tasks to solve the difficulties of short emotion identification datasets and the high dataset annotation costs encountered by small businesses. The BBMEJ model is suitable for many datasets. When an insufficient quantity of relevant datasets prevents the model from achieving the desired training results, the prediction accuracy of the model can be enhanced by fine-tuning it with additional datasets prior to training. Due to the number of parameters in the Bert model, fine-tuning requires a lot of data, which drives up the cost of fine-tuning. We present the Bert Tail Attention Fine-Tuning (BTAFT) method to make fine-tuning work better. Our experimental findings demonstrate that the BTAFT fine-tuning approach performs better in terms of the prediction effect than fine-tuning all parameters. Our model obtains a small sample prediction accuracy of 0.636, which is better than the ideal baseline of 0.064. The Macro-F1 (F1) evaluation metrics significantly exceed other models.

EI-HCR: An Efficient End-to-End Hybrid Consistency Regularization Algorithm for Semisupervised Remote Sensing Image Segmentation

Recently, remote sensing image (RSI) semantic segmentation technology has advanced greatly, with the fully supervised process achieving particularly strong performance. However, the technology depends heavily on dataset labels, leading to high annotation costs. To alleviate this problem, we propose a novel efficient end-to-end hybrid consistency regularization algorithm (EI-HCR) for the semi-supervised semantic segmentation of RSI, wherein only a few labeled images and a large number of unlabeled images are effectively used. First, we devise data perturbation (DP) consistency regularization (CR), which includes a data mix-up method to combine unlabeled and labeled images. Then, we employ teacher and student networks to conduct model perturbation (MP) CR. Both segmentation results are regarded as pseudo-labels for each other. In the end, the semi-supervised loss is composed of DP and MP consistency loss, and supervises network training along with the fully supervised loss. More importantly, we first combine the characteristics of knowledge distillation to make the student network more lightweight, efficiently reducing the model inference time. Experimental results demonstrate the effectiveness of EI-HCR on the ISPRS Vaihingen and Massachusetts Buildings datasets. With only 5% of the labeled images, EI-HCR can achieve the same accuracy as the fully supervised training with 50% of the labeled images, and the number of student model parameters is only 9.64 M, indicating the method’s great advantages over other algorithms.

Robust Keypoint Detection and Matching on Fisheye Images by Self-Supervised Learning.

Accurate image feature point detection and matching are essential to computer vision tasks such as panoramic image stitching and 3D reconstruction. However, ordinary feature point approaches cannot be directly applied to fisheye images due to their large distortion, which makes the ordinary camera model unable to adapt. To address such a problem, this paper proposes a self-supervised learning method for feature point detection and matching on fisheye images. This method utilizes a Siamese network to automatically learn the correspondence of feature points across transformed image pairs to avoid high annotation costs. Due to the scarcity of the fisheye image dataset, a two-stage viewpoint transform pipeline is also adopted for image augmentation to increase the data variety. Furthermore, this method adopts both deformable convolution and contrastive learning loss to improve the feature extraction and description of distorted image regions. Compared with traditional feature point detectors and matchers, this method has been demonstrated with superior performance on fisheye images.

Unsupervised domain adaptation using modified cycle generative adversarial network for aerial image classification

The unsupervised domain adaptation (UDA) of aerial image classification is of great significance in the traffic management and public safety applications. Due to the differences in the distribution of source and target domain datasets from different sensors and cities, and the acquisition difficulty and high annotation cost of target domain datasets, existing domain adaptive methods have difficulty obtaining high accuracy and stability. In this work, a modified cycle generative adversarial network (CycleGAN) (MGAN) based on the multiscale residual block (MSRB) and multipooling coordinate attention (MPCA) modules is proposed to generate fake images, which is the most critical stage of the UDA method. In the MGAN, the MSRB module can extract the multiscale information of the image, and the MPCA module can fuse the position and channel information in the image. These two modules better extract the features of edges, regions and semantic classes, which makes the generated fake images better retain the information of the source domain and possess the style of the target domain. Otherwise, the Deeplabv3+ model is used to obtain a pretrained and final segmentation model in the first and third stages. In the experiment, the proposed method is validated on the aerial images of the Potsdam and Vaihingen datasets in the International Society for Photogrammetry and Remote Sensing (ISPRS) two-dimensional semantic labeling benchmark and improves overall accuracy from 52% to 60% on the segmentation of the target domain, compared with the existing optimal method; it also achieves an accuracy of 77% for buildings. The results demonstrate that the proposed method can achieve a higher accuracy and stability.

Semi-supervised cross-modal hashing via modality-specific and cross-modal graph convolutional networks

Cross-modal hashing maps heterogeneous multimedia data into Hamming space for retrieving relevant samples across modalities, which has received great research interests due to its rapid retrieval and low storage cost. In real-world applications, due to high manual annotation cost of multi-media data, we can only make use of limited number of labeled data with rich unlabeled data. In recent years, several semi-supervised cross-modal hashing (SCH) methods have been presented. However, how to fully explore and jointly utilize the modality-specific (complementarity) and modality-shared (correlation) information for retrieval has not been well studied for existing SCH works. In this paper, we propose a novel SCH approach named Modality-specific and Cross-modal Graph Convolutional Networks (MCGCN). The network architecture contains two modality-specific channels and a cross-modal channel to learn modality-specific and shared representations for each modality, respectively. Graph convolutional network (GCN) is leveraged in these three channels to explore intra-modal and inter-modal similarity, and perform semantic information propagation from labeled data to unlabeled data. Modality-specific and shared representations for each modality are fused with attention scheme. To further reduce the modality gap, a discriminative model is designed, learning to classify the modality of representations, and network training is guided by adversarial scheme. Experiments on two widely used multi-modal datasets demonstrate MCGCN outperforms state-of-the-art semi-supervised/supervised cross-modal hashing methods.

Ranking products through online reviews considering the mass assignment of features based on BERT and [formula omitted]-rung orthopair fuzzy set theory

A product ranking method is an effective tool that can analyze a significant number of online product reviews to recommend suitable products to consumers. However, existing product ranking methods have two main limitations: (1) the high manual annotation costs and (2) the inability to express consumers’ purchasing decisions because the information is limited to a single feature of each product. To overcome the limitations, this paper proposes a novel product ranking method considering the mass assignment of features based on bidirectional encoder representations using transformers (BERT) and q-rung orthopair fuzzy set theory. First, BERT is adopted to identify sentiment orientations of online product reviews and product features from online product reviews. Subsequently, the product features are clustered into groups and the relative frequencies of product features are obtained. Second, the relative frequencies of product features are transformed into q-rung orthopair fuzzy numbers based on mass assignment theory. Third, the q-rung orthopair fuzzy numbers are aggregated by the q-rung orthopair fuzzy generalized weighted Heronian mean operator to rank the products. Finally, we implement the method using a case study of six different phones to verify its feasibility. Using the case study, we also perform comparisons and sensitivity analyses, which demonstrate the superiority of our method.

RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging.

AI-powered Medical Imaging has recently achieved enormous attention due to its ability to provide fast-paced healthcare diagnoses. However, it usually suffers from a lack of high-quality datasets due to high annotation cost, inter-observer variability, human annotator error, and errors in computer-generated labels. Deep learning models trained on noisy labelled datasets are sensitive to the noise type and lead to less generalization on the unseen samples. To address this challenge, we propose a Robust Stochastic Knowledge Distillation (RoS-KD) framework which mimics the notion of learning a topic from multiple sources to ensure deterrence in learning noisy information. More specifically, RoS-KD learns a smooth, well-informed, and robust student manifold by distilling knowledge from multiple teachers trained on overlapping subsets of training data. Our extensive experiments on popular medical imaging classification tasks (cardiopulmonary disease and lesion classification) using real-world datasets, show the performance benefit of RoS-KD, its ability to distill knowledge from many popular large networks (ResNet-50, DenseNet-121, MobileNet-V2) in a comparatively small network, and its robustness to adversarial attacks (PGD, FSGM). More specifically, RoS-KD achieves > 2% and > 4% improvement on F1-score for lesion classification and cardiopulmonary disease classification tasks, respectively, when the underlying student is ResNet-18 against recent competitive knowledge distillation baseline. Additionally, on cardiopulmonary disease classification task, RoS-KD outperforms most of the SOTA baselines by ~1% gain in AUC score.

Research on Application of Intelligent Corpus Annotation of Entity Extraction with Construction of Knowledge Graph

The purpose of this paper is to solve the problem of big data and small samples caused by the high manual annotation cost of a military corpus. The deep learning algorithm of entity extraction in the military field was organically combined with the method of bootstrapping loop iteration to complete a study on the application of intelligent corpus annotation of military field entities. With the experimental research showing that using a small number of military field entity corpus annotations for RoBERTa pretraining word vectors and BiLSTM-CRF models and based on the bootstrapping algorithm idea to complete 3 rounds of loop iterations and 10 rounds of cross-validation joint-voting model iterations, the best entity extraction model evaluation F value reached up to 91.5%. Finally, the 60M intelligent corpus annotation application testing was completed using the best model of iteration of this round, with a total of 178,177 sentences of military field corpus intelligently labeled, the number of entities that should be labeled reaching 417,734. Therefore, this is an efficient way of construction and evaluation of intelligent corpus annotation model in the military entity extraction field. The findings of this paper provide an effective way of how to complete the labeled corpus. The research serves as a first step for future research, for example, the construction of knowledge graphs and military intelligent Q&A.

LIMU-BERT

Deep learning greatly empowers Inertial Measurement Unit (IMU) sensors for a wide range of sensing applications. Most existing works require substantial amounts of wellcurated labeled data to train IMU-based sensing models, which incurs high annotation and training costs. Compared with labeled data, unlabeled IMU data are abundant and easily accessible. This article presents a novel representation learning model that can make use of unlabeled IMU data and extract generalized rather than task-specific features. With the representations learned via our model, task-specific models trained with limited labeled samples can achieve superior performances in typical IMU sensing applications, such as Human Activity Recognition (HAR).

Stream-based active learning with linear models

The proliferation of automated data collection schemes and the advances in sensorics are increasing the amount of data we are able to monitor in real-time. However, given the high annotation costs and the time required by quality inspections, data is often available in an unlabeled form. This is fostering the use of active learning for the development of soft sensors and predictive models. In production, instead of performing random inspections to obtain product information, labels are collected by evaluating the information content of the unlabeled data. Several query strategy frameworks for regression have been proposed in the literature but most of the focus has been dedicated to the static pool-based scenario. In this work, we propose a new strategy for the stream-based scenario, where instances are sequentially offered to the learner, which must instantaneously decide whether to perform the quality check to obtain the label or discard the instance. The approach is inspired by the optimal experimental design theory and the iterative aspect of the decision-making process is tackled by setting a threshold on the informativeness of the unlabeled data points. The proposed approach is evaluated using numerical simulations and the Tennessee Eastman Process simulator. The results confirm that selecting the examples suggested by the proposed algorithm allows for a faster reduction in the prediction error.

Active Mask-Box Scoring R-CNN for Sonar Image Instance Segmentation

Instance segmentation of sonar images is an effective method for underwater target recognition. However, the mismatch among positioning accuracy found by boxIoU and classification confidence, which is used as NMS score in current instance segmentation models; and the high annotation cost of sonar images, are two major problems in the task. To tackle these problems, in this paper, we present a novel instance segmentation method called Mask-Box Scoring R-CNN and embedded it in our proposed deep active learning framework. For the mismatch problem between boxIoU and NMS score, Mask-Box Scoring R-CNN uses a boxIoU head to predict the quality of the bounding boxes. We amend the non-maximum suppression (NMS) score predicted by BoxIoU to preserve high-quality bounding boxes in inference flow. To deal with the annotating problem, we propose a triplets-measure-based active learning (TBAL) method and a balanced-sampling method applicable for deep learning. The TBAL method evaluates the amount of information of unlabeled samples from the aspects of classification confidence, positioning accuracy, and mask quality. The balanced-sampling method selects hard samples from the dataset to train the model to improve performance. The experimental results show that Mask-Box Scoring R-CNN achieves improvements of 1% in boxAP and 1.3% boxAP on our sonar image dataset compared with Mask Scoring R-CNN and Mask R-CNN, respectively. The active learning framework with TBAL and balanced sampling can achieve a competitive performance with less labeled samples than other frameworks, which can better facilitate underwater target recognition.

Source-free unsupervised domain adaptation for cross-modality abdominal multi-organ segmentation

Domain adaptation is crucial for transferring the knowledge from the source labeled CT dataset to the target unlabeled MR dataset in abdominal multi-organ segmentation. Meanwhile, it is highly desirable to avoid the high annotation cost related to the target dataset and protect the source dataset privacy. Therefore, we propose an effective source-free unsupervised domain adaptation method for cross-modality abdominal multi-organ segmentation without source dataset access. The proposed framework comprises two stages. In the first stage, the feature map statistics-guided model adaptation combined with entropy minimization is developed to help the top segmentation network reliably segment the target images. The pseudo-labels output from the top segmentation network are used to guide the style compensation network to generate source-like images. The pseudo-labels output from the middle segmentation network is used to supervise the learning progress of the desired model (bottom segmentation network). In the second stage, the circular learning and pixel-adaptive mask refinement are used to further improve the desired model performance. With this approach, we achieved satisfactory abdominal multi-organ segmentation performance, outperforming the existing state-of-the-art domain adaptation methods. The proposed approach can be easily extended to situations in which target annotation data exist. With only one labeled MR volume, the performance can be leveled with that of supervised learning. Furthermore, the proposed approach is proven to be effective for source-free unsupervised domain adaptation in reverse direction.

Self-Supervised Learning Methods for Label-Efficient Dental Caries Classification.

High annotation costs are a substantial bottleneck in applying deep learning architectures to clinically relevant use cases, substantiating the need for algorithms to learn from unlabeled data. In this work, we propose employing self-supervised methods. To that end, we trained with three self-supervised algorithms on a large corpus of unlabeled dental images, which contained 38K bitewing radiographs (BWRs). We then applied the learned neural network representations on tooth-level dental caries classification, for which we utilized labels extracted from electronic health records (EHRs). Finally, a holdout test-set was established, which consisted of 343 BWRs and was annotated by three dental professionals and approved by a senior dentist. This test-set was used to evaluate the fine-tuned caries classification models. Our experimental results demonstrate the obtained gains by pretraining models using self-supervised algorithms. These include improved caries classification performance (6 p.p. increase in sensitivity) and, most importantly, improved label-efficiency. In other words, the resulting models can be fine-tuned using few labels (annotations). Our results show that using as few as 18 annotations can produce ≥45% sensitivity, which is comparable to human-level diagnostic performance. This study shows that self-supervision can provide gains in medical image analysis, particularly when obtaining labels is costly and expensive.

Semi-supervised segmentation of echocardiography videos via noise-resilient spatiotemporal semantic calibration and fusion.

We present a novel model for left ventricle endocardium segmentation from echocardiography video, which is of great significance in clinical practice and yet a challenging task due to (1) the severe speckle noise in echocardiography videos, (2) the irregular motion of pathological heart, and (3) the limited training data caused by high annotation cost. The proposed model has three compelling characteristics. First, we propose a novel adaptive spatiotemporal semantic calibration method to align the feature maps of consecutive frames, where the spatiotemporal correspondences are figured out based on feature maps instead of pixels, thereby mitigating the adverse effects of speckle noise in the calibration. Second, we further learn the importance of each feature map of neighbouring frames to the current frame from the temporal perspective so as to distinctively rather than uniformly harness the temporal information to tackle the irregular and anisotropic motions. Third, we integrate these techniques into the mean teacher semi-supervised architecture to leverage a large amount of unlabeled data to improve the segmentation accuracy. We extensively evaluate the proposed method on two public echocardiography video datasets (EchoNet-Dynamic and CAMUS), where the average dice coefficient on the left ventricular endocardium segmentation achieves 92.87% and 93.79%, respectively. Comparisons with state-of-the-art methods also demonstrate the effectiveness of the proposed method by achieving a better segmentation performance with a faster speed.

IDPS Signature Classification Based on Active Learning With Partial Supervision From Network Security Experts

As intrusion detection and prevention systems (IDPSs) grow in importance, the cost of managing signatures, which are pattern files of malicious communications, continues to increase. To ensure the optimal operation of an IDPS, network security experts need to classify the signatures generated over time according to their importance (low, medium, and high). Although machine learning approaches can be used to automatically classify signatures instead of human experts, there are several challenges to applying them, including (a) high annotation costs, (b) security incidents caused by classification errors, and (c) classification accuracy decreases due to domain shifts. To overcome these challenges, we propose a system based on active learning that collaborates with experts to periodically classify received signatures. The signatures are sorted by uncertainty sampling; some are transferred to experts, and the rest are automatically classified. The experts classify the transferred signatures and add them to the training dataset, and the classification model is retrained. After training, the new signatures that have not yet been labeled are classified. The proposed system executes this workflow each time it receives signatures. For evaluation purposes, a real dataset was collected monthly with the help of the experts. Experiments are conducted on this real dataset to evaluate the proposed system in a simulation case. An analysis is then performed by comparing several variants of the proposed system. The results show that the system with Monte Carlo dropout (MC-Dropout) performs best. We also show that this variation has two effects: it transfers more samples with “medium” importance to the experts and mitigates imbalances in the training dataset.

Deep Anomaly Generation: An Image Translation Approach of Synthesizing Abnormal Banded Chromosome Images

Advances in deep-learning-based pipelines have led to breakthroughs in a variety of microscopy image diagnostics. However, a sufficiently big training data set is usually difficult to obtain due to high annotation costs. In the case of banded chromosome images, the creation of big enough libraries is difficult for multiple pathologies due to the rarity of certain genetic disorders. Generative Adversarial Networks (GANs) have proven to be effective in generating synthetic images and extending training data sets. In our work, we implement a conditional adversarial network that allows generation of realistic single chromosome images following user-defined banding patterns. To this end, an image-to-image translation approach based on self- generated 2D chromosome segmentation label maps is used. Our validation shows promising results when synthesizing chromosomes with seen as well as unseen banding patterns. We believe that this approach can be exploited for data augmentation of chromosome data sets with structural abnormalities. Therefore, the proposed method could help to tackle medical image analysis problems such as data simulation, segmentation, detection, or classification in the field of cytogenetics.

When BERT meets Bilbo: a learning curve analysis of pretrained language model on disease classification

BackgroundNatural language processing (NLP) tasks in the health domain often deal with limited amount of labeled data due to high annotation costs and naturally rare observations. To compensate for the lack of training data, health NLP researchers often have to leverage knowledge and resources external to a task at hand. Recently, pretrained large-scale language models such as the Bidirectional Encoder Representations from Transformers (BERT) have been proven to be a powerful way of learning rich linguistic knowledge from massive unlabeled text and transferring that knowledge to downstream tasks. However, previous downstream tasks often used training data at such a large scale that is unlikely to obtain in the health domain. In this work, we aim to study whether BERT can still benefit downstream tasks when training data are relatively small in the context of health NLP.MethodWe conducted a learning curve analysis to study the behavior of BERT and baseline models as training data size increases. We observed the classification performance of these models on two disease diagnosis data sets, where some diseases are naturally rare and have very limited observations (fewer than 2 out of 10,000). The baselines included commonly used text classification models such as sparse and dense bag-of-words models, long short-term memory networks, and their variants that leveraged external knowledge. To obtain learning curves, we incremented the amount of training examples per disease from small to large, and measured the classification performance in macro-averaged F_{1} score.ResultsOn the task of classifying all diseases, the learning curves of BERT were consistently above all baselines, significantly outperforming them across the spectrum of training data sizes. But under extreme situations where only one or two training documents per disease were available, BERT was outperformed by linear classifiers with carefully engineered bag-of-words features.ConclusionAs long as the amount of training documents is not extremely few, fine-tuning a pretrained BERT model is a highly effective approach to health NLP tasks like disease classification. However, in extreme cases where each class has only one or two training documents and no more will be available, simple linear models using bag-of-words features shall be considered.