Cost Of Manual Labeling Research Articles

Event Graph Neural Network for Opinion Target Classification of Microblog Comments

Opinion target classification of microblog comments is one of the most important tasks for public opinion analysis about an event. Due to the high cost of manual labeling, opinion target classification is generally considered as a weak-supervised task. This article attempts to address the opinion target classification of microblog comments through an event graph convolution network (EventGCN) in a weak-supervised manner. Specifically, we take microblog contents and comments as document nodes, and construct an event graph with three typical relationships of event microblogs, including the co-occurrence relationship of event keywords extracted from microblogs, the reply relationship of comments, and the document similarity. Finally, under the supervision of a small number of labels, both word features and comment features can be represented well to complete the classification. The experimental results on two event microblog datasets show that EventGCN can significantly improve the classification performance compared with other baseline models.

Semi-supervised partial multi-label classification with low-rank and manifold constraints

• A novel method for simultaneous processing of redundant and empty annotation in multi-label classification scenarios. • L ion is a unified framework for both estimating latent labels and training prediction models, simultaneously. • The low-rank constraint could capture local label correlations for recovering latent labels more accurately. • The manifold constraint could spread the supervision over linked neighboring instances even for unlabeled ones. Multi-Label Classification (MLC), processing data annotated with multiple labels simultaneously, is usually based on the assumption that the instances are all correctly labeled with the ground-truth labels. However this assumption is often not satisfied in real-world scenarios due to the high cost of manual labeling. In this paper, we consider a classification scenario in which only a part of the instances are redundantly annotated, and call this scenario a Semi-supervised Partial Multi-Label Classification (SPMLC). We propose a novel method L ion (semi-supervised partial multi-label classification with Low-rank and manIfOld coNstraints) to handle this particular scenario. Especially, the not exactly known ground-truth labels are regarded as latent labels, and the prediction is achieved by estimating the confidences of latent labels ( i.e., the latent ground-truth confidences) in future data. L ion jointly learns the latent ground-truth confidences and the prediction model. The low-rank and manifold constraints are utilized to capture local label correlations and neighboring structure of instances, so as to accurately estimate the latent ground-truth confidences. Extensive experimental results validate the effectiveness of L ion .

Improved Fake Reviews Detection Model Based on Vertical Ensemble Tri-Training and Active Learning

People’s increasingly frequent online activity has generated a large number of reviews, whereas fake reviews can mislead users and harm their personal interests. In addition, it is not feasible to label reviews on a large scale because of the high cost of manual labeling. Therefore, to improve the detection performance by utilizing the unlabeled reviews, this article proposes a fake reviews detection model based on vertical ensemble tri-training and active learning (VETT-AL). The model combines the features of review text with the user behavior features as feature extraction. In the VETT-AL algorithm, the iterative process is divided into two parts: vertical integration within the group and horizontal integration among the groups. The intra-group integration is to integrate three original classifiers by using the previous iterative models of the classifiers. The inter-group integration is to adopt the active learning based on entropy to select the data with the highest confidence and label it, and as the result of that, the second generation classifiers are trained by the traditional process to improve the accuracy of the label. Experimental results show that the proposed model has a good classification performance.

Anomaly Detection Algorithm Based on Semi-Supervised Collaborative Strategy

This paper analyzes the existing credit card anomaly detection classification algorithms, summarizes the parts that can be improved, and proposes an outlier detection classification algorithm based on the unsupervised algorithm and active learning decision trees. Since the 1990s, machine learning technology has been widely used in the field of credit card fraud detection. Among them, the supervised learning expert system used for classification tends to perform better than the unsupervised learning model. The good performance of supervised learning methods requires high learning costs. The training process is usually serial or partially parallel. Therefore, the computing power, time, and manual labeling cost for learning need to be considered as algorithm selection factors. However, manual labeling usually involves the counter or ATM sending information to the back-end large-value transaction authorization department for manual review, and then the employee judges whether it is an abnormal transaction based on experience. This rigorous process will produce a very large time delay. Combining this feature, this article selects the appropriate unsupervised outlier detection method, selects part of the training data, and uses a small part of the more valuable data for annotation learning. Experiments show that this method can improve the accuracy while saving the time cost of training. Compared with the training time, the added time of classification is negligible.

HAL: Hybrid active learning for efficient labeling in medical domain

The success of the deep convolutional neural networks in computer vision tasks mainly relies on massive labeled training data. However, in the field of medical images, it is difficult to construct large labeled datasets since the labeling of medical images is time-consuming, labor-intensive, and medical expertise demanded. To meet the challenge, we propose a hybrid active learning framework HAL for efficient labeling in the medical domain, which integrates active learning into deep learning to reduce the cost of manual labeling and take the advantages of deep neural networks. The proposed HAL utilizes a hybrid sampling strategy considering both sample diversity and prediction loss simultaneously. The effectiveness and efficiency of proposed HAL are validated on three medical image datasets. The experimental results show that the proposed HAL outperforms several state-of-the-art active learning methods. On the Hyper-Kvasir Dataset, with only 10% of the labels, the HAL achieves 95% performance of the deep learning method trained on the entire dataset. The quantitative and qualitative analysis proves that HAL can greatly reduce the number of labels needed for training a deep neural network, which is robust to address efficient labeling problems even with imbalanced data distribution.

Tracking Based Semi-Automatic Annotation for Scene Text Videos

Recently, video scene text detection has received increasing attention due to its comprehensive applications. However, the lack of annotated scene text video datasets has become one of the most important problems, which hinders the development of video scene text detection. The existing scene text video datasets are not large-scale due to the expensive cost caused by manual labeling. In addition, the text instances in these datasets are too clear to be a challenge. To address the above issues, we propose a tracking based semi-automatic labeling strategy for scene text videos in this paper. We get semi-automatic scene text annotation by labeling manually for the first frame and tracking automatically for the subsequent frames, which avoid the huge cost of manual labeling. Moreover, a paired low-quality scene text video dataset named Text-RBL is proposed, consisting of raw videos, blurry videos, and low-resolution videos, labeled by the proposed convenient semi-automatic labeling strategy. Through an averaging operation and bicubic down-sampling operation over the raw videos, we can efficiently obtain blurry videos and low-resolution videos paired with raw videos separately. To verify the effectiveness of Text-RBL, we propose a baseline model combined with the text detector and tracker for video scene text detection. Moreover, a failure detection scheme is designed to alleviate the baseline model drift issue caused by complex scenes. Extensive experiments demonstrate that Text-RBL with paired low-quality videos labeled by the semi-automatic method can significantly improve the performance of the text detector in low-quality scenes.

Multitask Learning for Ship Detection From Synthetic Aperture Radar Images

Ship detection from synthetic aperture radar (SAR) images is inherently subject to the special imaging mechanism of SAR. In recent years, deep-learning-based techniques for detecting objects from optical images have rapidly advanced and promoted the development of SAR image detection technology. However, the strong speckle noise in SAR images degrades low-level feature learning in shallow layers, hindering the higher level learning of semantic features for object detection. In view of the problems encountered in direct end-to-end feature learning for object detection and the close relationship between objects and auxiliary cues, a multitask learning-based object detector (MTL-Det) is proposed in this article to distinguish ships in SAR images. The proposed approach models the ship detection problem, not as a single object detection task, but as three cooperative tasks. The model involves two auxiliary subtasks that are focused on learning object-specific cues ( e.g ., texture and shape) for the ship detection task, which is constrained by the pseudoground truth generated by the main task. Assisted by auxiliary subtasks, the low-level features are robust to speckle noise and reliably support high-level feature learning. Compared with traditional single-task-based object detectors, more discriminative object-specific features are learned by multitask learning without the extra cost of manual labeling. The experiments conducted in this study help demonstrate the advantages of MTL-Det in improving the ship detection performance on two SAR datasets: high-resolution SAR images dataset and large-scale SAR ship detection dataset-v1.0.

Fault Judgment of Transmission Cable Based on Multi-Channel Data Fusion and Transfer Learning

Non-intrusive transmission cable monitoring is the latest advanced measurement technology for smart grids. It only samples the voltage on a certain part of the transmission cable, and uses intelligent algorithms to identify the quality, which has obvious advantages of low construction and maintenance costs. This paper established a model based on multi-channel data fusion and transfer learning to classify the quality of transmission cable. First, we used the ANSYS Maxwell simulation platform to obtain ten kinds of specific fault data, which solved the time cost of manual labeling. Then, we performed multi-channel data fusion on the original data, which strengthened the expression of important features and was more conducive to the training of the model. Next, we used Depthwise Separable Convolution (DSC) to speed up the learning of the model, and improve the accuracy of the classification. Finally, we transferred the model trained with simulation data into the real scene, realized the transfer from multi classes to two classes, the effectiveness was proved in experiments. The accuracy of the model built in the article to classify the quality of the transmission cables is 98.1%.

Research on Bearing Fault Diagnosis Method Based on an Adaptive Anti-Noise Network under Long Time Series.

In the era of big data, longer time series fault signals will not only be easy to copy and store, but also reduce the labor cost of manual labeling, which can better meet the needs of industrial big data. Aiming to effectively extract the key classification information from a longer time series of bearing vibration signals and achieve high diagnostic accuracy under noise and different load conditions. The one-dimensional adaptive long sequence convolutional network (ALSCN) is proposed. ALSCN can better extract features directly from high-dimensional original signals without manually extracting features and relying on expert knowledge. By adding two improved multi-scale modules, ALSCN can not only extract important features efficiently from noise signals, but also alleviate the problem of losing key information due to continuous down-sampling. Moreover, a Bayesian optimization algorithm is constructed to automatically find the best combination of hyperparameters in ALSCN. Based on two bearing data sets, the model is compared with traditional model such as SVM and deep learning models such as convolutional neural networks (CNN) et al. The results prove that ALSCN has a higher diagnostic accuracy rate on 5120-dimensional sequences under −5 signal to noise ratio (SNR) with better generalization.

A semi-supervised approach for extracting TCM clinical terms based on feature words

BackgroundA semi-supervised model is proposed for extracting clinical terms of Traditional Chinese Medicine using feature words.MethodsThe extraction model is based on BiLSTM-CRF and combined with semi-supervised learning and feature word set, which reduces the cost of manual annotation and leverage extraction results.ResultsExperiment results show that the proposed model improves the extraction of five types of TCM clinical terms, including traditional Chinese medicine, symptoms, patterns, diseases and formulas. The best F1-value of the experiment reaches 78.70% on the test dataset.ConclusionsThis method can reduce the cost of manual labeling and improve the result in the NER research of TCM clinical terms.

Crowd-Assisted Disaster Scene Assessment with Human-AI Interactive Attention

The recent advances of mobile sensing and artificial intelligence (AI) have brought new revolutions in disaster response applications. One example is disaster scene assessment (DSA) which leverages computer vision techniques to assess the level of damage severity of the disaster events from images provided by eyewitnesses on social media. The assessment results are critical in prioritizing the rescue operations of the response teams. While AI algorithms can significantly reduce the detection time and manual labeling cost in such applications, their performance often falls short of the desired accuracy. Our work is motivated by the emergence of crowdsourcing platforms (e.g., Amazon Mechanic Turk, Waze) that provide unprecedented opportunities for acquiring human intelligence for AI applications. In this paper, we develop an interactive Disaster Scene Assessment (iDSA) scheme that allows AI algorithms to directly interact with humans to identify the salient regions of the disaster images in DSA applications. We also develop new incentive designs and active learning techniques to ensure reliable, timely, and cost-efficient responses from the crowdsourcing platforms. Our evaluation results on real-world case studies during Nepal and Ecuador earthquake events demonstrate that iDSA can significantly outperform state-of-the-art baselines in accurately assessing the damage of disaster scenes.

Aceso: PICO-guided Evidence Summarization on Medical Literature.

Evidence-Based Medicine (EBM) aims to apply the best available evidence gained from scientific methods to clinical decision making. A generally accepted criterion to formulate evidence is to use the PICO framework, where PICO stands for Problem/Population, Intervention, Comparison, and Outcome. Automatic extraction of PICO-related sentences from medical literature is crucial to the success of many EBM applications. In this work, we present our Aceso system, which automatically generates PICO-based evidence summaries from medical literature. In Aceso 1, we adopt an active learning paradigm, which helps to minimize the cost of manual labeling and to optimize the quality of summarization with limited labeled data. An UMLS2Vec model is proposed to learn a vector representation of medical concepts in UMLS 2, and we fuse the embedding of medical knowledge with textual features in summarization. The evaluation shows that our approach is better on identifying PICO sentences against state-of-the-art studies and outperforms baseline methods on producing high-quality evidence summaries.

Semi-Supervised Hyperspectral Image Classification via Spatial-Regulated Self-Training

Because there are many unlabeled samples in hyperspectral images and the cost of manual labeling is high, this paper adopts semi-supervised learning method to make full use of many unlabeled samples. In addition, those hyperspectral images contain much spectral information and the convolutional neural networks have great ability in representation learning. This paper proposes a novel semi-supervised hyperspectral image classification framework which utilizes self-training to gradually assign highly confident pseudo labels to unlabeled samples by clustering and employs spatial constraints to regulate self-training process. Spatial constraints are introduced to exploit the spatial consistency within the image to correct and re-assign the mistakenly classified pseudo labels. Through the process of self-training, the sample points of high confidence are gradually increase, and they are added to the corresponding semantic classes, which makes semantic constraints gradually enhanced. At the same time, the increase in high confidence pseudo labels also contributes to regional consistency within hyperspectral images, which highlights the role of spatial constraints and improves the HSIc efficiency. Extensive experiments in HSIc demonstrate the effectiveness, robustness, and high accuracy of our approach.

Data Augmentation Based on Wasserstein Generative Adversarial Nets Under Few Samples

Aiming at the problem of low accuracy of image classification under the condition of few samples, an improved method based on Wasserstein Generative Adversarial Nets is proposed. The small data sets are augmented by generating target samples through minimax game between the generator and the discriminator. Firstly, label information is introduced into the network to save the cost of manual labeling for the generated images, and then classifier loss is introduced to further improve the quality of generated images, and the classification accuracy is improved after expanding the data set.

Exploiting Typicality for Selecting Informative and Anomalous Samples in Videos.

In this paper, we present a novel approach to find informative and anomalous samples in videos exploiting the concept of typicality from information theory. In most video analysis tasks, selection of the most informative samples from a huge pool of training data in order to learn a good recognition model is an important problem. Furthermore, it is also useful to reduce the annotation cost as it is time-consuming to annotate unlabeled samples. Typicality is a simple and powerful technique which can be applied to compress the training data to learn a good classification model. In a continuous video clip, an activity shares a strong correlation with its previous activities. We assume that the activity samples that appear in a video form a Markov chain. We explicitly show how typicality can be utilized in this scenario. We compute an atypical score for a sample using typicality and the Markovian property, which can be applied to two challenging vision problems-(a) sample selection for learning activity recognition models, and (b) anomaly detection. In the first case, our approach leads to a significant reduction of manual labeling cost while achieving similar or better recognition performance compared to a model trained with the entire training set. For the latter case, the atypical score has been exploited in identifying anomalous activities in videos where our results demonstrate the effectiveness of the proposed framework over other recent strategies.

SemTra: A semi-supervised approach to traffic flow labeling with minimal human effort

Network traffic classification is an essential component for service differentiation, network design and management and security systems. The limitations of traditional port-based and payload methods have been addressed by recent promising studies which rely on the analysis of the statistics of traffic flows and the use of machine learning techniques. However, due to the high cost of manual labeling, it is hard to obtain sufficient, reliable and up-to-date labeled data for effective IP traffic classification. This paper proposes a novel semi-supervised approach, called SemTra, which automatically alleviates the shortage of labeled flows for machine learning by exploiting the advantages of both supervised and unsupervised models. In particular, SemTra involves the following: (i) generating multi-view representations of the original data based on dimensionality reduction methods to have strong discrimination ability, (ii) incorporating the generated representations into the ensemble clustering model to provide a combined clustering output with better quality and stability, (iii) adapting the concept of self-training to iteratively utilize the few labeled data along with unlabeled within local and global viewpoints; and (iiii) obtaining the final class decision by combining the decisions of mapping strategy of clusters, the local self-training and global self-training approaches. Extensive experiments were carried out to compare the effectiveness of SemTra over representative semi-supervised methods using sixteen network traffic datasets.

Adverse event detection by integrating twitter data and VAERS

BackgroundVaccine has been one of the most successful public health interventions to date. However, vaccines are pharmaceutical products that carry risks so that many adverse events (AEs) are reported after receiving vaccines. Traditional adverse event reporting systems suffer from several crucial challenges including poor timeliness. This motivates increasing social media-based detection systems, which demonstrate successful capability to capture timely and prevalent disease information. Despite these advantages, social media-based AE detection suffers from serious challenges such as labor-intensive labeling and class imbalance of the training data.ResultsTo tackle both challenges from traditional reporting systems and social media, we exploit their complementary strength and develop a combinatorial classification approach by integrating Twitter data and the Vaccine Adverse Event Reporting System (VAERS) information aiming to identify potential AEs after influenza vaccine. Specifically, we combine formal reports which have accurately predefined labels with social media data to reduce the cost of manual labeling; in order to combat the class imbalance problem, a max-rule based multi-instance learning method is proposed to bias positive users. Various experiments were conducted to validate our model compared with other baselines. We observed that (1) multi-instance learning methods outperformed baselines when only Twitter data were used; (2) formal reports helped improve the performance metrics of our multi-instance learning methods consistently while affecting the performance of other baselines negatively; (3) the effect of formal reports was more obvious when the training size was smaller. Case studies show that our model labeled users and tweets accurately.ConclusionsWe have developed a framework to detect vaccine AEs by combining formal reports with social media data. We demonstrate the power of formal reports on the performance improvement of AE detection when the amount of social media data was small. Various experiments and case studies show the effectiveness of our model.

Protein Function Prediction Based on Active Semi‐supervised Learning

In our study, the active learning and semisupervised learning methods are comprehensively used for label delivery of proteins with known functions in Proteinprotein interaction (PPI) network so as to predict the functions of unknown proteins. Because the real PPI network is generally observed with overlapping protein nodes with multiple functions, the mislabeling of overlapping protein may result in accumulation of prediction errors. For this reason, prior to executing the label delivery process of semi-supervised learning, the adjacency matrix is used to detect overlapping proteins. As the topological structure description of interactive relation between proteins, PPI network is observed with party hub protein nodes that play an important role, in co-expression with its neighborhood. Therefore, to reduce the manual labeling cost, party hub proteins most beneficial for improvement of prediction accuracy are selected for class labeling and the labeled party hub proteins are added into the labeled sample set for semisupervised learning later. As the experimental results of real yeast PPI network show, the proposed algorithm can achieve high prediction accuracy with few labeled samples.

Word Sense Disambiguation by Learning Decision Trees from Unlabeled Data

In this paper we describe a machine learning approach to word sense disambiguation that uses unlabeled data. Our method is based on selective sampling with committees of decision trees. The committee members are trained on a small set of labeled examples which are then augmented by a large number of unlabeled examples. Using unlabeled examples is important because obtaining labeled data is expensive and time-consuming while it is easy and inexpensive to collect a large number of unlabeled examples. The idea behind this approach is that the labels of unlabeled examples can be estimated by using committees. Using additional unlabeled examples, therefore, improves the performance of word sense disambiguation and minimizes the cost of manual labeling. Effectiveness of this approach was examined on a raw corpus of one million words. Using unlabeled data, we achieved an accuracy improvement up to 20.2%.