Proportion Of Label Research Articles

Multi-Class Learning from Label Proportions for Bank Customer Classification

In this paper, we study multi-class learning from label proportions, and apply it to bank customer classification, in order to provide advice for banks to better manage customer relationships. We attempt to apply the multi-class extreme learning machine (ELM) to learning from label proportions (LLP). With a structure similar to neural network, ELM possesses higher computational speed and better generalization ability. As a result, it is suitable to deal with large-scale and multi-class problems. LLP is a learning problem, which classifies training data into bags where only the label proportions of each class in each bag is available. Furthermore, in order to maintain the stable model accuracy when the bag sizes increasing, we manage to add small number of labeled samples in our model, called LLP-ELM with in a semi-supervised learning framework. The experiments prove that our improvement has advantages in the case of large bag sizes. In practical, it is worth to consider to apply the proposed algorithm to multi-class learning from label proportions for bank customer classification and other multi-class scenarios.

Non-negative Matrix Factorization Based Learning from Label Proportions for Vehicle Loan Default Detection

In traditional supervised learning, the model is trained through a large number of known labels. In reality, the amount of data increases rapidly, and the cost of manual annotation is high. Besides, under the restriction of privacy protection, it is infeasible to obtain labels of a large number of samples. However, the proportion of samples in a certain class can be easily obtained. Thanks to label proportional learning, we can obtain the instance-level classifier merely based on the proportional information. In addition to this, label proportional learning with minor instance-level labeling can potentially improve the accuracy of the model. Hence, in this paper, we propose an improved model based on non-negative matrix factorization to solve semi-supervised label proportional learning, called semi-supervised proportion Matrix Factorization (SPMF). In order to effectively predict car default customers, our method leverage a small number of known labels and the remaining sample label proportional information to construct a classification model. In our experiment, customers defaults can be predicted and high prediction accuracy can be guaranteed. This study is potentially useful for Internet auto financial institutions to properly avoid default risks and establish a more reliable credit rating mechanism. Particularly, our approach provides an alternative way to effectively solve default customer identification under the privacy protection constraint.

A Study on Mobile Customer Churn Based on Learning from Soft Label Proportions

In real-life scenarios, manual labeling is costly and inefficient. besides sometimes there is privacy restrictions on the user’s information, which makes it difficult to detect customers losses in the mobile customer field. We cannot directly obtain information about a user is lost or not. However, the business manager can estimate the proportion of lost users based on his business experience. Besides, considering that the proportion information we obtain in reality is estimated and not completely accurate, we propose a soft label proportional learning method based on the learning from label proportions (LLP). In detail, we modify proportional loss term in the loss function by introducing a slack variable. In this way, the proportional loss within a certain range can be regarded as zero when the estimated proportional information is uncertain. The proportional information can be determined in a range by adjusting the parameter. The experimental results show the effectiveness of the proposed method, which provides a new way to effectively solve the problem of customer churn analysis in the field of mobile communication.

Pol-SAR Based Oil Spillage Classification With Various Scenarios of Prior Knowledge

Oil slicks from ships or oil platforms cause serious damage to the marine and coastal environment and ecosystems. To monitor such spill events, fully polarimetric (Pol-SAR) synthetic aperture radar (SAR) has been widely used in improving oil spillage detection performance of the image segmentation methods and machine learning methods. A key factor that greatly affects the performance of the existing methods is the use of prior knowledge, such as the manual labels of oil slicks and clean seawater. In some scenarios, a large number of trustworthy manual labels of oil slicks have been accumulated. In other scenarios, however, there are few manual labels. To efficiently utilize the manual labels in various scenarios, we propose an oil spillage identification method, named RampOC. We design cost-sensitive risk minimization model to tackle the problem of imbalance of oil slicks and seawater, employ the ramp loss function to deal with various cases of the manual labels, and design a kernel-based CCCP algorithm to train a classifier by solving a series of convex dual programming. The trained classifier is applicable to various cases, including the normal case that a certain number of the labels of oil slicks are labeled, the case that a large proportion of labels have gone missing or incorrect, and the case that no labels of oil slicks are known. In all of these cases, RampOC has proven to be effective to detect the ocean oil spillage regions. All the MATLAB source codes could be freely downloaded from https://github.com/Isaac-QiXing/rampOC.

A classification algorithm based on weighted ML-kNN for multi-label data

The ML-kNN algorithm uses naive Bayesian classification to modify the traditional kNN algorithm to solve multi-label classification problems. However, the ML-kNN algorithm is prone to misjudgement or incomplete judgment of the unseen instance's label set in two special cases: when the number of labels in the training set is not balanced and when the training instances are unevenly distributed in space. Therefore, a weighted ML-kNN algorithm (i.e., wML-kNN) is proposed in this paper. The main idea is to assign different weights to each label according to the proportion of labels and mutual information of the spatial distribution of unseen instances to training instances. This method can reduce the probability of misjudgement of the unseen instance's label set. A comparative study was conducted on four multi-label datasets that included review classification and three other published benchmark multi-label datasets: yeast gene function analysis, natural scene classification, and musical sentiment classification. The results show that the performance of the wML-kNN algorithm is better than the other four multi-label learning algorithms, including ML-kNN.

Bag-Level Aggregation for Multiple-Instance Active Learning in Instance Classification Problems.

A growing number of applications, e.g., video surveillance and medical image analysis, require training recognition systems from large amounts of weakly annotated data, while some targeted interactions with a domain expert are allowed to improve the training process. In such cases, active learning (AL) can reduce labeling costs for training a classifier by querying the expert to provide the labels of most informative instances. This paper focuses on AL methods for instance classification problems in multiple instance learning (MIL), where data are arranged into sets, called bags, which are weakly labeled. Most AL methods focus on single-instance learning problems. These methods are not suitable for MIL problems because they cannot account for the bag structure of data. In this paper, new methods for bag-level aggregation of instance informativeness are proposed for multiple instance AL (MIAL). The aggregated informativeness method identifies the most informative instances based on classifier uncertainty and queries bags incorporating the most information. The other proposed method, called cluster-based aggregative sampling, clusters data hierarchically in the instance space. The informativeness of instances is assessed by considering bag labels, inferred instance labels, and the proportion of labels that remain to be discovered in clusters. Both proposed methods significantly outperform reference methods in extensive experiments using benchmark data from several application domains. Results indicate that using an appropriate strategy to address MIAL problems yields a significant reduction in the number of queries needed to achieve the same level of performance as single-instance AL methods.

Weakly Supervised Classification For High Energy Physics

We present a classification algorithm that applies the machine learning paradigm of Learning from Label Proportions (LLP) [1] to enable learning on unlabelled data. Our algorithm, Weakly Supervised Classification, receives as its only input the class proportions of batches of data but makes per-instance classification decisions matching the performance of fully supervised approaches. We apply our model to the problem of Quark-Gluon tagging and show that it is robust to underlying mismodelling of the simulated data unlike fully supervised learning.

Unsupervised Learning for Brain-Computer Interfaces Based on Event-Related Potentials: Review and Online Comparison [Research Frontier

One of the fundamental challenges in brain-computer interfaces (BCIs) is to tune a brain signal decoder to reliably detect a user's intention. While information about the decoder can partially be transferred between subjects or sessions, optimal decoding performance can only be reached with novel data from the current session. Thus, it is preferable to learn from unlabeled data gained from the actual usage of the BCI application instead of conducting a calibration recording prior to BCI usage. We review such unsupervised machine learning methods for BCIs based on event-related potentials of the electroencephalogram. We present results of an online study with twelve healthy participants controlling a visual speller. Online performance is reported for three completely unsupervised learning methods: (1) learning from label proportions, (2) an expectation-maximization approach and (3) MIX, which combines the strengths of the two other methods. After a short ramp-up, we observed that the MIX method not only defeats its two unsupervised competitors but even performs on par with a state-of-the-art regularized linear discriminant analysis trained on the same number of data points and with full label access. With this online study, we deliver the best possible proof in BCI that an unsupervised decoding method can in practice render a supervised method unnecessary. This is possible despite skipping the calibration, without losing much performance and with the prospect of continuous improvement over a session. Thus, our findings pave the way for a transition from supervised to unsupervised learning methods in BCIs based on eventrelated potentials.

SyMIL: MinMax Latent SVM for Weakly Labeled Data.

Designing powerful models able to handle weakly labeled data are a crucial problem in machine learning. In this paper, we propose a new multiple instance learning (MIL) framework. Examples are represented as bags of instances, but we depart from standard MIL assumptions by introducing a symmetric strategy (SyMIL) that seeks discriminative instances in positive and negative bags. The idea is to use the instance the most distant from the hyper-plan to classify the bag. We provide a theoretical analysis featuring the generalization properties of our model. We derive a large margin formulation of our problem, which is cast as a difference of convex functions, and optimized using concave-convex procedure. We provide a primal version optimizing with stochastic subgradient descent and a dual version optimizing with one-slack cutting-plane. Successful experimental results are reported on standard MIL and weakly supervised object detection data sets: SyMIL significantly outperforms competitive methods (mi/MI/Latent-SVM), and gives very competitive performance compared to state-of-the-art works. We also analyze the selected instances of symmetric and asymmetric approaches on weakly supervised object detection and text classification tasks. Finally, we show complementarity of SyMIL with recent works on learning with label proportions on standard MIL data sets.

Learning from label proportions on high-dimensional data

Learning from label proportions (LLP), in which the training data is in the form of bags and only the proportion of each class in each bag is available, has attracted wide interest in machine learning. However, how to solve high-dimensional LLP problem is still a challenging task. In this paper, we propose a novel algorithm called learning from label proportions based on random forests (LLP-RF), which has the advantage of dealing with high-dimensional LLP problem. First, by defining the hidden class labels inside target bags as random variables, we formulate a robust loss function based on random forests and take the corresponding proportion information into LLP-RF by penalizing the difference between the ground truth and estimated label proportion. Second, a simple but efficient alternating annealing method is employed to solve the corresponding optimization model. At last, various experiments demonstrate that our algorithm can obtain the best accuracies on high-dimensional data compared with several recently developed methods.

Learning from noisy label proportions for classifying online social data

Inferring latent attributes (e.g., demographics) of social media users is important to improve the accuracy and validity of social media analysis methods. While most existing approaches use either heuristics or supervised classification, recent work has shown that accurate classification models can be trained using supervision from population statistics. These learning with label proportion (LLP) models are fit on bags of instances and then applied to individual accounts. However, it is well known that many social media sites such as Twitter are not a representative sample of the population; thus, there are many sources of noise in these label proportions (e.g., sampling bias). This can in turn degrade the quality of the resulting model. In this paper, we investigate classification algorithms that use population statistical constraints such as demographics, names, and social network followers to fit classifiers to predict individual user attributes. We propose LLP methods that explicitly model the noise inherent in these label proportions. On several real and synthetic datasets, we find that combining these enhancements together can significantly reduce averaged classification error by 7%, resulting in methods that are robust to noise in the provided label proportions.

Learning from label proportions with pinball loss

Learning from label proportions is a new kind of learning problem which has drawn much attention in recent years. Different from the well-known supervised learning, it considers instances in bags and uses the label proportion of each bag instead of instance. As obtaining the instance label is not always feasible, it has been widely used in areas like modeling voting behaviors and spam filtering. However, learning from label proportions still suffers great challenges due to the inference of noise, the improper partition of bags and so on. In this paper, we propose a novel learning from label proportions method based on pinball loss, called “pSVM-pin”, to address the above issues. The pinball loss is introduced to generate an effective classifier in order to eliminate the impact of noise. Experimental results prove the precision of pSVM-pin compared with competing methods.

Adaboost-LLP: A Boosting Method for Learning With Label Proportions.

How to solve the classification problem with only label proportions has recently drawn increasing attention in the machine learning field. In this paper, we propose an ensemble learning strategy to deal with the learning problem with label proportions (LLP). In detail, we first give a loss function based on different weights for LLP, and then construct the corresponding weak classifier, at the same time, estimate its conditional probabilities by a standard logistic function. At last, by introducing the maximum likelihood estimation, we propose a new anyboost learning system for LLP (called Adaboost-LLP). Unlike traditional methods, our method does not make any restrictive assumptions on training set; at the same time, compared with alter- SVM, Adaboost-LLP exploits more extra weight information and uses multiple weak classifiers that can be solved efficiently to combine a strong classifier. All experiments show that our method outperforms the existing methods in both accuracy and training time.

Learning from label proportions for SAR image classification

Synthetic aperture radar (SAR) image classification plays a key role in SAR interpretation. Due to the cost and difficulty of truth labeling for SAR images, the newly labeled samples available for image classification are very limited. This paper focuses on defining a new sample labeling method to solve the problem of truth acquisition for training data in SAR image classification. An efficient classification framework for high-resolution SAR images is presented in this paper, which is built on learning from uncertain labels. We use grid labeling for rapid training data acquisition by assigning a label to a group of neighboring pixels at a time. A novel SVM-based learning model is proposed to optimize the uncertain training data within the constraints of label proportions in each group and then predict the label of each sample for the test data based on the optimized training set. This work intends to explore a rapid labeling method called grid labeling for efficient training set definition and apply it to large-scale SAR image classification. The model demonstrates good performance in both accuracy and efficiency for scene interpretation of high-resolution SAR images.

Chronic fluoxetine rescues changes in plasma membrane density of 5-HT1A autoreceptors and serotonin transporters in the olfactory bulbectomy rodent model of depression

Reduced serotonin (5-HT) neurotransmission is postulated to underlie the pathogenesis of depression. The serotonin transporter (SERT) and 5-HT1A auto-receptors act in concert to ensure homeostasis of serotonin (5-HT) neurotransmission and regulation of their cell surface expression represent efficient mechanisms to maintain this homeostasis. Thus, we investigated the changes in the subcellular distribution of SERT and 5-HT1A receptors (5-HT1AR) in the rat olfactory bulbectomy model of depression using immuno-gold labeling and electron microscopy, and examined the effect of chronic treatment with the antidepressant, fluoxetine, a serotonin reuptake inhibitor, on the subcellular distribution of SERT and 5-HT1AR. The density of plasma membrane labeling of 5-HT1A auto-receptors on dendrites of dorsal raphe neurons was increased after bulbectomy, but the 5-HT1A hetero-receptor membrane labeling on dendrites of CA3 hippocampal neurons was not. The density of membrane labeling of SERTs was increased both in dendrites of dorsal raphe neuron and axon terminals in the hippocampus after bulbectomy. However, the proportion of 5-HT1AR and SERT membrane labeling relative to total labeling was unchanged, suggesting an increase in protein levels. The increases in 5-HT1AR and SERTs membrane labeling induced by bulbectomy were reversed by chronic fluoxetine treatment, and these changes were associated with a reduction in the relative proportion of membrane versus total labeling, consistent with a protein shift between subcellular compartments. Our findings support the hypothesis that changes in efficacy of serotonergic neurotransmission in this model of depression depends on both activity and density of cell surface-expressed SERT and 5-HT1A auto-receptors.

Melatonin Inhibits Neural Cell Apoptosis and Promotes Locomotor Recovery via Activation of the Wnt/β-Catenin Signaling Pathway After Spinal Cord Injury.

The spinal cord is highly sensitive to spinal cord injury (SCI) by external mechanical damage, resulting in irreversible neurological damage. Activation of the Wnt/β-catenin signaling pathway can effectively reduce apoptosis and protect against SCI. Melatonin, an indoleamine originally isolated from bovine pineal tissue, exerts neuroprotective effects after SCI through activation of the Wnt/β-catenin signaling pathway. In this study, we demonstrated that melatonin exhibited neuroprotective effects on neuronal apoptosis and supported functional recovery in a rat SCI model by activating the Wnt/β-catenin signaling pathway. We found that melatonin administration after SCI significantly upregulated the expression of low-density lipoprotein receptor related protein 6 phosphorylation (p-LRP-6), lymphoid enhancer factor-1 (LEF-1) and β-catenin protein in the spinal cord. Melatonin enhanced motor neuronal survival in the spinal cord ventral horn and improved the locomotor functions of rats after SCI. Melatonin administration after SCI also reduced the expression levels of Bax and cleaved caspase-3 in the spinal cord and the proportion of terminal deoxynucleotidyl transferase-mediated deoxyuridine triphosphate nick end labeling (TUNEL) positive cells, but increased the expression level of Bcl-2. These results suggest that melatonin attenuated SCI by activating the Wnt/β-catenin signaling pathway.

Learning from label proportions in brain-computer interfaces: Online unsupervised learning with guarantees.

ObjectiveUsing traditional approaches, a brain-computer interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g., by subject-to-subject transfer of a pre-trained classifier or unsupervised adaptive classification methods which learn from scratch and adapt over time. While such heuristics work well in practice, none of them can provide theoretical guarantees. Our objective is to modify an event-related potential (ERP) paradigm to work in unison with the machine learning decoder, and thus to achieve a reliable unsupervised calibrationless decoding with a guarantee to recover the true class means.MethodWe introduce learning from label proportions (LLP) to the BCI community as a new unsupervised, and easy-to-implement classification approach for ERP-based BCIs. The LLP estimates the mean target and non-target responses based on known proportions of these two classes in different groups of the data. We present a visual ERP speller to meet the requirements of LLP. For evaluation, we ran simulations on artificially created data sets and conducted an online BCI study with 13 subjects performing a copy-spelling task.ResultsTheoretical considerations show that LLP is guaranteed to minimize the loss function similar to a corresponding supervised classifier. LLP performed well in simulations and in the online application, where 84.5% of characters were spelled correctly on average without prior calibration.SignificanceThe continuously adapting LLP classifier is the first unsupervised decoder for ERP BCIs guaranteed to find the optimal decoder. This makes it an ideal solution to avoid tedious calibration sessions. Additionally, LLP works on complementary principles compared to existing unsupervised methods, opening the door for their further enhancement when combined with LLP.

Improving zero-training brain-computer interfaces by mixing model estimators

Objective. Brain-computer interfaces (BCI) based on event-related potentials (ERP) incorporate a decoder to classify recorded brain signals and subsequently select a control signal that drives a computer application. Standard supervised BCI decoders require a tedious calibration procedure prior to every session. Several unsupervised classification methods have been proposed that tune the decoder during actual use and as such omit this calibration. Each of these methods has its own strengths and weaknesses. Our aim is to improve overall accuracy of ERP-based BCIs without calibration. Approach. We consider two approaches for unsupervised classification of ERP signals. Learning from label proportions (LLP) was recently shown to be guaranteed to converge to a supervised decoder when enough data is available. In contrast, the formerly proposed expectation maximization (EM) based decoding for ERP–BCI does not have this guarantee. However, while this decoder has high variance due to random initialization of its parameters, it obtains a higher accuracy faster than LLP when the initialization is good. We introduce a method to optimally combine these two unsupervised decoding methods, letting one method’s strengths compensate for the weaknesses of the other and vice versa. The new method is compared to the aforementioned methods in a resimulation of an experiment with a visual speller. Main results. Analysis of the experimental results shows that the new method exceeds the performance of the previous unsupervised classification approaches in terms of ERP classification accuracy and symbol selection accuracy during the spelling experiment. Furthermore, the method shows less dependency on random initialization of model parameters and is consequently more reliable. Significance. Improving the accuracy and subsequent reliability of calibrationless BCIs makes these systems more appealing for frequent use.

Deep Collective Inference

Collective inference is widely used to improve classification in network datasets. However, despite recent advances in deep learning and the successes of recurrent neural networks (RNNs), researchers have only just recently begun to study how to apply RNNs to heterogeneous graph and network datasets. There has been recent work on using RNNs for unsupervised learning in networks (e.g., graph clustering, node embedding) and for prediction (e.g., link prediction, graph classification), but there has been little work on using RNNs for node-based relational classification tasks. In this paper, we provide an end-to-end learning framework using RNNs for collective inference. Our main insight is to transform a node and its set of neighbors into an unordered sequence (of varying length) and use an LSTM-based RNN to predict the class label as the output of that sequence. We develop a collective inference method, which we refer to as Deep Collective Inference (DCI), that uses semi-supervised learning in partially-labeled networks and two label distribution correction mechanisms for imbalanced classes. We compare to several alternative methods on seven network datasets. DCI achieves up to a 12% reduction in error compared to the best alternative and a 25% reduction in error on average — over all methods, for all label proportions.

Learning with label proportions based on nonparallel support vector machines

Learning a classifier from groups of unlabeled data, only knowing, for each group, the proportions of data with particular labels, is an important branch of classification tasks that are conceivable in many practical applications. In this paper, we proposed a novel solution for the problem of learning with label proportions (LLP) based on nonparallel support vector machines, termed as proportion-NPSVM, which can improve the classifiers to be a pair of nonparallel classification hyperplanes. The unique property of our method is that it only needs to solve a pair of smaller quadratic programming problems. Moreover, it can efficiently incorporate the known group label proportions with the latent unknown observation labels into one optimization model under a large-margin framework. Compared to the existing approaches, there are several advantages shown as follows: 1) it does not need to make restrictive assumptions on the training data; 2) nonparallel classifiers can be achieved without computing the large inverse matrices; 3) the optimization model can be effectively solved by using the alternative strategy with SMO technique or SOR method; 4) proportion-NPSVM has better generalization ability. Sufficient experimental results on both binary-classes and multi-classes data sets show the efficiency of our proposed method in classification accuracy, which prove the state-of-the-art method for LLP problems compared with competing algorithms.