Supervised Classification Task Research Articles

Unsupervised Exemplar-Based Learning for Improved Document Image Classification

Many recent state-of-the-art approaches for document image classification are based on supervised feature learning that requires a large amount of labeled training data. In real-world problem of document image classification, the available amount of labeled data is limited and scarce while a large amount of unlabeled data is often available at almost no cost. In this paper, we present an approach for learning visual features for document analysis in an unsupervised way, which improves the document image classification performance without increasing the amount of annotated data. The proposed approach trains a neural network model on an auxiliary task in which every training example is associated with a different label (exemplar) and expanded to multiple images through a data augmentation technique. Thus, the learned model, which is trained in an unsupervised way, is used to boost the document classification performance. In fact, this learned model has proved to be consistently efficient in two different settings: i) as an unsupervised feature extractor to represent document images for an unsupervised classification task (i.e., clustering); and ii) in the parameters initialization of a supervised classification task trained with a small amount of annotated data. We perform experiments on the Tobacco-3482 dataset and demonstrate the capability of our approach to improve i) the unsupervised classification accuracy up to 2.4%; and ii) the supervised classification accuracy by 1.5% without any extra data or by 5% when using 3000 additional not annotated samples.

Ambulatory assessment of phonotraumatic vocal hyperfunction using glottal airflow measures estimated from neck-surface acceleration.

Phonotraumatic vocal hyperfunction (PVH) is associated with chronic misuse and/or abuse of voice that can result in lesions such as vocal fold nodules. The clinical aerodynamic assessment of vocal function has been recently shown to differentiate between patients with PVH and healthy controls to provide meaningful insight into pathophysiological mechanisms associated with these disorders. However, all current clinical assessment of PVH is incomplete because of its inability to objectively identify the type and extent of detrimental phonatory function that is associated with PVH during daily voice use. The current study sought to address this issue by incorporating, for the first time in a comprehensive ambulatory assessment, glottal airflow parameters estimated from a neck-mounted accelerometer and recorded to a smartphone-based voice monitor. We tested this approach on 48 patients with vocal fold nodules and 48 matched healthy-control subjects who each wore the voice monitor for a week. Seven glottal airflow features were estimated every 50 ms using an impedance-based inverse filtering scheme, and seven high-order summary statistics of each feature were computed every 5 minutes over voiced segments. Based on a univariate hypothesis testing, eight glottal airflow summary statistics were found to be statistically different between patient and healthy-control groups. L1-regularized logistic regression for a supervised classification task yielded a mean (standard deviation) area under the ROC curve of 0.82 (0.25) and an accuracy of 0.83 (0.14). These results outperform the state-of-the-art classification for the same classification task and provide a new avenue to improve the assessment and treatment of hyperfunctional voice disorders.

Fusion of Polarimetric Features and Structural Gradient Tensors for VHR PolSAR Image Classification

This paper proposes a fast texture based supervised classification framework for fully polarimetric synthetic aperture radar (PolSAR) images with very high spatial resolution (VHR). With the development of recent polarimetric radar remote sensing technologies, the acquired images contain not only rich polarimetric characteristics but also high spatial content. Thus, the notion of geometrical structures and heterogeneous textures within VHR PolSAR data becomes more and more significant. Moreover, when the spatial resolution is increased, we need to deal with large-size image data. In this paper, our motivation is to characterize textures by incorporating (fusing) both polarimetric and structural features, and then use them for classification purpose. First, polarimetric features from the weighted coherency matrix and local geometric information based on the Di Zenzo structural tensors are extracted and fused using the covariance approach. Then, supervised classification task is performed by using Riemannian distance measure relevant for covariance-based descriptors. In order to accelerate the computational time, we propose to perform texture description and classification only on characteristic points, not all pixels from the image. Experiments conducted on the VHR F-SAR data as well as the AIRSAR Flevoland image using the proposed framework provide very promising and competitive results in terms of terrain classification and discrimination.

Assessing Information Transmission in Data Transformations with the Channel Multivariate Entropy Triangle.

Data transformation, e.g., feature transformation and selection, is an integral part of any machine learning procedure. In this paper, we introduce an information-theoretic model and tools to assess the quality of data transformations in machine learning tasks. In an unsupervised fashion, we analyze the transformation of a discrete, multivariate source of information into a discrete, multivariate sink of information related by a distribution . The first contribution is a decomposition of the maximal potential entropy of , which we call a balance equation, into its (a) non-transferable, (b) transferable, but not transferred, and (c) transferred parts. Such balance equations can be represented in (de Finetti) entropy diagrams, our second set of contributions. The most important of these, the aggregate channel multivariate entropy triangle, is a visual exploratory tool to assess the effectiveness of multivariate data transformations in transferring information from input to output variables. We also show how these decomposition and balance equations also apply to the entropies of and , respectively, and generate entropy triangles for them. As an example, we present the application of these tools to the assessment of information transfer efficiency for Principal Component Analysis and Independent Component Analysis as unsupervised feature transformation and selection procedures in supervised classification tasks.

Missing Data Imputation for Supervised Learning

ABSTRACTMissing data imputation can help improve the performance of prediction models in situations where missing data hide useful information. This paper compares methods for imputing missing categorical data for supervised classification tasks. We experiment on two machine learning benchmark datasets with missing categorical data, comparing classifiers trained on non-imputed (i.e., one-hot encoded) or imputed data with different levels of additional missing-data perturbation. We show imputation methods can increase predictive accuracy in the presence of missing-data perturbation, which can actually improve prediction accuracy by regularizing the classifier. We achieve results comparable to the state-of-the-art on the Adult dataset with missing-data perturbation and -nearest-neighbors (-NN) imputation.

Rmcfs: An R Package for Monte Carlo Feature Selection and Interdependency Discovery

We describe the R package rmcfs that implements an algorithm for ranking features from high dimensional data according to their importance for a given supervised classification task. The ranking is performed prior to addressing the classification task per se. This R package is the new and extended version of the MCFS (Monte Carlo feature selection) algorithm where an early version was published in 2005. The package provides an easy R interface, a set of tools to review results and the new ID (interdependency discovery) component. The algorithm can be used on continuous and/or categorical features (e.g., gene expression and phenotypic data) to produce an objective ranking of features with a statistically well-defined cutoff between informative and non-informative ones. Moreover, the directed ID graph that presents interdependencies between informative features is provided.

Training Support Vector Machines with privacy-protected data

In this paper, we address a machine learning task using encrypted training data. Our basic scenario has three parties: Data Owners, who own private data; an Application, which wants to train and use an arbitrary machine learning model on the Users’ data; and an Authorization Server, which provides Data Owners with public and secret keys of a partial homomorphic cryptosystem (that protects the privacy of their data), authorizes the Application to get access to the encrypted data, and assists it in those computations not supported by the partial homomorphism. As machine learning model, we have selected the Support Vector Machine (SVM) due to its excellent performance in supervised classification tasks. We evaluate two well known SVM algorithms, and we also propose a new semiparametric SVM scheme better suited for the privacy-protected scenario. At the end of the paper, a performance analysis regarding the accuracy and the complexity of the developed algorithms and protocols is presented.

Name usage pattern in the synonym ambiguity problem in bibliographic data

Individuals often appear with multiple names when considering large bibliographic datasets, giving rise to the synonym ambiguity problem. Although most related works focus on resolving name ambiguities, this work focus on classifying and characterizing multiple name usage patterns--the root cause for such ambiguity. By considering real examples bibliographic datasets, we identify and classify patterns of multiple name usage by individuals, which can be interpreted as name change, rare name usage, and name co-appearance. In particular, we propose a methodology to classify name usage patterns through a supervised classification task and show that different classes are robust (across datasets) and exhibit significantly different properties. We show that the collaboration network structure emerging around nodes corresponding to ambiguous names from different name usage patterns have strikingly different characteristics, such as their common neighborhood and degree evolution. We believe such differences in network structure and in name usage patterns can be leveraged to design more efficient name disambiguation algorithms that target the synonym problem.

Deep convolutional neural networks for semi-supervised learning from synthetic aperture sonar (SAS) images

Advancements in deep neural networks for computer-vision tasks have the potential to improve automatic target recognition (ATR) in synthetic aperture sonar (SAS) imagery. Many of the recent improvements in computer vision have been made possible by densely labeled datasets such as ImageNet. In contrast, SAS datasets typically contain far fewer labeled samples than unlabeled samples—often by several orders of magnitude. Yet unlabeled SAS data contain information useful for both generative and discriminative tasks. Here results are shown from semi-supervised ladder networks for learning to classify and localize in SAS images from very few labels. We perform end-to-end training concurrently with unlabeled and labeled samples and find that the unsupervised-learning task improves classification accuracy. Ladder networks are employed to adapt fully convolutional networks used for pixelwise prediction based on supervised training to semi-supervised semantic segmentation and target localization by pixel-level classification of whole SAS images. Using this approach, we find improved segmentation and better generalization in new SAS environments compared to purely supervised learning. We hypothesize that utilizing large unsupervised data in conjunction with the supervised classification task helps the network generalize by learning more invariant hierarchical features. [Work supported by the Office of Naval Research.]

A Multilayer Perceptron Artificial Neural Networks Based a Preprocessing and Hybrid Optimization Task for Data Mining and Classification

Artificial neural networks (ANNs) optimization represent an attractive area that attract many researchers in different disciplines, this in the aim to improve the performance of this model. In literature, there is no fix theory that illustrates how to construct this non linear model. Thus, all proposed construction was based on empirical illustration. Multilayer perceptron (MLP) is one of the most used models in ANNs area. It was described as a good non linear approximator with a power ability to lean well non linear system, and most of research was limited to a 3 layers MLP, by describing that 3 layers are sufficient to have good approximation. In this context we are interested to this model construction for solving supervised classification tasks in data mining. This construction requires a preprocessing phase that seems to scribe be important for the final performance. This paper present a process of MLP construction based on two phases: a preparation phase and an optimization phase. The first one describes a process of data cleaning, discretization, normalization, expansion, reduction and features selection. The second phase aims to optimize the set of weights based on some combination of hybrid algorithms such back-propagation algorithm, a local search and different evolution. An empirical illustration will be done to in order to validate the proposed model. At the end, a comparison with others known classifiers will be done to justify the validity of the proposed model.

Overlapping area hyperspheres for kernel-based similarity method

Measuring similarity between sets of objects is a key step in a wide areas of machine learning. Popular examples include general classification framework and numerous applications in computer vision. In this paper, we propose a kernel-based similarity method which is inspired from an interesting biological behavior of trees and induced mathematically by formulating it as a quadratic optimization problem in a reproducing kernel Hilbert space (RKHS). The proposed method is compared to the maximum mean discrepancy, a recent and challenging kernel similarity method. We conduct and present several numerical experiments on synthetic data as well as real-word image data. The proposed method yields favorable performances in terms of classification performances in the context of supervised classification tasks on the challenging Caltech101 dataset and other datasets such as USPS and ETH80. Furthermore, the efficiency of the proposed method in the context of image segmentation through unsupervised clustering of superpixels has been also asserted.

The robustness of majority voting compared to filtering misclassified instances in supervised classification tasks

Removing or filtering outliers and mislabeled instances prior to training a learning algorithm has been shown to increase classification accuracy, especially in noisy data sets. A popular approach is to remove any instance that is misclassified by a learning algorithm. However, the use of ensemble methods has also been shown to generally increase classification accuracy. In this paper, we extensively examine filtering and ensembling. We examine 9 learning algorithms individually and ensembled together as filtering algorithms as well as the effects of filtering in the 9 chosen learning algorithms on a set of 54 data sets. We compare the filtering results with using a majority voting ensemble. We find that the majority voting ensemble significantly outperforms filtering unless there are high amounts of noise present in the data set. Additionally, for most cases, using an ensemble of learning algorithms for filtering produces a greater increase in classification accuracy than using a single learning algorithm for filtering.

Classification of news-related tweets

It is important to obtain public opinion about a news article. Microblogs such as Twitter are popular and an important medium for people to share ideas. An important portion of tweets are related to news or events. Our aim is to find tweets about newspaper reports and measure the popularity of these reports on Twitter. However, it is a challenging task to match informal and very short tweets with formal news reports. In this study, we formulate this problem as a supervised classification task. We propose to form a training set using tweets containing a link to the news and the content of the same news article. We preprocess tweets by removing unnecessary words and symbols and apply stemming by means of morphological analysers. We apply binary classifiers and anomaly detection to this task. We also propose a textual similarity-based approach. We observed that preprocessing of tweets increases accuracy. The textual similarity method obtains results with the highest recognition rate. Success increases in some cases when report text is used with tweets containing a link to the news report within the training set of classification studies. We propose that this study, which is made directly in consideration of tweet texts that measure the trends of national newspaper reports on social media, has a higher significance when compared to Twitter analyses made by using a hashtag. Given the limited number of scientific studies on Turkish tweets, this study makes a contribution to the literature.

Open set recognition for automatic target classification with rejection

Training sets for supervised classification tasks are usually limited in scope and only contain examples of a few classes. In practice, classes that were not seen in training are given labels that are always incorrect. Open set recognition (OSR) algorithms address this issue by providing classifiers with a rejection option for unknown samples. In this work, we introduce a new OSR algorithm and compare its performance to other current approaches for open set image classification.

Integrating Features for Accelerometer-based Activity Recognition

Activity recognition is the problem of predicting the current action of a person through the motion sensors worn on the body. The problem is usually approached as a supervised classification task where a discriminative model is learned from known samples and a new query is assigned to a known activity label using learned model. The challenging issue here is how to feed this classifier with a fixed number of features where the real input is a raw signal of varying length. In this study, we consider three possible feature sets, namely time-domain, frequency domain and wavelet-domain statistics, and their combinations to represent motion signal obtained from accelerometer reads worn in chest through a mobile phone. In addition to a systematic comparison of these feature sets, we also provide a comprehensive evaluation of some preprocessing steps such as filtering and feature selection. The results determine that feeding a random forest classifier with an ensemble selection of most relevant time-domain and frequency-domain features extracted from raw data can provide the highest accuracy in a real dataset.

Perceptron-like computation based on biologically-inspired neurons with heterosynaptic mechanisms

Perceptrons are one of the fundamental paradigms in artificial neural networks and a key processing scheme in supervised classification tasks. However, the algorithm they provide is given in terms of unrealistically simple processing units and connections and therefore, its implementation in real neural networks is hard to be fulfilled. In this work, we present a neural circuit able to perform perceptron’s computation based on realistic models of neurons and synapses. The model uses Wang-Buzsaki neurons with coupling provided by axodendritic and axoaxonic synapses (heterosynapsis). The main characteristics of the feedforward perceptron operation are conserved, which allows to combine both approaches: whereas the classical artificial system can be used to learn a particular problem, its solution can be directly implemented in this neural circuit. As a result, we propose a biologically-inspired system able to work appropriately in a wide range of frequencies and system parameters, while keeping robust to noise and error.

Simultaneous feature selection and Gaussian mixture model estimation for supervised classification problems

A new expectation maximization (EM) algorithm for time-critical supervised classification tasks in remote sensing is proposed. Compared to standard EM and other approaches, it has the following advantages: (1) No knowledge about the class distributions is needed. (2) The number of components is estimated. (3) It does not require careful initialization. (4) Singular estimates are avoided due to the ability of pruning components. (5) The best discriminating features are identified simultaneously. (6) The features are identified by incorporating Mahalanobis distances.Three experiments are carried out in order to demonstrate the relevance of the method. The main findings are the following: (1) Feature selection is very important in terms of prediction quality of models. (2) In the experiments the proposed method estimates better models than other state-of-the-art methods. (3) The incorporation of Mahalanobis distances is very valuable for the identification of relevant features. (4) The proposed method is more robust than the compared methods. (5) In case of complex data distributions the new approach is able to provide better results.

Web Multimedia Object Classification Using Cross-Domain Correlation Knowledge

Given a collection of web images with the corresponding textual descriptions, in this paper, we propose a novel cross-domain learning method to classify these web multimedia objects by transferring the correlation knowledge among different information sources. Here, the knowledge is extracted from unlabeled objects through unsupervised learning and applied to perform supervised classification tasks. To mine more meaningful correlation knowledge, instead of using commonly used visual words in the traditional bag-of-visual-words (BoW) model, we discover higher level visual components (words and phrases) to incorporate the spatial and semantic information into our image representation model, i.e., bag-of-visual-phrases (BoP). By combining the enriched visual components with the textual words, we calculate the frequently co-occurring pairs among them to construct a cross-domain correlated graph in which the correlation knowledge is mined. After that, we investigate two different strategies to apply such knowledge to enrich the feature space where the supervised classification is performed. By transferring such knowledge, our cross-domain transfer learning method can not only handle large scale web multimedia objects, but also deal with the situation that the textual descriptions of a small portion of web images are missing. Empirical experiments on two different datasets of web multimedia objects are conducted to demonstrate the efficacy and effectiveness of our proposed cross-domain transfer learning method.

Sow-activity classification from acceleration patterns: A machine learning approach

This paper describes a supervised learning approach to sow-activity classification from accelerometer measurements. In the proposed methodology, pairs of accelerometer measurements and activity types are considered as labeled instances of a usual supervised classification task. Under this scenario sow-activity classification can be approached with standard machine learning methods for pattern classification. Individual predictions for elements of times series of arbitrary length are combined to classify it as a whole. An extensive comparison of representative learning algorithms, including neural networks, support vector machines, and ensemble methods, is presented. Experimental results are reported using a data set for sow-activity classification collected in a real production herd. The data set, which has been widely used in related works, includes measurements from active (Feeding, Rooting, Walking) and passive (Lying Laterally, Lying Sternally) activities. When classifying 1-s length observations, the best method achieved an average recognition rate of 74.64%, for the five activities. When classifying 2-min length time series, the performance of the best model increased to 80%. This is an important improvement from the 64% average recognition rate for the same five activities obtained in previous work. The pattern classification approach was also evaluated in alternative scenarios, including distinguishing between active and passive categories, and a multiclass setting. In general, better results were obtained when using a tree-based logitboost classifier. This method proved to be very robust to noise in observations. Besides its higher performance, the suggested method is more flexible than previous approaches, since time series of any length can be analyzed.

Determining dependence relations using a new score based on imprecise probabilities

In the analysis of data, the discovery of dependence relations can play a very important role. Our principal aim in this paper is to present a new score to determine when two categorical variables are independent. It can be resumed as an interval-valued score that is based on the Heckerman, Geiger, and Chickering's score, which can be used in supervised classification task. We carry out an empirical comparison with different scores to determine when two binary variables are independent. Also, we have considered the following measures: the Bayesian score metric, the Bayesian information criterion BIC, the p-value of the Chi-square test for independence and the upper entropy score based on imprecise probabilities. We will see that our new score has a behavior that it is more similar to statistical tests from small samples and to Bayesian procedures for large samples. We find this behavior very appropriate for some types of problems.