Supervised Classification Problem Research Articles

Identifying e-Commerce in Enterprises by means of Text Mining and Classification Algorithms

Monitoring specific features of the enterprises, for example, the adoption of e-commerce, is an important and basic task for several economic activities. This type of information is usually obtained by means of surveys, which are costly due to the amount of personnel involved in the task. An automatic detection of this information would allow consistent savings. This can actually be performed by relying on computer engineering, since in general this information is publicly available on-line through the corporate websites. This work describes how to convert the detection of e-commerce into a supervised classification problem, where each record is obtained from the automatic analysis of one corporate website, and the class is the presence or the absence of e-commerce facilities. The automatic generation of similar data records requires the use of several Text Mining phases; in particular we compare six strategies based on the selection of best words and best n-grams. After this, we classify the obtained dataset by means of four classification algorithms: Support Vector Machines; Random Forest; Statistical and Logical Analysis of Data; Logistic Classifier. This turns out to be a difficult case of classification problem. However, after a careful design and set-up of the whole procedure, the results on a practical case of Italian enterprises are encouraging.

A novel semi supervised approach for text classification

Text categorization, also known as text classification is a supervised classification problem. It aims to assign a predefined class label or group to a new or unknown text document. Most of the time we need a collection of large data from each class to train the classifier. It may be noted that, it is very hard or expensive to collect labelled text data. In most cases we assign the label manually which is neither cost effective nor efficient. In this paper, we have introduced a semi-supervised classification approach where the learner needs very small amount of labelled data with a large amount of unlabeled data to assign a class label to a new or unknown text document. The proposed method uses Kohonen self organizing map (SOM) for labelling the unlabeled data and three classifiers namely support vector machine (SVM), Naive Bayes (NB), and decision tree (DT): classification and regression tree (CART) for observing the accuracy of classification. The experimental results obtained show the effectiveness of our proposed method.

Extended Box Clustering for Classification Problems

In this work we address a technique, based on elementary convex sets called box hulls, for effectively grouping finite point sets into non-convex objects, called box clusters. The proposed clustering approach is based on homogeneity conditions, not according to some distance measure, and it is situated inside the theoretical framework of Supervised clustering. This approach extends the so-called (convex) box clustering, originally developed in the context of the logical analysis of data, to non-convex geometry. We briefly discuss the topological properties of these clusters and introduce a family of hypergraphs, called incompatibility hypergraphs; the main aim for these hypergraphs is their role in clustering algorithms, even if they have strong theoretical properties as shown in other works in literature. We also discuss of supervised classification problems and generalized Voronoi diagrams are considered to define a classifier based on box clusters. Finally, computational experiments on real world data are used to show the efficacy of our methods both in terms of clustering and accuracy.

Federated learning of predictive models from federated Electronic Health Records

BackgroundIn an era of “big data,” computationally efficient and privacy-aware solutions for large-scale machine learning problems become crucial, especially in the healthcare domain, where large amounts of data are stored in different locations and owned by different entities. Past research has been focused on centralized algorithms, which assume the existence of a central data repository (database) which stores and can process the data from all participants. Such an architecture, however, can be impractical when data are not centrally located, it does not scale well to very large datasets, and introduces single-point of failure risks which could compromise the integrity and privacy of the data. Given scores of data widely spread across hospitals/individuals, a decentralized computationally scalable methodology is very much in need. ObjectiveWe aim at solving a binary supervised classification problem to predict hospitalizations for cardiac events using a distributed algorithm. We seek to develop a general decentralized optimization framework enabling multiple data holders to collaborate and converge to a common predictive model, without explicitly exchanging raw data. MethodsWe focus on the soft-margin l1-regularized sparse Support Vector Machine (sSVM) classifier. We develop an iterative cluster Primal Dual Splitting (cPDS) algorithm for solving the large-scale sSVM problem in a decentralized fashion. Such a distributed learning scheme is relevant for multi-institutional collaborations or peer-to-peer applications, allowing the data holders to collaborate, while keeping every participant's data private. ResultsWe test cPDS on the problem of predicting hospitalizations due to heart diseases within a calendar year based on information in the patients Electronic Health Records prior to that year. cPDS converges faster than centralized methods at the cost of some communication between agents. It also converges faster and with less communication overhead compared to an alternative distributed algorithm. In both cases, it achieves similar prediction accuracy measured by the Area Under the Receiver Operating Characteristic Curve (AUC) of the classifier. We extract important features discovered by the algorithm that are predictive of future hospitalizations, thus providing a way to interpret the classification results and inform prevention efforts.

Multiple kernel learning using single stage function approximation for binary classification problems

ABSTRACTIn this paper, the multiple kernel learning (MKL) is formulated as a supervised classification problem. We dealt with binary classification data and hence the data modelling problem involves the computation of two decision boundaries of which one related with that of kernel learning and the other with that of input data. In our approach, they are found with the aid of a single cost function by constructing a global reproducing kernel Hilbert space (RKHS) as the direct sum of the RKHSs corresponding to the decision boundaries of kernel learning and input data and searching that function from the global RKHS, which can be represented as the direct sum of the decision boundaries under consideration. In our experimental analysis, the proposed model had shown superior performance in comparison with that of existing two stage function approximation formulation of MKL, where the decision functions of kernel learning and input data are found separately using two different cost functions. This is due to the fact that single stage representation helps the knowledge transfer between the computation procedures for finding the decision boundaries of kernel learning and input data, which inturn boosts the generalisation capacity of the model.

A Comparative Analysis of a Novel Anomaly Detection Algorithm with Neural Networks

Anomaly Detection is an important research domain of Pattern Recognition due to its effects of classification and clustering problems. In this paper, an anomaly detection algorithm is proposed using different primitive cost functions such as Normal Perceptron, Relaxation Criterion, Mean Square Error (MSE) and Ho-Kashyap. These criterion functions are minimized to locate the decision boundary in the data space so as to classify the normal data objects and the anomalous data objects. The authors proposed algorithm uses the concept of supervised classification, though it is very different from solving normal supervised classification problems. This proposed algorithm using different criterion functions has been compared with the accuracy of the Neural Networks (NN) in order to bring out a comparative analysis between them and discuss some advantages.

Brillouin Optical Time-Domain Analyzer Assisted by Support Vector Machine for Ultrafast Temperature Extraction

Brillouin optical time-domain analyzer (BOTDA) assisted by support vector machine (SVM) for ultrafast temperature extraction is proposed and experimentally demonstrated. The temperature extraction is treated as a supervised classification problem and the Brillouin gain spectrum (BGS) is classified into each temperature class according to the support vectors and hyperplane of the SVM model after training. Ideal pseudo-Voigt curve-based BGS is used to train the SVM to get the support vectors and hyperplane. The performance of SVM is investigated in both simulation and experiment under various conditions for BGS collection. Both simulation and experiment results show that SVM is more robust to a wide range of signal-to-noise ratios, averaging times, pump pulse widths, frequency scanning steps, and temperatures. In addition to better accuracy, the processing speed for temperature extraction using SVM is 100 times faster than that using conventional Lorentzian curve and pseudo-Voigt curve fitting techniques in our experiment. The fast processing speed together with good accuracy and robustness makes SVM a highly competitive candidate for future high-speed BOTDA sensors.

A comparison of fitness-case sampling methods for genetic programming

Genetic programming (GP) is an evolutionary computation paradigm for automatic program induction. GP has produced impressive results but it still needs to overcome some practical limitations, particularly its high computational cost, overfitting and excessive code growth. Recently, many researchers have proposed fitness-case sampling methods to overcome some of these problems, with mixed results in several limited tests. This paper presents an extensive comparative study of four fitness-case sampling methods, namely: Interleaved Sampling, Random Interleaved Sampling, Lexicase Selection and Keep-Worst Interleaved Sampling. The algorithms are compared on 11 symbolic regression problems and 11 supervised classification problems, using 10 synthetic benchmarks and 12 real-world data-sets. They are evaluated based on test performance, overfitting and average program size, comparing them with a standard GP search. Comparisons are carried out using non-parametric multigroup tests and post hoc pairwise statistical tests. The experimental results suggest that fitness-case sampling methods are particularly useful for difficult real-world symbolic regression problems, improving performance, reducing overfitting and limiting code growth. On the other hand, it seems that fitness-case sampling cannot improve upon GP performance when considering supervised binary classification.

Graph Regularized Restricted Boltzmann Machine.

The restricted Boltzmann machine (RBM) has received an increasing amount of interest in recent years. It determines good mapping weights that capture useful latent features in an unsupervised manner. The RBM and its generalizations have been successfully applied to a variety of image classification and speech recognition tasks. However, most of the existing RBM-based models disregard the preservation of the data manifold structure. In many real applications, the data generally reside on a low-dimensional manifold embedded in high-dimensional ambient space. In this brief, we propose a novel graph regularized RBM to capture features and learning representations, explicitly considering the local manifold structure of the data. By imposing manifold-based locality that preserves constraints on the hidden layer of the RBM, the model ultimately learns sparse and discriminative representations. The representations can reflect data distributions while simultaneously preserving the local manifold structure of data. We test our model using several benchmark image data sets for unsupervised clustering and supervised classification problem. The results demonstrate that the performance of our method exceeds the state-of-the-art alternatives.

Robust pattern decoding in shape-coded structured light

Decoding is a challenging and complex problem in a coded structured light system. In this paper, a robust pattern decoding method is proposed for the shape-coded structured light in which the pattern is designed as grid shape with embedded geometrical shapes. In our decoding method, advancements are made at three steps. First, a multi-template feature detection algorithm is introduced to detect the feature point which is the intersection of each two orthogonal grid-lines. Second, pattern element identification is modelled as a supervised classification problem and the deep neural network technique is applied for the accurate classification of pattern elements. Before that, a training dataset is established, which contains a mass of pattern elements with various blurring and distortions. Third, an error correction mechanism based on epipolar constraint, coplanarity constraint and topological constraint is presented to reduce the false matches. In the experiments, several complex objects including human hand are chosen to test the accuracy and robustness of the proposed method. The experimental results show that our decoding method not only has high decoding accuracy, but also owns strong robustness to surface color and complex textures.

Optimization of Gene Set Annotations Using Robust Trace-Norm Multitask Learning.

Gene set enrichment (GSE) is a useful tool for analyzing and interpreting large molecular datasets generated by modern biomedical science. The accuracy and reproducibility of GSE analysis are heavily affected by the quality and integrity of gene sets annotations. In this paper, we propose a novel method, robust trace-norm multitask learning, to solve the optimization problem of gene set annotations. Inspired by the binary nature of annotations, we convert the optimization of gene set annotations into a weakly supervised classification problem and use discriminative logistic regression to fit these datasets. Then, the output of logistic regression can be used to measure the probability of the existence of annotations. In addition, the optimization of each row of the annotation matrix can be treated as an independent weakly classification task, and we use the multitask learning approach with trace-norm regularization to optimize all rows of annotation matrix simultaneously. Finally, the experiments on simulated and real data demonstrate the effectiveness and good performance of the proposed method.

Improving specific class mapping from remotely sensed data by cost-sensitive learning

ABSTRACTIn many remote-sensing projects, one is usually interested in a small number of land-cover classes present in a study area and not in all the land-cover classes that make-up the landscape. Previous studies in supervised classification of satellite images have tackled specific class mapping problem by isolating the classes of interest and combining all other classes into one large class, usually called others, and by developing a binary classifier to discriminate the class of interest from the others. Here, this approach is called focused approach. The strength of the focused approach is to decompose the original multi-class supervised classification problem into a binary classification problem, focusing the process on the discrimination of the class of interest. Previous studies have shown that this method is able to discriminate more accurately the classes of interest when compared with the standard multi-class supervised approach. However, it may be susceptible to data imbalance problems present in the training data set, since the classes of interest are often a small part of the training set. A result the classification may be biased towards the largest classes and, thus, be sub-optimal for the discrimination of the classes of interest. This study presents a way to minimize the effects of data imbalance problems in specific class mapping using cost-sensitive learning. In this approach errors committed in the minority class are treated as being costlier than errors committed in the majority class. Cost-sensitive approaches are typically implemented by weighting training data points accordingly to their importance to the analysis. By changing the weight of individual data points, it is possible to shift the weight from the larger classes to the smaller ones, balancing the data set. To illustrate the use of the cost-sensitive approach to map specific classes of interest, a series of experiments with weighted support vector machines classifier and Landsat Thematic Mapper data were conducted to discriminate two types of mangrove forest (high-mangrove and low-mangrove) in Saloum estuary, Senegal, a United Nations Educational, Scientific and Cultural Organisation World Heritage site. Results suggest an increase in overall classification accuracy with the use of cost-sensitive method (97.3%) over the standard multi-class (94.3%) and the focused approach (91.0%). In particular, cost-sensitive method yielded higher sensitivity and specificity values on the discrimination of the classes of interest when compared with the standard multi-class and focused approaches.

Nonparametric regression on contaminated functional predictor with application to hyperspectral data

Regressing nonparametrically a scalar response on a contaminated random curve observed at some measurement grid may be a hard task. To address this common statistical situation, a kernel presmoothing step is achieved on the noisy functional predictor. After that, the kernel estimator of the regression operator is built using the smoothed functional covariate instead of the original corrupted one. The rate of convergence is stated for this nested-kernel estimator with special attention to high-dimensional setting (i.e. the size of the measurement grid is much larger than the sample size). The proposed method is applied to simulated datasets in order to assess its finite-sample properties. Our methodology is further illustrated on a real hyperspectral dataset involving a supervised classification problem.

A new nearest neighbor classification method based on fuzzy set theory and aggregation operators

The Fuzzy Nearest Neighbor Classification (FuzzyNNC) has been successfully used, as a tool to deal with supervised classification problems. It has significantly increased the classification accuracy by considering the uncertainty associated with the class labels of the training patterns. Nevertheless, FuzzyNNC's limited methods fail to efficiently handle the imprecision in features measurement and the uncertainty induced by the choice of the distance measure and the number of neighbors in the decision rule. In this paper, we propose a new method called Fuzzy Analogy-based Classification (FABC) to tackle the FuzzyNNC limitations. In this work, we exploit the fuzzy linguistic modeling and approximate reasoning materials in order to endow FABC with intelligent capabilities, like imprecision tolerance, optimization, adaptability and trade-off. Hence, our approach is composed of two main steps. Firstly, we describe the domain features using fuzzy linguistic variables. Secondly, we define the classification process using two intelligent aggregation operators. The first one allows the optimization of the similarity evaluation, by defining the adequate features to be considered. The second one integrates a trade-off strategy within the decision rule, by using a global voting approach with compensation property. The integration of such mechanisms will increase the classification accuracy and make the FuzzyNNC approach more useful for classification problems where imprecision and uncertainty are unavoidable. The proposed FABC is validated on the most known datasets, representing various classification difficulties and compared to the many extensions of the FuzzyNNC approach. The results obtained show that our proposed FABC method can be adapted to different classification problems and improve the classification accuracy. Thus, the FABC has the best rank value against the comparison methods with high significant level. Moreover, we conclude that our optimized similarity and global voting rule are more robust to handle the uncertainty in the classification process than those used by the comparison methods.

Optimization approaches to Supervised Classification

The Supervised Classification problem, one of the oldest and most recurrent problems in applied data analysis, has always been analyzed from many different perspectives. When the emphasis is placed on its overall goal of developing classification rules with minimal classification cost, Supervised Classification can be understood as an optimization problem. On the other hand, when the focus is in modeling the uncertainty involved in the classification of future unknown entities, it can be formulated as a statistical problem. Other perspectives that pay particular attention to pattern recognition and machine learning aspects of Supervised Classification have also a long history that has lead to influential insights and different methodologies.In this review, two approaches to Supervised Classification strongly related to optimization theory will be discussed and compared. In particular, we will review methodologies based on Mathematical Programming models that optimize observable criteria linked to the true objective of misclassification error (or cost) minimization, and approaches derived from the minimization of known bounds on the true misclassification error. The former approach is known as the Mathematical Programming approach to Supervised Classification, while the latter is in the origin of the well known Classification Support Vector Machines.Throughout the review two-group as well as general multi-group problems will be considered, and the review will conclude with a discussion of the most promising research directions in this area.

Dynamic statistical classification

We consider the statistical supervised classification problem from adynamical systems approach. We assume that two classes exist and that, for each one, a multivariate normal distribution determines the probability to be in a certain region in the n dimensional real vector space. These density functions are the potentials of corresponding gradient vector fields for each class; we construct a “classifying vector field” as a suitable weighted mean ofthem. From data known in the literature, we estimate the population parameters, and the classes are successfully distinguished; we compute and present confusion matrices. A one and two-dimensional analysis is given.

Target curricula via selection of minimum feature sets

We consider the effect of introducing a curriculum of targets when training Boolean models on supervised Multi Label Classification (MLC) problems. In particular, we consider how to order targets i...

A new topological entropy-based approach for measuring similarities among piecewise linear functions

In this paper we present a novel methodology based on a topological entropy, the so-called persistent entropy, for addressing the comparison between discrete piecewise linear functions. The comparison is certified by the stability theorem for persistent entropy that is presented here. The theorem is used in the implementation of a new algorithm. The algorithm transforms a discrete piecewise linear function into a filtered simplicial complex that is analyzed via persistent homology and persistent entropy. Persistent entropy is used as a discriminant feature for solving the supervised classification problem of real long-length noisy signals of DC electrical motors. The quality of classification is stated in terms of the area under receiver operating characteristic curve (AUC=93.87%).

Vine copula classifiers for the mind reading problem

In this paper we introduce vine copulas to model probabilistic dependencies in supervised classification problems. Vine copulas allow the representation of the dependence structure of multidimensional distributions as a factorization of bivariate pair-copulas. The flexibility of this model lies in the fact that we can mix different types of pair-copulas in a factorization, which allows covering a wide range of types of dependencies, i.e., from independence to much more complex forms of bivariate correlations. This property motivates us to use vine copulas as classifiers, particularly for problems for which the type and strength of bivariate interactions between the variables show a great variability. This is the case of brain signal classification problems where information is represented as multiple time series, each one recorded from different brain region. Our experimental results on a real-word Mind Reading Problem reveal that vine copula-based classifiers perform competitively compared to the four best classification methods presented at the Mind Reading Challenge Competition 2011.

Machine learning models, epistemic set-valued data and generalized loss functions: An encompassing approach

We study those problems where the goal is to find “optimal” models with respect to some specific criterion, in regression and supervised classification problems. Alternatives to the usual expected loss minimization criterion are proposed, and a general framework where this criterion can be seen as a particular instance of a general family of criteria is provided.In the new setting, each model is formally identified with a random variable that associates a loss value to each individual in the population. Based on this identification, different stochastic orderings between random variables lead to different criteria to compare pairs of models. Our general setting encompasses the classical criterion based on the minimization of the expected loss, but also other criteria where a numerical loss function is not available, and therefore the computation of its expectation does not make sense.The presentation of the new framework is divided into two stages. First, we consider the new framework under standard situations about the sample information, where both the collection of attributes and the response variables are observed with precision. Then, we assume that just incomplete information about them (expressed in terms of set-valued data sets) is provided. We cast some comparison criteria from the recent literature on learning methods from low-quality data as particular instances of our general approach.