Model For Classification Problems Research Articles

Meta-classifier free negative sampling for extreme multilabel classification

Negative sampling is a common approach for making the training of deep models in classification problems with very large output spaces, known as extreme multilabel classification (XMC) problems, tractable. Negative sampling methods aim to find per instance negative labels with higher scores, known as hard negatives, and limit the computations of the negative part of the loss to these labels. Two well-known methods for negative sampling in XMC models are meta-classifier-based and Maximum Inner product Search (MIPS)-based adaptive methods. Owing to their good prediction performance, methods which employ a meta classifier are more common in contemporary XMC research. On the flip side, they need to train and store the meta classifier (apart from the extreme classifier), which can involve millions of additional parameters. In this paper, we focus on the MIPS-based methods for negative sampling. We highlight two issues which may prevent deep models trained by these methods to undergo stable training. First, we argue that using hard negatives excessively from the beginning of training leads to unstable gradient. Second, we show that when all the negative labels in a MIPS-based method are restricted to only those determined by MIPS, training is sensitive to the length of intervals for pre-processing the weights in the MIPS method. To mitigate the aforementioned issues, we propose to limit the labels selected by MIPS to only a few and sample the rest of the needed labels from a uniform distribution. We show that our proposed MIPS-based negative sampling can reach the performance of LightXML, a transformer-based model trained by a meta classifier, while there is no need to train and store any additional classifier. The code for our experiments is available at https://github.com/xmc-aalto/mips-negative-sampling.

General Performance Score for classification problems

Several performance metrics are currently available to evaluate the performance of Machine Learning (ML) models in classification problems. ML models are usually assessed using a single measure because it facilitates the comparison between several models. However, there is no silver bullet since each performance metric emphasizes a different aspect of the classification. Thus, the choice depends on the particular requirements and characteristics of the problem. An additional problem arises in multi-class classification problems, since most of the well-known metrics are only directly applicable to binary classification problems. In this paper, we propose the General Performance Score (GPS), a methodological approach to build performance metrics for binary and multi-class classification problems. The basic idea behind GPS is to combine a set of individual metrics, penalising low values in any of them. Thus, users can combine several performance metrics that are relevant in the particular problem based on their preferences obtaining a conservative combination. Different GPS-based performance metrics are compared with alternatives in classification problems using real and simulated datasets. The metrics built using the proposed method improve the stability and explainability of the usual performance metrics. Finally, the GPS brings benefits in both new research lines and practical usage, where performance metrics tailored for each particular problem are considered.

FPCANet: Fisher discrimination for Principal Component Analysis Network

With the development of Deep Learning (DL) in recent years, integrating traditional machine learning methods with DL has received a lot of attention. One of such representative work is the Principal Component Analysis Network (PCANet), which adopts Principal Component Analysis (PCA) to learn convolutional kernels (or filters) for image classification. Nevertheless, PCANet does not use the discriminative information during learning filters. In this paper, based on PCA in the PCANet, we propose a new model called Fisher PCA (FPCA) which combines Fisher Linear Discriminant Analysis (LDA) with PCA. To facilitate the practical calculation, a approximate model of FPCA is given by introducing a intermediate variable. Theoretically, we analyze the relationship between the original FPCA model and its approximate model, and give a convergence analysis of the approximate model. Additionally, stacking the approximate model of FPCA, we also construct a deep network named FPCA Network (FPCANet). Extensive experiments are conducted to compare FPCANet with other state-of-the-art models for classification problems. The results show that the proposed FPCANet can learn features with more discriminative information, and thus demonstrating its competitive performances on classification tasks.

A case study on partitioning data for classification

Designing accurate model for classification problem is a real concern in context of machine learning. The various factors such as inclusion of excellent samples in the training set, the number of samples as well as the proportion of each class type in the set (that would be sufficient for designing model) play important roles in this purpose. In this article, an investigation is introduced to address the question of what proportion of the samples should be devoted to the training set for developing a better classification model. The experimental results on several datasets, using C4.5 classifier, shows that any equidistributed data partitioning in between (20%, 80%) and (30%, 70%) may be considered as the best sample partition to build classification model irrespective to domain, size and class imbalanced.

A bilateral-truncated-loss based robust support vector machine for classification problems

Support vector machine (SVM) is sensitive to outliers or noise in the training dataset. Fuzzy SVM (FSVM) and the bilateral-weighted FSVM (BW-FSVM) can partly overcome this shortcoming by assigning different fuzzy membership degrees to different training samples. However, it is a difficult task to set the fuzzy membership degrees of the training samples. To avoid setting fuzzy membership degrees, from the beginning of the BW-FSVM model, this paper outlines the construction of a bilateral-truncated-loss based robust SVM (BTL-RSVM) model for classification problems with noise. Based on its equivalent model, we theoretically analyze the reason why the robustness of BTL-RSVM is higher than that of SVM and BW-FSVM. To solve the proposed BTL-RSVM model, we propose an iterative algorithm based on the concave---convex procedure and the Newton---Armijo algorithm. A set of experiments is conducted on ten real world benchmark datasets to test the robustness of BTL-RSVM. The statistical tests of the experimental results indicate that compared with SVM, FSVM and BW-FSVM, the proposed BTL-RSVM can significantly reduce the effects of noise and provide superior robustness.

Inferring robust decision models in multicriteria classification problems: An experimental analysis

Recent research on robust decision aiding has focused on identifying a range of recommendations from preferential information and the selection of representative models compatible with preferential constraints. This study presents an experimental analysis on the relationship between the results of a single decision model (additive value function) and the ones from the full set of compatible models in classification problems. Different optimization formulations for selecting a representative model are tested on artificially generated data sets with varying characteristics.

Environmental risk assessment in the Tuscany region: a proposal

AbstractIn this paper we present a first attempt to define a statistically coherent protocol for Environmental Risk Assessment (ERA), considered as a classification problem. Our approach moves from an idea developed in the pattern recognition literature. Several independent classifiers, each working on a subset of the covariates space, produce a set of corresponding units classifications. Then a gate, modulating the partial results, produces a final, combined classification. We propose a combined classification strategy based on rank transformations and Bayesian mixture classifiers. Although the use of Bayesian mixture models in classification problems is quite common in many fields of application, the novelty of our proposal concerns the use of truncated Gaussian components to model the behaviour of the rank variables in a multidimensional setting. We approach the general problem by partitioning the covariate space into several subspaces, each one representing one environmental dimension; we consider three environmental dimensions (Air, Water and Waste), represented by several pressure indicators. The evaluation of environmental risk for the Tuscany municipalities is our motivating example. Copyright © 2007 John Wiley & Sons, Ltd.

Design of hierarchical fuzzy model for classification problem using GAs

This paper proposes a new hierarchical fuzzy model (HFM) to solve the classification problem. The developed classification model comprises of two stages; one is to generate the fuzzy IF-THEN rules for each subsystem and the other is to determine the classification unit. For the classification problem, number of rules and the correct classification rate are the fundamental requirements. In this paper, we also advance two genetic algorithms (GAs) to tune the HFM. One is used to determine the combination of the input features for each subsystem on the HFM and the other is to reduce the number of rules in each fuzzy subsystem. The performance has been tested by simulations on the well known Wine and Iris databases. Simulations demonstrate that the proposed HFM under a few rules can provide sufficiently high classification rate even with higher feature dimensions.

An improved error signal for the backpropagation model for classification problems

Most supervised neural networks are trained by minimizing the mean squared error of the training set. But there are problems of using mean squared error especially whenever the target output is equal to the actual output in which the error signal tends to zero. This will lead to the instability of the internal structure of the network as well. In this paper, we discuss an improved convergence rates of standard backpropagation model with some modifications in its learning strategies. A modified backpropagation model is experimented on XOR problem, data of profitability analysis and Kuala Lumpur Composite Index (KLCI) at Kuala Lumpur Stock Exchange (KLSE) and handwritten/handprinted digits. The results are compared with standard backpropagation model which is based on mean square errors

Review of Categorical Models for Classification Issues in Accounting and Finance

Recent studies have extensively used the logit or probit models for classification problems in accounting and finance. More than 289 articles in prestigious journals have used these or similar methods from 1989 through 1996. This paper reviews several categorical techniques and compares the performance of logit or probit with alternative procedures. Intuitive and mathematical explanations of how the models examined differ in terms of underlying assumptions and other attributes are provided. The alternative techniques are applied to two substantive research questions: predicting bankruptcy and auditors' consistency judgements. Four empirical criteria provide some evidence that the exponential generalized beta of the second kind (EGB2), lomit, and burrit (all new to the accounting and finance literature) improve the log-likelihood functions, and the explanatory power, compared with logit and other models. EGB2, lomit and burrit also provide significantly better classifications and predictions than logit and other techniques.

Artificial Intelligence Frontiers in Statistics: AI and Statistics III

Part 1 Statistical expert systems: DEXPERT - an expert system for the design of experiments, T.J. Lorenzen, et al inside two commercially available statistical expert systems, J.F.M. Raes expert system for simulation modelling and sectoral forecasting, M. Ollivier, et al an architecture for knowledge based statistical support systems, A. Prat, et al enhancing explanation capabilities of statistical expert systems through hypertext, P. Hietala measurement scales as metadata, D.J. Hand. Part 2 Belief networks: on the design of belief networks for knowledge-based systems, B. Abramson lack-of-information based control in expert systems, R. Almond adaptive importance sampling for Bayesian networks applied to filtering problems, A.R. Runnalls intelligent arc addition, belief propagation and utilization of parallel processors by probabilistic inference engines, A. Ranjbar and M. McLeish a new method for representing and solving Bayesian decision problems, P.P. Shenoy. Part 3 Learning: inferring causal structure in mixed populations, C. Glymour, et al a knowledge acquisition inductive system guided by empirical interpretation of derived results, K. Tsujino and S. Nishida incorporating statistical techniques into empirical symbolic learning systems, F. Esposito, et al learning classification trees, W. Buntine an analysis of two probabilistic model induction techniques, S.L. Crawford and M. Fung. Part 4 Neural networks: a robust back propagation algorithm for function approximation, D.S. Chen and R.C. Jain maximum likelihood training of neural networks, H. Gish a connectionist knowledge acquisition tool - CONDKAT, A. Ultsch, et al connectionist, rule-based and Bayesian decision aids - an empirical comparison, S. Schwartz, et al. Part 5 Text manipulation: statistical approaches to aligning sentences and identifying word correspondences in parallel texts - a report on work in progress, W.A. Gale and K.W. Church probabilistic text understanding, R.P. Goldman and E. Charniak the application of machine learning techniques in subject classification, I. Kavanagh et al. Part 6 Other areas: a statistical semantics for causation, J. Pearl and T.S. Verma admissible stochastic complexity models for classification problems, P. Smyth combining the probability judgements of experts - statistical and artificial intelligence approaches, L.A. Cox randomness and independence in nonmonotonic reasoning, E. Neufeld consistent regions in probabilistic logic when using different norms, D. Bouchaffra a decision theoretic approach to controlling the cost of planning, L. Hartman.