Classification Problem In Discriminant Analysis Research Articles

On the binary classification problem in discriminant analysis using linear programming methods

This paper is centred on a binary classification problem in which it is desired to assign a new object with multivariate features to one of two distinct populations as based on historical sets of samples from two populations. A linear discriminant analysis framework has been proposed, called the minimised sum of deviations by proportion (MSDP) to model the binary classification problem. In the MSDP formulation, the sum of the proportion of exterior deviations is minimised subject to the group separation constraints, the normalisation constraint, the upper bound constraints on proportions of exterior deviations and the sign unrestriction vis-à-vis the non-negativity constraints. The two-phase method in linear programming is adopted as a solution technique to generate the discriminant function. The decision rule on group-membership prediction is constructed using the apparent error rate. The performance of the MSDP has been compared with some existing linear discriminant models using a previously published dataset on road casualties. The MSDP model was more promising and well suited for the imbalanced dataset on road casualties.

A piecewise linear programming approach to the two-group discriminant problem – an adaptation to Fisher’s linear discriminant function model

Most existing linear programming (LP) models have optimization objectives that are very different from Fisher’s linear discriminant function (FLDF). An LP technique that adapts to FLDF to solve the two-group classification problem is desirable, as FLDF is one of the most popular classification rules. Therefore, this paper introduces a piecewise linear programming (PLP- p) approach that has an optimization objective very similar to that of FLDF to solve the two-group classification problem in discriminant analysis. Moreover, the paper compares the classificatory performance between FLDF and the new PLP- p model, and shows that the results from both approaches are as good as each other when applied to three published data sets. However, the new PLP- p is more flexible than FLDF in terms of adding different types of constraints and weighting individual observations. The results of a simulation experiment confirm the value of our proposed approach.

A simple weighting scheme for classification in two-group discriminant problems

This paper introduces a new weighted linear programming model, which is simple and has strong intuitive appeal for two-group classifications. Generally, in applying weights to solve a classification problem in discriminant analysis where the relative importance of every observation is known, larger weights (penalties) will be assigned to those more important observations. The perceived importance of an observation is measured here as the willingness of the decision-maker to misclassify this observation. For instance, a decision-maker is least willing to see a classification rule that misclassifies a top financially strong firm to the group that contains bankrupt firms. Our weighted-linear programming model provides an objective-weighting scheme whereby observations can be weighted according to their perceived importance. The more important this observation, the heavier its assigned weight. Results of a simulation experiment that uses contaminated data show that the weighted linear programming model consistently and significantly outperforms existing linear programming and standard statistical approaches in attaining higher average hit-ratios in the 100 replications for each of the 27 cases tested. Scope and purpose Generally, in applying weights to solve a discriminant problem where the relative importance of every observation is known, larger weights (penalties) will be assigned to those more important observations. However, if decision-makers do not have prior or additional information about the observations, it is very difficult to assign weights to the observations. Subjective judgements from decision-makers may be a way of obtaining those weights. An alternative way is to suggest an objective weighting scheme for obtaining classification weights of observations from the data matrix of the training sample. We suggest a new approach, which provides an objective weighting scheme whereby individual observations can be weighted according to their perceived importance. The more important the observation, the heavier its assigned weight will be. The ‘importance’ of individual observation is first determined in one of two stages of our model using more than one discriminant function. Simulation experiments are run to test this new approach.

Aggregating expert predictions in a networked environment

This paper describes an approach for combining the classifications or predictions of n local experts into a single composite prediction. We describe a Java-based application that allows a user to select up to n prediction experts that provide information for assigning an object to one of two predetermined groups. An advantage of this type of application is that it is capable of interacting with the Internet in a relatively seamless way. We examine the accuracy and robustness of our technique by comparing the classification accuracy of our technique, a maximum entropy-based aggregation technique, and four classification methods on a real-world, two-group data-set concerned with bank failure prediction. The classificaiton methods studied in this work include Quinlan's C4.5 decision-tree classifier, logistic regression, mahalanobis distance measures, and a neural network classifier. Our model includes a fundamental component (i.e., a transaction manager) that helps improve the general performance of applications that perform network-based classification. This component is found to provide reliable and secure connections along with ways to direct traffic across the Internet. Our results suggest three major contributions: (1) a transaction manager increases the flexibility of a network-based classifier since it is capable of transacting with one or more specific types of prediction expert(s) over the Internet; (2) our approach tends to be more accurate than the individual classification methods we examined; and, (3) our approach can outperform a recently introduced statistically based aggregation technique. Scope and purpose The emergence of the Internet has produced a need for employing new types of programming and research tools that are capable of accessing information resources located throughout the world. There is only a limited amount of research available in this area and this work describes a network-based tool that solves a two-group classification problem. The two-group classification problem in discriminant analysis is concerned with developing a rule for predicting to which of k=2 mutually exclusive groups an observation of unknown origin belongs. This problem commonly occurs in business and other areas, and a plethora of statistical and artificial intelligence (AI) techniques exist to help decision-makers effectively analyze their data. A number of recent studies have compared the classificatory performance of various AI techniques to the more traditional statistical techniques, however, decision makers are left in somewhat of a quandary about which of the many available classification techniques to use to solve a specific classification problem. This paper proposes a new aggregation technique that focuses on combining or aggregating the predictions from multiple classification techniques into a single composite prediction. Our approach provides a simple method for aggregating expert predictions coming from remote locations by combining Java and Common Object Request Broker Architecture (CORBA) into a general classification tool. Object-oriented models developed using Java are platform independent and can be easily modified. CORBA provides the services necessary to establish and manage network connections. Computational results show that our technique outperforms a recently introduced maximum entropy-based aggregation technique using a real-world data set.

Minimizing deviations from the group mean: A new linear programming approach for the two-group classification problem

This paper proposes a new linear programming approach to solve the two-group classification problem in discriminant analysis. This new approach is based on an idea from cluster analysis that objects within the same group should be more similar than objects between groups. Consequently, it is desirable for the classification score of an object to be nearer to its mean classification score, but further from the mean classification score of the other group. This objective is accomplished by minimizing the total deviation of the classification scores of the objects from their group mean scores in a linear programming approach. When applied to an actual managerial problem and simulated data, the proposed linear programming approach performs well both in groups separation and group-membership predictions of new objects. Moreover, this new approach has an advantage of obtaining more stable classification function across different samples than most of the existing linear programming approaches.

Combining Neural Networks and Statistical Predictions to Solve the Classification Problem in Discriminant Analysis*

A number of recent studies have compared the performance of neural networks (NNs) to a variety of statistical techniques for the classification problem in discriminant analysis. The empirical results of these comparative studies indicate that while NNs often outperform the more traditional statistical approaches to classification, this is not always the case. Thus, decision makers interested in solving classification problems are left in a quandary as to what tool to use on a particular data set. We present a new approach to solving classification problems by combining the predictions of a well‐known statistical tool with those of an NN to create composite predictions that are more accurate than either of the individual techniques used in isolation.

Second order mathematical programming formulations for discriminant analysis

This paper introduces a nonparametric formulation based on mathematical programming (MP) for solving the classification problem in discriminant analysis, which differs from previously proposed MP-based models in that, even though the final discriminant function is linear in terms of the parameters to be estimated, the formulation is quadratic in terms of the predictor (attribute) variables. Including second order (i.e., quadratic and cross-product) terms of the attribute variables in the model is similar in concept to the usual treatment of multiple predictor variables in statistical methods such as Fisher's linear discriminant analysis, and allows an analysis of how including nonlinear terms and interaction effects affect the predictive ability of the estimated classification function. Using simulation experiments involving data conditions for which nonlinear classifiers are appropriate, the classificatory performance of this class of second order MP models is compared with that of existing statistical (linear and quadratic) and first order MP-based formulations. The results of these experiments show that the proposed formulation appears to be a very attractive alternative to previously introduced linear and quadratic statistical and linear MP-based classification methods.

On the classification gap in mathematical programming-based approaches to the discriminant problem

This article proposes a mathematical-programming-based approach to solve the classification problem in discriminant analysis which explicitly considers the classification gap. The procedure consists of two distinct phases and initially treats the classification gap as a fuzzy set in which the classification rule is not yet established. The nature of the classification gap is examined and a variety of methods are discussed which can be applied to identify the most appropriate classification rule over the fuzzy set. The proposed methodology has several potential advantages. First, it offers a more refined approach to the classification problem, facilitating careful analysis of the fuzzy region where the classification decision may not be obvious. Secondly, the two-phase approach enables the analysis of larger data sets when using computer-intensive procedures such as mixed-integer programming. Finally, because of the restricted choice of separating hyperplanes in phase 2, the approach appears to be more robust than other classification techniques with respect to outlier-contaminated data conditions. The robustness issue and computational advantage of our proposed methodology are illustrated using a limited simulation experiment.

A comparison of the classical and the linear programming approaches to the classification problem in discriminant analysis

Several mathematical programming approaches to the classification problem in discriminant analysis have recently been introduced. This paper empirically compares these newly introduced classification techniques with Fisher's linear discriminant analysis (FLDA), quadratic discriminant analysis (QDA), logit analysis, and several rank-based procedures for a variety of symmetric and skewed distributions. The percent of correctly classified observations by each procedure in a holdout sample indicate that while under some experimental conditions the linear programming approaches compete well with the classical procedures, overall, however, their performance lags behind that of the classical procedures.

Mathematical Programming Approaches for the Classification Problem in Two-Group Discriminant Analysis.

The authors introduce mathematical programming formulations as new approaches to solve the classification problem in discriminant analysis. These formulations have recently emerged as powerful alternatives to the existing methods of maximizing correct classification of entities into groups. The research literature on mathematical programming formulations is reviewed and summarized. An illustration using a real-world classification problem is provided. issues relevant to potential users of these formulations as well as fruitful future research avenues are discussed.

Classification performance of mathematical programming techniques in discriminant analysis: Results for small and medium sample sizes

AbstractThe performance on small and medium‐size samples of several techniques to solve the classification problem in discriminant analysis is investigated. The techniques considered are two widely used parametric statistical techniques (Fisher's linear discriminant function and Smith's quadratic function), and a class of recently proposed nonparametric estimation techniques based on mathematical programming (linear and mixed‐integer programming). A simulation study is performed, analyzing the relative performance of the above techniques in the two‐group case, for various small sample sizes, moderate group overlap and across six different data conditions. Training samples as well as validation samples are used to assess the classificatory performance of the techniques. The degree of group overlap and sample sizes selected for analysis in this paper are of interest in practice because they closely reflect conditions of many real data sets. The results of the experiment show that Smith's nonlinear quadratic function tends to be superior on the training samples and validation samples when the variances–covariances across groups are heterogeneous, while the mixed‐integer technique performs best on the training samples when the variances–covariances are equal, and on validation samples with equal variances and discrete uniform independent variables. The mixed‐integer technique and the quadratic discriminant function are also found to be more sensitive than the other techniques to the sample size, giving disproportionally inaccurate results on small samples.

Solving the Classification Problem in Discriminant Analysis Via Linear and Nonlinear Programming Methods*

ABSTRACTThis paper demonstrates the feasibility of applying nonlinear programming methods to solve the classification problem in discriminant analysis. The application represents a useful extension of previously proposed linear programming‐based solutions for discriminant analysis. The analysis of data obtained by conducting a Monte Carlo simulation experiment shows that these new procedures are promising. Future research that should promote application of the proposed methods for solving classification problems in a business decision‐making environment is discussed.