Problem Of Discriminant Analysis Research Articles

Support vector machine with discriminative low‐rank embedding

AbstractSupport vector machine (SVM) is a binary classifier widely used in machine learning. However, neglecting the latent data structure in previous SVM can limit the performance of SVM and its extensions. To address this issue, the authors propose a novel SVM with discriminative low‐rank embedding (LRSVM) that finds a discriminative latent low‐rank subspace more suitable for SVM classification. The extension models of LRSVM are introduced by imposing different orthogonality constraints to prevent computational inaccuracies. A detailed derivation of the authors’ iterative algorithms are given that is essentially for solving the SVM on the low‐rank subspace. Additionally, some theorems and properties of the proposed models are presented by the authors. It is worth mentioning that the subproblems of the proposed algorithms are equivalent to the standard or the weighted linear discriminant analysis (LDA) problems. This indicates that the projection subspaces obtained by the authors’ algorithms are more suitable for SVM classification compared to those from the LDA method. The convergence analysis for the authors proposed algorithms are also provided. Furthermore, the authors conduct experiments on various machine learning data sets to evaluate the algorithms. The experiment results show that the authors’ algorithms perform significantly better than other algorithms, which indicates their superior abilities on classification tasks.

Sparse Linear Discriminant Analysis Based on lq Regularization

Linear discriminant analysis plays an important role in feature extraction, data dimensionality reduction, and classification. With the progress of science and technology, the data that need to be processed are becoming increasingly large. However, in high-dimensional situations, linear discriminant analysis faces two problems: the lack of interpretability of the projected data since they all involve all p features, which are linear combinations of all features, as well as the singularity problem of the within-class covariance matrix. There are three different arguments for linear discriminant analysis: multivariate Gaussian model, Fisher discrimination problem, and optimal scoring problem. To solve these two problems, this article establishes a model for solving the kth discriminant component, which first transforms the original model of Fisher discriminant problem in linear discriminant analysis by using a diagonal estimated matrix for the within-class variance in place of the original within-class covariance matrix, which overcomes the singularity problem of the matrix and projects it to an orthogonal projection space to remove its orthogonal constraints, and subsequently an lq norm regularization term is added to enhance its interpretability for the purpose of dimensionality reduction and classification. Finally, an iterative algorithm for solving the model and a convergence analysis are given, and it is proved that the sequence generated by the algorithm is descended and converges to a local minimum of the problem for any initial value.

Criterion of solvability of the discriminant analysis problem underlying the medical organization management model

The article considers the task of building an effective and efficient organization of the management processes of a medical organization. A distinctive feature is noted - social orientation as the main function, and not just profit maximization with limited resources. This approach requires taking into account not only financial and resource components, but also the impact of many other external factors. However, the authors, realizing the difficulty of describing them, propose to take into account the effect of the "social orientation of the institution" in an implicit form when describing the financial condition of the organization. Taking into account implicit, including external, factors is possible when implementing models not only of statistical and financial analysis, as is usually accepted, but also their extensions to some image recognition methods, which often have a simple computer implementation. In particular, the formalization of the financial condition model of the organization is proposed, which reduces to the problem of mathematical programming with state vectors of both internal and external factors. The possibility of considering a decision-making model that would take into account the influence of possible external factors is also considered, while the "informality" of the problem in the sense of the absence of classical methods of solution is indicated. In this regard, it is proposed to reduce the model to the task of discriminant analysis, which is essentially a system of inequalities. Methods for solving systems of linear inequalities are known, but they are difficult to implement. For this reason, it is proposed to add an optimality criterion (objective function) to the discriminant analysis model, which allows to obtain a linear programming problem that already has simpler computer implementations of solutions. However, this possibility is present only in the case of linear constraints, which is a somewhat special case of the general problem. To solve the problem, in its broadest sense, it is proposed not only to reduce it to mathematical programming, and then to discriminant analysis, but also an effective recognition algorithm indicating the solvability of the problem. The process of constructive problem solving often requires a lot of time and effort, even with the powerful development of modern technologies. With the help of the given recognition algorithm, the discriminant analysis model in some formulation is output in a matrix form, which allows us to conclude when the model has the existence of a separating functional. The authors also propose the use of majority committee methods. The derived algorithm is quite simple to implement, but at the same time effective and efficient in practice, which makes it universal for this kind of tasks.

A branch and bound method solving the max–min linear discriminant analysis problem

Fisher linear discriminant analysis (FLDA or LDA) is a well-known technique for dimension reduction and classification. The method was first formulated in 1936 by Fisher in the one-dimensional setting. In this paper, we will examine the LDA problem using a different objective function. Instead of maximizing the sum of all distances between all classes, we will define an objective function that will maximize the minimum separation among all distances between all classes. This leads to a difficult nonconvex optimization problem. We present a branch and bound method for the problem in the case where the reduction is to the one-dimensional space.

On Discriminant Analysis with some Bivariate Exponential Distributions

Discriminant analysis has found relevance in many areas of science and engineering. Its purpose is to assign an observation x to one of two or more groups on the basis of the value of x. Many problems of discriminant analysis have assumed that the underlying distribution is the multivariate normal in which x is assumed to have means

IMPROVING THE ALGORITHM FOR MANAGING THE PROVISION OF MEDICAL SERVICES BASED ON COMMITTEE METHODS

In matters of management in social and economic systems, medical issues are becoming increasingly important, and the processes of diagnosis of diseases are becoming the most important task of various health care institutions. Incorrect diagnosis of diseases leads not only to the fact that the patient's treatment becomes ineffective, and sometimes even harmful, but also leads to a significant increase in the costs associated with expensive procedures for further analysis and treatment of the patient. It is for this reason that it is necessary to improve the algorithm of the diagnostic procedure, including two qualitatively new blocks based on the developed methods of latent analysis procedures based on Committee and discriminant analysis. Methods of Committee and discriminant analysis as an auxiliary device are also effective and effective in solving the problem of diagnosing any diseases, including neurological and vascular diseases, as particularly dangerous and significantly impairing the standard of living of the population and leading to increased mortality, as well as worsens the statistical results of health care institutions. The improved algorithm of the diagnostic procedure proposed in the article is supplemented by the authors with two stages, which are added if the picture of symptoms is blurred and the preliminary diagnosis is not presented, or when several diagnoses fit the existing symptoms. This usually occurs due to conflicting, redundant information that corresponds to a variety of diseases. In this case, the method of differential diagnosis is used, which consists in re-interviewing, examining the patient, studying his medical history, prescribing additional clinical, functional and laboratory tests, which in some cases are excessive and even harmful to the patient, as well as increase the total cost of treating the disease, and usually increase the costs of the health care institution as a whole.. To resolve this situation, you can use discriminant analysis methods, which will allow you to find out the necessary diagnosis by assigning factors to a particular disease. However, often there is a situation related to the inconsistency of data that does not allow us to solve the problem of discriminant analysis using standard methods. In this case, you can use current mathematical methods, such as decision trees, the support vector method, and neural networks, but they are poorly interpreted and difficult to understand. therefore, we propose a method of analysis, which is improved by the authors in order to reduce the size of the problem, which helps reduce the number of factors necessary for the diagnosis of diseases. This method allows you to eliminate the inconsistency of data and unambiguously make a diagnosis, some features of which can later be identified by other standard clinical or instrumental methods. Thus, we can conclude that the use of methods of Committee and discriminant analysis in the diagnosis of diseases is not only an auxiliary, but also a mandatory attribute, in order to improve the quality of medical services. Moreover, the main methodological position of the application of these methods is to use the search for latent risk factors for diseases, which allows increasing the proportion of timely diagnosed patients. Also, do not forget about the provision of correct information collection and processing.

Ratio sum formula for dimensionality reduction

High-dimensional data analysis often suffers the so-called curse of dimensionality. Therefore, dimensionality reduction is usually carried out on the high-dimensional data before the actual analysis, which is a common and efficient way to eliminate this effect. And the popular trace ratio criterion is an extension of the original linear discriminant analysis (LDA) problem, which involves a search of a transformation matrix W to embed high-dimensional space into a low-dimensional space to achieve dimensionality reduction. However, the trace ratio criterion tends to obtain projection direction with very small variance, which the subset after the projection is diffcult to present the most representative information of the data with maximum efficiency. In this paper, we target on this problem and propose the ratio sum formula for dimensionality reduction. Firstly, we analyze the impact of this trend. Then in order to solve this problem, we propose a new ratio sum formula as well as the solution. In the end, we perform experiments on the Yale-B, ORL, and COIL-20 data sets. The theoretical studies and actual numerical analysis confirm the effectiveness of the proposed method.

Statistical process fault isolation using robust nonnegative garrote

Abstract Fault isolation is an essential procedure in multivariate statistical process monitoring, which is used to locate the detected fault. Fault isolation identifies the crucial variables responsible for the detected fault. Accurate isolation results are useful for process engineers in diagnosing the root cause of the fault. Recent studies have revealed the equivalence between the fault isolation task and the variable selection problem in discriminant analysis. Inspired by this idea, a nonnegative garrote-based fault isolation strategy is developed to identify the criticality of each process variable to the detected fault, which is further revised to a more robust version by adopting a robust nonnegative garrote. The critical variables can be identified even when the historical process data are contaminated by outliers using the method proposed in this study. The Tennessee Eastman process was used to illustrate the validity of the proposed method.

On the binary classification problem in discriminant analysis using linear programming methods

This paper is centred on a binary classification problem in which it is desired to assign a new object with multivariate features to one of two distinct populations as based on historical sets of samples from two populations. A linear discriminant analysis framework has been proposed, called the minimised sum of deviations by proportion (MSDP) to model the binary classification problem. In the MSDP formulation, the sum of the proportion of exterior deviations is minimised subject to the group separation constraints, the normalisation constraint, the upper bound constraints on proportions of exterior deviations and the sign unrestriction vis-à-vis the non-negativity constraints. The two-phase method in linear programming is adopted as a solution technique to generate the discriminant function. The decision rule on group-membership prediction is constructed using the apparent error rate. The performance of the MSDP has been compared with some existing linear discriminant models using a previously published dataset on road casualties. The MSDP model was more promising and well suited for the imbalanced dataset on road casualties.

Efficient computation for differential network analysis with applications to quadratic discriminant analysis

Differential network analysis is an important statistical problem with wide applications. Many statisticians focus on binary problems and propose to perform such analysis by obtaining sparse estimates of the difference between precision matrices. These methods are supported by excellent theoretical properties and practical performance. However, efficient computation for these methods remains a challenging problem. A novel algorithm referred to as the SMORE algorithm is proposed for differential network analysis. The SMORE algorithm has low storage cost and high computation speed, especially in the presence of strong sparsity. In the meantime, the SMORE algorithm provides a unified framework for binary and multiple network problems. In addition, the SMORE algorithm can be applied in high-dimensional quadratic discriminant analysis problems as well, leading to a new approach for multiclass high-dimensional quadratic discriminant analysis. Numerical studies confirm the stability and the efficiency of the proposed SMORE algorithm in both differential network analysis and quadratic discriminant analysis.

Supervised Dimensionality Reduction Methods via Recursive Regression.

In this article, the recursive problems of both orthogonal linear discriminant analysis (OLDA) and orthogonal least squares regression (OLSR) are investigated. Different from other works, the associated recursive problems are addressed via a novel recursive regression method, which achieves the dimensionality reduction in the orthogonal complement space heuristically. As for the OLDA, an efficient method is developed to obtain the associated optimal subspace, which is closely related to the orthonormal basis of the optimal solution to the ridge regression. As for the OLSR, the scalable subspace is introduced to build up an original OLSR with optimal scaling (OS). Through further relaxing the proposed problem into a convex parameterized orthogonal quadratic problem, an effective approach is derived, such that not only the optimal subspace can be achieved but also the OS could be obtained automatically. Accordingly, two supervised dimensionality reduction methods are proposed via obtaining the heuristic solutions to the recursive problems of the OLDA and the OLSR.

Perturbation analysis of an eigenvector-dependent nonlinear eigenvalue problem with applications

The eigenvector-dependent nonlinear eigenvalue problem arises in many important applications, such as the discretized Kohn–Sham equation in electronic structure calculations and the trace ratio problem in linear discriminant analysis. In this paper, we perform a perturbation analysis for the eigenvector-dependent nonlinear eigenvalue problem, which gives upper bounds for the distance between the solution to the original nonlinear eigenvalue problem and the solution to the perturbed nonlinear eigenvalue problem. A condition number for the nonlinear eigenvalue problem is introduced, which reveals the factors that affect the sensitivity of the solution. Furthermore, two computable error bounds are given for the nonlinear eigenvalue problem, which can be used to measure the quality of an approximate solution. Numerical results on practical problems, such as the Kohn–Sham equation and the trace ratio optimization, indicate that the proposed upper bounds are sharper than the state-of-the-art bounds.

Tri-Direction 2D-Fisher Discriminant Analysis (T2D-FDA) for Feature Extraction

A new image feature extraction method for face recognition called Tri-direction 2D-Fisher Discriminant Analysis (T2D-FDA) is proposed to deal with the Small Sample Size (SSS) problem in conventional 1D-Fisher Discriminant Analysis (1D-FDA). Moreover, the essence of T2D-FDA is investigated, and the equivalence of the left-multiplying 2D-FDA of the original image matrices and the left-multiplying D2D-FDA of diagonal image matrices is verified if each column is viewed as a computational unit. Different from the 1D-LDA based approaches, the T2D-FDA is based on 2D image matrices rather than column vectors, so the image matrix does not need to be transformed into a long vector before feature extraction. The advantage arising in this way is that the SSS problem does not exist anymore because within-class scatter matrices constructed is full-rank in T2D-FDA. The proposed method is applied to face recognition where only few training images exist for each subject. Experiment results show T2D-FDA outperforms the current linear subspace methods in SSS problem on four public databases: ORL, Yale, AR and FERET face databases.

Bayesian Discriminant Analysis Using a High Dimensional Predictor

We consider the problem of Bayesian discriminant analysis using a high dimensional predictor. In this setting, the underlying precision matrices can be estimated with reasonable accuracy only if some appropriate additional structure like sparsity is assumed. We induce a prior on the precision matrix through a sparse prior on its Cholesky decomposition. For computational ease, we use shrinkage priors to induce sparsity on the off-diagonal entries of the Cholesky decomposition matrix and exploit certain conditional conjugacy structure. We obtain the contraction rate of the posterior distribution for the mean and the precision matrix respectively using the Euclidean and the Frobenius distance, and show that under some milder restriction on the growth of the dimension, the misclassification probability of the Bayesian classification procedure converges to that of the oracle classifier for both linear and quadratic discriminant analysis. Extensive simulations show that the proposed Bayesian methods perform very well. An application to identify cancerous breast tumorbased on image data obtained using find needle aspirate is considered.

Self-Weighted Supervised Discriminative Feature Selection.

In this brief, a novel self-weighted orthogonal linear discriminant analysis (SOLDA) problem is proposed, and a self-weighted supervised discriminative feature selection (SSD-FS) method is derived by introducing sparsity-inducing regularization to the proposed SOLDA problem. By using the row-sparse projection, the proposed SSD-FS method is superior to multiple sparse feature selection approaches, which can overly suppress the nonzero rows such that the associated features are insufficient for selection. More specifically, the orthogonal constraint ensures the minimal number of selectable features for the proposed SSD-FS method. In addition, the proposed feature selection method is able to harness the discriminant power such that the discriminative features are selected. Consequently, the effectiveness of the proposed SSD-FS method is validated theoretically and experimentally.

Multispectral palmprint recognition using Pascal coefficients-based LBP and PHOG descriptors with random sampling

Local binary pattern (LBP) algorithm and its variants have been used extensively to analyse the local textural features of digital images with great success. Numerous extensions of LBP descriptors have been suggested, focusing on improving their robustness to noise and changes in image conditions. In our research, inspired by the concepts of LBP feature descriptors and a random sampling subspace, we propose an ensemble learning framework, using a variant of LBP constructed from Pascal’s coefficients of n-order and referred to as a multiscale local binary pattern. To address the inherent overfitting problem of linear discriminant analysis, PCA was applied to the training samples. Random sampling was used to generate multiple feature subsets. In addition, in this work, we propose a new feature extraction technique that combines the pyramid histogram of oriented gradients and LBP, where the features are concatenated for use in the classification. Its performance in recognition was evaluated using the Hong Kong Polytechnic University database. Extensive experiments unmistakably show the superiority of the proposed approach compared to state-of-the-art techniques.

A method of reducing the number of members of the committee systems of linear inequalities

The problem of discriminant analysis under mild conditions is reduced to a system of linear inequalities. However, this system can turn out to be inconsistent, which happens rather frequently. This is when the method of committees is used. The quality of a committee is improved as the number of its members drops. A method of reducing the number of members of committees, if this is fundamentally possible, is considered.

Multivariate fault isolation of batch processes via variable selection in partial least squares discriminant analysis

In recent years, multivariate statistical monitoring of batch processes has become a popular research topic, wherein multivariate fault isolation is an important step aiming at the identification of the faulty variables contributing most to the detected process abnormality. Although contribution plots have been commonly used in statistical fault isolation, such methods suffer from the smearing effect between correlated variables. In particular, in batch process monitoring, the high autocorrelations and cross-correlations that exist in variable trajectories make the smearing effect unavoidable. To address such a problem, a variable selection-based fault isolation method is proposed in this research, which transforms the fault isolation problem into a variable selection problem in partial least squares discriminant analysis and solves it by calculating a sparse partial least squares model. As different from the traditional methods, the proposed method emphasizes the relative importance of each process variable. Such information may help process engineers in conducting root-cause diagnosis.

Способ оценки прогностической силы бинарного показателя

The paper considers the solution of one of the types of classification problems in data analysis. Let us assume that the set of objects under consideration is in some way divided into two groups (we call this a regular partition). Along with this, each of the objects in view has a certain binary indicator measured – for each of the objects it has only values 0 or 1. It is required to estimate the confidence to assign the object to one of the groups of the regular partition using the knowledge of this indicator. This problem is a variant of the so called discriminant analysis problem, where the rule for assigning an object to one of the groups is called prognostic. So, the introduced numerical characteristic of the indicator of the informational content is called the prognostic power of it. The characteristic is introduced by estimating the differences between the regular partition of the set and the partition constructed by the binary indicator being studied. The magnitude of the difference is determined by calculating the cluster metric previously introduced in the work of the first author. This characteristic is compared with the correlation coefficient and the relative risk ratio commonly used in such cases.DOI 10.14258/izvasu(2017)4-15

Kernel Locality Preserving Symmetrical Weighted Fisher Discriminant Analysis based subspace approach for expression recognition

This paper mainly focuses on dimensional reduction of fused dataset of holistic and geometrical face features vectors by solving singularity problem of linear discriminant analysis and maximizing the Fisher ratio in nonlinear subspace region with the preservation of local discriminative features. The combinational feature vector space is projected into low dimensional subspace using proposed Kernel Locality Preserving Symmetrical Weighted Fisher Discriminant Analysis (KLSWFDA) method. Matching score level fusion technique has been applied on projected subspace and combinational entire Gabor subspace is framed. Euclidean distance metric (L2) and support vector machine (SVM) classifier has been implemented to recognize and classify the expressions. Performance of proposed approach is evaluated and compared with state of art approaches. Experimental results on JAFFE, YALE and FD expression database demonstrate the effectiveness of the proposed approach.