Minimax Probability Machine Research Articles

Determination of stability of slope using Minimax Probability Machine

This article examines the capability of Minimax Probability Machine (MPM) for the determination of stability of slope. MPM is constructed within a probabilistic framework. This study uses MPM as classification and regression tools. Unit weight (γ), cohesion (c), angle of internal friction (φ), slope angle (β), height (H) and pore water pressure coefficient (ru) have been used as inputs of the MPM model. The outputs of MPM are stability status of slope and factor of safety (F). The results of MPM have been compared with the artificial neural network models. The experimental results demonstrate that the developed MPM is a promising tool for the determination of stability of slope.

A unified classification model for modeling of seismic liquefaction potential of soil based on CPT

The evaluation of liquefaction potential of soil due to an earthquake is an important step in geosciences. This article examines the capability of Minimax Probability Machine (MPM) for the prediction of seismic liquefaction potential of soil based on the Cone Penetration Test (CPT) data. The dataset has been taken from Chi–Chi earthquake. MPM is developed based on the use of hyperplanes. It has been adopted as a classification tool. This article uses two models (MODEL I and MODEL II). MODEL I employs Cone Resistance (qc) and Cyclic Stress Ratio (CSR) as input variables. qc and Peak Ground Acceleration (PGA) have been taken as inputs for MODEL II. The developed MPM gives 100% accuracy. The results show that the developed MPM can predict liquefaction potential of soil based on qc and PGA.

Determination of stability of epimetamorphic rock slope using Minimax Probability Machine

The article employs Minimax Probability Machine (MPM) for the prediction of the stability status of epimetamorphic rock slope. The MPM gives a worst-case bound on the probability of misclassification of future data points. Bulk density (d), height (H), inclination (β), cohesion (c) and internal friction angle (φ) have been used as input of the MPM. This study uses the MPM as a classification technique. Two models {Linear Minimax Probability Machine (LMPM) and Kernelized Minimax Probability Machine (KMPM)} have been developed. The generalization capability of the developed models has been checked by a case study. The experimental results demonstrate that MPM-based approaches are promising tools for the prediction of the stability status of epimetamorphic rock slope.

A DC programming approach for feature selection in the Minimax Probability Machine

AbstractThis paper presents a new feature selection framework based on the L0-norm, in which data are summarized by their moments of the class conditional densities. However, discontinuity of the L0-norm makes it difficult to find the optimal solution. We apply a proper approximation of the L0-norm and a bound on the misclassification probability involving the mean and covariance of the dataset, to derive a robust difference of convex functions (DC) program formulation, while the DC optimization algorithm is used to solve the problem effectively. Furthermore, a kernelized version of this problem is also presented in this work. Experimental results on both real and synthetic datasets show that the proposed formulations can select fewer features than the traditional Minimax Probability Machine and the L1-norm state.

Performance assessment of genetic programming (GP) and minimax probability machine regression (MPMR) for prediction of seismic ultrasonic attenuation

The determination of seismic attenuation (s) (dB/cm) is a challenging task in earthquake science. This article employs genetic programming (GP) and minimax probability machine regression (MPMR) for prediction of s. GP is developed based on genetic algorithm. MPMR maximizes the minimum probability of future predictions being within some bound of the true regression function. Porosity (n) (%), permeability (k) (millidarcy), grain size (d) (μm), and clay content (c) (%) have been considered as inputs of GP and MPMR. The output of GP and MPMR is s. The developed GP gives an equation for prediction of s. The results of GP and MPMR have been compared with the artificial neural network. This article gives robust models based on GP and MPMR for prediction of s.

Training Effective Node Classifiers for Cascade Classification

Cascade classifiers are widely used in real-time object detection. Different from conventional classifiers that are designed for a low overall classification error rate, a classifier in each node of the cascade is required to achieve an extremely high detection rate and moderate false positive rate. Although there are a few reported methods addressing this requirement in the context of object detection, there is no principled feature selection method that explicitly takes into account this asymmetric node learning objective. We provide such an algorithm here. We show that a special case of the biased minimax probability machine has the same formulation as the linear asymmetric classifier (LAC) of Wu et al. (linear asymmetric classifier for cascade detectors, 2005). We then design a new boosting algorithm that directly optimizes the cost function of LAC. The resulting totally-corrective boosting algorithm is implemented by the column generation technique in convex optimization. Experimental results on object detection verify the effectiveness of the proposed boosting algorithm as a node classifier in cascade object detection, and show performance better than that of the current state-of-the-art.

Laplacian minimax probability machine

In this paper, we propose a Laplacian minimax probability machine, which is a semi-supervised version of minimax probability machine based on the manifold regularization framework. We also show that the proposed method can be kernelized on the basis of a theorem similar to the representer theorem for non-linear cases. Experiments confirm that the proposed methods achieve competitive results, as compared to existing graph-based learning methods such as the Laplacian support vector machine and the Laplacian regularized least square, for publicly available datasets from the UCI machine learning repository.

A Unified Classification Model Based on Robust Optimization

A wide variety of machine learning algorithms such as the support vector machine (SVM), minimax probability machine (MPM), and Fisher discriminant analysis (FDA) exist for binary classification. The purpose of this letter is to provide a unified classification model that includes these models through a robust optimization approach. This unified model has several benefits. One is that the extensions and improvements intended for SVMs become applicable to MPM and FDA, and vice versa. For example, we can obtain nonconvex variants of MPM and FDA by mimicking Perez-Cruz, Weston, Hermann, and Schölkopf's (2003) extension from convex ν-SVM to nonconvex Eν-SVM. Another benefit is to provide theoretical results concerning these learning methods at once by dealing with the unified model. We give a statistical interpretation of the unified classification model and prove that the model is a good approximation for the worst-case minimization of an expected loss with respect to the uncertain probability distribution. We also propose a nonconvex optimization algorithm that can be applied to nonconvex variants of existing learning methods and show promising numerical results.

Robust extreme learning machine

The output weights computing of extreme learning machine (ELM) encounters two problems, the computational and outlier robustness problems. The computational problem occurs when the hidden layer output matrix is a not full column rank matrix or an ill-conditioned matrix because of randomly generated input weights and biases. An existing solution to this problem is Singular Value Decomposition (SVD) method. However, the training speed is still affected by the large complexity of SVD when computing the Moore–Penrose (MP) pseudo inverse. The outlier robustness problem may occur when the training data set contaminated with outliers then the accuracy rate of ELM is extremely affected. This paper proposes the Extended Complete Orthogonal Decomposition (ECOD) method to solve the computational problem in ELM weights computing via ECODLS algorithm. And the paper also proposes the other three algorithms, i.e. the iteratively reweighted least squares (IRWLS-ELM), ELM based on the multivariate least-trimmed squares (MLTS-ELM), and ELM based on the one-step reweighted MLTS (RMLTS-ELM) to solve the outlier robustness problem. However, they also encounter the computational problem. Therefore, the ECOD via ECODLS algorithm is also used successfully in the three proposed algorithms. The experiments of regression problems were conducted on both toy and real-world data sets. The outlier types are one-sided and two-sided outliers. Each experiment was randomly contaminated with outliers, of one type only, with 10%, 20%, 30%, 40%, and 50% of the total training data size. Meta-metrics evaluation was used to measure the outlier robustness of the proposed algorithms compared to the existing algorithms, i.e. the minimax probability machine regression (MPMR) and the ordinary ELM. The experimental results showed that ECOD can effectively replace SVD. The ECOD is robust to the not full column rank or the ill-conditional problem. The speed of the ELM training using ECOD is also faster than the ordinary training algorithm. Moreover, the meta-metrics measure showed that the proposed algorithms are less affected by the increasing number of outliers than the existing algorithms.

One class boundary method classifiers for application in a video-based fall detection system

In this study, the authors introduce a video-based robust fall detection system for monitoring an elderly person in a smart room environment. Video features, namely the centroid and orientation of a voxel person, are extracted. The boundary method, which is an example of one class classification technique, is then used to determine whether the incoming features lie in the `fall region` of the feature space, and thereby effectively distinguishing a fall from other activities, such as walking, sitting, standing, crouching or lying. Four different types of boundary methods, k -centre, k th nearest neighbour, one class support vector machine and single class minimax probability machine (SCMPM) are assessed on representative test datasets. The comparison is made on the following three aspects: (i) true positive rate, false positive rate and geometric means in detection. (ii) Robustness to noise in the training dataset. (iii) The computational time for the test phase. From the comparison results, the authors show that the SCMPM achieves the best overall performance. By applying one class classification techniques with three-dimensional (3-d) features, the authors can obtain a more efficient fall detection system with acceptable performance, as shown in the experimental part; besides, it can avoid the drawbacks of other traditional fall detection methods.

Simultaneous feature selection and classification via Minimax Probability Machine

Simultaneous feature selection and classification via Minimax Probability Machine

Simultaneous feature selection and classification via Minimax Probability Machine

This paper presents a novel method for simultaneous feature selection and classification by incorporating a robust L1-norm into the objective function of Minimax Probability Machine (MPM). A fractional programming framework is derived by using a bound on the misclassification error involving the mean and covariance of the data. Furthermore, the problems are solved by the Quadratic Interpolation method. Experiments show that our methods can select fewer features to improve the generalization compared to MPM, which illustrates the effectiveness of the proposed algorithms.

A biased minimax probability machine-based scheme for relevance feedback in image retrieval

In recent years, minimax probability machines (MPMs) have demonstrated excellent performance in a variety of pattern recognition problems. At the same time various machine learning methods have been applied on relevance feedback tasks in content-based image retrieval (CBIR). One of the problems in typical techniques for relevance feedback is that they treat the relevant feedback and irrelevant feedback equally. Since the negative instances largely outnumber the positive instances, the assumption that they are balanced is incorrect as the data are biased. In this paper we study how biased minimax probability machine (BMPM), a variation of MPM, can be applied for relevance feedback in image retrieval tasks. Different from previous methods, this model directly controls the accuracy of classification of the future data to construct biased classifiers. Hence, it provides a rigorous treatment on imbalanced dataset. Mathematical formulation and explanations are provided to demonstrate the advantages. Experiments are conducted to evaluate the performance of our proposed framework, in which encouraging and promising experimental results are obtained.

Maxi–Min Margin Machine: Learning Large Margin Classifiers Locally and Globally

In this paper, we propose a novel large margin classifier, called the maxi-min margin machine M(4). This model learns the decision boundary both locally and globally. In comparison, other large margin classifiers construct separating hyperplanes only either locally or globally. For example, a state-of-the-art large margin classifier, the support vector machine (SVM), considers data only locally, while another significant model, the minimax probability machine (MPM), focuses on building the decision hyperplane exclusively based on the global information. As a major contribution, we show that SVM yields the same solution as M(4) when data satisfy certain conditions, and MPM can be regarded as a relaxation model of M(4). Moreover, based on our proposed local and global view of data, another popular model, the linear discriminant analysis, can easily be interpreted and extended as well. We describe the M(4) model definition, provide a geometrical interpretation, present theoretical justifications, and propose a practical sequential conic programming method to solve the optimization problem. We also show how to exploit Mercer kernels to extend M(4) for nonlinear classifications. Furthermore, we perform a series of evaluations on both synthetic data sets and real-world benchmark data sets. Comparison with SVM and MPM demonstrates the advantages of our new model.

Robust BMPM training based on second-order cone programming and its application in medical diagnosis

The Biased Minimax Probability Machine (BMPM) constructs a classifier which deals with the imbalanced learning tasks. It provides a worst-case bound on the probability of misclassification of future data points based on reliable estimates of means and covariance matrices of the classes from the training data samples, and achieves promising performance. In this paper, we develop a novel yet critical extension training algorithm for BMPM that is based on Second-Order Cone Programming (SOCP). Moreover, we apply the biased classification model to medical diagnosis problems to demonstrate its usefulness. By removing some crucial assumptions in the original solution to this model, we make the new method more accurate and robust. We outline the theoretical derivatives of the biased classification model, and reformulate it into an SOCP problem which could be efficiently solved with global optima guarantee. We evaluate our proposed SOCP-based BMPM (BMPMSOCP) scheme in comparison with traditional solutions on medical diagnosis tasks where the objectives are to focus on improving the sensitivity (the accuracy of the more important class, say "ill" samples) instead of the overall accuracy of the classification. Empirical results have shown that our method is more effective and robust to handle imbalanced classification problems than traditional classification approaches, and the original Fractional Programming-based BMPM (BMPMFP).

利用得分层多实例融合改进虹膜识别性能

Fusion of multiple instances within a modality for biometric verification performance improvement has received considerable attention. In this letter, we present an iris recognition method based on multi-instance fusion, which combines the left and right irises of an individual at the matching score level. When fusing, a novel fusion strategy using minimax probability machine (MPM) is applied to generate a fused score for the final decision. The experimental results on CASIA and UBIRIS databases show that the proposed method can bring obvious performance improvement compared with the single-instance method. The comparison among different fusion strategies demonstrates the superiority of the fusion strategy based on MPM.

Minimax Probability Machine with Genetic Feature Optimized for Intrusion Detection

Minimax Probability Machine with Genetic Feature Optimized for Intrusion Detection

FEATURE SELECTION BASED ON MINIMUM ERROR MINIMAX PROBABILITY MACHINE

Feature selection is an important task in pattern recognition. Support Vector Machine (SVM) and Minimax Probability Machine (MPM) have been successfully used as the classification framework for feature selection. However, these paradigms cannot automatically control the balance between prediction accuracy and the number of selected features. In addition, the selected feature subsets are also not stable in different data partitions. Minimum Error Minimax Probability Machine (MEMPM) has been proposed for classification recently. In this paper, we outline MEMPM to select the optimal feature subset with good stability and automatic balance between prediction accuracy and the size of feature subset. The experiments against feature selection with SVM and MPM show the advantages of the proposed MEMPM formulation in stability and automatic balance between the feature subset size and the prediction accuracy.

Prediction of chaotic time series based on modified minimax probability machine regression

Long-term prediction of chaotic time series is very difficult, for the chaos restricts predictability. In this paper a new method is studied to model and predict chaotic time series based on minimax probability machine regression (MPMR). Since the positive global Lyapunov exponents lead the errors to increase exponentially in modelling the chaotic time series, a weighted term is introduced to compensate a cost function. Using mean square error (MSE) and absolute error (AE) as a criterion, simulation results show that the proposed method is more effective and accurate for multistep prediction. It can identify the system characteristics quite well and provide a new way to make long-term predictions of the chaotic time series.

A comparative study of Minimax Probability Machine-based approaches for face recognition

Automatic face recognition is a challenging problem in the biometric recognition area. Minimax Probability Machine (MPM) and its extension, Minimum Error Minimax Probability Machine, have shown advantages in the machine learning literature. In this paper, we incorporate the MPM-based approaches into our face recognition system for further study. To test the performance of our new system, we compare the MPM-based approaches with SVM, a PCA-based and a LDA-based algorithms on the FERET database for both verification and identification. The experimental results demonstrate that MPM-based approaches are promising for automatic face recognition.