Multiple Hyperplanes Research Articles

An improved equilibrium efficient frontier data envelopment analysis approach for evaluating decision-making units with fixed-sum outputs

In recent years, significant development has been made in efficiency evaluation for Decision-Making Units (DMUs) with fixed-sum outputs. The Generalized Equilibrium Efficient Frontier Data Envelopment Analysis (GEEFDEA) approach introduced by Yang et al. (2015) is one of the most representative methods. In the GEEFDEA approach, all DMUs are adjusted to become efficient under the same set of weights, indicating that the Equilibrium Efficient Frontier (EEF) constructed consists of only one hyperplane. However, in practical scenarios under the Variable Return to Scale (VRS) assumption, the production frontier always consists of multiple hyperplanes, presenting a more complex shape. To fill this research gap, we propose an improved Equilibrium Efficient Frontier Data Envelopment Analysis approach. Our approach allows DMUs to have different weights for inputs, variable-sum outputs, and fixed-sum outputs, resulting in an EEF with multiple hyperplanes. It is noted that our new approach uses a non-linear model to obtain the EEF. We show that the model can be linearized in the case of a single fixed-sum output. In situations involving multiple fixed-sum outputs, we propose an algorithm based on the Expectation Maximization (EM) mechanism to solve the model to obtain an EEF. Finally, we illustrate the advantages of our new approach through a numerical example and a case study in the global motor vehicle industry.

Incremental estimation of low-density separating hyperplanes for clustering large data sets

An efficient unsupervised method for obtaining low-density hyperplane separators is proposed. The method is based on a modified stochastic gradient descent applied on a convolution of the empirical distribution function with a smoothing kernel. Low-density hyperplanes are motivated by the fact that they avoid intersecting high density regions, and so tend to pass between high density clusters, thus separating them from one another, while keeping the individual clusters intact. Multiple hyperplanes can be combined in a hierarchical model to obtain a complete clustering solution. A simple post-processing of solutions induced by large collections of hyperplanes yields an efficient and accurate clustering method, capable of automatically selecting the number of clusters. Experiments show that the proposed method is highly competitive in terms of both speed and accuracy when compared with relevant benchmarks. Code is available in the form of an R package at https://github.com/DavidHofmeyr/iMDH.

Research on Tower Mechanical Fault Classification Method Based on Multiclass Central Segmentation Hyperplane Support Vector Machine Improvement Algorithm

In this paper, a classification recognition algorithm for tower mechanical faults is proposed, and a multiclass central segmentation hyperplane support vector machine (CSH-SVM) is proposed to improve the existing multiclass support vector machine for problems in which a certain sample satisfies multiple hyperplanes at the same time. The tilt angle change and wind direction data were extracted using the tilt sensors and anemometers attached to the tower, and the temperature and humidity sensors, as well as real-time rainfall and water accumulation information, were combined to construct a sample of the original dataset during the operation of the tower. The unbalanced samples were improved using the synthetic minority oversampling technique (SMOTE) algorithm to construct a balanced dataset suitable for machine learning and improve the prediction accuracy of machine learning. At the same time, the support vector machine hyperplane under the one-vs-all classification principle was additionally computed, and the new hyperplane was computed via the existing hyperplane not only to solve the classification problem of the transition area under the one-vs-all classification so that the samples located in this area no longer meet two hyperplane equations at the same time, but also to reduce the probability of incorrect classification to a certain extent. Through verification, CSH-SVM can classify 15 out of 77 misclassified samples into the correct category with slightly higher computational power than the traditional one-vs-all classification SVM, which can improve the classification prediction accuracy for unbalanced tower mechanical failure datasets and make an accurate judgment on the current state of the tower through the tower data as to when the tower may generate mechanical failure, thus reducing economic loss and personal safety threats.

Evolving multi-label fuzzy classifier with advanced robustness respecting human uncertainty

Multi-label classification has attracted much attention in the machine learning community to address the problem of assigning single samples to more than one class at the same time. We propose an evolving multi-label fuzzy classifier (EFC-ML-FWU) which is able to self-adapt and self-evolve its structure and consequent parameters in the form of multiple hyper-planes with new incoming multi-label samples in an incremental, single-pass manner and which especially addresses the intrinsic curse of dimensionality as well as human label uncertainty problems, often apparent in multi-label classification problems, to ensure the advanced robustness of the learned structure and parameters. The former is achieved by integrating feature weights into the learning process, specifically designed for online multi-label classification problems in an incremental manner, measuring the impact of features with respect to their discriminatory power. The features are integrated (i) into the rule evolution criterion, leading to a shrinkage of distances along unimportant dimensions, which reduces the likelihood of unnecessary rule evolution and thus decreases over-fitting due to the curse of dimensionality, (ii) into the first consequent learning part by a variable-regularized RFWLS approach realized through an incremental coordinate descent algorithm, and (iii) into the second consequent learning part employing correlation-based preservation learning by using weight-based thresholds (extending the classical Lipschitz constant-based threshold) within soft shrinkage operations to optimize a feature-based weighted L1-norm on the consequent parameters. Uncertainty in class labels is handled by an integration of sample weights, where lower weights indicate a higher uncertainty in the labels carried by a sample. This leads to (i) a weighted updating of the incremental feature weights, (ii) a weighted update of the rule antecedent space through a weighted incremental clustering process, and (iii) a specific weighted update of the consequent parameters exploring a single-label and a multi-label view of uncertainty. Our approach was evaluated on several data sets from the MULAN repository and showed significantly improved classification accuracy and average precision trend lines compared to alternative (evolving) one-versus-rest or classifier chaining concepts, and especially improved the native EFC-ML method without feature weights and uncertainty handling with performance gains up to 17% in the AUC of the accuracy trends. Furthermore, interesting insights into an improved robustness of the multi-label classifier (i) in the case of wrong labels due to low user experience levels and (ii) in the case of low label certainties but potentially correct labels were obtained.

MCMS-STM: An Extension of Support Tensor Machine for Multiclass Multiscale Object Recognition in Remote Sensing Images

The support tensor machine (STM) extended from support vector machine (SVM) can maintain the inherent information of remote sensing image (RSI) represented as tensor and obtain effective recognition results using a few training samples. However, the conventional STM is binary and fails to handle multiclass classification directly. In addition, the existing STMs cannot process objects with different sizes represented as multiscale tensors and have to resize object slices to a fixed size, causing excessive background interferences or loss of object’s scale information. Therefore, the multiclass multiscale support tensor machine (MCMS-STM) is proposed to recognize effectively multiclass objects with different sizes in RSIs. To achieve multiclass classification, by embedding one-versus-rest and one-versus-one mechanisms, multiple hyperplanes described by rank-R tensors are built simultaneously instead of single hyperplane described by rank-1 tensor in STM to separate input with different classes. To handle multiscale objects, multiple slices of different sizes are extracted to cover the object with an unknown class and expressed as multiscale tensors. Then, M-dimensional hyperplanes are established to project the input of multiscale tensors into class space. To ensure an efficient training of MCMS-STM, a decomposition algorithm is presented to break the complex dual problem of MCMS-STM into a series of analytic sub-optimizations. Using publicly available RSIs, the experimental results demonstrate that the MCMS-STM achieves 89.5% and 91.4% accuracy for classifying airplanes and ships with different classes and sizes, which outperforms typical SVM and STM methods.

An Approach to Supervised Learning: Dynamic Multi-Hyperplane Partitioning

Supervised learning has tremendous applications in cancer prediction, patient treatment, business, and engineering. Datasets for supervised learning are often corrupted by noise and non-biological effects, leading to model-overfitting and performance degradation in current methods of binary classification. In this research, we develop a new classification method that fully exploits unique characteristics, such as persistent outliers, anomalies from a batch effect, and hidden relationships between features and their classes in the datasets, hence improving classification performance of current methods. The proposed method, called dynamic multi-hyperplane partitioning (DMP), learns the model by using subclassifiers, which are random in number and each of which uses multiple hyperplanes for decision boundaries. We also develop a method to transform samples to improve classification performance of DMP. We prove that, under a mild condition, accuracy of DMP is as good as or supersedes that of support vector machine (SVM). We test DMP on comprehensive datasets, which span diverse fields of applications, and compare accuracy, sensitivity, specificity, F-measure, and the receiver operating characteristic of DMP to those of competitive baselines, including SVM, random forest, Bayes classifier, gradient boosting tree, and deep-belief nets and neural nets. From the comparison, the proposed method is most accurate in nine out of eleven datasets, when using the mean values alone for comparison. DMP achieves 100% accuracy, 100% sensitivity, and 100% specificity in three datasets. As a generalization, we perform statistical test of difference, at significance levels of 0.05, 0.01 and 0.001. From statistical tests, DMP is the most accurate or one of the most accurate classifiers in nine out of eleven benchmark datasets., and is not the most accurate classifier in the remaining two datasets. The DMP learning method is accurate, simple to implement, and does not require fine-tuning of parameters, making it attractive for binary classification. This research has practical applications and leads to a timely and accurate approach to binary classification in diverse fields.

ACM sheaves on the double plane

The goal of this paper is to start a study of aCM and Ulrich sheaves on non-integral projective varieties. We show that any aCM vector bundle of rank two on the double plane is a direct sum of line bundles. As a by-product, any aCM vector bundle of rank two on a sufficiently high dimensional quadric hypersurface also splits. We consider aCM and Ulrich vector bundles on multiple hyperplanes and prove the existence of such bundles that do not split if the multiple hyperplane is linearly embedded into a sufficiently high dimensional projective space. Then we restrict our attention to the double plane and give a classification of aCM sheaves of rank at most 3 / 2 3/2 on the double plane and describe the family of isomorphism classes of them.

A visualization method of four-dimensional polytopes by oval display of parallel hyperplane slices

A method to visualize polytopes in a four-dimensional euclidian space (x, y, z, w) is proposed. A polytope is sliced by multiple hyperplanes that are parallel to each other and separated by uniform intervals. Since the hyperplanes are perpendicular to the w-axis, the resulting multiple slices appear in the three-dimensional (x, y, z) space and they are shown by the standard computer graphics. The polytope is rotated extrinsically in the four-dimensional space by means of a simple input method based on keyboard typings. The multiple slices are placed on a parabola curve in the three-dimensional world coordinates. The slices in a view window form an oval appearance. Both the simple and the double rotations in the four-dimensional space are applied to the polytope. All slices synchronously change their shapes when a rotation is applied to the polytope. The compact display in the oval of many slices with the help of quick rotations facilitate a grasp of the four-dimensional configuration of the polytope.

One-pass online learning: A local approach

Online learning is very important for processing sequential data and helps alleviate the computation burden on large scale data as well. Especially, one-pass online learning is to predict a new coming sample׳s label and update the model based on the prediction, where each coming sample is used only once and never stored. So far, existing one-pass online learning methods are globally modeled and do not take the local structure of the data distribution into consideration, which is a significant factor of handling the nonlinear data separation case. In this work, we propose a local online learning (LOL) method, a multiple hyperplane Passive Aggressive algorithm integrated with online clustering, so that all local hyperplanes are learned jointly and working cooperatively. This is achieved by formulating a common component as information traffic among multiple hyperplanes in LOL. A joint optimization algorithm is proposed and theoretical analysis on the cumulative error is also provided. Extensive experiments on 11 datasets show that LOL can learn a nonlinear decision boundary, overall achieving notably better performance without using any kernel modeling and second order modeling.

Connected sum at infinity and Cantrell-Stallings hyperplane unknotting

We give a general treatment of the somewhat unfamiliar operation on manifolds called Connected Sum at Infinity, or CSI for short. A driving ambition has been to make the geometry behind the well definition and basic properties of CSI as clear and elementary as possible. CSI then yields a very natural and elementary proof of a remarkable theorem of J. C. Cantrell and J. R. Stallings. It asserts unknotting of proper embeddings of euclidean (m-1)-space in euclidean m-space with m not equal to 3, for all three classical manifold categories: topological, piecewise linear, and differentiable. It is one of the few major theorems whose statement and proof can be the same for all three categories. We give it the acronym HLT, which is short for Hyperplane Linearization Theorem. The topological version of the HLT immediately implies B. Mazur's topological Schoenflies theorem. We can thus claim that the Cantrell-Stallings theorem, as we present it, is an enhancement of the topological Schoenflies theorem that has exceptional didactic value. We also prove a classification of multiple codimension 1 hyperplane embeddings in eucldiean m-space for m not equal to 3. Namely, they are classified by countable simplicial trees with one edge for each hyperplane (planar trees for m=2). This result is called the Multiple Hyperplane Linearization Theorem, or MHLT for short. We give an exposition of C. Greathouse's Slab Theorem, and in conclusion some possibly novel proofs of the 2-dimensional MHLT and related results classifying contractible 2-manifolds with boundary.

Query Dependent Learning to Model Based on Ordered Multiple Hyperplanes

This paper proposes a ranking model that trains different hyperplanes for different queries and optimizes hyperplanes with the order relations.It aims at solving the problem of most existing rank methods that do not consider the significant differences between queries and only resort to a single function that is time consuming.Next,a weighted voting method is proposed to aggregate the ranking lists of the hyperplanes as the final rank.The weights reflect the degree of precision.Effectiveness is tested by the benchmark data set LETOR OHSUMED and is compare with other ranking models.The proposed method shows improved ranking performance with a significant reduction of training time.

QoRank: A query-dependent ranking model using LSE-based weighted multiple hyperplanes aggregation for information retrieval

Ranking is a core problem for information retrieval since the performance of the search system is directly impacted by the accuracy of ranking results. Ranking model construction has been the focus of both the fields of information retrieval and machine learning, and learning to rank in particular has attracted much interest. Many ranking models have been proposed, for example, RankSVM is a state-of-the-art method for learning to rank and has been empirically demonstrated to be effective. However, most of the proposed methods do not consider about the significant differences between queries, only resort to a single function in ranking. In this paper, we present a novel ranking model named QoRank, which performs the learning task dependent on queries. We also propose a LSE (least-squares estimation) -based weighted method to aggregate the ranking lists produced by base decision functions as the final ranking. Comparison of QoRank with other ranking techniques is conducted, and several evaluation criteria are employed to evaluate its performance. Experimental results on the LETOR OHSUMED data set show that QoRank strikes a good balance of accuracy and complexity, and outperforms the baseline methods. © 2010 Wiley Periodicals, Inc.

Time-Adaptive Support Vector Machines

In this work we propose an adaptive classification method able both to learn and to follow the temporal evolution of a drifting concept. With that purpose we introduce a modified SVM classifier, created using multiple hyperplanes valid only at small temporal intervals (windows). In contrast to other strategies proposed in the literature, our method learns all hyperplanes in a global way, minimizing a cost function that evaluates the error committed by this family of local classifiers plus a measure associated to the VC dimension of the family. We also show how the idea of slowly changing classifiers can be applied to non-linear stationary concepts with results similar to those obtained with normal SVMs using gaussian kernels.

Minimax Input Shaper Design Using Linear Programming

The focus of this paper is on the design of robust input shapers where the maximum value of the cost function over the domain of uncertainty is minimized. This nonlinear programming problem is reformulated as a linear programming problem by approximating a n-dimensional hypersphere with multiple hyperplanes (as in a geodesic dome). A recursive technique to approximate a hypersphere to any level of accuracy is developed using barycentric coordinates. The proposed technique is illustrated on the spring-mass-dashpot and the benchmark floating oscillator problem undergoing a rest-to-rest maneuver. It is shown that the results of the linear programming problem are nearly identical to that of the nonlinear programming problem.

Evolving fuzzy classifiers using different model architectures

In this paper we present two novel approaches for on-line evolving fuzzy classifiers, called eClass and FLEXFIS-Class. Both methods can be applied with different model architectures, including single model (SM) with class labels as consequents, classification hyper-planes as consequents, and multi-model (MM) architecture. Additionally, eClass can have a multi-input–multi-output (MIMO) architecture with multiple hyper-planes as consequents of each fuzzy rule. The difference between MM and MIMO architectures is that the former one applies one separate and independent fuzzy rule-based (FRB) classifier for each class and is using an indicator labelling scheme, while the latter one applies a single FRB where the rules are MIMO rather than MISO. Both, eClass and FLEXFIS-Class methods are designed to work on a per-sample basis and are thus one-pass, incremental. Additionally, their structure (FRB) is evolving rather than fixed. It adapts their parameters in antecedent and consequent parts with any newly loaded sample. A special emphasis is placed on advanced issues for improving accuracy and robustness, including a thorough comparison between global and local learning of consequent functions, a novel approach for detecting of and reacting on drifts in the data streams and an enhanced outlier treatment strategy. The methods and their extensions according to the advanced issues are evaluated on one benchmark problem of handwritten images recognition as well as on a real-life problem of image classification framework, where images should be classified into good and bad ones during an on-line and interactive production process.

Deming least-squares fit to multiple hyperplanes

A method is derived to fit a set of multidimensional experimental data points having a priori uncertainties and possibly also covariances in all coordinates to a straight line, plane, or hyperplane of any dimensionality less than the number of coordinates. The least-squares formulation used is that of Deming, which treats all coordinates on an equal basis. Experimentalists needing to fit a linear model to data of this kind have usually performed multiple independent fits in subspaces of the full data space such that each fit has only one dependent coordinate. That procedure does not guarantee mutual consistency of the fits. The present method can be thought of as providing multiple such hyperplane fits in a single simultaneous and therefore consistent solution. As examples, the method is applied to a straight-line fit in three dimensions to synthetic data and to an analysis of xenon isotopes in a lunar rock.

Support Vector Machine Implementations for Classification & Clustering

BackgroundWe describe Support Vector Machine (SVM) applications to classification and clustering of channel current data. SVMs are variational-calculus based methods that are constrained to have structural risk minimization (SRM), i.e., they provide noise tolerant solutions for pattern recognition. The SVM approach encapsulates a significant amount of model-fitting information in the choice of its kernel. In work thus far, novel, information-theoretic, kernels have been successfully employed for notably better performance over standard kernels. Currently there are two approaches for implementing multiclass SVMs. One is called external multi-class that arranges several binary classifiers as a decision tree such that they perform a single-class decision making function, with each leaf corresponding to a unique class. The second approach, namely internal-multiclass, involves solving a single optimization problem corresponding to the entire data set (with multiple hyperplanes).ResultsEach SVM approach encapsulates a significant amount of model-fitting information in its choice of kernel. In work thus far, novel, information-theoretic, kernels were successfully employed for notably better performance over standard kernels. Two SVM approaches to multiclass discrimination are described: (1) internal multiclass (with a single optimization), and (2) external multiclass (using an optimized decision tree). We describe benefits of the internal-SVM approach, along with further refinements to the internal-multiclass SVM algorithms that offer significant improvement in training time without sacrificing accuracy. In situations where the data isn't clearly separable, making for poor discrimination, signal clustering is used to provide robust and useful information – to this end, novel, SVM-based clustering methods are also described. As with the classification, there are Internal and External SVM Clustering algorithms, both of which are briefly described.

Video object segmentation and tracking using /spl psi/-learning classification

As a requisite of the emerging content-based multimedia technologies, video object (VO) extraction is of great importance. This paper presents a novel semiautomatic segmentation and tracking method for single VO extraction. Unlike traditional approaches, the proposed method formulates the separation of the VO from the background as a classification problem. Each frame is divided into small blocks of uniform size, which are called object blocks if the centering pixels belong to the object, or background blocks otherwise. After a manual segmentation of the first frame, the blocks of this frame are used as the training samples for the object-background classifier. A newly developed learning tool called /spl psi/-learning is employed to train the classifier which outperforms the conventional Support Vector Machines in linearly nonseparable cases. To deal with large and complex objects, a multilayer approach constructing a so-called hyperplane tree is proposed. Each node of the tree represents a hyperplane, responsible for classifying only a subset of the training samples. Multiple hyperplanes are thus needed to classify the entire set. Through the combination of the multilayer scheme and /spl psi/-learning, one can avoid the complexity of nonlinear mapping as well as achieve high classification accuracy. During the tracking phase, the pixel in the center of every block in a successive frame is classified by a sequence of hyperplanes from the root to a leaf node of the hyperplane tree, and the class of the block is identified accordingly. All the object blocks thus form the object of interest, whose boundary unfortunately is stair-like due to the block effect. In order to obtain the pixel-wise boundary in a cost efficient way, a pyramid boundary refining algorithm is designed, which iteratively selects a few informative pixels for class label checking, and reduces uncertainty about the actual boundary of the object. The proposed method has been applied on video sequences with various spatial and temporal characteristics, and experimental results demonstrate it to be effective, efficient, and robust.

Ellipsoid Set Refinement by Simultaneous Use of Multiple Hyperplane Cuts

Let Hn+, Hn-, n = 1,2,… denote a sequence of pairs of parallel hyperplanes in Rm, and, for each n, let Hn denote the set of points on the open "hyperstrip" between the planes Hn+ andHn-. It is assumed that the polytopic set Pk = ∩n=1kHn is not empty for any k. Further let εn, n = 1,2,… denote a corresponding sequence of hyperellipsoids, and let εn ∈ Rm denote the open hyperellipsoidal set bounded by εn. This paper derives a mathematical formulation for fitting a new, smaller hypervolume, hyperellipsoid, εn, to the intersection ε1 ∩ H1 ∩ H2 ∩ … ∩ Hk = ε1 ∩ Pk. The prevailing method for solving this problem involves sequential refinement of ellipsoids as hyperstrips are considered one at a time. The simultaneous use of multiple hyperstrips results in a better fit to the polytope Pk than that achieved by sequential refinement. The problem arises in numerous control and signal processing problems - in particular the broad class of ellipsoid bounding algorithms used for identification and classification.

Support Vector Machine Decision Trees with Rare Event Detection

Model selection and rare event detection are two major problems frequently encountered in support vector learning. This paper proposes a new support vector learning algorithm, to be referred to as Linear Support Vector Machine Decision Tree (LSVM-DT), that solves the two problems. It consists of a binary tree structure with linear support vector machines in all tree nodes and class labels in all leaves. During training, multiple linear hyperplanes are constructed while traversing down the tree. The LSVM-DT is capable of separating both linearly and nonlinearly separable data. The only parameter that needs to be chosen by the user is the regularization parameter C , thus eliminating the model selection problem. Its built-in rare event detection mechanism allows the LSVM-DT to solve classification problems with underrepresented or unproportional classes. This phenomenon occurs within the tree even if the classes initially have equal prior probabilities. Experiments with different data sets show that the LSV...