Monotonic Classification Research Articles

Evolutionary Fuzzy Rule-Based Methods for Monotonic Classification

In data science applications, it is very often to require predictive models satisfying monotonicity with respect to the explanatory variables involved in the dataset. In ordinal classification or regression, this occurs when the output variable or class label do not decrease when input variables increase, or vice versa. This problem is commonly known as monotonic classification, and most existing classification techniques are not able to manage this kind of constraints or they require first to monotonize the data. In the literature, the monotonicity has been considered in linguistic fuzzy models, fuzzy-inference methods, and fuzzy rule-based control systems. However, to the best of our knowledge, there is no fuzzy rule-based system designed to produce monotonic fuzzy rule-based models for classification problems. In this paper, we propose to incorporate some mechanisms based on monotonicity indexes for addressing such problems in two popular and competitive evolutionary fuzzy systems algorithms for classification and regression tasks: FARC-HD and FSmogfs $^e$ + Tun $^e$ . In addition, the proposals are able to handle any kind of classification dataset without the necessity of preprocessing. The quality of our approaches is analyzed using statistical analysis and comparing with well-known monotonic classifiers.

Partially monotonic decision trees

In multicriteria decision tasks, certain features are linearly ordered according to the decision and are called criteria, whereas others, called regular attributes, are not. In practice, regular attributes and criteria coexist in most classification tasks. In this paper, we propose a rank-inconsistent rate that distinguishes attributes from criteria. Furthermore, it represents the directions of the monotonic relationships between criteria and decisions. We design a partially monotonic decision tree algorithm to extract decision rules for partially monotonic classification tasks. Experimental results show that the proposed algorithm is effective and efficient.

Fusing Complete Monotonic Decision Trees

Monotonic classification is a kind of classification task in which a monotonicity constraint exist between features and class, i.e., if sample $x_i$ has a higher value in each feature than sample $x_j$ , it should be assigned to a class with a higher level than the level of $x_j$ 's class. Several methods have been proposed, but they have some limits such as with limited kind of data or limited classification accuracy. In our former work, the classification accuracy on monotonic classification has been improved by fusing monotonic decision trees, but it always has a complex classification model. This work aims to find a monotonic classifier to process both nominal and numeric data by fusing complete monotonic decision trees. Through finding the completed feature subsets based on discernibility matrix on ordinal dataset, a set of monotonic decision trees can be obtained directly and automatically, on which the rank is still preserved. Fewer decision trees are needed, which will serve as base classifiers to construct a decision forest fused complete monotonic decision trees. The experiment results on 10 datasets demonstrate that the proposed method can reduce the number of base classifiers effectively and then simplify classification model, and obtain good classification performance simultaneously.

A Method for Monotonic Classification Based on Decision Forest

A Method for Monotonic Classification Based on Decision Forest

Prototype selection to improve monotonic nearest neighbor

Student surveys occupy a central place in the evaluation of courses at teaching institutions. At the end of each course, students are requested to evaluate various aspects such as activities, methodology, coordination or resources used. In addition, a final qualification is given to summarize the quality of the course. The prediction of this final qualification can be accomplished by using monotonic classification techniques. The outcome offered by these surveys is particularly significant for faculty and teaching staff associated with the course.The monotonic nearest neighbor classifier is one of the most relevant algorithms in monotonic classification. However, it does suffer from two drawbacks, (a) inefficient execution time in classification and (b) sensitivity to no monotonic examples. Prototype selection is a data reduction process for classification based on nearest neighbor that can be used to alleviate these problems. This paper proposes a prototype selection algorithm called Monotonic Iterative Prototype Selection (MONIPS) algorithm. Our objective is two-fold. The first one is to introduce MONIPS as a method for obtaining monotonic solutions. MONIPS has proved to be competitive with classical prototype selection solutions adapted to monotonic domain. Besides, to further demonstrate the good performance of MONIPS in the context of a student survey about taught courses.

RULEM: A novel heuristic rule learning approach for ordinal classification with monotonicity constraints

In many real world applications classification models are required to be in line with domain knowledge and to respect monotone relations between predictor variables and the target class, in order to be acceptable for implementation. This paper presents a novel heuristic approach, called RULEM, to induce monotone ordinal rule based classification models. The proposed approach can be applied in combination with any rule- or tree-based classification technique, since monotonicity is guaranteed in a post-processing step. RULEM checks whether a rule set or decision tree violates the imposed monotonicity constraints and existing violations are resolved by inducing a set of additional rules which enforce monotone classification. The approach is able to handle non-monotonic noise, and can be applied to both partially and totally monotone problems with an ordinal target variable. Two novel justifiability measures are introduced which are based on RULEM and allow to calculate the extent to which a classification model is in line with domain knowledge expressed in the form of monotonicity constraints. An extensive benchmarking experiment and subsequent statistical analysis of the results on 14 public data sets indicates that RULEM preserves the predictive power of a rule induction technique while guaranteeing monotone classification. On the other hand, the post-processed rule sets are found to be significantly larger which is due to the induction of additional rules. E.g., when combined with Ripper a median performance difference was observed in terms of PCC equal to zero and an average difference equal to −0.66%, with on average 5 rules added to the rule sets. The average and minimum justifiability of the original rule sets equal respectively 92.66% and 34.44% in terms of the RULEMF justifiability index, and 91.28% and 40.1% in terms of RULEMS, indicating the effective need for monotonizing the rule sets.

Monotonic classification extreme learning machine

Monotonic classification problems mean that both feature values and class labels are ordered and monotonicity relationships exist between some features and the decision label. Extreme Learning Machine (ELM) is a single-hidden layer feedforward neural network with fast training rate and good generalization capability, but due to the existence of training error, ELM cannot be directly used to handle monotonic classification problems. This work proposes a generalization of ELM for processing the monotonic classification, named as Monotonic Classification Extreme Learning Machine (MCELM) in which the monotonicity constraints are imposed to the original ELM model. Mathematically, MCELM is a quadratic programming problem in which the monotonicity relationships are considered as constraints and the training error is the objective to be minimized. The mathematical model of MCELM not only can make the generated classifier monotonic but also can minimize the classification error. MCELM does not need to tune parameters iteratively, and therefore, keeps the advantage of extremely fast training which is the essential characteristic of ELM. MCELM does not require that the monotonic relationships existing between features and the output are consistent, which essentially relaxes the assumption of consistent monotonicity used in most existing approaches to handling monotonic classification problems. In comparison with exiting approaches to handling monotonic classification, MCELM can indeed generate a monotonicity-reserving classifier which experimentally shows a much better generalization capability on both artificial and real world datasets.

Multivariate decision trees with monotonicity constraints

Classification with monotonicity constraint is a fundamental task in social analysis, management and decision making, where a monotonic function guarantees that objects with better feature values are not assigned with worse decisions. In practice, an object may be better than another on some attributes, while worse on others. These objects are considered to be incomparable. Incomparable object pairs limit the performance of a monotone classifier. In this work, we design an algorithm to combat this issue by constructing multivariate decision trees with monotonicity constraints (MMT). The classification model is naturally deemed as an oblique tree as it discovers partitions via oblique hyperplane in the input space. Our algorithm generates the projections of the objects which are used to split the data by improved splitting criteria with rank mutual information (RMI) or rank Gini impurity (RGI). Moreover, an improved algorithm with L1-regularization is also proposed to compute the optimal subsets of features in the process of constructing the trees, which leads to a more compact tree. Experimental results show that the proposed algorithm improves the classification performance in monotone classification tasks. The proposed algorithm is also effective even if data is contaminated by non-monotonic noisy samples.

Current prospects on ordinal and monotonic classification

Ordinal classification covers those classification tasks where the different labels show an ordering relation, which is related to the nature of the target variable. In addition, if a set of monotonicity constraints between independent and dependent variables has to be satisfied, then the problem is known as monotonic classification. Both issues are of great practical importance in machine learning. Ordinal classification has been widely studied in specialized literature, but monotonic classification has received relatively low attention. In this paper, we define and relate both tasks in a common framework, providing proper descriptions, characteristics, and a categorization of existing approaches in the state-of-the-art. Moreover, research challenges and open issues are discussed, with focus on frequent experimental behaviours and pitfalls, commonly used evaluation measures and the encouragement in devoting substantial research efforts in specific learning paradigms.

Hyperrectangles Selection for Monotonic Classification by Using Evolutionary Algorithms

AbstractIn supervised learning, some real problems require the response attribute to represent ordinal values that should increase with some of the explaining attributes. They are called classification problems with monotonicity constraints. Hyperrectangles can be viewed as storing objects in Rn which can be used to learn concepts combining instance-based classification with the axis-parallel rectangle mainly used in rule induction systems. This hybrid paradigm is known as nested generalized exemplar learning. In this paper, we propose the selection of the most effective hyperrectangles by means of evolutionary algorithms to tackle monotonic classification. The model proposed is compared through an exhaustive experimental analysis involving a large number of data sets coming from real classification and regression problems. The results reported show that our evolutionary proposal outperforms other instance-based and rule learning models, such as OLM, OSDL, k-NN and MID; in accuracy and mean absolute error...

MoNGEL: monotonic nested generalized exemplar learning

In supervised prediction problems, the response attribute depends on certain explanatory attributes. Some real problems require the response attribute to represent ordinal values that should increase with some of the explaining attributes. They are called classification problems with monotonicity constraints. In this paper, we aim at formalizing the approach to nested generalized exemplar learning with monotonicity constraints, proposing the monotonic nested generalized exemplar learning (MoNGEL) method. It accomplishes learning by storing objects in $${\mathbb {R}}^n$$Rn, hybridizing instance-based learning and rule learning into a combined model. An experimental analysis is carried out over a wide range of monotonic data sets. The results obtained have been verified by non-parametric statistical tests and show that MoNGEL outperforms well-known techniques for monotonic classification, such as ordinal learning model, ordinal stochastic dominance learner and k-nearest neighbor, considering accuracy, mean absolute error and simplicity of constructed models.

Rank discrimination measures for enforcing monotonicity in decision tree induction

Monotone classification is a relatively recent topic in machine learning in which the classification function to learn is asked to guarantee a sort of monotonicity of the class with respect to attribute values. Nevertheless, real datasets are quite far from being monotone and this can sharply limit the performance of purely monotone classifiers while standard classifiers are simply insensitive to monotonicity. Here we focus on rank discrimination measures to be used in decision tree induction, i.e., functions able to measure the discrimination power of an attribute with respect to the class taking into account the monotonicity of the class with respect to the attribute. Three new measures are studied in detail and a hierarchical construction model is derived allowing the formal definition of a general rank discrimination measure. Our measures have been compared with other well-known proposals, quantifying both the accuracy and the monotonicity of the resulting binary decision tree classifiers.

Feature selection for monotonic classification via maximizing monotonic dependency

AbstractMonotonic classification is a special task in machine learning and pattern recognition. As to monotonic classification, it is assumed that both features and decision are ordinal and there is the monotonicity constraints between the features and decision. Little work has been focused on feature selection for this type of tasks although a number of feature selection algorithms have been introduced for nominal classification problems. However these techniques can not be applied to monotonic classification as they do not consider the monotonicity constraints. In this work, we present a technique to compute the quality of features for monotonic classification. Using gradient directing search method, this method trains a feature weight vector by maximizing the fuzzy monotonic dependency, which was defined in fuzzy preference rough sets. We conduct some experiments to compare the classification performances of the proposed method with some other techniques. The experimental results show the effectiveness...

Rank Entropy-Based Decision Trees for Monotonic Classification

In many decision making tasks, values of features and decision are ordinal. Moreover, there is a monotonic constraint that the objects with better feature values should not be assigned to a worse decision class. Such problems are called ordinal classification with monotonicity constraint. Some learning algorithms have been developed to handle this kind of tasks in recent years. However, experiments show that these algorithms are sensitive to noisy samples and do not work well in real-world applications. In this work, we introduce a new measure of feature quality, called rank mutual information (RMI), which combines the advantage of robustness of Shannon's entropy with the ability of dominance rough sets in extracting ordinal structures from monotonic data sets. Then, we design a decision tree algorithm (REMT) based on rank mutual information. The theoretic and experimental analysis shows that the proposed algorithm can get monotonically consistent decision trees, if training samples are monotonically consistent. Its performance is still good when data are contaminated with noise.

Feature Selection for Monotonic Classification

Monotonic classification is a kind of special task in machine learning and pattern recognition. Monotonicity constraints between features and decision should be taken into account in these tasks. However, most existing techniques are not able to discover and represent the ordinal structures in monotonic datasets. Thus, they are inapplicable to monotonic classification. Feature selection has been proven effective in improving classification performance and avoiding overfitting. To the best of our knowledge, no technique has been specially designed to select features in monotonic classification until now. In this paper, we introduce a function, which is called rank mutual information, to evaluate monotonic consistency between features and decision in monotonic tasks. This function combines the advantages of dominance rough sets in reflecting ordinal structures and mutual information in terms of robustness. Then, rank mutual information is integrated with the search strategy of min-redundancy and max-relevance to compute optimal subsets of features. A collection of numerical experiments are given to show the effectiveness of the proposed technique.

Optimal monotone relabelling of partially non-monotone ordinal data

Noise in multi-criteria data sets can manifest itself as non-monotonicity. Work on the remediation of such non-monotonicity is rather scarce. Nevertheless, errors are often present in real-life data sets, and several monotone classification algorithms are unable to use such partially non-monotone data sets. Fortunately, as we will show here, it is possible to restore monotonicity in an optimal way, by relabelling part of the data set. By exploiting the properties of a (minimum) flow network, and identifying pleasing properties of some maximum cuts, an elegant single-pass optimal ordinal relabelling algorithm is formulated.

Large-margin feature selection for monotonic classification

Monotonic classification plays an important role in the field of decision analysis, where decision values are ordered and the samples with better feature values should not be classified into a worse class. The monotonic classification tasks seem conceptually simple, but difficult to utilize and explain the order structure in practice. In this work, we discuss the issue of feature selection under the monotonicity constraint based on the principle of large margin. By introducing the monotonicity constraint into existing margin based feature selection algorithms, we design two new evaluation algorithms for monotonic classification. The proposed algorithms are tested with some artificial and real data sets, and the experimental results show its effectiveness.

Decision trees for ordinal classification

In many classification problems the domains of the attributes and the classes are linearly ordered. For such problems the classification rule often needs to be order-preserving or monotonic as we call it. Since the known decision tree methods generate non-monotonic trees, these methods are not suitable for monotonic classification problems. We provide an order-preserving tree-generation algorithm for multi-attribute classification problems with $k$ linearly ordered classes, and an algorithm for repairing non-monotonic decision trees. The performance of these algorithms is tested on a real-world financial dataset and on random monotonic datasets.

A parallel monotonic classification machine

Current parallel machines, such as array processors and Prolog or Lisp machines, are not optimized to execute monotonic classification tasks. A new special-purpose parallel computer is proposed, which efficiently solves this important type of problem from examples it encounters. After the study of sufficient examples, the macine is capable of classifying objects which it might have never seen before. What makes the machine unique is its low gate count when compared with its counterparts.

Learning bias in neural networks and an approach to controlling its effect in monotonic classification

As a learning machine, a neural network using the backpropagation training algorithm is subject to learning bias. This results in unpredictability of boundary generation behavior in pattern recognition applications, especially in the case of small training sample size. It is suggested that in a large class of pattern recognition problems such as managerial and other problems possessing monotonicity properties, the effect of learning bias can be controlled by using multiarchitecture monotonic function neural networks.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>