Monotonic Classification Research Articles

Fusing multiple interval-valued fuzzy monotonic decision trees

As a powerful knowledge mining technique for ordinal classification tasks, dominance-based rough set theory has many advantages but also some issues. Sensitivity to noisy information means that even a single mislabeled sample can lead to substantial fluctuations in the approximation calculations. Another issue is that most monotonic classifiers can only handle real-valued data and cannot directly deal with interval-valued data, which is common in practical applications. To address these issues, a tree-based fusion learning method for monotonic classification of interval-valued attributes, named FM-IFMDT, has been proposed. Its functional structure comprises three components: (i) The proposed robust β-precision interval-valued fuzzy dominance neighborhood rough set model (β-IFDNRS) can adaptively identify and filter noise samples. (ii) The interval dominance discernibility matrix based on β-IFDNRS is developed for feature selection and can generate a set of complete and diverse feature subsets. (iii) The novel interval-valued fuzzy monotonic decision tree (IFMDT) based on the probability distribution is trained on each feature subset and is used as the base classifier of the weighted voting fusion model. Extensive experiments show the proposed fusion learning method has significant advantages.

Metric learning for monotonic classification: turning the space up to the limits of monotonicity

This paper presents, for the first time, a distance metric learning algorithm for monotonic classification. Monotonic datasets arise in many real-world applications, where there exist order relations in the input and output variables, and the outputs corresponding to ordered pairs of inputs are also expected to be ordered. Monotonic classification can be addressed through several distance-based classifiers that are able to respect the monotonicity constraints of the data. The performance of distance-based classifiers can be improved with the use of distance metric learning algorithms, which are able to find the distances that best represent the similarities among each pair of data samples. However, learning a distance for monotonic data has an additional drawback: the learned distance may negatively impact the monotonic constraints of the data. In our work, we propose a new model for learning distances that does not corrupt these constraints. This methodology will also be useful in identifying and discarding non-monotonic pairs of samples that may be present in the data due to noise. The experimental analysis conducted, supported by a Bayesian statistical testing, demonstrates that the distances obtained by the proposed method can enhance the performance of several distance-based classifiers in monotonic problems.

Self-adaptive interval dominance-based feature selection for monotonic classification of interval-valued attributes

Self-adaptive interval dominance-based feature selection for monotonic classification of interval-valued attributes

Active Antinoise Fuzzy Dominance Rough Feature Selection Using Adaptive K-Nearest Neighbors

Feature selection methods with anti-noise performance are effective dimensionality reduction methods for classification tasks with noise. However, there are few studies on robust feature selection methods for monotonic classification tasks. The fuzzy dominance rough set (FDRS) model is a nontrivial knowledge acquisition tool, which is widely used in feature selection of monotonic classification tasks. Nonetheless, this model has been proved in practice to be generally poorly fault-tolerance, and only one noisy sample can cause huge interference in acquiring knowledge. In view of these two issues, this paper firstly designs an adaptive <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -nearest neighbors strategy to calculate the density of samples. The noisy samples are identified according to their densities, and then an active anti-noise fuzzy dominance rough set model is proposed. Then, in the active anti-noise fuzzy dominance rough approximation space, the class-separability is evaluated by the approximation operators of the proposed model, and the feature-redundancy is evaluated by the fuzzy ranking conditional mutual information. On this basis, a feature evaluation index is designed comprehensively considering class-separability and feature-redundancy. Finally, a feature selection algorithm is designed to select the feature subset with the highest classification performance. The experimental results show that the proposed algorithm has better robustness and classification performance.

Fuzzy rough feature selection using a robust non-linear vague quantifier for ordinal classification

Ordinal classification is a common classification problem, which widely exists in multi-attribute decision making problems. The dominance-based rough set approach (DRSA) is a knowledge acquisition tool for ordinal classification tasks. Nevertheless, the existence of noise information in the collected data greatly reduces the accuracy of the DRSA and some extended model. An effective way to solve this issue is to improve the robustness of these models. In addition, most existing feature selection strategies based on dominance rough sets mainly focus on the monotonic classification consistency between features and decision, but ignoring the classification information provided by features combination. Motivated by these two issues, this paper proposes a robust fuzzy dominance rough set model to combat noise interference and develop a feature selection approach based on the proposed model for ordinal classification tasks. First, a non-linear vague quantifier is adopted to construct the robust model, and related dependency function is introduced. Second, the rank entropy based uncertainty measures are explored to characterize the contribution of features combination for ordinal classification. Based on the proposed uncertainty measures, a new feature evaluation index is presented. Meanwhile, the corresponding feature selection algorithm is designed. Finally, numerical comparative experiments are performed to indicate that the proposed model and feature selection algorithm have good capability.

Feature Selection Considering Multiple Correlations Based on Soft Fuzzy Dominance Rough Sets for Monotonic Classification

Monotonic classification is a common task in the field of multicriteria decision-making, in which features and decision obey a monotonic constraint. The dominance-based rough set theory is an important mathematical tool for knowledge acquisition in monotonic classification tasks (MCTs). However, existing dominance-based rough set models are very sensitive to noise information, and only a misclassified sample will lead to large errors in acquiring knowledge. This unstable phenomenon does not meet the requirements of practical applications. On the other hand, feature selection is supposedly an effective dimensionality reduction approach for classification tasks. In the real world, feature combinations with multiple correlations can often provide important classification information, where the multiple correlations include redundancy, complementarity, and interaction between features. To the best of our knowledge, most of the existing feature selection methods for MCTs only consider the relevance between features and decision, while ignoring the multiple correlations. To overcome these two drawbacks, in this article, we propose a robust fuzzy dominance rough set model, and develop a feature selection method that considers multiple correlations based on the robust model for MCTs. First, a soft fuzzy dominance rough set (SFDRS) with robustness is proposed. Second, a feature evaluation index considering multiple correlations is presented. Finally, a feature selection algorithm based on SFDRS is designed to select an optimal feature subset. Extensive experiments are conducted on 12 public datasets, and the results show that the SFDRS model has good robustness and the proposed feature selection algorithm has excellent classification performance.

Self-adaptive weighted interaction feature selection based on robust fuzzy dominance rough sets for monotonic classification

Feature selection is an effective dimensionality reduction technique for classification tasks. Monotonic classification task (MCT) is a special classification task in which features and decision obey a monotonic constraint, which is widely used in various application fields. Most of the existing feature selection methods for MCTs are based on some dominance rough set models. However, these methods only consider the correlation between features and decisions, and ignore the interaction between features. Moreover, these dominance rough set models are not robust to noisy information. Thus, to make up for these two deficiencies, we propose a novel feature selection approach based on an improved robust fuzzy dominance rough set for MCTs. First, a β−precision fuzzy dominance neighborhood rough set (β−PFDNRS) with robustness is proposed. Second, a feature evaluation index is presented, which not only considers the correlation between features and decision, but also considers the interaction between features. Finally, a β−PFDNRS based self-adaptive weighted interaction feature selection (β−SWIFS) algorithm is designed to select a feature sequence. Extensive experiments are conducted on fifteen public datasets, and the results show that the β−PFDNRS model has good robustness and the proposed feature selection algorithm has an excellent classification performance.

Incremental Feature Selection Using a Conditional Entropy Based on Fuzzy Dominance Neighborhood Rough Sets

Incremental feature selection approaches can improve the efficiency of feature selection used for dynamic datasets, which has attracted increasing research attention. Nevertheless, there is currently no work on incremental feature selection approaches for dynamic ordered data. Moreover, the monotonic classification effect of ordered data is easily affected by noise, so a robust feature evaluation metric is needed for feature selection algorithm. Motivated by these two issues, we investigate incremental feature selection approaches using a new conditional entropy with robustness for dynamic ordered data in this study. First, we propose a new rough set model, i.e., fuzzy dominance neighborhood rough sets (FDNRS). Second, a conditional entropy with robustness is defined based on FDNRS model, which is used as evaluation metric for features and combined with a heuristic feature selection algorithm. Finally, two incremental feature selection algorithms are designed on the basis of the above researches. Experiments are performed on ten public datasets to evaluate the robustness of the proposed metric and the performance of the incremental algorithms. Experimental results verify that the proposed metric is robust and our incremental algorithms are effective and efficient for updating reducts in dynamic ordered data.

Novel fuzzy rank discrimination measures for monotonic ordinal feature selection

Classification with ordinal monotonic constraints refers to the sample categories are preference-ordered, and monotonicity constraints between the sample categories and class labels needs to be guaranteed in the learned classification model. Feature selection aims at finding highly discriminative features to avoid degradation of learning performance, which has increasingly gathered attention in the field of monotonic classification. In this paper, we propose novel fuzzy rank discrimination measures to evaluate monotonic consistency for feature selection in ordinal dataset. The proposed measure fulfill the monotonicity property in the sense that value of measure does not decrease when a new candidate feature is added to the selected feature subset, which guarantees the optimal feature subset can be achieved without evaluating all possible combinations of features explicitly. A wrapper approach based on the feature ranking in terms of minimal-redundancy and maximal-relevance criterion is implemented to evaluate the performance of the proposed method for monotonic feature selection. The superiority of the proposed fuzzy rank discrimination measure over the existing measure, in terms of robustness and classification accuracy, is established extensively on several real-life monotonic datasets by using four monotonic classifiers.

Monotonic relation-constrained Takagi-Sugeno-Kang fuzzy system

The Takagi-Sugeno-Kang fuzzy system has wide applications across different areas, e.g., regression, classification and decision making, attributed to its high precision and interpretability. However, the existing Takagi-Sugeno-Kang fuzzy system is not an ideal solution to some special scenarios, particularly for those that are constrained monotonically. To this end, a monotonic relation-constrained Takagi-Sugeno-Kang fuzzy system classifier is proposed in this paper. The proposed method introduces a monotonic relation between the inputs and the outputs, where the objective function is expressed in a monotonically constrained form and a strategy for generating monotonicity constraint pairs is developed. Furthermore, to address the convexity loss caused by the increasing monotonicity constraints, the proposed method introduces the Tikhonov regularization strategy to ensure the uniqueness and boundedness of the solution. The results from extensive experiments show that the proposed method exhibits better classification performance than the original Takagi-Sugeno-Kang fuzzy system and state-of-the-art monotonic classification methods in handling monotonic datasets.

Class-weighted neural network for monotonic imbalanced classification

In real life scenarios, classification problems with the characters of monotonicity constraints and imbalanced class distribution widely exist. However, at present, the research on this kind of problem is still rare. Traditional algorithms designed only for monotonic classification and imbalanced classification are not available for monotonic imbalanced classification. So far, there is only one approach specially designed for monotonic imbalanced classification problems, which is based on the resampling technique. In this paper, from the algorithmic point of view, we propose a weighted single-hidden-layer feedforward neural network (WMCS-SLFN) based on multi-objective genetic algorithm, where both the monotonicity constraints and the imbalanced distribution are considered. Additionally, in order to improve the generalization capability of WMCS-SLFN, we put forward a selective ensemble strategy for WMCS-SLFN based on the 0–1 knapsack problem, which can generate an ensemble of WMCS-SLFN with the optimal prediction accuracy under the monotonicity constraints. Contrast experiments conducted on eight monotonic imbalanced datasets verify the effectiveness of our proposed methods, and moreover, the experimental results analyzed by Wilcoxon statistical test highlight the advantage of our work significantly.

Data-Driven Preference Learning Methods for Value-Driven Multiple Criteria Sorting with Interacting Criteria

The learning of predictive models for data-driven decision support has been a prevalent topic in many fields. However, construction of models that would capture interactions among input variables is a challenging task. In this paper, we present a new preference learning approach for multiple criteria sorting with potentially interacting criteria. It employs an additive piecewise-linear value function as the basic preference model, which is augmented with components for handling the interactions. To construct such a model from a given set of assignment examples concerning reference alternatives, we develop a convex quadratic programming model. Because its complexity does not depend on the number of training samples, the proposed approach is capable for dealing with data-intensive tasks. To improve the generalization of the constructed model on new instances and to overcome the problem of overfitting, we employ the regularization techniques. We also propose a few novel methods for classifying nonreference alternatives in order to enhance the applicability of our approach to different data sets. The practical usefulness of the proposed approach is demonstrated on a problem of parametric evaluation of research units, whereas its predictive performance is studied on several monotone classification problems. The experimental results indicate that our approach compares favourably with the classical UTilités Additives DIScriminantes (UTADIS) method and the Choquet integral-based sorting model. Summary of Contribution. The paper tackles vital challenges at the intersections of multiple criteria decision analysis and machine learning, showing how computationally advanced techniques can be used for faithfully representing human preferences and dealing with complex decision problems. Specifically, we propose a novel preference learning method for multiple criteria sorting problems. The introduced approach incorporates convex quadratic programming to construct a value-based preference model based on large sets of preference statements. In this way, we extend the applicability of decision analysis methods to preferences derived from historical data or observation of users' behavior in addition to the preference judgments explicitly revealed by the decision-makers. The method's practical usefulness is illustrated on a variety of real-world datasets from fields such as higher education, medicine, human resources, and housing market. Its potential for supporting better decision-making is enhanced by both an interpretable form of the assumed model handling interactions between criteria as well as a high predictive performance demonstrated in the extensive computational experiments.

PERFORMANCE VALIDATION OF PRIOR QUANTIZATION TECHNIQUES IN OUTLIERS CLASSIFICATION USING WDBC DATASET

Data mining is the process of analyzing enormous data and summarizing it into the useful knowledge discovery and the task of data mining approaches is growing quickly, particularly classification techniques very efficient, way to classifying the data, which is important in the decision-making process for medical practitioners. This study presents the quantization and validation (OQV) techniques for fast outlier detection in large size WDBC data sets. The distance metrics utilization makes the algorithm as the linear one for various objects and assures the sequential scanning. The inclusion of direct quantization technique and the cluster explicit discovery assures the simplicity and the economical. The comparative analysis of proposed OQV techniques with the triangular boundary-based classification and the Weighing-based Feature Selection and Monotonic Classification (WFSMC) regarding the accuracy, precision, recall and the number of attributes assures an effectiveness of OQV for large size datasets.

Learning rule sets and Sugeno integrals for monotonic classification problems

In some variants of the supervised classification setting, the domains of attributes and the set of classes are totally ordered sets. The task of learning a classifier that is nondecreasing w.r.t. each attribute is called monotonic classification. Several kinds of models can be used in this task; in this paper, we focus on decision rules. We propose a method for learning a set of decision rules that optimally fits the training data while favoring short rules over long ones. We give new results on the representation of sets of if-then rules by extensions of Sugeno integrals to distinct attribute domains, where local utility functions are used to map attribute domains to a common totally ordered scale. We study whether such qualitative extensions of Sugeno integral provide compact representations of large sets of decision rules.

Shape-restricted support vector machine (SR-SVM): a SVM classifier taking supplementary shape information of input

Many classification problems contain shape information from input features, such as monotonic, convex, and concave. In this research, we propose a new classifier, called Shape-Restricted Support Vector Machine (SR-SVM), which takes the component-wise shape information to enhance classification accuracy. There exists vast research literature on monotonic classification covering monotonic or ordinal shapes. Our proposed classifier extends to handle convex and concave types of features, and combinations of these types. While standard SVM uses linear separating hyperplanes, our novel SR-SVM essentially constructs non-parametric and nonlinear separating planes subject to component-wise shape restrictions. We formulate SR-SVM classifier as a convex optimization problem and solve it using an active-set algorithm. The approach applies basis function expansions on the input and effectively utilizes the standard SVM solver. We illustrate our methodology using simulation and real world examples, and show that SR-SVM improves the classification performance with additional shape information of input.

Entity Matching with Quality and Error Guarantees

Given two sets of entities X and Y , entity matching aims to decide whether x and y represent the same entity for each pair (x, y) ∈ X × Y . In many scenarios, the only way to ensure perfect accuracy is to launch a costly inspection procedure on every (x, y), whereas performing the procedure |X| · |Y | times is prohibitively expensive. It is, therefore, important to design an algorithm that carries out the procedure on only some pairs, and renders the verdicts on the other pairs automatically with as few mistakes as possible. This article describes an algorithm that achieves the purpose using the methodology of active monotone classification. The algorithm ensures an asymptotically optimal tradeoff between the number of pairs inspected and the number of mistakes made.

Monotonic classification: An overview on algorithms, performance measures and data sets

Currently, knowledge discovery in databases is an essential first step when identifying valid, novel and useful patterns for decision making. There are many real-world scenarios, such as bankruptcy prediction, option pricing or medical diagnosis, where the classification models to be learned need to fulfill restrictions of monotonicity (i.e. the target class label should not decrease when input attributes values increase). For instance, it is rational to assume that a higher debt ratio of a company should never result in a lower level of bankruptcy risk. Consequently, there is a growing interest from the data mining research community concerning monotonic predictive models. This paper aims to present an overview of the literature in the field, analyzing existing techniques and proposing a taxonomy of the algorithms based on the type of model generated. For each method, we review the quality metrics considered in the evaluation and the different data sets and monotonic problems used in the analysis. In this way, this paper serves as an overview of monotonic classification research in specialized literature and can be used as a functional guide for the field.

Label noise filtering techniques to improve monotonic classification

The monotonic ordinal classification has increased the interest of researchers and practitioners within machine learning community in the last years. In real applications, the problems with monotonicity constraints are very frequent. To construct predictive monotone models from those problems, many classifiers require as input a data set satisfying the monotonicity relationships among all samples. Changing the class labels of the data set (relabeling) is useful for this. Relabeling is assumed to be an important building block for the construction of monotone classifiers and it is proved that it can improve the predictive performance.In this paper, we will address the construction of monotone datasets considering as noise the cases that do not meet the monotonicity restrictions. For the first time in the specialized literature, we propose the use of noise filtering algorithms in a preprocessing stage with a double goal: to increase both the monotonicity index of the models and the accuracy of the predictions for different monotonic classifiers. The experiments are performed over 12 datasets coming from classification and regression problems and show that our scheme improves the prediction capabilities of the monotonic classifiers instead of being applied to original and relabeled datasets. In addition, we have included the analysis of noise filtering process in the particular case of wine quality classification to understand its effect in the predictive models generated.

On the Entropy Formulas and Solitons for the Ricci-Harmonic Flow

This paper is concerned with entropy monotonicity formulas and classification of gradient almost soliton for Ricci-harmonic flow. The new entropy formulas introduced here are monotone in general along the flow and constant exactly on shrinker or expander as the case may be. The consequence of this is the nonexistence of periodic solutions except those that are gradient solitons. Furthermore, the paper discusses gradient almost Ricci-harmonic soliton with respect to a fixed metric, not minding the dynamical nature of the flow but treating the defining elliptic equations and relying on analytic techniques. Finally, we are able to classify gradient almost Ricci-harmonic solitons.

A fast rank mutual information based decision tree and its implementation via Map‐Reduce

SummaryTo address the time‐consuming problem for the confirmation of splitting attributes and splitting points in classic rank mutual information based decision trees, this paper establishes a fast rank mutual information based decision tree (FRMIDT) for classification problems. First, the proposed FRMIDT algorithm improves the velocity by a max‐relevance and min‐redundancy criterion to remove the redundant attributes in each tree node building. Then, the fuzzy c‐means algorithm is employed to confirm the splitting points for further acceleration. Meanwhile, a parallel implementation is developed in the framework of Map‐Reduce (MR‐FRMIDT) for medium or large‐scale data classification. Several comparative studies are conducted on UCI benchmark data sets. In contrast to the classic rank mutual information based decision tree on 12 data sets, the proposed FRMIDT model effectively reduces the computational time on the premise of keeping testing accuracy. Furthermore, the proposed FRMIDT algorithm is comparable through comparing FRMIDT with other traditional decision tree classifiers including BFT, C4.5, LAD, NBT, and SC. Meanwhile, the comparison with 7 different popular splitting measures based monotonic decision trees on several data sets illustrates the effectiveness of FRMIDT in monotonic classification. At last, the experimental analysis on other 6 data sets shows that the proposed MR‐FRMIDT is feasible and has a good parallel performance on reducing execution time and avoiding memory restrictions.