Feature Set Partitioning Research Articles

Ensemble multi-view feature set partitioning method for effective multi-view learning

Ensemble multi-view feature set partitioning method for effective multi-view learning

Enhancing multi-view ensemble learning with zig-zag pattern-based feature set partitioning

Enhancing multi-view ensemble learning with zig-zag pattern-based feature set partitioning

A review of feature set partitioning methods for multi-view ensemble learning

Since the present era is entirely computer and Internet of Things (IoT) oriented, enormous amounts of data are produced quickly from many sources. Machine learning’s primary function, which is widely applicable to these high-dimensional datasets, is classification. One of the well-known approaches for improving classification effectiveness is ensemble learning. The splitting of huge datasets vertically based on the feature set partitioning (FSP) method is known as views. Multi-view ensemble learning (MEL) is the process of applying different classification models to these views and then combining their predictions. The FSP methods used to create views are the backbone of MEL. This review has been done to study, present, analyze, and collectively compare various FSP methods. Several issues and challenges related to FSP and MEL are also highlighted in this paper. The classification performance of recent FSP methods for the MEL framework is compared as an auxiliary experiment using fifteen high-dimensional datasets. A nonparametric statistical analysis is used for the comparison of FSP methods.

Collaboration graph for feature set partitioning in data classification

The curse of dimensionality of features in data classification is still an open issue. An approach to solve this problem is to partition features into several sub-sets of features hence the data classification task for every subset is performed. Then, an ensemble of these classifications are reported as the result of the classification problem. However, the feature set partitioning into sub-sets of features is still an area of research interest. Thus, in this paper, an innovative framework is proposed in which, first, a collaboration measure between each two features is defined and measured. Then, the collaboration graph, consisted of features as nodes and measured collaborations as edges’ weights, is generated according to the collaboration measures calculated. After that, a community detection method is used to find the graph communities. The communities are considered as the feature subsets and a base classifier is trained for each subset based on the corresponding training data of the subsets. Then, the ensemble classifier is created by a combination of base classifiers according to the AdaBoost Aggreagation. The simulation results of the proposed approach over the real and synthetic datasets indicate that the proposed approach considerably increases the classification accuracy in comparison to previous methods.

Non-sequential partitioning approaches to decision tree classifier

Decision tree is a well-known classifier which is widely used in real-world applications. It is easy to interpret, however it suffers from instability and lower classification performance for high-dimensionality datasets due to curse of dimensionality. Feature set partitioning is a novel concept to address the higher dimensionality problem by dividing the feature set into subsets (blocks). Many of the existing partitioning based decision tree approaches are sequential in nature, which lack logical relationships amongst the features. In this work, we propose novel non-sequential feature set partitioning methods by exploiting the ideas of Ferrer Diagram and Bell Triangle to create feature blocks with a mix of low, medium, and high correlation features. The experimental results on 11 UCI and KEEL datasets demonstrate the superiority of the proposed partitioning methods, upto 5% higher classification accuracy, over NBTree, BFTree, Serial-CMFP partitioning method, and classical decision tree techniques. The proposed methods also exhibit improved stability as compared to other decision tree methods.

Multi-view ensemble learning: an optimal feature set partitioning for high-dimensional data classification

Multi-view ensemble learning has the potential to address issues related to the high dimensionality of data. It attempts to utilize all the relevant only discarding the irrelevant features. The view of a dataset is the sub-table of the training data with respect to a subset of the feature set. The problem of discarding the irrelevant features and obtaining subsets of the relevant features is useful for dimension reduction and dealing with the problem of having fewer training examples than even the reduced set of relevant features. A feature set partitioning resulting in the blocks of relevant features may not yield multiple-view-based classifiers with good classification performance. In this work the optimal feature set partition approach has been proposed. Further, the ensemble learning from views aims to maximize the performance of the classifier. The experiments study the performance of random feature set partitioning, attribute bagging, view generation using attribute clustering, view construction using genetic algorithm and OFSP proposed method. The blocks of relevant feature subsets are used to construct the multi-view classifier ensemble using K-nearest neighbor, Naive Bayesian and support vector machine algorithm applied to sixteen high-dimensional data sets from UCI machine learning repository. The performance parameters considered for comparison are classification accuracy, disagreement among the classifiers, execution time and percentage reduction of attributes.

Multi-view Ensemble Learning Using Optimal Feature Set Partitioning: An Extended Experiments and Analysis in Low Dimensional Scenario

Abstract Multi-view ensemble learning (MEL) has successfully addressed the issue related to high dimensionality of the data. It exploits the information of views of the data. To obtain views of data, an optimal feature set partitioning (OFSP) method [1] has been shown performance enhancement of MEL. Results of the experiments carried out on datasets and their statistical analysis show the effectiveness for classification problem in high dimensional scenario. In this work, classification performance of MEL using OFSP method has been analyzed in low dimensional situations. Therefore, experiments are performed on low dimension datasets using K-Nearest Neighbor (KNN), Naive Bayesian (NB) and Support Vector Machine (SVM) classification algorithms. The experimental results and their statistical analysis show that OFSP method is also effective in low dimensional environment.

Privacy-preserving data mining: A feature set partitioning approach

In privacy-preserving data mining (PPDM), a widely used method for achieving data mining goals while preserving privacy is based on k-anonymity. This method, which protects subject-specific sensitive data by anonymizing it before it is released for data mining, demands that every tuple in the released table should be indistinguishable from no fewer than k subjects. The most common approach for achieving compliance with k-anonymity is to replace certain values with less specific but semantically consistent values. In this paper we propose a different approach for achieving k-anonymity by partitioning the original dataset into several projections such that each one of them adheres to k-anonymity. Moreover, any attempt to rejoin the projections, results in a table that still complies with k-anonymity. A classifier is trained on each projection and subsequently, an unlabelled instance is classified by combining the classifications of all classifiers. Guided by classification accuracy and k-anonymity constraints, the proposed data mining privacy by decomposition (DMPD) algorithm uses a genetic algorithm to search for optimal feature set partitioning. Ten separate datasets were evaluated with DMPD in order to compare its classification performance with other k-anonymity-based methods. The results suggest that DMPD performs better than existing k-anonymity-based algorithms and there is no necessity for applying domain dependent knowledge. Using multiobjective optimization methods, we also examine the tradeoff between the two conflicting objectives in PPDM: privacy and predictive performance.

Genetic algorithm-based feature set partitioning for classification problems

Feature set partitioning generalizes the task of feature selection by partitioning the feature set into subsets of features that are collectively useful, rather than by finding a single useful subset of features. This paper presents a novel feature set partitioning approach that is based on a genetic algorithm. As part of this new approach a new encoding schema is also proposed and its properties are discussed. We examine the effectiveness of using a Vapnik–Chervonenkis dimension bound for evaluating the fitness function of multiple, oblivious tree classifiers. The new algorithm was tested on various datasets and the results indicate the superiority of the proposed algorithm to other methods.