Standard UCI Datasets Research Articles

K-Means text clustering method based on Decision Grey Wolf Optimization

Aiming at the problem that the traditional algorithm is easy to fall into local optimum in the process of text clustering, which leads to inaccurate text clustering results, a text clustering method based on decision grey wolf optimization K-Means is proposed to cluster the text data set and the standard UCI data set respectively. Afterword segmentation, stop words removal, feature extraction, and text vectorization of text data, the powerful optimization ability of the Decision Gray Wolf Optimization (DGWO) algorithm is used for global optimization, and the clustering center in K-Means algorithm is replaced by the location of wolves. The position of the wolf group is updated by iterative optimization to obtain the optimal clustering center, to perform text clustering. The experimental results show that compared with the traditional method, the precision, recall, and F-Measure of the text data clustering are improved by 49.22%, 51.15%, and 48.98% respectively. The precision, recall, and F-Measure of UCI data clustering are increased by 23.92%, 25.40%, and 24.70% respectively, and the text clustering results are more reliable.

Research on multi-granularity sequential three-way decisions based on the fuzzy T-equivalence relation

The fuzzy decision-theoretic rough set (FDTRS) is a data processing method based on the fuzzy theory designed to handle decision problems with uncertainty and fuzziness. It simplifies the complexity of decision-making and provides decision strategies by integrating multiple sources of information. This approach makes the decision process more scientific and rational. The mainstream FDTRS model breaks the over restrictive limits of classical decision-theoretic rough sets, computes the fuzzy similarity relation between two objects utilizing the Gaussian kernel function, which is another major breakthrough and innovation in the field of decision-theoretic rough sets. However, this model suffers from the problems of insufficient decision information, single perspective and high decision-making cost when dealing with dynamic, complex and uncertain problems. Emerging in recent years, sequential three-way decisions not only provides a flexible mechanism to implement the idea of progressive computation, but also deals with dynamic, complex and uncertain problems. To this end, this paper proposes a sequential three-way decision model based on the fuzzy T-equivalence relation (FTE-based sequential three-way decision model) from the perspective of sequential three-way decisions, and gives relevant definitions and examples to effectively solve the problem of insufficient information. Then in order to solve the single-view problem, we extend the sequential three-way decisions to the case of multi-granularity, and propose FTE-based multi-granularity sequential three-way decisions in four aggregation strategies with the corresponding properties and algorithms. In addition, the rationality of models are tentatively analyzed through specific examples. Finally, six sets of UCI standard datasets are used to validate the proposed models, which demonstrate their good performance. These models contribute to the field of fuzzy decision-theoretic rough sets and sequential three-way decisions.

Density peak clustering based on improved dung beetle optimization and mahalanobis metric

The density peak clustering algorithm (DPC) quickly divides each cluster based on high-density peak points and shows better clustering performance. In order to address the issue that the local density is constrained by the preset cut-off distance in DPC and the Euclidean distance cannot capture the possible correlation between different features, a DPC algorithm based on improved dung beetle optimization (IDBO) and Mahalanobis metric is proposed, called IDBO-MDDPC. The IDBO algorithm enhances the ball dung beetle individual by incorporating nonlinear dynamic factors to increase the search and development capabilities of the algorithm and by incorporating an adaptive cosine wave inertial weight strategy to more precisely determine the optimal position of the thief dung beetle in order to improve the convergence speed and accuracy of the algorithm. The IDBO algorithm is simulated on eight benchmark functions, and the results demonstrate that it is superior to other comparison algorithms in terms of convergence speed and accuracy. In the DPC algorithm, the Mahalanobis metric is used to capture the correlation between features to improve clustering performance. The IDBO algorithm is integrated with the DPC algorithm, and the F-Measure evaluation index is used to design the objective function so that the optimal value of the cut-off distance can be automatically selected. In order to evaluate the efficiency of the algorithm, three sets of artificially synthesized datasets and five sets of UCI standard datasets were chosen for studies. Experimental results show that the IDBO-MDDPC algorithm can automatically determine a better cut-off distance value and ensure higher clustering accuracy.

A ReliefF guided binary equalization optimizer based on Z-type and reverse Z-type transfer functions for feature selection

Feature selection (FS) refers to the process of finding the most relevant feature subset according to some criteria. A ReliefF guided novel binary equilibrium optimizer (RG-NBEO) feature selection method was put forward to solve the FS problem. Z-type transfer functions and reverse Z-type transfer functions are proposed to convert the continuous searching space into binary searching space. The ReliefF guidance strategy was adopted to achieve the increasing and decreasing of features in the iterative process. The proposed feature selection method was tested by using 10 UCI standard datasets. Firstly, the Z-type and Rz-type transfer functions with different parameters are compared, and the transfer function with best performance is selected to compare with S-, V- and U-type transfer functions. Then the proposed transfer function is applied to five wrapped feature selection algorithms, and the simulation results are analyzed statistically. Finally, the proposed transfer function is applied to other machine learning algorithms to verify the applicability of the proposed function. Simulation results verify the effectiveness of the proposed Z-type and Rz-type transfer function and ReliefF guided binary equalization optimizer.

A Robust Adaptive Hierarchical Learning Crow Search Algorithm for Feature Selection

Feature selection is a multi-objective problem, which can eliminate irrelevant and redundant features and improve the accuracy of classification at the same time. Feature selection is a great challenge to balance the conflict between the two goals of selection accuracy and feature selection ratio. The evolutionary algorithm has been proved to be suitable for feature selection. Recently, a new meta-heuristic algorithm named the crow search algorithm has been applied to the problem of feature selection. This algorithm has the advantages of few parameters and achieved good results. However, due to the lack of diversity in late iterations, the algorithm falls into local optimal problems. To solve this problem, we propose the adaptive hierarchical learning crow search algorithm (AHL-CSA). Firstly, an adaptive hierarchical learning technique was used to adaptive divide the crow population into several layers, with each layer learning from the top layer particles and the topmost layer particles learning from each other. This strategy encourages more exploration by lower individuals and more exploitation by higher individuals, thus improving the diversity of the population. In addition, in order to make full use of the search information of each level in the population and reduce the impact of local optimization on the overall search performance of the algorithm, we introduce an information sharing mechanism to help adjust the search direction of the population and improve the convergence accuracy of the algorithm. Finally, different difference operators are used to update the positions of particles at different levels. The diversity of the population is further improved by using different difference operators. The performance of the method was tested on 18 standard UCI datasets and compared with eight other representative algorithms. The comparison of experimental results shows that the proposed algorithm is superior to other competitive algorithms. Furthermore, the Wilcoxon rank-sum test was used to verify the validity of the results.

Improved interval type-2 fuzzy K-means clustering based on adaptive iterative center with new defuzzification method

Interval Type-2 Fuzzy K-means (IT2FKM) is an efficient clustering algorithm, which mainly focuses on further describes the uncertainty of the data sets by introducing interval type-2 fuzzy membership function. Since the estimation center obtained by two membership functions is an interval type-1 fuzzy set, it is necessary to use the defuzzification method to obtain a crisp cluster center. The existing IT2FKM algorithms adopt the generalized centroid method (GC) in the process of type-reduction and defuzzification for iterative clustering center. But unfortunately, this method simply uses a single numerical value 0.5 as the type-reduction coefficient to balance and synthesize for the left value and the right value of an interval cluster center, while ignoring the distribution of the clusters. This defect makes IT2FKM fail to update the optimal cluster center when facing with imbalanced clusters, which affect the accuracy in some cases. To address this failing, in this paper, some adverse effects caused by generalized centroid method when facing with imbalanced clusters are analyzed in detail. Thereafter, an improved type-reduction method of iterative cluster centers for IT2FKM clustering is proposed considering the imbalanced clustering distributions. On the basis of which, an enhanced IT2FKM algorithm is further developed. The enhanced IT2FKM algorithm has an average classification accuracy of 97.5% when faced with four datasets with different degrees imbalance, which is about 3.25% higher than other clustering algorithms. The effectiveness and superiority of the proposed algorithm are further demonstrated by some experimental analysis of UCI standard datasets.

A Hybrid Particle Swarm Optimization Algorithm with Dynamic Adjustment of Inertia Weight Based on a New Feature Selection Method to Optimize SVM Parameters

Support vector machine (SVM) is a widely used and effective classifier. Its efficiency and accuracy mainly depend on the exceptional feature subset and optimal parameters. In this paper, a new feature selection method and an improved particle swarm optimization algorithm are proposed to improve the efficiency and the classification accuracy of the SVM. The new feature selection method, named Feature Selection-score (FS-score), performs well on data sets. If a feature makes the class external sparse and the class internal compact, its FS-score value will be larger and the probability of being selected will be greater. An improved particle swarm optimization model with dynamic adjustment of inertia weight (DWPSO-SVM) is also proposed to optimize the parameters of the SVM. By improving the calculation method of the inertia weight of the particle swarm optimization (PSO), inertia weight can decrease nonlinearly with the number of iterations increasing. In particular, the introduction of random function brings the inertia weight diversity in the later stage of the algorithm and the global searching ability of the algorithm to avoid falling into local extremum. The experiment is performed on the standard UCI data sets whose features are selected by the FS-score method. Experiments demonstrate that our algorithm achieves better classification performance compared with other state-of-the-art algorithms.

Crow search algorithm with time varying flight length Strategies for feature selection

Feature Selection (FS) is an efficient technique use to get rid of irrelevant, redundant and noisy attributes in high dimensional datasets while increasing the efficacy of machine learning classification. The CSA is a modest and efficient metaheuristic algorithm which has been used to overcome several FS issues. The flight length (fl) parameter in CSA governs crows' search ability. In CSA, fl is set to a fixed value. As a result, the CSA is plagued by the problem of being hoodwinked in local minimum. This article suggests a remedy to this issue by bringing five new concepts of time dependent fl in CSA for feature selection methods including linearly decreasing flight length, sigmoid decreasing flight length, chaotic decreasing flight length, simulated annealing decreasing flight length, and logarithm decreasing flight length. The proposed approaches' performance is assessed using 13 standard UCI datasets. The simulation result portrays that the suggested feature selection approaches overtake the original CSA, with the chaotic-CSA approach beating the original CSA and the other four proposed approaches for the FS task.

High Dimensional Data Processing Based on Optimized DPC Algorithm

With the rapid growth of science and technology, machine learning and big data analysis have developed more and more difficult. Similarly, data also become difficult to process and classify due to the fact that the data dimension is becoming larger and larger. Furthermore, aiming at the defect due to the amount of data the clustering using speedy examine, search, and discovery of density peaks (also known as DPC) does not adjust to data sets with large dimensions (high-dimensional). Therefore, in this paper, we suggest an optimization procedure, which we pronounce as t-dpc, and is founded on the t-sne dimension lessening technique, and which can also optimize the technique for estimation of the Gaussian kernel function, using unified measurement criteria in solving density. In the simulation based experiments, using two different data sets i.e., the UCI standard data set, and the artificial data set, the proposed DPC procedure is associated with the classical t-dpc algorithm. The empirical evaluation and investigational outcomes illustrate that the proposed method of t-dpc not merely acclimatizes to the high-dimensional data sets, nevertheless it also increases the effectiveness of the classical DPC technique.

An adaptive and altruistic PSO-based deep feature selection method for Pneumonia detection from Chest X-rays

Pneumonia is one of the major reasons for child mortality especially in income-deprived regions of the world. Although it can be detected and treated with very less sophisticated instruments and medication, Pneumonia detection still remains a major concern in developing countries. Computer-aided based diagnosis (CAD) systems can be used in such countries due to their lower operating costs than professional medical experts. In this paper, we propose a CAD system for Pneumonia detection from Chest X-rays, using the concepts of deep learning and a meta-heuristic algorithm. We first extract deep features from the pre-trained ResNet50, fine-tuned on a target Pneumonia dataset. Then, we propose a feature selection technique based on particle swarm optimization (PSO), which is modified using a memory-based adaptation parameter, and enriched by incorporating an altruistic behavior into the agents. We name our feature selection method as adaptive and altruistic PSO (AAPSO). The proposed method successfully eliminates non-informative features obtained from the ResNet50 model, thereby improving the Pneumonia detection ability of the overall framework. Extensive experimentation and thorough analysis on a publicly available Pneumonia dataset establish the superiority of the proposed method over several other frameworks used for Pneumonia detection. Apart from Pneumonia detection, AAPSO is further evaluated on some standard UCI datasets, gene expression datasets for cancer prediction and a COVID-19 prediction dataset. The overall results are satisfactory, thereby confirming the usefulness of AAPSO in dealing with varied real-life problems. The supporting source codes of this work can be found at https://github.com/rishavpramanik/AAPSO.

MSGWO-MKL-SVM: A Missing Link Prediction Method for UAV Swarm Network Based on Time Series

Missing link prediction technology (MLP) is always a hot research area in the field of complex networks, and it has been extensively utilized in UAV swarm network reconstruction recently. UAV swarm is an artificial network with strong randomness, in the face of which prediction methods based on network similarity often perform poorly. To solve those problems, this paper proposes a Multi Kernel Learning algorithm with a multi-strategy grey wolf optimizer based on time series (MSGWO-MKL-SVM). The Multiple Kernel Learning (MKL) method is adopted in this algorithm to extract the advanced features of time series, and the Support Vector Machine (SVM) algorithm is used to determine the hyperplane of threshold value in nonlinear high dimensional space. Besides that, we propose a new measurable indicator of Multiple Kernel Learning based on cluster, transforming a Multiple Kernel Learning problem into a multi-objective optimization problem. Some adaptive neighborhood strategies are used to enhance the global searching ability of grey wolf optimizer algorithm (GWO). Comparison experiments were conducted on the standard UCI datasets and the professional UAV swarm datasets. The classification accuracy of MSGWO-MKL-SVM on UCI datasets is improved by 6.2% on average, and the link prediction accuracy of MSGWO-MKL-SVM on professional UAV swarm datasets is improved by 25.9% on average.

Privacy-Preserving Decision Trees Training and Prediction

In the era of cloud computing and machine learning, data has become a highly valuable resource. Recent history has shown that the benefits brought forth by this data driven culture come at a cost of potential data leakage. Such breaches have a devastating impact on individuals and industry, and lead the community to seek privacy preserving solutions. A promising approach is to utilize Fully Homomorphic Encryption ( \( \mathsf {FHE } \) ) to enable machine learning over encrypted data, thus providing resiliency against information leakage. However, computing over encrypted data incurs a high computational overhead, thus requiring the redesign of algorithms, in an “ \( \mathsf {FHE } \) -friendly” manner, to maintain their practicality. In this work we focus on the ever-popular tree based methods, and propose a new privacy-preserving solution to training and prediction for trees over data encrypted with homomorphic encryption. Our solution employs a low-degree approximation for the step-function together with a lightweight interactive protocol, to replace components of the vanilla algorithm that are costly over encrypted data. Our protocols for decision trees achieve practical usability demonstrated on standard UCI datasets encrypted with fully homomorphic encryption. In addition, the communication complexity of our protocols is independent of the tree size and dataset size in prediction and training, respectively, which significantly improves on prior works. 1

Multi-category intuitionistic fuzzy twin support vector machines with an application to plant leaf recognition

The intuitionistic fuzzy twin support vector machine for multi-categorization is developed in this study, which incorporates both structural and empirical risk concepts. In this method, each training pattern is first aggregated with the appropriate membership and non-membership degrees, which describe the position of a pattern in relation to its class centre and surrounding circumstances in input or feature space, and then the separating hyperplane is constructed using the kernel function and convex quadratic programming. Empirical findings on an artificial and thirteen UCI standard datasets show that it outperforms well-known existing methods including improved support vector machines, K-nearest neighbour, logistic regression, decision trees, random forests, and multilayer perceptrons. Furthermore, the suggested classifier with linear, polynomial, and Gaussian kernels has been used to identify the leaves of various plants, where the shape, texture, and margin data are extracted from the leaf in order to categorize the plant species. The method’s generalization capacity is demonstrated by the classification results on two leaf datasets of thirty and one hundred species, respectively. To compare the suggested method’s prediction capacity with others, statistical analysis is performed using two non-parametric tests, Friedman and Wilcoxon, with a 5% threshold of significance. The results show that the proposed method yields better performance for both linear and non-linear kernels.

Inequality distance hyperplane multiclass support vector machines

In this study, inequality distance hyperplane multiclass support vector machines (IDH-MSVM) algorithm is proposed on the basis of multiclassification support vector machine (MSVM) which was proposed by J. Weston and C. Watkins in 1999. It only needs to solve a single objective optimization problem to deal with multiclassification problems. For original MSVM, the hyperplane distance refers to the classification interval between classical margin and hyperplane, which is equality. However, the IDH-MSVM introduces parameters to adjust distance between every classification hyperplane and classical margin, which makes the hyperplane distance inequality. The effectiveness of the proposed method is experimented on UCI standard data sets and compared with several multiclassification algorithms. Experimental results show that this method has a better classification effect on multiclassification data.

AIEOU: Automata-based improved equilibrium optimizer with U-shaped transfer function for feature selection

High dimensional data has become an essential challenge to address in Data Science and Machine Learning. Reducing the number of dimensions by excluding noisy, irrelevant, or correlated dimensions is often referred to as the feature selection (FS). The ultimate goal in FS is to identify an optimal set of dimensions (features) to realize better classification accuracy, decrease the computational time and optimize the memory requirements with the help of some methods. Recently, optimization algorithms have gained popularity in different fields because of their flexibility and effectiveness. Equilibrium optimizer (EO) is a physics-based meta-heuristic algorithm, which is inspired from a well-mixed dynamic mass balance on a control volume that has good exploration and exploitation capabilities. In this work, an improved version of EO is proposed with the inclusion of learning based automata to find proper values of its parameters and Adaptive β Hill Climbing (AβHC) to find a better equilibrium pool. The method is used as a feature selector, evaluated on 18 standard UCI datasets with the help of K-nearest neighbors (KNN) classifier, and compared with eight state-of-the-art methods including classical and hybrid meta-heuristic algorithms. Moreover, the proposed methods is applied on high dimensional Microarray datasets which generally contain a few samples but large number of features, and often lead to ‘curse of dimensionality’. The obtained results illustrate the supremacy of the proposed method over the other state-of-the-art methods mentioned in literature. The source code of this work is publicly available at https://github.com/ahmed-shameem/Feature_selection.

Adaptive Multi-level Backward Tracking for Sequential Feature Selection

In the past few decades, the large amount of available data has become a major challenge in data mining and machine learning. Feature selection is a significant preprocessing step for selecting the most informative features by removing irrelevant and redundant features, especially for large datasets. These selected features play an important role in information searching and enhancing the performance of machine learning models. In this research, we propose a new technique called One-level Forward Multi-level Backward Selection (OFMB). The proposed algorithm consists of two phases. The first phase aims to create preliminarily selected subsets. The second phase provides an improvement on the previous result by an adaptive multi-level backward searching technique. Hence, the idea is to apply an improvement step during the feature addition and an adaptive search method on the backtracking step. We have tested our algorithm on twelve standard UCI datasets based on k-nearest neighbor and naive Bayes classifiers. Their accuracy was then compared with some popular methods. OFMB showed better results than the other sequential forward searching techniques for most of the tested datasets.

Binary Jaya algorithm based on binary similarity measure for feature selection

Feature selection (FS) has become an indispensable data preprocessing task because of the huge amount of high dimensional data being generated by current technologies. These high dimensional data contains irrelevant, redundant, and noisy features that deteriorate classification accuracy. FS reduces dimensionality by removing the unwanted features thus improves classification accuracy. FS can be considered as a binary optimization problem. In order to solve this problem, this work proposes a new wrapper feature selection technique based on the Jaya algorithm. Three binary variants of the Jaya algorithm are proposed, the first and second ones are based on transfer functions namely BJaya-S and BJaya-V. The third variant (BJaya-JS) explores the search space on the basis of the Jaccard Similarity index. In addition, a probability-based local search technique, namely Neighbourhood Search is proposed to balance the exploration and exploitation. The variants of Jaya algorithm are evaluated and the best variant is selected. The best variant is further compared with six state-of-the-art feature selection techniques. All the performances are tested on 18 high dimensional standard UCI datasets. Experimental result comparison shows that the proposed feature selection technique performs better than other competitors.

Chaotic vortex search algorithm: metaheuristic algorithm for feature selection

The Vortex Search Algorithm (VSA) is a meta-heuristic algorithm that has been inspired by the vortex phenomenon proposed by Dogan and Olmez in 2015. Like other meta-heuristic algorithms, the VSA has a major problem: it can easily get stuck in local optimum solutions and provide solutions with a slow convergence rate and low accuracy. Thus, chaos theory has been added to the search process of VSA in order to speed up global convergence and gain better performance. In the proposed method, various chaotic maps have been considered for improving the VSA operators and helping to control both exploitation and exploration. The performance of this method was evaluated with 24 UCI standard datasets. In addition, it was evaluated as a Feature Selection (FS) method. The results of simulation showed that chaotic maps (particularly the Tent map) are able to enhance the performance of the VSA. Furthermore, it was clearly shown the fitness of the proposed method in attaining the optimal feature subset with utmost accuracy and the least number of features. If the number of features is equal to 36, the percentage of accuracy in VSA and the proposed model is 77.49 and 92.07. If the number of features is 80, the percentage of accuracy in VSA and the proposed model is 36.37 and 71.76. If the number of features is 3343, the percentage of accuracy in VSA and the proposed model is 95.48 and 99.70. Finally, the results on Real Application showed that the proposed method has higher percentage of accuracy in comparison to other algorithms.

S-shaped versus V-shaped transfer functions for binary Manta ray foraging optimization in feature selection problem

Feature selection (FS) is considered as one of the core concepts in the areas of machine learning and data mining which immensely impacts the performance of classification model. Through FS, irrelevant or partially relevant features can be eliminated which in turn helps in enhancing the performance of the model. Over the years, researchers have applied different meta-heuristic optimization techniques for the purpose of FS as these overcome the limitations of traditional optimization approaches. Going by the trend, we introduce a new FS approach based on a recently proposed meta-heuristic algorithm called Manta ray foraging optimization (MRFO) which is developed following the food foraging nature of the Manta rays, one of the largest known marine creatures. As MRFO is apposite for continuous search space problems, we have adapted a binary version of MRFO to fit it into the problem of FS by applying eight different transfer functions belonging to two different families: S-shaped and V-shaped. We have evaluated the eight binary versions of MRFO on 18 standard UCI datasets. Of these, the best one is considered for comparison with 16 recently proposed meta-heuristic FS approaches. The results show that MRFO outperforms the state-of-the-art methods in terms of both classification accuracy and number of features selected. The source code of this work is available in https://github.com/Rangerix/MetaheuristicOptimization .

Feature Selection with Binary Symbiotic Organisms Search Algorithm for Email Spam Detection

One method to increase classifier accuracy is using Feature Selection (FS). The main idea in the FS is reducing complexity, eliminating irrelevant information, and deleting a subset of input features that either have little information or have no information for prediction. In this paper, three efficient binary methods based on the Symbiotic Organisms Search (SOS) algorithm were presented for solving the FS problem. In the first and second methods, several S_shaped and V_shaped transfer functions were used for the binarization of the SOS, respectively. These methods were called BSOSS and BSOSV. In the third method, two new operators called Binary Mutualism Phase (BMP) and Binary Commensalism Phase (BCP) were presented for binarization of the SOS, named Efficient Binary SOS (EBSOS). The proposed methods were run on 18 standard UCI datasets and compared to the base and important meta-heuristic algorithms. The test results showed that the EBSOS method has the best performance among the three proposed methods for the binarization of the SOS. Finally, the EBSOS method was compared to the Genetic Algorithm (GA), Binary Bat Algorithm (BBA), Binary Particle Swarm Optimization (BPSO) Algorithm, Binary Flower Pollination Algorithm (BFPA), Binary Grey Wolf Optimizer (BGWO) Algorithm, Binary Dragonfly Algorithm (BDA), and Binary Chaotic Crow Search Algorithm (BCCSA). In addition, the EBSOS method was executed on the spam email dataset with the KNN, NB, SVM, and MLP classifiers. The results showed that the EBSOS method has better performance compared to other methods in terms of feature count and accuracy criteria. Furthermore, it was practically evaluated on spam email detection in particular.