Multi-Cluster Feature Selection Research Articles

Parkinson’s Disease Detection by Using Machine Learning Method based on Local Classification on Class Boundary

Parkinson’s disease (PD) detection has long been an important task in medical intelligence. Recognition methods based on speech signals show great potential in Parkinson’s disease diagnosis. In this paper, based on an efficient machine learning method for Parkinson’s disease detection, we take the use of test data incorporates an efficient Secure Two-Party Computing (S2PC) protocol to protect the privacy of patients. We present two key components, the secure use of data and a local classification methodology, including the description of class boundaries. We conducted experiments on two datasets to validate our proposed method, and the results show well data security protection ability compared to some more sophisticated methods. And the performance of Local Classification on Class Boundary(LCCB) and Hyperplane K-Nearest Neighbor(HKNN) is significantly better than that of both Support Vector Machines(SVM) and Random Forest(RF). When the number of selected features is from 400 to 500, HKNN and LCCB are roughly equal where the accuracy of HKNN is 95.2%, and LCCB has the rate of 94.7%. Then we use Multi-Cluster Feature Selection(MCFS) to analyze and select the important features from D2 dataset. It shows that even if only two features are selected, the boundaries of the two categories are also clear and easy to distinguish.

An Unsupervised Abnormal Power Consumption Detection Method Combining Multi-Cluster Feature Selection and the Gaussian Mixture Model

An Unsupervised Abnormal Power Consumption Detection Method Combining Multi-Cluster Feature Selection and the Gaussian Mixture Model

Fault diagnosis of rotating machines based on modified hierarchical fluctuation dispersion entropy and multi-cluster feature selection

Fault diagnosis of rotating machines based on modified hierarchical fluctuation dispersion entropy and multi-cluster feature selection

Application of improved bubble entropy and machine learning in the adaptive diagnosis of rotating machinery faults

This paper proposes an improved bubble entropy algorithm called Improved Hierarchical Refined Composite Multiscale Multichannel Bubble Entropy (IHRCMMCBE) to characterize the fault characteristics of rotating machinery. By introducing the refined composite multiscale analysis algorithm, the improved hierarchical decomposition algorithm, and the multi-channel data analysis method, the bubble entropy algorithm can more fully characterize the fault characteristics. Then, this method is combined with the machine learning algorithm to realize automatic fault diagnosis of rotating machinery. Multi-Cluster Feature Selection (MCFS) is used for feature selection to screen out useless features and improve feature extraction efficiency and feature quality. The MCFS method has excellent performance and does not need parameters, so it is suitable for adaptive fault diagnosis. Meanwhile, the EigenClass classifier using the concept of generalized eigenvalue is used to realize fault recognition. Besides, the recently proposed Gorilla Troops Optimizer (GTO) algorithm is used to improve EigenClass, and a GTO-optimized EigenClass (GTO-EigenClass) classifier is proposed to adaptively select parameters and achieve adaptive fault identification. The proposed fault diagnosis method of rotating machinery based on the improved bubble entropy is independent of data and application scenarios and is hardly affected by subjective factors. The ablation experiment proves the effectiveness of each improvement of bubble entropy. Also, the performance verification experiments and the comparative experiments prove that the method can effectively characterize different types of rotating machinery, different fault types, fault degrees, and composite faults, and it can achieve higher calculation efficiency and classification accuracy. When the sample length is 1024, the average accuracy of bearing and gearbox fault identification can reach 97.85 % and 97.57 % respectively, and when the sample length is 2048, the identification accuracy can reach 100 %.

Screening of serum exosome markers for colorectal cancer based on Boruta and multi-cluster feature selection algorithms

Screening of serum exosome markers for colorectal cancer based on Boruta and multi-cluster feature selection algorithms

A comprehensive working condition identification scheme for rolling bearings based on modified CEEMDAN as well as modified hierarchical amplitude-aware permutation entropy

As a pivotal part of a machine driven system, the health states of rolling bearings usually determine the normal operation of a whole item of equipment. Consequently, it is important to make accurate and timely judgments as to the operating conditions of rolling bearings. In this paper, a synthesized diagnosis technology, including fault pre-judgment and identification for rolling bearings is proposed. In the first section, a threshold value is defined on the basis of the sensitivity of amplitude-aware permutation entropy (AAPE) to bearing faults. Whether the bearing has defects is judged is based on this value. If a defect exists, a feature extraction scheme combining the modified complete ensemble empirical mode decomposition with adaptive noise (MCEEMDAN) and the modified hierarchical AAPE (MHAAPE) is adopted, to fully mine the hidden state features. Firstly, the scheme uses MCEEMDAN, which benefits from a good time-frequency decomposition capability, to divide the signal of trouble into a group of intrinsic mode functions (IMFs). Second, the MHAAPE of each IMF component is computed to form the candidate state features. Then, multi cluster feature selection is employed to compress the high-dimensional fault features to form the low-dimensional sensitive feature vectors required for subsequent classification. Finally, the sensitive feature vectors are input into a random forest classifier for training and classification, so as to ascertain the different fault types and severity. In addition, different contrastive methods are tested based on experimental data. The experiment results indicate that, compared to contrastive methods, the proposed scheme enjoys better performance, which can effectively judge whether the bearing is healthy and accurately identify different fault states in bearings.

Research on Fault Diagnosis Method of Rolling Bearing Based on Feature Optimization and Self-Adaptive SVM

Aiming at the problems of the difficulty of extracting the fault sensitive features in the mixed domain of bearings under complex working conditions and the lack of self-adaptability of support vector machine (SVM) parameters, to solve these problems, we investigate a diagnosis method combining optimal feature selection and self-adaptive SVM. Firstly, the optimal feature space through multicluster feature selection (MCFS) is constructed. Then, the differential evolution (DE) strategy is used to improve the search performance of gray wolf optimizer (GWO) to improve the diagnosis accuracy of SVM and make it strong self-adaptability. The experimental results show that, with a small number of sensitive features containing significant category differences, the proposed method not only guarantees the construction of the optimal feature space under the minimum feature dimension, but also greatly improves the accuracy of fault recognition. Simultaneously, compared with the traditional feature reduction method (PCA), the proposed MCFS combined diagnosis model improves the accuracy from 98.5% to 100% with 1.5%. Concurrently, compared with MPA-SVM, GWO-SVM, and PSO-SVM, the convergence performance is improved by 61.54%, 78.26%, and 92.64%, respectively. It can seek the best classification performance in the shortest time. The effectiveness and superiority of the proposed method are fully verified.

Multi-Cluster Feature Selection Based on Isometric Mapping

This letter presents an unsupervised feature selection method based on machine learning. Feature selection is an important component of artificial intelligence, machine learning, which can effectively solve the curse of dimensionality problem. Since most of the labeled data is expensive to obtain, this paper focuses on the unsupervised feature selection method. The distance metric of traditional unsupervised feature selection algorithms is usually based on Euclidean distance, and it is maybe unreasonable to map high-dimensional data into low-dimensional space by using Euclidean distance. Inspired by this, this paper combines manifold learning to improve the multi-cluster unsupervised feature selection algorithm. By using geodesic distance, we propose a multi-cluster feature selection based on isometric mapping (MCFS-I) algorithm to perform unsupervised feature selection adaptively for multiple clusters. Experimental results show that the proposed method consistently improves the clustering performance compared to the existing competing methods.

Amazonian cacao-clone nibs discrimination using NIR spectroscopy coupled to naïve Bayes classifier and a new waveband selection approach

Near-Infrared Spectroscopy (NIRS) has shown to be helpful in the study of rice, tea, cocoa, and other foods due to its versatility and reduced sample treatment. However, the high complexity of the data produced by NIR sensors makes necessary pre-treatments such as feature selection techniques that produce compact profiles. Supervised and unsupervised techniques have been tested, creating different subsets of features for classification, which affect the performance of the classifiers based on such compact profiles. In this sense, we propose and test a new covering array feature selection (CAFS) algorithm coupled to the naïve Bayes classifier (NBC) to discriminate among Amazonian cacao nibs from six cacao clones. The CAFS wrapper approach looks for the wavebands that maximize the F1-score, and then, are more relevant for classification. For this purpose, cacao pods of six varieties were collected, and their grains were extracted and processed (fermented, dried, roasted, and milled) to obtain cacao nibs. Then from each clone NIR spectral profiles in the range of 1100–2500 nm were extracted, and relevant wavebands were selected using the proposed CAFS algorithm. For comparison, two standard feature selection techniques were implemented the multi-cluster feature selection MCFS and the eigenvector centrality feature selection ECFS. Then, based on the different selected variables, three NBCs were built and compared among them through statistical metrics. The results showed that using the wavebands selected by CAFS, the NBC performed an average accuracy of 99.63%; being this superior to the 94.92% and 95.79% for ECFS and MCFS respectively. These results showed that the wavebands selected by the proposed CAFS algorithm allowed obtaining a better fit concerning other feature selection methods reported in the literature.

Multivariate hierarchical multiscale fluctuation dispersion entropy: Applications to fault diagnosis of rotating machinery

With a view to completing the fault diagnosis of rotating machinery efficiently and accurately, this paper presents a novel fault diagnosis model that combines multivariate hierarchical multiscale fluctuation dispersion entropy (MHMFDE), multi-cluster feature selection (MCFS) and gray wolf optimization-based kernel extreme learning machine(GWO-KELM). Firstly, MHMFDE is presented to capture the high-dimensional fault features hidden in the attained multichannel vibration signals. Integrating the multiscale entropy-based method and the hierarchical entropy-based method that are currently popular in the domain of fault identification, MHMFDE can simultaneously extract affluent fault features from multivariate vibration signals in-depth as well as overcoming the problem of information loss in the existing single-channel data analysis methods. Afterward, MCFS is used to pick sensitive features from the attained raw fault features to form the sensitive feature vectors, thereby reducing the impact of redundant features. Finally, GWO-KELM is adopted to quantitatively analyze the diagnostic effect. Three examples reveal that the presented approach enjoys excellent performance in the fault diagnosis domain. Especially for the identification of compound faults of rotating machinery, the performance of the presented method is significantly superior to that of existing methods.

Generalized refined composite multiscale fuzzy entropy and multi-cluster feature selection based intelligent fault diagnosis of rolling bearing

Extracting the failure related information from vibration signals is a very important aspect of vibration-based fault detection for rolling bearing Multiscale entropy and its improvement, multiscale fuzzy entropy (MFE), are significant complexity measure tools of time series. They have been successfully applied to extract vibration failure features for rolling bearing condition monitoring . However, MFE over different scales will fluctuate with increase of scale factor. A new nonlinear dynamic parameter termed generalized refined composite multiscale fuzzy entropy (GRCMFE) is firstly developed to enhance the performance of MSE and MFE in data complexity measurement. Then three algorithms are developed and compared with MSE and MFE, as well as two algorithms of generalized MFE to verify the availability and superiority by analyzing two kinds of noise signals. In addition, based on three algorithms of GRCMFE, a novel fault diagnosis approach for rolling bearing is proposed with linking multi-cluster feature selection for supervised learning and the gravitational search algorithm optimized support vector machine for failure pattern recognition. Last, the proposed fault diagnostic approach was utilized to analyze two kinds of bearing test data sets. Analysis results indicate that our proposed fault diagnosis approach could effectively extract nonlinear dynamic complexity information and gets the highest identifying rate and the best performance among the comparative approaches.

A Cheap Feature Selection Approach for the K-Means Algorithm.

The increase in the number of features that need to be analyzed in a wide variety of areas, such as genome sequencing, computer vision, or sensor networks, represents a challenge for the K -means algorithm. In this regard, different dimensionality reduction approaches for the K -means algorithm have been designed recently, leading to algorithms that have proved to generate competitive clusterings. Unfortunately, most of these techniques tend to have fairly high computational costs and/or might not be easy to parallelize. In this article, we propose a fully parallelizable feature selection technique intended for the K -means algorithm. The proposal is based on a novel feature relevance measure that is closely related to the K -means error of a given clustering. Given a disjoint partition of the features, the technique consists of obtaining a clustering for each subset of features and selecting the m features with the highest relevance measure. The computational cost of this approach is just O(m·max{n·K,logm}) per subset of features. We additionally provide a theoretical analysis on the quality of the obtained solution via our proposal and empirically analyze its performance with respect to well-known feature selection and feature extraction techniques. Such an analysis shows that our proposal consistently obtains the results with lower K -means error than all the considered feature selection techniques: Laplacian scores, maximum variance, multicluster feature selection, and random selection while also requiring similar or lower computational times than these approaches. Moreover, when compared with feature extraction techniques, such as random projections, the proposed approach also shows a noticeable improvement in both error and computational time.

Improved Ensemble Classification for Evolving Data Streams

A major challenge for evolving data stream classification is feature evolution where features of stream instances are dynamically changing as they progress. Existing classification methods considered feature evolution either for fixed-size data or of limited degree with presumed dependence to history, making them unable to work effectively on evolving data streams of unbounded size and arbitrary feature evolution. Particularly, for evolving data streams containing instances of multiple labels, classification coping with feature evolution faces significant challenges. In this article, we present efficient ensemble methods for classifying evolving data streams of both single label and multiple labels through effective model coupling. For single-label classification, we present an improved unsupervised classification algorithm that applies multi-cluster feature selection (MCFS), which was originally proposed for static data classification, in the DXMiner framework to handle each window of instances in a dynamic stream. Our method generates an optimal feature subset and achieves a high classification accuracy. We further improve the time complexity of the feature selection process in MCFS by applying the Ball-tree searching technique. For multi-label classification, we propose an effective fixed-size ensemble classifier based on multi-label KNN, which works only for static multi-label data classification, by incorporating a weight adaptation strategy among the classifiers in the ensemble to dynamically update the model and cope with arbitrary feature evolution of stream instances as the stream progresses. Extensive experiment results on real-life data streams show that our algorithms outperform the existing results for single-label and multi-label classification in classification accuracy and efficiency.

Evaluation of Sub-Selection Methods for Assessing Climate Change Impacts on Low-Flow and Hydrological Drought Conditions

A challenge for climate impact studies is the identification of a sub-set of climate model projections from the many typically available. Sub-selection has potential benefits, including making large datasets more meaningful and uncovering underlying relationships. We examine the ability of seven sub-selection methods to capture low flow and drought characteristics simulated from a large ensemble of climate models for two catchments. Methods include Multi-Cluster Feature Selection (MCFS), Unsupervised Discriminative Features Selection (UDFS), Diversity-Induced Self-Representation (DISR), Laplacian score (LScore), Structure Preserving Unsupervised Feature Selection (SPUFS), Non-convex Regularized Self-Representation (NRSR) and Katsavounidis–Kuo–Zhang (KKZ). We find that sub-selection methods perform differently in capturing varying aspects of the parent ensemble, i.e. median, lower or upper bounds. They also vary in their effectiveness by catchment, flow metric and season, making it very difficult to identify a best sub-selection method for widespread application. Rather, researchers need to carefully judge sub-selection performance based on the aims of their study, the needs of adaptation decision making and flow metrics of interest, on a catchment by catchment basis.

Enhancing Obstructive Apnea Disease Detection Using Dual-Tree Complex Wavelet Transform-Based Features and the Hybrid "K-Means, Recursive Least-Squares" Learning for the Radial Basis Function Network.

Background:The obstructive sleep apnea (OSA) detection has become a hot research topic because of the high risk of this disease. In this paper, we tested some powerful and low computational signal processing techniques for this task and compared their results with the recent achievements in OSA detection.Methods:The Dual-tree complex wavelet transform (DT-CWT) is used in this paper to extract feature coefficients. From these coefficients, eight non-linear features are extracted and then reduced by the Multi-cluster feature selection (MCFS) algorithm. The remaining features are applied to the hybrid “K-means, RLS” RBF network which is a low computational rival for the Support vector machine (SVM) networks family.Results and Conclusion:The results showed suitable OSA detection percentage near 96% with a reduced complexity of nearly one third of the previously presented SVM based methods.

Systematic framework for short text classification based on improved TWE and supervised MCFS topic merging strategy

Text classification task can help people to discover valuable information hidden in the text set. Many previous studies have achieved excellent results in traditional text classification tasks. With the development of new social media, a large number of short texts appear on the Internet. Due to the sparsity of the short text, many classification algorithms which achieve excellent results on long texts hardly achieve satisfactory results on short texts. Therefore, it is important to find a method for calculating effective vector representations of words and overcoming the feature sparseness problem. Based on the above, we carry out the work from improving the quality of word vector representation and enhancing the effect of classification. This paper proposes a systematic framework for improving short text classification performance. In our framework, we first build a topic model with Latent Dirichlet Allocation (LDA) on a universal dataset from Wikipedia and use this model to perform topic inference on short texts. Then, an improved scheme of topical word embedding (TWE) is proposed to learn the vector representations of both words and topics, which use the word in the current word-topic pair to predict the contextual words and the topic in the same word-topic pair to predict its contextual topics. In addition, the supervised Multi-Cluster Feature Selection algorithm (MCFS) is employed to execute topic selection, and we propose a topic merging strategy that is based on the MCFS. At the end of topic selection and merging, short text matrices are generated using the vector representations of both words and topics, and these matrices are fed into a convolution neural network (CNN). On an open short text classification dataset, we compared the proposed framework with various baselines, and the experimental results indicate the effectiveness of our method.

Modified Hierarchical Multiscale Dispersion Entropy and its Application to Fault Identification of Rotating Machinery

The rotating machinery possesses complicated structures and various fault types, whose health state monitoring is essential for the normal production and operation of the equipment. To distinguish different working states of rotating machinery efficiently and accurately, this article presents a novel approach for extracting fault features of vibration signals called modified hierarchical multiscale dispersion entropy (MHMDE). And on this basis, an innovative approach for fault diagnosis of rotating machinery based on MHMDE, multi-cluster feature selection (MCFS) and particle swarm optimization kernel extreme learning machine (PSO-KELM) is developed. Firstly, MHMDE is employed to extract the high-dimensional fault features of rotating machinery. This approach can effectively overcome the shortcomings that multi-scale entropy only focuses on the information in the low-frequency components but discards the high-frequency information, as well as the significant dropping of efficiency if the number of hierarchical layers of hierarchical entropy is large. Then MCFS is employed to screen the sensitive features from the high-dimensional fault features. Finally, the sensitive feature vectors are input into the PSO-KELM-based fault classifier to complete the rotating machinery fault diagnosis. It is proved that the presented approach can effectively identify different fault states of rotating machinery through three typical examples. Meanwhile, the presented approach is compared with multi-scale dispersion entropy (MDE) and hierarchical dispersion entropy (HDE), etc. The results show that the presented approach possesses more superior performance.

Elman Neural Network Soft-Sensor Model of PVC Polymerization Process Optimized by Chaos Beetle Antennae Search Algorithm

The conversion rate of vinyl chloride monomer (VCM) is an important product quality indicator in the process of Polyvinyl chloride (PVC) polymerization. Due to the complexity of the PVC polymerization process and the limitation of site conditions, it is difficult to obtain the VCM conversion rate online in real time.Therefore, this article puts forward a soft-sensor model based on Beetle Antennae Search Algorithm (BAS) to optimize Elman neural network(Elman). Firstly, Multi-Cluster Feature Selection (MCFS) is used to reduce the dimensionality of the high-dimensional input variables, so that we get auxiliary variables of the soft-sensor model. Then, using Elman neural network as a soft-sensor model, and it is trained by the proposed optimization algorithm, which combines the chaotic map and the Beetle Antennae Search Algorithm (CBAS). The simulation results show that the model can significantly improve the prediction accuracy of the VCM conversion rate while realizing the real-time control of the PVC polymerization production process.

Classification of Depression Patients and Normal Subjects Based on Electroencephalogram (EEG) Signal Using Alpha Power and Theta Asymmetry.

Depression or Major Depressive Disorder (MDD) is a mental illness which negatively affects how a person thinks, acts or feels. MDD has become a major disease affecting millions of people presently. The diagnosis of depression is questionnaire based and is not based on any objective criteria. In this paper, feature extracted from EEG signal are used for the diagnosis of depression. Alpha, alpha1, alpha2, beta, delta and theta power and theta asymmetry was used as feature. Alpha1, alpha2 along with theta asymmetry was also used as a feature. Multi-Cluster Feature Selection (MCFS) was used for feature selection when feature combination was used. The classifiers used were Support Vector Machine (SVM), Logistic Regression (LR), Naïve-Bayesian (NB) and Decision Tree (DT). Alpha2 showed higher classification accuracy than alpha1 and alpha power in all applied classifier. From t-test it was found that there was a significant difference in the theta power of left and right hemisphere of normal subjects, but there was no significant difference in depression patients. Average theta asymmetry in normal subjects is higher than MDD patients but the difference in theta asymmetry in normal subjects and MDD patients is not significant. The combination of alpha2 and theta asymmetry showed the highest classification accuracy of 88.33% in SVM.

Multivariate Analysis and Machine Learning for Ripeness Classification of Cape Gooseberry Fruits

This paper explores five multivariate techniques for information fusion on sorting the visual ripeness of Cape gooseberry fruits (principal component analysis, linear discriminant analysis, independent component analysis, eigenvector centrality feature selection, and multi-cluster feature selection.) These techniques are applied to the concatenated channels corresponding to red, green, and blue (RGB), hue, saturation, value (HSV), and lightness, red/green value, and blue/yellow value (L*a*b) color spaces (9 features in total). Machine learning techniques have been reported for sorting the Cape gooseberry fruits’ ripeness. Classifiers such as neural networks, support vector machines, and nearest neighbors discriminate on fruit samples using different color spaces. Despite the color spaces being equivalent up to a transformation, a few classifiers enable better performances due to differences in the pixel distribution of samples. Experimental results show that selection and combination of color channels allow classifiers to reach similar levels of accuracy; however, combination methods still require higher computational complexity. The highest level of accuracy was obtained using the seven-dimensional principal component analysis feature space.