Feature Selection For Clustering Research Articles

Micromagnetic and quantitative prediction of yield strength and tensile strength in DP590 steels based on ReliefF + Clustering feature selection method

The correlation between multiple patterns of micromagnetic signatures and the mechanical properties (yield strength (Rp) and tensile strength (Rm) of high-strength steel (referred to as DP590 steel in Chinese standards) was investigated in this study. Feedforward neural network (FF-NN) models with carefully selected magnetic features as input nodes were established for quantitative prediction of yield strength and tensile strength. The accuracy FF-NN models highly relied on the quality of calibration specimens and the way of selecting magnetic features. The variations of the measured target properties were used to evaluate the quality of the calibration specimens. The specimens with similar yield strength (or tensile strength) were merged to share the same target properties in the model training process. The results demonstrated that merging proper target properties (label) could improve the performance of the models in quantitative prediction of yield strength and tensile strength in DP590 steels. In addition, the performances of FF-NN models combined with the algorithms of ReliefF and ReliefF + clustering were evaluated. The comparison results proved that the FF-NN models employing input nodes selection strategy of ReliefF + clustering realized the advantages of smaller dimensions of input nodes, less training time consumption at the cost of slight accuracy reduction.

Prediction of drug amount in Parkinson's disease using hybrid machine learning systems and radiomics features

AbstractParkinson's disease (PD) is progressive and heterogeneous. Levodopa is widely prescribed to control PD, and its long‐term‐treatment leads to dyskinesia in a dose‐dependent manner. Interpretation of clinical trials comparing different drug treatments for PD is complicated by different dose intensities employed: higher doses of levodopa produce better symptomatic control but more late complications. Thus, the dose must be recalibrated and reduced gradually. Since recommendations for gradually reducing Levodopa are currently lacking and estimation of Levodopa amount can help doctor to correctly prescribe drug amount, this study aims to predict Levodopa amount and incremental doses using Hybrid Machine Learning Systems (HMLS) and a mixture of radiomics and clinical features. We selected 264 patients from PPMI and obtained 950 features including imaging and nominating features. We generated seven datasets constructed from the dataset in years 0 and 1, which linked with outcomes, (O1) patients being on/off drug in year 1, (O2) dose amount in year 1, and (O3‐8) incremental dose from 1st to 2nd, 2nd to 3rd, 3rd to 4th, 4th to 5th, 1st to 4th, 1st to 5th year. HMLSs included 10 feature extraction/9 feature selection algorithms followed by 10 prediction algorithms. To predict O1, timeless dataset + Random Forest + ReliefA had the highest accuracy~88.5% ± 2.2%, and external testing~91.6%. Furthermore, to predict O2, timeless dataset + Minimum Redundancy Maximum Relevance Algorithm (MRMR) + K Nearest Neighbor Regressor (KNN‐R) achieved a mean absolute error (MAE) ~ 47.5 ± 13.6 ([30.3:850 milligram]) and external testing~31.9. To predict dose increments (O3‐8), HMLSs: Unsupervised Feature Selection with Ordinal Locality + KNNR, ReliefA + KNNR, ReliefA + KNNR, Local Learning‐based Clustering Feature Selection + KNNR, MRMR + KNNR, and MRMR + KNNR applied to timeless datasets resulted in MAEs ~ 0.42 ± 0.18, 0.10 ± 0.09, 0.04 ± 0.01, 0.24 ± 0.15, 0.25 ± 0.05, and 0.33 ± 0.26 ([0.23:29.7]), respectively. Moreover, their external testing confirmed our findings. We demonstrated that timeless datasets including a mixture of clinical and imaging features, linked with appropriate HMLSs, significantly improve prediction performances.

New Algorithm of Traditional Chinese Medicine and Protection of Intangible Cultural Heritage Based on Big Data Deep Learning.

Traditional Chinese medicine (TCM) is a summary of the diagnosis and treatment experience formed by the working people in the long-term struggle against diseases, so it is very important to protect the intangible cultural heritage of TCM. How to extract valuable knowledge accurately and conveniently from the massive medical records of TCM is one of the important issues in the current research on the development of TCM. Due to the large amount of data of TCM medical records, many feature attributes, and diverse patterns, the existing classification technology has high computational complexity, low mining efficiency, and poor universality. Therefore, this paper proposed to quantify the medical records of TCM and obtained the main symptoms according to the improved hierarchical clustering feature selection algorithm. This paper also proposed a support vector machine (SVM) classification method using improved particle swarm algorithm to classify TCM information, which not only improves the efficiency and accuracy of TCM information classification but also discovers the potential dialectical and symptom patterns in diagnosis and treatment, so that the intangible cultural heritage protection of TCM can be developed sustainably. This paper showed that the information acquisition accuracy of the improved algorithm was very high. Before the improved algorithm was used, the accuracy of information mining for TCM was 67.90% at the highest and 65.53% at the lowest, but after using the improved algorithm, the accuracy rate of information mining for TCM was 88.02% at the highest and 82.45% at the lowest. It can be seen that using the improved algorithm to mine TCM information can quickly process effective information.

A comprehensive working condition identification scheme for rolling bearings based on modified CEEMDAN as well as modified hierarchical amplitude-aware permutation entropy

As a pivotal part of a machine driven system, the health states of rolling bearings usually determine the normal operation of a whole item of equipment. Consequently, it is important to make accurate and timely judgments as to the operating conditions of rolling bearings. In this paper, a synthesized diagnosis technology, including fault pre-judgment and identification for rolling bearings is proposed. In the first section, a threshold value is defined on the basis of the sensitivity of amplitude-aware permutation entropy (AAPE) to bearing faults. Whether the bearing has defects is judged is based on this value. If a defect exists, a feature extraction scheme combining the modified complete ensemble empirical mode decomposition with adaptive noise (MCEEMDAN) and the modified hierarchical AAPE (MHAAPE) is adopted, to fully mine the hidden state features. Firstly, the scheme uses MCEEMDAN, which benefits from a good time-frequency decomposition capability, to divide the signal of trouble into a group of intrinsic mode functions (IMFs). Second, the MHAAPE of each IMF component is computed to form the candidate state features. Then, multi cluster feature selection is employed to compress the high-dimensional fault features to form the low-dimensional sensitive feature vectors required for subsequent classification. Finally, the sensitive feature vectors are input into a random forest classifier for training and classification, so as to ascertain the different fault types and severity. In addition, different contrastive methods are tested based on experimental data. The experiment results indicate that, compared to contrastive methods, the proposed scheme enjoys better performance, which can effectively judge whether the bearing is healthy and accurately identify different fault states in bearings.

Clustering as feature selection method in spam classification: uncovering sick-leave sellers

Purpose This paper aims to propose a novel way of using textual clustering as a feature selection method. It is applied to identify the most important keywords in the profile classification. The method is demonstrated through the problem of sick-leave promoters on Twitter. Design/methodology/approach Four machine learning classifiers were used on a total of 35,578 tweets posted on Twitter. The data were manually labeled into two categories: promoter and nonpromoter. Classification performance was compared when the proposed clustering feature selection approach and the standard feature selection were applied. Findings Radom forest achieved the highest accuracy of 95.91% higher than similar work compared. Furthermore, using clustering as a feature selection method improved the Sensitivity of the model from 73.83% to 98.79%. Sensitivity (recall) is the most important measure of classifier performance when detecting promoters’ accounts that have spam-like behavior. Research limitations/implications The method applied is novel, more testing is needed in other datasets before generalizing its results. Practical implications The model applied can be used by Saudi authorities to report on the accounts that sell sick-leaves online. Originality/value The research is proposing a new way textual clustering can be used in feature selection.

Classification of Right/Left Hand Motor Imagery by Effective Connectivity Based on Transfer Entropy in Electroencephalogram Signal.

The right and left-hand motor imagery (MI) analysis based on the electroencephalogram (EEG) signal can directly link the central nervous system to a computer or a device. This study aims to identify a set of robust and nonlinear effective brain connectivity features quantified by transfer entropy (TE) to characterize the relationship between brain regions from EEG signals and create a hierarchical feature selection and classification for discrimination of right and lefthand MI tasks. TE is calculated among EEG channels as the distinctive, effective connectivity features. TE is a model-free method that can measure nonlinear effective connectivity and analyze multivariate dependent directed information flow among neural EEG channels. Then four feature subset selection methods namely relief-F, Fisher, Laplacian, and local learningbased clustering (LLCFS) algorithms are used to choose the most significant effective connectivity features and reduce redundant information. Finally, support vector machine (SVM) and linear discriminant analysis (LDA) methods are used for classification. Results show that the best performance in 29 healthy subjects and 60 trials is achieved using the TE method via the Relief-F algorithm as feature selection and support vector machine (SVM) classification with 91.02% accuracy. The TE index and a hierarchical feature selection and classification can be useful for the discrimination of right- and left-hand MI tasks from multichannel EEG signals. Effective connectivity features were extracted from electroencephalogram (EEG) to analyze relationships between regions.Four feature selection methods used to select most significant effective features.Support vector machine (SVM) used for discrimination of right and left hand motor imagery (MI) task. In this study, we investigated brain activity using effective connectivity during MI task based on EEG signals. The motor imagery task can accomplish the same goal as motor execution, since they are both activated by the same brain area. Transfer entropy, coherence, and Granger casualty were employed to extract the features. Differential patterns of activity between the left vs. right MI task showed activity around the motor area rather than other areas. In order to reduce redundant information and select the most significant effective connectivity features, four feature subset selection algorithms are used: Relief-F, Fisher, Laplacian, and learning-based clustering feature selection (LLCFS). Then, support vector machine (SVM) and linear discriminant analysis (LDA) are used to classify left and right hand MI task. Comparison of three different connectivity methods showed that TE index had the highest classification accuracy, and could be useful for the discrimination of right and left hand MI tasks from multichannel EEG signals.

Comparison of Methods for Feature Selection in Clustering of High-Dimensional RNA-Sequencing Data to Identify Cancer Subtypes.

Cancer subtype identification is important to facilitate cancer diagnosis and select effective treatments. Clustering of cancer patients based on high-dimensional RNA-sequencing data can be used to detect novel subtypes, but only a subset of the features (e.g., genes) contains information related to the cancer subtype. Therefore, it is reasonable to assume that the clustering should be based on a set of carefully selected features rather than all features. Several feature selection methods have been proposed, but how and when to use these methods are still poorly understood. Thirteen feature selection methods were evaluated on four human cancer data sets, all with known subtypes (gold standards), which were only used for evaluation. The methods were characterized by considering mean expression and standard deviation (SD) of the selected genes, the overlap with other methods and their clustering performance, obtained comparing the clustering result with the gold standard using the adjusted Rand index (ARI). The results were compared to a supervised approach as a positive control and two negative controls in which either a random selection of genes or all genes were included. For all data sets, the best feature selection approach outperformed the negative control and for two data sets the gain was substantial with ARI increasing from (−0.01, 0.39) to (0.66, 0.72), respectively. No feature selection method completely outperformed the others but using the dip-rest statistic to select 1000 genes was overall a good choice. The commonly used approach, where genes with the highest SDs are selected, did not perform well in our study.

A hybrid model for credit risk assessment: empirical validation by real-world credit data

This paper examines which hybridization strategy is more suitable for credit risk assessment in the dynamic financial world. As such, we use extensive new data sets and develop different hybrid models by combining traditional statistical and modern artificial intelligence methods based on classification and clustering feature selection approaches. We find that a multilayer perceptron (MLP) combined with discriminant analysis or logistic regression (LR) can significantly improve classification accuracy compared with other single and hybrid classifiers. In particular, the findings of our empirical analysis, statistical significance test and expected cost of misclassification test confirm the superiority of the clustering-based LR + MLP hybrid classifier in improving prediction accuracy in maximum performance criteria. To check the efficiency and viability of the proposed model, we use three imbalanced data sets: Chinese farmer credit, Chinese small and medium-sized enterprise (SME) credit and German credit. We also use Australian credit data for further authentication and a robustness check. The first two data sets are private and high dimensional, whereas the second two are mostly used, publicly available and low dimensional. Thus, our findings are relevant for many areas of credit risk, such as SME, farmer and consumer credit risk modeling.

A survey on feature selection approaches for clustering

The massive growth of data in recent years has led challenges in data mining and machine learning tasks. One of the major challenges is the selection of relevant features from the original set of available features that maximally improves the learning performance over that of the original feature set. This issue attracts researchers’ attention resulting in a variety of successful feature selection approaches in the literature. Although there exist several surveys on unsupervised learning (e.g., clustering), lots of works concerning unsupervised feature selection are missing in these surveys (e.g., evolutionary computation based feature selection for clustering) for identifying the strengths and weakness of those approaches. In this paper, we introduce a comprehensive survey on feature selection approaches for clustering by reflecting the advantages/disadvantages of current approaches from different perspectives and identifying promising trends for future research.

Feature Selection for Clustering of Homicide Rates in the Brazilian State of Goias

Homicide is recognized as one of the most violent types of crime. In some countries, it is a hard problem to tackle because of its high occurrence and the lack of research on it. In Brazil, this problem is even harder, since this country is responsible for about 10% of the homicides in the world. Some Brazilian states suffer from the rise of homicide rates, like the state of Goi´as, in which its homicide rate increased from 24.5 per 100,000 in 2002 to 42.6 per 100,000 in 2014, becoming one of the five most violent states of Brazil, despite of having few population. This paper aims at applying clustering algorithms and feature selection models on criminal data concerning homicides and socio-economic variables in the state of Goi´as. We employed three clustering algorithms: K-means, Densitybased, and Hierarchical; as well as two feature selection models: Univariate Selection and Feature Importance. Our results indicate that homicide rates are more recurrent in large urban centers, although these cities have the best socio-economic indicators. Population and the educational level of the adult population were the variables which most influenced the results. K-means clustering brought the optimum outcomes, and Univariate Selection better selected attributes of the database.

A new method for classifying nuts using image processing and k‐means++ clustering

AbstractIn this study, nuts are determined using image processing techniques and classified by k‐means and k‐means++ methods in rapid time. Firstly, pixel area (PA) and square area (SA) features are extracted by an image processing technique. Then, the obtained data are classified by the k‐means and k‐means++ clustering methods. We present a new method called the pixel area and square area method (PASAM), which is realized by clustering feature selection of PA and SA values. Our proposed method is applied on numerous real RGB images of nuts for different cases. Experimental results show that k‐means++ clustering based on PASAM is more efficient than other techniques such as k‐means and k‐means++ clustering based on PA and SA. The nuts in the working environment are determined and classified with 100% accuracy by PASAM.Practical applicationsThe proposed research distinguishes small, medium and big nuts based on image processing and PASAM. This method is appropriate to use to separate all vegetables and fruits for which classification is important.

Unified Simultaneous Clustering and Feature Selection for Unlabeled and Labeled Data.

This paper proposes a novel feature selection method, namely, unified simultaneous clustering feature selection (USCFS). A regularized regression with a new type of target matrix is formulated to select the most discriminative features among the original features from labeled or unlabeled data. The regression with -norm regularization allows the projection matrix to represent an effective selection of discriminative features. For unsupervised feature selection, the target matrix discovers label-like information not from the original data points but rather from projected data points, which are of a reduced dimensionality. Without the aid of an affinity graph-based local structure learning method, USCFS allows the target matrix to capture latent cluster centers via orthogonal basis clustering and to simultaneously select discriminative features guided by latent cluster centers. When class labels are available, the target matrix is also able to find latent class labels by regarding the ground-truth class labels as an approximate guide. Hence, supervised feature selection is realized using these latent class labels, which may differ from the ground-truth class labels. Experimental results demonstrate the effectiveness of the proposed method. Specifically, the proposed method outperforms the state-of-the-art methods on diverse real-world data sets for both the supervised and the unsupervised feature selection.

Gamers’ involvement detection from EEG data with cGAAM – A method for feature selection for clustering

This paper reports the results of an experiment to identify EEG patterns specific to different levels of player involvement when playing a video game. To obtain unbiased results, we based our patterns on both raw EEG data and expert knowledge. We used a three-step procedure to identify patterns. First, we looked for clusters in the reduced feature space extracted from EEG data. Next, we assigned experts’ interpretations to the clusters. Finally, we analysed relations between features used to form the clusters and the class labels provided by experts. The most challenging part of the procedure was feature selection simultaneous with unsupervised classification. To accomplish this task, we developed a new approach for simultaneous feature selection and clustering based on modified GAAM (genetic algorithm with aggressive mutation). When the cGAAM algorithm was applied to EEG data, it returned the feature subsets that (a) were highly consistent across subjects and (b) provided 50% more compact clusters than clusters built over the feature subsets returned by a forward selection search strategy.The main cognitive outcome of EEG signal analysis was a set of patterns differentiating players’ involvement in a game. Conclusions included: 1. For a majority of subjects the most discriminative features were activity in the theta band in the left and right frontal areas, and activity in the delta band in the left frontal area; 2. All three features significantly differentiated between low and high, or medium and high engagement; 3. All subjects showed positive correlations between selected feature values and levels of engagement.

Mixed integer linear programming and heuristic methods for feature selection in clustering

This paper studies the problem of selecting relevant features in clustering problems, out of a data-set in which many features are useless, or masking. The data-set comprises a set U of units, a set V of features, a set R of (tentative) cluster centres and distances for every , , . The feature selection problem consists of finding a subset of features such that the total sum of the distances from the units to the closest centre is minimised. This is a combinatorial optimisation problem that we show to be NP-complete, and we propose two mixed integer linear programming formulations to calculate the solution. Some computational experiments show that if clusters are well separated and the relevant features are easy to detect, then both formulations can solve problems with many integer variables. Conversely, if clusters overlap and relevant features are ambiguous, then even small problems are unsolved. To overcome this difficulty, we propose two heuristic methods to find that, most of the time, one of them, called q-vars, calculates the optimal solution quickly. Then, the q-vars heuristic is combined with the k-means algorithm to cluster some simulated data. We conclude that this approach outperforms other methods for clustering with variable selection that were proposed in the literature.

Cluster feature selection in high-dimensional linear models

This paper concerns with variable screening when highly correlated variables exist in high-dimensional linear models. We propose a novel cluster feature selection (CFS) procedure based on the elastic net and linear correlation variable screening to enjoy the benefits of the two methods. When calculating the correlation between the predictor and the response, we consider highly correlated groups of predictors instead of the individual ones. This is in contrast to the usual linear correlation variable screening. Within each correlated group, we apply the elastic net to select variables and estimate their parameters. This avoids the drawback of mistakenly eliminating true relevant variables when they are highly correlated like LASSO [R. Tibshirani, Regression shrinkage and selection via the lasso, J. R. Stat. Soc. Ser. B 58 (1996) 268–288] does. After applying the CFS procedure, the maximum absolute correlation coefficient between clusters becomes smaller and any common model selection methods like sure independence screening (SIS) [J. Fan and J. Lv, Sure independence screening for ultrahigh dimensional feature space, J. R. Stat. Soc. Ser. B 70 (2008) 849–911] or LASSO can be applied to improve the results. Extensive numerical examples including pure simulation examples and semi-real examples are conducted to show the good performances of our procedure.

A new hybrid filter–wrapper feature selection method for clustering based on ranking

Feature selection is a common task in areas such as Pattern Recognition, Data Mining, and Machine Learning since it can help to improve prediction quality, reduce computation time and build more understandable models. Although feature selection for supervised classification has been widely studied, feature selection in the absence of class labels, namely feature selection for clustering or unsupervised feature selection, has been less addressed. Most existing unsupervised feature selection approaches suffer from the called “Bias of Criterion Values to Dimension,” which arises when feature subsets with different cardinality are evaluated by an internal evaluation clustering criterion. In this paper, we introduce a new hybrid filter–wrapper method for clustering, which combines the spectral feature selection framework using the Laplacian Score ranking and a modified Calinski–Harabasz index. The proposed method in the filter stage sorts the features according to their relevance, while in the wrapper stage, through our modified Calinski–Harabasz index that takes into account the cardinality of the feature subsets under evaluation, evaluates the features considering them as a subset rather than individually by using two well-known selection strategies. Experiments on different datasets show that the proposed method alleviates the “Bias of Criterion Values to Dimension” and, identifies and selects more relevant features than those selected by other reported hybrid filter–wrapper feature selection methods for clustering. Additionally, we also contrast our results against other filter and wrapper methods of the state-of-the-art.

Feature selection for clustering using instance-based learning by exploring the nearest and farthest neighbors

Feature selection for clustering is an active research topic and is used to identify salient features that are helpful for data clustering. While partitioning a dataset into clusters, a data instance and its nearest neighbors will belong to the same cluster, and this instance and its farthest neighbors will belong to different clusters. We propose a new Feature Selection method to identify salient features that are useful for maintaining the instance’s Nearest neighbors and Farthest neighbors (referred to here as FSNF). In particular, FSNF uses the mutual information criterion to estimate feature salience by considering maintainability. Experiments on benchmark datasets demonstrate the effectiveness of FSNF within the context of cluster analysis.

Feature selection for clustering categorical data with an embedded modelling approach

AbstractResearch on the problem of feature selection for clustering continues to develop. This is a challenging task, mainly due to the absence of class labels to guide the search for relevant features. Categorical feature selection for clustering has rarely been addressed in the literature, with most of the proposed approaches having focused on numerical data. In this work, we propose an approach to simultaneously cluster categorical data and select a subset of relevant features. Our approach is based on a modification of a finite mixture model (of multinomial distributions), where a set of latent variables indicate the relevance of each feature. To estimate the model parameters, we implement a variant of the expectation‐maximization algorithm that simultaneously selects the subset of relevant features, using a minimum message length criterion. The proposed approach compares favourably with two baseline methods: a filter based on an entropy measure and a wrapper based on mutual information. The results obtained on synthetic data illustrate the ability of the proposed expectation‐maximization method to recover ground truth. An application to real data, referred to official statistics, shows its usefulness.

Text Categorization based on Clustering Feature Selection

In this paper, we discuss a text categorization method based on k-means clustering feature selection. K-means is classical algorithm for data clustering in text mining, but it is seldom used for feature selection. For text data, the words that can express correct semantic in a class are usually good features. We use k-means method to capture several cluster centroids for each class, and then choose the high frequency words in centroids as the text features for categorization. The words extracted by k-means not only can represent each class clustering well, but also own high quality for semantic expression. On three normal text databases, classifiers based on our feature selection method exhibit better performances than original classifiers for text categorization.

Mode selective damping of power system electromechanical oscillations for large power systems using supplementary remote signals

This paper presents the design of local decentralized power system stabilizer (PSS) controllers, using selected suitable remote signals as supplementary inputs, for a separate better damping of specific inter-area modes, for large-scale power systems. System identification technique is used for deriving lower order state-space models suitable for control design. The lower-order model is identified by probing the network in open loop with low-energy pulses or random signals. The identification technique is then applied to signal responses, generated by time-domain simulations of the large-scale model, to obtain reduced-order model. Lower-order equivalent models, thus obtained, are used to design each local PSS controller separately for each of the inter-area modes of interest. The PSS controller uses only those local and remote input signals in which the assigned single inter-area mode is most observable and is located at a generator which is most effective in controlling that mode. The PSS controller, designed for a particular single inter-area mode, also works mainly in a frequency band given by the natural frequency of the assigned mode. The locations of the local PSS controllers are obtained based on the amplitude gains of the frequency responses of the best-suited measurement to the inputs of all generators in the interconnected system. For the selection of suitable local and supplementary remote input signals, the features or measurements from the whole system are pre-selected first by engineering judgment and then using a clustering feature selection technique. Final selection of local and remote input signals is based on the degree of observability of the considered single mode in them. To provide robust behavior, H∞ control theory together with an algebraic Riccati equation approach has been applied to design the controllers. The effectiveness of the resulting PSS controllers is demonstrated through digital simulation studies conducted on a sixteen-machine, three-area test power system.