Clustering-based Feature Extraction Research Articles

CFEM: a cluster based feature extraction method for network intrusion detection

cFEM: a cluster based feature extraction method for network intrusion detection

A feature extraction method based on spectral segmentation and integration of hyperspectral images

In response to the curse of dimensionality in hyperspectral images (HSIs), to date, numerous dimensionality reduction methods have been proposed among which the feature extraction (FE) methods are of particular interest. This paper introduces a new supervised pixel-based FE called spectral segmentation and integration (SSI). In SSI, the spectral signature curve (SSC) of the pixels are identically divided into some non-overlapping segments, called channels. The existing bands in each channel are then integrated using a mean-weighted operator, leading to some new features in a very lower number than the original bands. SSI applies a particle swarm optimization (PSO) algorithm to globally search and locate the optimum positions and widths of the channels. For the sake of evaluation and comparison, the features provided by the proposed SSI method were applied to the well-known SVM classifier. The results were compared to not only a most recent pixel-based FE method, namely, spectral region splitting but also six conventional FE methods, including nonparametric weighted feature extraction, decision boundaries feature extraction, clustering-based feature extraction, semi-supervised local discriminant analysis, band correlation clustering and principal component analysis. Experimental results, obtained on two HSIs, proved the superiority of the proposed SSI.

Arrhythmia Recognition and Classification Using Combined Parametric and Visual Pattern Features of ECG Morphology

ECG is a non-invasive tool used to detect cardiac arrhythmias. Many arrhythmias classification solutions with various ECG features have been reported in literature. In this work, a new method combined with a novel morphological feature is proposed for accurate recognition and classification of arrhythmias. First, the events of the ECG signals are detected. Then, parametric features of ECG morphology, i.e., amplitude, interval and duration, are extracted from selected ECG regions. Next, a novel feature for analyzing QRS complex morphology changes as visual patterns as well as a new clustering-based feature extraction algorithm is proposed. Finally, the feature vectors are applied to three well-known classifiers (neural network, SVM, and KNN) for automatic diagnosis. The proposed method was assessed with all fifteen types of heartbeats as recommended by the Association for Advancement of Medical Instrumentation from the MIT-BIH arrhythmia database and achieved the best overall accuracy of 97.70% based on KNN, using the combined parametric and visual pattern features of ECG Morphology. The accuracies for the six main types - normal (N), left bundled branch blocks (L), right bundled branch blocks (R), premature ventricular contractions (V), atrial premature beats (A) and paced beats (P) are 97.79%, 99.50%, 99.59%, 97.69%, 89.70%, and 99.92%, respectively. Comparisons with peer works prove a marginal progress in automatic heart arrhythmia classification performance.

An unsupervised feature extraction method based on band correlation clustering for hyperspectral image classification using limited training samples

ABSTRACTDue to the advancement of sensor technology, hyperspectral sensors are now able to record reflectance of the earth’s surface in hundreds of bands. This makes hyperspectral images a rich source for a diverse range of applications, including classification tasks. Based on Hughes phenomenon, classification accuracy decreases as dimensionality increases while a limited number of training samples are within reach. In a hyperspectral image adjacent and non-adjacent bands may have a high correlation; therefore removing this redundancy reduces the amount of data for further analysis including classification. Extraction of a few features which lead to high classification accuracy is a need. In this article, we propose an unsupervised feature extraction method called band correlation clustering (BCC). The proposed method includes three main steps: 1) Calculation of the bands’ correlation coefficient and generating correlation coefficient matrix, 2) Grouping the bands based on correlation coefficient matrix using the k-means clustering algorithm, and 3) Mean calculation of each cluster as a newly extracted feature. The proposed method is evaluated by two different criteria: classification accuracy and time consumption. For performance evaluation, the extracted features are fed to two supervised classifiers of non-parametric SVM and parametric ML. The results are compared with the ones obtained by unsupervised extracted features using principle component analysis (PCA), independent component analysis (ICA), and minimum noise fraction (MNF) and supervised ones using generalized discriminant analysis (GDA) and clustering-based feature extraction (CBFE). Comparison of the results shows promising performance for proposed BCC both in terms of computational costs and classification accuracy.

Clustering Based Feature Extraction for Image Forgery Detection

Clustering Based Feature Extraction for Image Forgery Detection

The Empirical Study of Semi-Supervised Deep Fuzzy C-Mean Clustering for Software Fault Prediction

Software fault prediction is a very consequent research topic for software quality assurance. The performance of fault prediction model depends on the features that are used to train it. Redundant and irrelevant features can hinder the performance of a classification model. In this paper, we propose an empirical study of two-stage data pre-processing technique on software fault prediction models. In the first stage, a novel semi-supervised deep Fuzzy C-Mean (DFCM) clustering-based feature extraction technique is proposed to create new features by utilizing deep multi-clusters of unlabeled and labeled data sets that tends to maximize intra-cluster class and intra-cluster feature by using FCM clustering. The FCM also utilizes to handle the class imbalance problem. In the second stage, we further ameliorate the prediction performance with coalescence of feature selection (using random-under sampling) to reduce the noisy data for classification. However, by the performance of the model results in the amalgamation of novel DFCM data pre-processing approach work better due to their ability to identify and amalgamation essential information in data features. An empirical study is designed on real-world software project (NASA & Eclipse) data set to evaluate the performance of DFCM by implemented different data pre-processing schemes on prediction models (C4.5, naive bayes, and 1-near neighbor (1-NN)), which are widely used in software fault prediction and further investigated the influencing factors in our approach. The result shows that the performance of the proposed DFCM feature extraction technique for data pre-processing is stable and effectiveness on all prediction models.

Fuzzy clustering-based feature extraction method for mental task classification

A brain computer interface (BCI) is a communication system by which a person can send messages or requests for basic necessities without using peripheral nerves and muscles. Response to mental task-based BCI is one of the privileged areas of investigation. Electroencephalography (EEG) signals are used to represent the brain activities in the BCI domain. For any mental task classification model, the performance of the learning model depends on the extraction of features from EEG signal. In literature, wavelet transform and empirical mode decomposition are two popular feature extraction methods used to analyze a signal having non-linear and non-stationary property. By adopting the virtue of both techniques, a theoretical adaptive filter-based method to decompose non-linear and non-stationary signal has been proposed known as empirical wavelet transform (EWT) in recent past. EWT does not work well for the signals having overlapped in frequency and time domain and failed to provide good features for further classification. In this work, Fuzzy c-means algorithm is utilized along with EWT to handle this problem. It has been observed from the experimental results that EWT along with fuzzy clustering outperforms in comparison to EWT for the EEG-based response to mental task problem. Further, in case of mental task classification, the ratio of samples to features is very small. To handle the problem of small ratio of samples to features, in this paper, we have also utilized three well-known multivariate feature selection methods viz. Bhattacharyya distance (BD), ratio of scatter matrices (SR), and linear regression (LR). The results of experiment demonstrate that the performance of mental task classification has improved considerably by aforesaid methods. Ranking method and Friedman’s statistical test are also performed to rank and compare different combinations of feature extraction methods and feature selection methods which endorse the efficacy of the proposed approach.

A flexible data-driven comorbidity feature extraction framework

Disease and symptom diagnostic codes are a valuable resource for classifying and predicting patient outcomes. In this paper, we propose a novel methodology for utilizing disease diagnostic information in a predictive machine learning framework. Our methodology relies on a novel, clustering-based feature extraction framework using disease diagnostic information. To reduce the data dimensionality, we identify disease clusters using co-occurrence statistics. We optimize the number of generated clusters in the training set and then utilize these clusters as features to predict patient severity of condition and patient readmission risk. We build our clustering and feature extraction algorithm using the 2012 National Inpatient Sample (NIS), Healthcare Cost and Utilization Project (HCUP) which contains 7 million hospital discharge records and ICD-9-CM codes. The proposed framework is tested on Ronald Reagan UCLA Medical Center Electronic Health Records (EHR) from 3041 Congestive Heart Failure (CHF) patients and the UCI 130-US diabetes dataset that includes admissions from 69,980 diabetic patients. We compare our cluster-based feature set with the commonly used comorbidity frameworks including Charlson's index, Elixhauser's comorbidities and their variations. The proposed approach was shown to have significant gains between 10.7–22.1% in predictive accuracy for CHF severity of condition prediction and 4.65–5.75% in diabetes readmission prediction.

Band Clustering-Based Feature Extraction for Classification of Hyperspectral Images Using Limited Training Samples

Feature extraction plays a central role in classification of hyperspectral data. We propose a clustering-based feature extraction (CBFE) method in this letter. The proposed method is supervised and only needs to calculate the first-order statistics. Thus, CBFE has better performance than some popular supervised feature extraction methods such as linear discriminant analysis, generalized discriminant analysis, and nonparametric weighted feature extraction in small sample size situation. In addition, CBFE works better than unsupervised approaches such as principal component analysis in classification applications. CBFE considers a vector associated with each band that is composed by the mean values of all classes in that band. Then, a clustering method such as k-means is run to group the similar bands in one cluster. The selected number of clusters is equal to the number of extracted features. Experiments carried out on two different hyperspectral data sets demonstrate that the CBFE has better performance in comparison with some conventional feature extraction methods.

A Novel Clustering-Based Feature Representation for the Classification of Hyperspectral Imagery

In this study, a new clustering-based feature extraction algorithm is proposed for the spectral-spatial classification of hyperspectral imagery. The clustering approach is able to group the high-dimensional data into a subspace by mining the salient information and suppressing the redundant information. In this way, the relationship between neighboring pixels, which was hidden in the original data, can be extracted more effectively. Specifically, in the proposed algorithm, a two-step process is adopted to make use of the clustering-based information. A clustering approach is first used to produce the initial clustering map, and, subsequently, a multiscale cluster histogram (MCH) is proposed to represent the spatial information around each pixel. In order to evaluate the robustness of the proposed MCH, four clustering techniques are employed to analyze the influence of the clustering methods. Meanwhile, the performance of the MCH is compared to three other widely used spatial features: the gray-level co-occurrence matrix (GLCM), the 3D wavelet texture, and differential morphological profiles (DMPs). The experiments conducted on four well-known hyperspectral datasets verify that the proposed MCH can significantly improve the classification accuracy, and it outperforms other commonly used spatial features.

Wavelets-Based Feature Extraction for Texture Classification

Texture classification is a necessary task in a wider variety of application areas such as manufacturing, textiles, and medicine. In this paper, we propose a novel wavelet-based feature extraction method for robust, scale invariant and rotation invariant texture classification. The method divides the 2-D wavelet coefficient matrices into 2-D clusters and then computes features from the energies inherent in these clusters. The features that contain the information effective for classifying texture images are computed from the energy content of the clusters, and these feature vectors are input to a neural network for texture classification. The results show that the discrimination performance obtained with the proposed cluster-based feature extraction method is superior to that obtained using conventional feature extraction methods, and robust to the rotation and scale invariant texture classification.

Machine fault diagnosis using a cluster-based wavelet feature extraction and probabilistic neural networks

In this paper, a cluster-based feature extraction from the coefficients of a discrete wavelet transform and probabilistic neural networks are proposed for machine fault diagnosis. The proposed approach first divides the matrix of wavelet coefficients into clusters, which are centered around the discriminative coefficient positions identified by an unsupervised procedure, based on the entropy value of coefficients from a set of representative signals. The features that contain the informative attributes of the signals are computed from the energy content of the obtained clusters. Then, machine faults are diagnosed based on these feature vectors using a probabilistic neural network. The experimental results from the application on bearing fault diagnosis have shown that the proposed approach is able to effectively extract important intrinsic information content of the test signals and increase the overall fault diagnostic accuracy, as compared to conventional methods.

A Cluster-Based Wavelet Feature Extraction Method for Machine Fault Diagnosis

In this paper, a cluster-based feature extraction from the coefficients of discrete wavelet transform is proposed for machine fault diagnosis. The proposed approach first divides the matrix of wavelet coefficients into clusters that are centered around the discriminative coefficient positions identified by an unsupervised procedure based on the entropy value of coefficients from a set of representative signals. The features that contain the informative attributes of the signals are computed from the energy content of so obtained clusters. Then machine faults are diagnosed based on these feature vectors using a neural network. The experimental results from the application on bearing fault diagnosis have shown that the proposed approach is able to effectively extract important intrinsic information content of the test signals, and increase the overall fault diagnostic accuracy as compared to conventional methods.

Facial expression recognition: A clustering-based approach

This paper describes a new clustering based feature extraction method for facial expression recognition. We demonstrate the effectiveness of this method and compare it with commonly used principal component analysis method and linear discriminant analysis method.