Proposed Feature Reduction Research Articles

Using Generic Direct M-SVM Model Improved by Kohonen Map and Dempster–Shafer Theory to Enhance Power Transformers Diagnostic

Many power transformers throughout the world are nearing or have gone beyond their theoretical design life. Since these important assets represent approximately 60% of the cost of the substation, monitoring their condition is necessary. Condition monitoring helps in the decision to perform timely maintenance, to replace equipment or extend its life after evaluating if it is degraded. The challenge is to prolong its residual life as much as possible. Dissolved Gas Analysis (DGA) is a well-established strategy to warn of fault onset and to monitor the transformer’s status. This paper proposes a new intelligent system based on DGA; the aim being, on the one hand, to overcome the conventional method weaknesses; and, on the other hand, to improve the transformer diagnosis efficiency by using a four-step powerful artificial intelligence method. (1) Six descriptor sets were built and then improved by the proposed feature reduction approach. Indeed, these six sets are combined and presented to a Kohonen map (KSOM), to cluster the similar descriptors. An averaging process was then applied to the grouped data, to reduce feature dimensionality and to preserve the complete information. (2) For the first time, four direct Multiclass Support Vector Machines (M-SVM) were introduced on the Generic Model basis; each one received the KSOM outputs. (3) Dempster–Shafer fusion was applied to the nine membership probabilities returned by the four M-SVM, to improve the accuracy and to support decision making. (4) An output post-processing approach was suggested to overcome the contradictory evidence problem. The achieved AUROC and sensitivity average percentages of 98.78–95.19% (p-value < 0.001), respectively, highlight the remarkable proposed system performance, bringing a new insight to DGA analysis.

Efficient Clustering-Based electrocardiographic biometric identification

The correct identification of individuals through different biometric traits is becoming increasingly important. Apart from traditional biomarkers (like fingerprints), many alternative measures have been proposed during the last two decades: electrocardiogram (ECG) and electroencephalogram (EEG) signals, iris or facial recognition, conductual traits, etc. Several works have shown that ECG-based recognition is a feasible alternative, either for stand-alone or multi-biometric recognition systems. In this paper, we propose a novel framework for ECG-based biometric identification, consisting of a simple and robust feature extraction approach and a clustering-based feature reduction method, that enables for an efficient and scalable biometric identification. The proposed feature reduction approach is a two phase method: it uses a clustering algorithm to group features according to their similarities first, and then clusters are represented in terms of a prototype vector and associated to the available subjects. On its side, the proposed time-domain feature extraction method is a semi-fiducial procedure, where the well-known Pan–Tompkins algorithm is first used to detect the R wave peaks of the QRS complexes, and then fixed-width time segments are selected for further dimensionality reduction and feature extraction. The resulting combined methods are efficient, robust, scalable and attain excellent results (with up-to 98.6% sensitivity) on all the subjects of the Physikalisch-Technische Bundesanstalt (PTB) database, regardless of their pathological or healthy status. Additionally, we also show how the existing Auto Correlation/Discrete Cosine Transform (AC/DCT)-based non-fiducial feature extraction method can be integrated within our framework, allowing us to attain up to 90.6% sensitivity on the Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) arrhythmia database. Since this database is much noisier and has a much lower sampling rate (360 Hz instead of 1000 Hz), we claim that this is a very good result.

Mutual Information-Driven Feature Reduction for Hyperspectral Image Classification

A hyperspectral image (HSI), which contains a number of contiguous and narrow spectral wavelength bands, is a valuable source of data for ground cover examinations. Classification using the entire original HSI suffers from the "curse of dimensionality" problem because (i) the image bands are highly correlated both spectrally and spatially, (ii) not every band can carry equal information, (iii) there is a lack of enough training samples for some classes, and (iv) the overall computational cost is high. Therefore, effective feature (band) reduction is necessary through feature extraction (FE) and/or feature selection (FS) for improving the classification in a cost-effective manner. Principal component analysis (PCA) is a frequently adopted unsupervised FE method in HSI classification. Nevertheless, its performance worsens when the dataset is noisy, and the computational cost becomes high. Consequently, this study first proposed an efficient FE approach using a normalized mutual information (NMI)-based band grouping strategy, where the classical PCA was applied to each band subgroup for intrinsic FE. Finally, the subspace of the most effective features was generated by the NMI-based minimum redundancy and maximum relevance (mRMR) FS criteria. The subspace of features was then classified using the kernel support vector machine. Two real HSIs collected by the AVIRIS and HYDICE sensors were used in an experiment. The experimental results demonstrated that the proposed feature reduction approach significantly improved the classification performance. It achieved the highest overall classification accuracy of 94.93% for the AVIRIS dataset and 99.026% for the HYDICE dataset. Moreover, the proposed approach reduced the computational cost compared with the studied methods.

A multi-scale feature selection approach for predicting benthic assemblages

Seafloor habitat maps are an important management tool used to delineate distinct regions of the seabed based on their biophysical properties. Spatially continuous bathymetry and backscatter-derived terrain features are commonly used as proxies for environmental conditions and processes that affect species distributions. Multi-scale approaches are increasingly applied to assess the relevant scales at which species co-occur. As the optimal scale(s) may be unknown, features can be calculated at multiple successive scales, yet this results in numerous highly correlated features that may negatively impact model interpretability. To address this increased dimensionality, feature selection approaches can be used to identify the most relevant features. Here, filter and wrapper approaches are assessed to select features from a highly dimensional multi-scale dataset. Terrain features describing the seabed were calculated across ten scales at two coastal sites in Placentia Bay, Newfoundland, Canada. Five species assemblages were identified using ground-truth underwater video sampling. Features predicting the presence of assemblages were assessed using the two selection methods, and the set of chosen features was modelled using three machine learning algorithms: extreme gradient boosting (XGB), random forest (RF), and support vector machines (SVM). The XGB model with features selected by scale-factor from the Boruta wrapper algorithm had the highest accuracy according to cross-validation- (61.67%, kappa 0.49). Bathymetry and terrain attributes were the most important predictors of assemblage occurrence across various analysis scales encompassing both broader and fine-scale variability of the seabed. The proposed feature reduction and selection approach improved the overall accuracy of predictions, and the resulting biological complexity captured in our habitat maps established baseline data for an ecologically significant coastal region.

Relevant and Non-Redundant Feature Selection for Cancer Classification and Subtype Detection.

Simple SummaryHere we introduce a new feature selection algorithm DTA, which selects important, non-redundant, and relevant features from diverse omics data. DTA selects non-redundant features by maximizing the similarity between each patient pair by an approximate k-cover algorithm. We successfully applied this algorithm to three different biological problems: (a) disease to healthy sample classification, (b) multiclass classification of different disease samples, and (c) disease subtypes detection. DTA outperformed other feature selection techniques in the binary classification of healthy and disease samples and multiclass classification of various diseases. It also improved the performance of a subtype detection algorithm by selecting the important features for few cancer types.Biologists seek to identify a small number of significant features that are important, non-redundant, and relevant from diverse omics data. For example, statistical methods such as LIMMA and DEseq distinguish differentially expressed genes between a case and control group from the transcript profile. Researchers also apply various column subset selection algorithms on genomics datasets for a similar purpose. Unfortunately, genes selected by such statistical or machine learning methods are often highly co-regulated, making their performance inconsistent. Here, we introduce a novel feature selection algorithm that selects highly disease-related and non-redundant features from a diverse set of omics datasets. We successfully applied this algorithm to three different biological problems: (a) disease-to-normal sample classification; (b) multiclass classification of different disease samples; and (c) disease subtypes detection. Considering the classification of ROC-AUC, false-positive, and false-negative rates, our algorithm outperformed other gene selection and differential expression (DE) methods for all six types of cancer datasets from TCGA considered here for binary and multiclass classification problems. Moreover, genes picked by our algorithm improved the disease subtyping accuracy for four different cancer types over state-of-the-art methods. Hence, we posit that our proposed feature reduction method can support the community to solve various problems, including the selection of disease-specific biomarkers, precision medicine design, and disease sub-type detection.

A feature reduction based reflected and exploited DDoS attacks detection system

The hacker attempts distributed denial of service (DDoS) attacks towards network resources to disturb or deny services. The hacker degrades the quality of service to legitimate users by performing reflection and exploitation based DDoS attacks with a trusted third party server that hides information of the attacker. It is, therefore, necessary to propose an intelligent intrusion detection system to detect reflection and exploitation based DDoS attacks efficiently and effectively. The present study proposes a feature reduction method by the combination of information gain (IG) and correlation (CR) feature selection techniques. This study presents a DDoS attack detection framework to detect reflection and exploitation based DDoS attacks in an efficient manner. The framework is tested on the latest DDoS evaluation (CICDDoS2019) dataset with J48 classifier. The feature reduction method obtains minimum and maximum reduction by 56 and 82.92% respectively, of the original features. The experimentation results show that the proposed framework outperforms using a reduced features subset. The validation of the proposed framework on knowledge discovery and data mining (KDD Cup 1999) dataset provides improvement in performance for binary and multi-level classification using feature reduction by 60.97% of the original features. The proposed feature reduction method is also compared to the relevant existing feature selection methods used for intrusion detection on CICDoS 2019 and KDD Cup 1999 datasets.

An efficient feature reduction method for the detection of DoS attack

Feature selection or reduction is a significant process for intrusion detection system (IDS) in finding optimal features. Irrelevant features present in the dataset increase load on computing resources and affect the performance of the system. The present study proposes a feature reduction method based on the combination of filter-based feature reduction algorithms, namely Information Gain Ratio (IGR), Correlation (CR), and ReliefF (ReF). The system initially obtains feature subsets for each classifier based on average weight and further Subset Combination Strategy (SCS) is applied. The proposed feature reduction method results in 24 reduced features for CICIDS 2017 DoS dataset. The proposed method shows an improved performance compared to the current state-of-the-art systems on CICIDS 2017 dataset. The proposed method has also been tested and compared with the current state-of-the-art systems on KDD Cup 99 dataset.

Effective subspace detection based on the measurement of both the spectral and spatial information for hyperspectral image classification

ABSTRACT Subspace detection from high dimensional hyperspectral image (HSI) data cube has become an important area of research for efficient identification of ground objects. Standard feature extraction method such as Principal Component Analysis (PCA) has some drawbacks as it depends solely on global variance of the dataset generated. Folded-PCA (FPCA), an improvement of PCA, offers more benefits over PCA as it envisages both local and global structures of image contents and requires less computation and memory. These superior properties make FPCA more effective for feature extraction in high dimensional remote sensing images e.g. HSIs. Therefore, the proposed feature reduction method combines FPCA feature extraction with Normalized Cross Cumulative Residual Entropy (NCCRE) feature selection, termed as FPCA-NCCRE, for efficient features’ subspace detection. NCCRE is utilised as a means of feature selection over the new features generated from FPCA to obtain a more informative subspace. It is experimented on a real mixed agricultural and an urban hyperspectral dataset. Finally, Kernel Support Vector Machine (KSVM) is implemented to calculate the classification accuracy using the detected subspace. From the experiments, it is observed that the proposed method outperforms the baseline approaches and obtains the highest accuracy of 97.67 and 98.57% on the two real hyperspectral images.

Efficient Clustering of Emails Into Spam and Ham: The Foundational Study of a Comprehensive Unsupervised Framework

The spread and adoption of spam emails in malicious activities like information and identity theft, malware propagation, monetary and reputational damage etc. are on the rise with increased effectiveness and diversification. Without doubt these criminal acts endanger the privacy of many users and businesses’. Several research initiatives have taken place to address the issue with no complete solution until now; and we believe an intelligent and automated methodology should be the way forward to tackle the challenges. However, till date limited studies have been conducted on the applications of purely unsupervised frameworks and algorithms in tackling the problem. To explore and investigate the possibilities, we intend to propose an anti-spam framework that fully relies on unsupervised methodologies through a multi-algorithm clustering approach. This article presents an in-depth analysis on the methodologies of the first component of the framework, examining only the domain and header related information found in email headers. A novel method of feature reduction using an ensemble of ‘unsupervised’ feature selection algorithms has also been investigated in this study. In addition, a comprehensive novel dataset of 100,000 records of ham and spam emails has been developed and used as the data source. Key findings are summarized as follows: I) out of six different clustering algorithms used – Spectral and K-means demonstrated acceptable performance while OPTICS projected the optimum clustering with an average of 3.5% better efficiency than Spectral and K-means, validated through a range of validations processes II) The other three algorithms- BIRCH, HDBSCAN and K-modes, did not fare well enough. III) The average balanced accuracy for the optimum three algorithms has been found to be ≈94.91%, and IV) The proposed feature reduction framework achieved its goal with high confidence.

High correlation-based banknote gradient assessment of ensemble classifier

This research presents a client and server-based mobile application for recognition and authentication of banknotes; the system extracted the shape context (SC), Scale Invariant Feature Transform (SIFT), gradient location and orientation histogram (GLOH), and Histogram of Gradient (HOG). It then reduces the feature set using Principal Component Analysis (PCA), Bag of Words and proposed two-dimension reduction approach based on low variance and high correlation filter. The classification was done using a 2-fold Weighted Majority Average (WMA) Ensemble technique with MPLNN and MCSVM as base classifiers. The application was built using Unity 3D; it was tested on Naira, USD, CAD and Euro banknotes and the experimental results proved that the implemented feature vector and the proposed feature reduction and classification technique presented the best results and with promising recognition accuracy, detection rate, and processing time.

Predicting Brain Age Based on Spatial and Temporal Features of Human Brain Functional Networks.

The organization of human brain networks can be measured by capturing correlated brain activity with functional MRI data. There have been a variety of studies showing that human functional connectivities undergo an age-related change over development. In the present study, we employed resting-state functional MRI data to construct functional network models. Principal component analysis was performed on the FC matrices across all the subjects to explore meaningful components especially correlated with age. Coefficients across the components, edge features after a newly proposed feature reduction method as well as temporal features based on fALFF, were extracted as predictor variables and three different regression models were learned to make prediction of brain age. We observed that individual's functional network architecture was shaped by intrinsic component, age-related component and other components and the predictive models extracted sufficient information to provide comparatively accurate predictions of brain age.

GARUDA: Gaussian dissimilarity measure for feature representation and anomaly detection in Internet of things

The objective of any anomaly detection system is to efficiently detect several types of malicious traffic patterns that cannot be detected by conventional firewall systems. Designing an efficient intrusion detection system has three primary challenges that include addressing high dimensionality problem, choice of learning algorithm, and distance or similarity measure used to find the similarity value between any two traffic patterns or input observations. Feature representation and dimensionality reduction have been studied and addressed widely in the literature and have also been applied for the design of intrusion detection systems (IDS). The choice of classifiers is also studied and applied widely in the design of IDS. However, at the heart of IDS lies the choice of distance measure that is required for an IDS to judge an incoming observation as normal or abnormal. This challenge has been understudied and relatively less addressed in the research literature both from academia and from industry. This research aims at introducing a novel distance measure that can be used to perform feature clustering and feature representation for efficient intrusion detection. Recent studies such as CANN proposed feature reduction techniques for improving detection and accuracy rates of IDS that used Euclidean distance. However, accuracies of attack classes such as U2R and R2L are not significantly promising. Our approach GARUDA is based on clustering feature patterns incrementally and then representing features in different transformation space through using a novel fuzzy Gaussian dissimilarity measure. Experiments are conducted on both KDD and NSL-KDD datasets. The accuracy and detection rates of proposed approach are compared for classifiers such as kNN, J48, naive Bayes, along with CANN and CLAPP approaches. Experiment results proved that proposed approach resulted in the improved accuracy and detection rates for U2R and R2L attack classes when compared to other approaches.

Feature Reduction for Power System Transient Stability Assessment Based on Neighborhood Rough Set and Discernibility Matrix

In machine learning-based transient stability assessment (TSA) problems, the characteristics of the selected features have a significant impact on the performance of classifiers. Due to the high dimensionality of TSA problems, redundancies usually exist in the original feature space, which will deteriorate the performance of classification. To effectively eliminate redundancies and obtain the optimal feature set, a new feature reduction method based on neighborhood rough set and discernibility matrix is proposed in this paper. First, 32 features are selected to structure the initial feature set based on system principle. An evaluation index based on neighborhood rough set theory is used to characterize the separability of classification problems in the specified feature space. By constructing the discernibility matrix of input features, a feature selection strategy is designed to find the optimal feature set. Finally, comparative experiments based on the proposed feature reduction method and several common feature reduction techniques used in TSA are applied to the New England 39 bus system and Australian simplified 14 generators system. The experimental results illustrate the effectiveness of the proposed feature reduction method.

Feature Selection Using Relative Fuzzy Entropy and Ant Colony Optimization Applied to Real-time Intrusion Detection System

Intrusion Detection System (IDS) is one of the most important component of network defense mechanism. In an attempt to detect network attacks, network traffic features need to be identified and both attack and normal data need to be profiled. This paper proposes a set of network traffic features that can be extracted for Real-Time Intrusion Detection. This paper also proposes Fuzzy Entropy based heuristic for Ant Colony Optimization (ACO) in-order to search for global best smallest set of network traffic features for Real-Time Intrusion Detection Data set. The proposed feature reduction algorithm was tested on standard bench-mark UCI data sets, and found to be efficient. Further the algorithm was applied to Real-Time IDS data set and found to produce promising results.

Mental Workload Recognition by Combining Wavelet Packet Transform and Kernel Spectral Regression Techniques

A Mental Workload (MWL) recognition system was developed based on psychophysiological data to assess temporal variations in MWL levels. Salient EEG features were first extracted by using fuzzy mutual-information-based wavelet-packet transform (FMI-WPT). Then we adopted the kernel spectral regression linear discriminant analysis (KSRDA) to reduce the EEG feature dimensionality and to simultaneously enhance the inter-class discrimination capacity of the MWL classifiers. By combining FMIWPT and KSRDA techniques, we designed, evaluated and compared different types of MWL classifiers. The results demonstrated a improvement of the MWL classification accuracy by the proposed feature reduction method and classifier design framework. Particularly, it was shown by extensive comparative studies that the KNN and SVM outperform other classifiers.

Vibration‐ and acoustic‐emissions based novelty detection of fretted bearings

PurposeThe purpose of this paper is to examine the use of a new feature reduction technique with novelty detection on vibration and acoustic‐emission sensors monitoring bearings mounted in the test benches of automotive manufacturers.Design/methodology/approachSignals from standard accelerometers and acoustic‐emission sensors were gathered from bearings operating under steady conditions on an accessory‐drive test bench. The bearings under test were subject to a variety of faults including fretting. These signals were processed and reduced to standard feature vectors, the dimensionality of which was reduced using a new principal‐component‐like technique optimized for novelty detection. The reduced data were analyzed with a novelty detection technique called the Support Vector Data Descriptor.FindingsThe classification results from these sensors, after being reduced with the proposed feature reduction technique, are substantially improved over those achievable with only standard novelty detection; nearly zero‐percent classification error was achieved.Research limitations/implicationsThe feature reduction technique depends, in part, on the availability of the fault type in question – potentially violating the normal novelty detection assumption of limited abnormal data. This may require the manufacturer to gather real or simulated fault data prior to running tests.Practical implicationsIncipient faults may be detectable at a much earlier stage in a manufacturer's component failure analysis. Test engineers may use this technique to reliably automate the fault detection process and enable improved root‐cause analysis through the earlier identification of faults.Originality/valueThe application of the feature reduction technique will provide manufacturers and researchers with a new means of improving fault detection in machinery components.

ECG arrhythmia classification using a probabilistic neural network with a feature reduction method

This paper presents an effective electrocardiogram (ECG) arrhythmia classification scheme consisting of a feature reduction method combining principal component analysis (PCA) with linear discriminant analysis (LDA), and a probabilistic neural network (PNN) classifier to discriminate eight different types of arrhythmia from ECG beats. Each ECG beat sample composed of 200 sampling points at a 360Hz sampling rate around an R peak is extracted from ECG signals. The feature reduction method is employed to find important features from ECG beats, and to improve the classification accuracy of the classifier. With the selected features, the PNN is then trained to serve as a classifier for discriminating eight different types of ECG beats. The average classification accuracy of the proposed scheme is 99.71%. Our experimental results have successfully validated that the integration of the PNN classifier with the proposed feature reduction method can achieve satisfactory classification accuracy.

Feature Selection and Extraction Methods for Power Systems Transient Stability Assessment Employing Computational Intelligence Techniques

One of the important aspects in achieving better performance for transient stability assessment (TSA) of power systems employing computational intelligence (CI) techniques is by incorporating feature reduction techniques. For small power system the number of features may be small but when larger systems are considered the number of features increased as the size of the systems increases. Apart from employing faster CI techniques to achieve faster and accurate TSA of power system, feature reduction techniques are needed in reducing the input features while preserving the needed information so as to make faster training of the CI technique. This paper presents feature reductions techniques used, namely correlation analysis and principle component analysis, in reducing number of input features presented to two CI techniques for TSA, namely probabilistic neural network (PNN) and least squares support vector machines (LS-SVM). The proposed feature reduction techniques are implemented and tested on the IEEE 39-bus test system and 87-bus Malaysia's power system. Numerical results are presented to demonstrate the performance of the feature reduction techniques and its effects on the accuracies and time taken for training the two CI techniques.

Improved feature reduction in input and feature spaces

In this paper, we present an improved feature reduction method in input and feature spaces for classification using support vector machines (SVMs). In the input space, we select a subset of input features by ranking their contributions to the decision function. In the feature space, features are ranked according to the weighted support vector in each dimension. By applying feature reduction in both input and feature spaces, we develop a fast non-linear SVM without a significant loss in performance. We have tested the proposed method on the detection of face, person, and car. Subsets of features are chosen from pixel values for face detection and from Haar wavelet features for person and car detection. The experimental results show that the proposed feature reduction method works successfully. In fact, our method performs better than the methods of using all the features and the Fisher's features in the detection of person and car. We also gain the advantage of speed.

Discriminative feature selection for speech recognition

Over the last several years, a major factor in reducing the error rate on most speech recognition systems has been the addition of new feature components to the frame vectors. However, because of the larger dimensionality of the frame feature vector, the number of model parameters and the computational requirements have also increased. To improve recognition performance, it is not feasible to indefinitely increase the size of the frame feature vectors, nor is it satisfactory or practical to run experiments on combinatorially chosen subsets of the feature set to pick the best performing subspace with the desired number of dimensions. It becomes clearly desirable to understand which components of the frame feature vector provide the greatest contribution to recognition performance and to discard the least useful components. Our feature ordering method allows new sets of features to be selectively incorporated into existing signal analysis methods. Discriminative analysis has been successfully used for hidden Markov model (HMM) parameter estimation. In this study, we use discriminative methods to perform feature selection of the frame feature space. The components of the feature vectors are rank ordered according to an objective criterion and only the most "significant" are used for recognition. The proposed feature reduction method has been applied to a 38 dimension vector consisting of 1 st and 2 nd order time derivatives of the frame energy and of the cepstral coefficients with their 1 st and 2 nd derivatives. Speaker independent recognition experiments with reduced feature sets were performed on three databases of high quality non-telephone speech and a data base of telephone-based speech recorded during a field trial. The dimension of the frame feature vectors, and hence the number of model parameters, were greatly reduced (by a factor of 2) without a significant loss of recognition performance.