Feature Reduction Method Research Articles

A Comprehensive Unsupervised Framework for Chronic Kidney Disease Prediction

The incidence, prevalence, and progression of chronic kidney disease (CKD) conditions have evolved over time, especially in countries that have varied social determinants of health. In most countries, diabetics and hypertension are the main causes of CKDs. The global guidelines classify CKD as a condition that results in decreased kidney function over time, as indicated by glomerular filtration rate (GFR) and markers of kidney damage. People with CKDs are likely to die at an early age. It is crucial for doctors to diagnose various conditions associated with CKD in an early stage because early detection may prevent or even reverse kidney damage. Early detection can provide better treatment and proper care to the patients. In many regional hospital/clinics, there is a shortage of nephrologists or general medical persons who diagnose the symptoms. This has resulted in patients waiting longer to get a diagnosis. Therefore, this research believes developing an intelligent system to classify a patient into classes of ‘CKD’ or ‘Non-CKD’ can help the doctors to deal with multiple patients and provide diagnosis faster. In time, organizations can implement the proposed machine learning framework in regional clinics that have lower medical expert retention, this can provide early diagnosis to patients in regional areas. Although, several researchers have tried to address the situation by developing intelligent systems using supervised machine learning methods, till date limited studies have used unsupervised machine learning algorithms. The primary aim of this research is to implement and compare the performance of various unsupervised algorithms and identify best possible combinations that can provide better accuracy and detection rate. This research has implemented five unsupervised algorithms, K-Means Clustering, DB-Scan, I-Forest, and Autoencoder. And integrating them with various feature selection methods. Integrating feature reduction methods with K-Means Clustering algorithm has achieved an overall accuracy of 99% in classifying the clinical data of CKD and Non-CKD.

Empirical Analysis of Feature Reduction in Deep Learning and Conventional Methods for Foot Image Classification

Deep learning algorithms are employed in many applications, especially in medical fields such as gait analysis and human pose detection for rehabilitation. However, creating the desired model with deep learning algorithms requires high memory and computing costs, which is problematic because deep learning technologies must be run on low-power devices such as edge computing equipment. To deal with these problems, feature reduction methods reduce the memory and energy costs. This paper presents an empirical analysis of deep learning with feature reduction. The method classifies foot images for knee rehabilitation using convolutional and dense autoencoders. The obtained results are compared with those of conventional methods (histograms of oriented gradients and local binary pattern algorithms). The features were classified and compared using support vector machine, k-nearest neighbor, and multilayer perceptron methods. The experimental results demonstrate that the conventional method uses fewer features than the deep learning method with higher accuracy because its algorithm projects pixels onto the histogram. In addition, using fewer features in deep learning layers maintains high accuracy, which is beneficial for edge computing implementations.

Semantic Network-Based Impervious Surface Extraction Method for Rural-Urban Fringe From High Spatial Resolution Remote Sensing Images

Impervious surfaces, as a key indicator of urban spatial environmental factors, have great significance in exploring the distribution law and spatial pattern of rural-urban fringe areas. To handle the increasingly rich feature information and complicated urban spatial structure in high spatial resolution remote sensing images (HSRRSIs), a semantic network model-guided extraction method for HSRRSI impervious surfaces in rural-urban fringes is proposed. The proposed method mainly includes three parts: First, construction of a semantic network model of ground covers in the rural-urban fringe and dimensionality reduction of its features. Second, optimization of a multi-scale segmentation algorithm based on the estimation of scale parameter 2 method and the fitness function. Third, proposal of a feature reduction method based on the ReliefF feature selection algorithm for spectral, texture, and geometry features to reduce the data redundancy in HSRRSIs. Finally, with the Geoeye-1 image of the rural-urban fringe of Zhanggong District as the data source, CART, RF, and SVM classifiers are used to extract the impervious surfaces of two different areas (named as Q1 and Q2), Q1 comprises the edge of rural-urban fringe with densely distributed industrial plants, and Q2 comprises a rural-urban fringe with a pronounced transition from urban to rural areas. Results show that the highest impervious surface extraction accuracy of the SVM classifier based on the semantic network model is obtained when the segmentation scale is at 210 and 215. The producer accuracy and overall accuracy for Q1 and Q2 are (94.27%, 86.41%) and (94.46%, 89.47%), respectively.

Improvement of the machine learning-based corrosion rate prediction model through the optimization of input features

The corrosion resistance of low-alloy steel seriously influences its performance, particularly as a class of materials widely used in marine environments. In this study, we collected the marine corrosion data of low-alloy steels and established corrosion rate prediction models with machine learning algorithms. Both the chemical composition of low-alloy steel and environmental factors were used as input features, and the random forest algorithm was selected as the modeling algorithm. Feature reduction methods, including the gradient boosting decision tree, Kendall correlation analysis and principal component analysis, were first applied to select the dominating factors on the corrosion rate. Then, we proposed two feature creation methods to convert the chemical composition features into a set of atomic and physical property features. As a result, the feature creation method crafted a model no longer limited to materials with specific chemical compositions. The machine learning-based corrosion rate prediction model also showed good prediction accuracy of the corrosion rate. This study improved the generalization ability of the corrosion rate prediction model and proved the feasibility of machine learning in corrosion resistance evaluation.

Improving the classification of invasive plant species by using continuous wavelet analysis and feature reduction techniques

The impacts of invasive plant species on the environment and economy make effectively detecting and mapping them crucial. Using leaf spectral reflectance and the advantages of continuous wavelet analysis (CWA), we aimed to utilize the CWA and features reduction techniques (principal component analysis (PCA), regularized random forest (RRF), and guided regularized random forest (GRRF)) and two famous classifiers (random forest (RF) and support vector machine (SVM)) to discriminate between five invasive plant species. The sample used in the study consisted of 562 leaves representing five species (Senna uniflora, Hyptis suaveolens, Parthenium hysterophorus, Prosopis juliflora, and Xanthium strumarium), which were collected from two sites. Both spectra (smoothed and original) were analyzed using CWA with different scales. 120 models of feature reduction methods (PCA, RRF, and GRRF) were established, combined with two classifiers (RF and SVM) and then compared. 90% of the smoothed CWA models (54 models) showed improvements in the overall accuracy values [1.18%, 19.38%] as compared to the smoothed spectra models alone. 94% of the non-smoothed CWA models (54 models) showed improvements in the overall accuracy values [0.18%, 19.38%] as compared to the non-smoothed spectra models alone. The highest overall accuracy was achieved at 98.87% with a model of CWA at scale 16 by using the GRRF and SVM; whereas, the models of smoothed and non-smoothed spectra without CWA had overall accuracies of 90.22% and 89.87%, respectively.Moreover, the models of CWA coupled with GRRF or RRF had better performance rates than the models of CWA with PCA. We concluded that the classification accuracy is improved when CWA with appropriate scales are used, and the feature selection process with the GRRF or RRF methods is also recommended for improving the classification performance.

The Performance of IoT Malware Detection Technique Using Feature Selection and Feature Reduction in Fog Layer

The rapid increase in the number of devices connected to internet (IoT) lead to expansion in the attacks that targeting these devices. One of these dangers attacks is malware which embedded with IoT devices that makes the detection for such malware is extremely challenging. The machine learning is one of the most effective techniques that used to detect different types of attacks in IoT environment. This technique includes three main stages: feature extraction, feature selection, and classification. The feature selection is the most important stage in ML technique because it contributes to minimizing the size of features which significantly accelerate the detection system. In this stage, most researchers trend to use one of three methods; feature selection, feature reduction and hybrid between feature selection and reduction. The present research aims to present a comparative study between the effect of using feature selection method and feature reduction method on the performance of the IoT malware detection system. The results showed that the proposed technique could achieved an accuracy about 97% when using feature selection method only. These results emphasize that feature selection method is more efficient than the feature reduction method in detection IoT malware.

Rolling Bearing Fault Diagnosis Based on Sensitive Feature Transfer Learning and Local Maximum Margin Criterion under Variable Working Condition

In real industrial scenarios, the working conditions of bearings are variable, and it is therefore difficult for data-driven diagnosis methods based on conventional machine-learning techniques to guarantee the desirable performance of diagnosis models, as the models assume that the distributions of both the training and testing data are the same. To enhance the performance of the fault diagnosis of bearings under different working conditions, a novel diagnosis framework inspired by feature extraction, transfer learning (TL), and feature dimensionality reduction is proposed in this work, and dual-tree complex wavelet packet transform (DTCWPT) is used for signal processing. Additionally, transferable sensitive feature selection by ReliefF and the sum of mean deviation (TSFSR) is proposed to reduce the redundant information of the original feature set, to select sensitive features for fault diagnosis, and to reduce the difference between the marginal distributions of the training and testing feature sets. Furthermore, a modified feature reduction method, the local maximum margin criterion (LMMC), is proposed to acquire low-dimensional mapping for high-dimensional feature spaces. Finally, bearing vibration signals collected from two test rigs are analyzed to demonstrate the adaptability, effectiveness, and practicability of the proposed diagnosis framework. The experimental results show that the proposed method can achieve high diagnosis accuracy and has significant potential benefits in industrial applications.

Online dynamic security assessment of wind integrated power system using SDAE with SVM ensemble boosting learner

The increasing trend of decarbonization, particularly high penetration of wind power generation (WPG), has resulted in an increased operational uncertainty of the power system. This emerging trend of wind integration requires novel and effective methods of power system security assessment. This paper proposes a method for dynamic security assessment (DSA) based upon stacked de-noising auto-encoder (SDAE) using support vector machine (SVM) ensemble with boosting learning approach. A two-step feature reduction method is introduced after screening the redundant features among original feature set using mutual information theory. Multi-level features of original input data are extracted using multi-layer SDAE. SVM ensemble classifier is used to perform classification with data from all hidden layers of SDAE. The output of multi-layer SDAE and SVM ensemble are combined together as an input to ensemble boosting learning method which gives the final DSA result. The proposed method is tested on modified New England 39 bus system and extended on simplified AC/DC hybrid power system of Shandong province, China. The effectiveness and accuracy of the proposed method is demonstrated by simulation results and by comparisons with other methods.

Effective subspace detection based on the measurement of both the spectral and spatial information for hyperspectral image classification

ABSTRACT Subspace detection from high dimensional hyperspectral image (HSI) data cube has become an important area of research for efficient identification of ground objects. Standard feature extraction method such as Principal Component Analysis (PCA) has some drawbacks as it depends solely on global variance of the dataset generated. Folded-PCA (FPCA), an improvement of PCA, offers more benefits over PCA as it envisages both local and global structures of image contents and requires less computation and memory. These superior properties make FPCA more effective for feature extraction in high dimensional remote sensing images e.g. HSIs. Therefore, the proposed feature reduction method combines FPCA feature extraction with Normalized Cross Cumulative Residual Entropy (NCCRE) feature selection, termed as FPCA-NCCRE, for efficient features’ subspace detection. NCCRE is utilised as a means of feature selection over the new features generated from FPCA to obtain a more informative subspace. It is experimented on a real mixed agricultural and an urban hyperspectral dataset. Finally, Kernel Support Vector Machine (KSVM) is implemented to calculate the classification accuracy using the detected subspace. From the experiments, it is observed that the proposed method outperforms the baseline approaches and obtains the highest accuracy of 97.67 and 98.57% on the two real hyperspectral images.

Intent based recognition of walking and ramp activities for amputee using sEMG based lower limb prostheses

The use of surface electromyogram (sEMG) has grown in the field of gait analysis, prostheses and exoskeleton. Surface electromyogram can directly reflect the human intention for locomotion modes and can be used as a source of control for lower limb prosthesis. The variations factors such as non-Gaussian nature of sEMG signal and mobility of amputees have been observed to degrade the activity recognition performance. This study investigates the properties of the sEMG signal with the purpose of determining the discriminant features to classify the feature space into various activities especially in the context of amputees. To address the variations in activity recognition performance, this study proposed the magnitude of bispectrum as a novel feature extraction method that is invariant to the variations factors and an unsupervised feature reduction method was used to extract the discriminant features. Furthermore, sEMG signals from eleven wearable sensors located on the lower limb muscles were recorded from six subjects including four able-bodies, one unilateral transtibial, and one unilateral transfemoral amputee during walking and ramp activities. Distinct muscles were selected using the L1-norm method. Effective classifier namely support vector machine and linear discriminant analysis were used to classify the multi-class sEMG signal patterns. The experimental results consistently showed an average accuracy of 99.7%. Further evaluation on three different types of prostheses revealed that the proposed method is more robust compared to the existing methods. The promising results of this study can be applied potentially in the control of lower limb wearable devices such as prostheses/exoskeletons.

Evolutionary feature transformation to improve prognostic prediction of hepatitis

Despite advances in Machine Learning (ML) algorithms, the clinical viability of ML-based decision support systems (DSS) to predict the prognosis of hepatitis remains limited. However, an appropriate feature selection could improve its reliability. Differently from conventional feature reduction methods, we hypothesised that applying feature reduction first and then augmenting the reduced feature space could improve classification performance further. Thus, a novel two-stage Genetic Algorithm (GA)-based feature transformation method, which involves both feature reduction and augmentation (2-Tra-GA), was developed, tested and validated. This 2-Tra-GA was later coupled to ML-based classifiers for a prognostic prediction. Clinical data with nineteen (N = 19) features on 320 patients with hepatitis obtained from the University of California-Irvine ML repository were utilised. When tested on these data, the GA-based feature reduction resulted in a reduced set with fifteen (N=15) features that led to the highest classification accuracy and reliability. Augmenting the reduced set by adding transformed features via an interpolation method (N=32 features in total, 15 reduced and 17 transformed) further improved the classification performance. Additionally, the performance of this novel hybrid algorithm was evaluated against classifiers alike and published studies. Applying feature reduction, then augmenting only such relevant features improved classification performance and computational efficiency, also over conventional wrapper-based feature selection methods. Thus, a novel hybrid DSS to improve the reliability of the prediction of prognosis for hepatitis is proposed. Findings also support the application of the proposed hybrid method to improve clinical decision making.

Word Embedding and Feature Reduction for Sentiment Analysis Using GA

Sentiment analysis (SA), also called as opinion mining is the technique for the removal of opinions of a specific entity or feature from reviews dataset. The opinions of other users help in decision making process of people. This paper studies different methods that are aimed at SA. These approaches vary from semantic based methods, machine learning, neural networks, syntactical methods with each having its own strength. Although hybrid approach also exists where the idea is to combine strengths of two or more methods to increase the accuracy. A framework in which sentiment analysis is done by using word embedding and feature reduction techniques is also proposed. Word embedding is a technique in which low-dimensional vector representation of words is provided. Feature reduction method is used with Support Vector Machine (SVM) classifier. The framework will perform sentiment analysis of user opinions by using a machine learning approach and provides a recommendation system for the ease of decision making for users. The proposed system in this paper has solved the scalability problem and improved the accuracy.

Evaluation of image processing technique as an expert system in mulberry fruit grading based on ripeness level using artificial neural networks (ANNs) and support vector machine (SVM)

Abstract Image processing and artificial intelligence (AI) techniques have been applied to analyze, evaluate and classify mulberry fruit according to their ripeness (unripe, ripe, and overripe). A total of 577 mulberries were graded by an expert and the images were captured by an imaging system. Then, the geometrical properties, color, and texture characteristics of each segmented mulberry was extracted using two feature reduction methods: Correlation-based Feature Selection subset (CFS) and Consistency subset (CONS). Artificial Neural Networks (ANN) and Support Vector Machine (SVM) were applied to classify mulberry fruit. ANN classification with the CFS subset feature extraction method resulted in the accuracy of 100 %, 100 %, and 99.1 % and the least mean square error (MSE) values of 9.2 × 10-10, 3.0 × 10-6, and 2.9 × 10-3 for training, validation, and test sets, respectively. The ANN structure with the CONS subset feature extraction method resulted in the acceptable model with the accuracy of 100 %, 98.9 %, and 98.3 % and calculated MSE values of 4.9 × 10-9, 3.0 × 10-3, and 3.1 × 10-3 for training, validation, and test sets, respectively. In general, the machine vision system combined with the ANN and SVM algorithms successfully classified mulberries based on maturity. Finally, the ANN model with four features (R, B, b*, and Cr) selected through the CONS subset method with the least number of inputs and acceptable high classification accuracy with low MSE value was proposed as the proper model for online applications.

Exploratory study on classification of lung cancer subtypes through a combined K-nearest neighbor classifier in breathomics

Accurate classification of adenocarcinoma (AC) and squamous cell carcinoma (SCC) in lung cancer is critical to physicians’ clinical decision-making. Exhaled breath analysis provides a tremendous potential approach in non-invasive diagnosis of lung cancer but was rarely reported for lung cancer subtypes classification. In this paper, we firstly proposed a combined method, integrating K-nearest neighbor classifier (KNN), borderline2-synthetic minority over-sampling technique (borderlin2-SMOTE), and feature reduction methods, to investigate the ability of exhaled breath to distinguish AC from SCC patients. The classification performance of the proposed method was compared with the results of four classification algorithms under different combinations of borderline2-SMOTE and feature reduction methods. The result indicated that the KNN classifier combining borderline2-SMOTE and feature reduction methods was the most promising method to discriminate AC from SCC patients and obtained the highest mean area under the receiver operating characteristic curve (0.63) and mean geometric mean (58.50) when compared to others classifiers. The result revealed that the combined algorithm could improve the classification performance of lung cancer subtypes in breathomics and suggested that combining non-invasive exhaled breath analysis with multivariate analysis is a promising screening method for informing treatment options and facilitating individualized treatment of lung cancer subtypes patients.

FuzzyFeatureRank. Bringing order into fuzzy classifiers through fuzzy expressions

This work presents FuzzyFeatureRank, a new feature reduction method inspired on PageRank to reduce the dimensionality of the feature space in supervised classification problems. More precisely, as it relies on a weighted directed graph, it is ultimately inspired on TextRank, a PageRank based method that adds weights to the edges to express the strength of the connections between nodes. The method is based on dividing each original feature used to describe the data into a set of fuzzy predicates and then ranking all of them by their ability to differentiate among classes in the light of the training set. In order to do that, both the information gained by each predicate and their redundancy with other already selected predicates are taken into account. The fuzzy predicates with the best scores can then be used as a reduced input to construct fuzzy classifiers that consider only the preselected predicates to build the antecedents of the fuzzy rules. The novelty of the proposal relies on being an approach halfway between feature selection and feature extraction approaches, being able to improve the discrimination ability of the original features but preserving the interpretability of the new features in the sense that they are fuzzy expressions. The experimental results support the suitability of the proposal.

Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review

In recent years, the rapid advances in machine learning (ML) and information fusion has made it possible to endow machines/computers with the ability of emotion understanding, recognition, and analysis. Emotion recognition has attracted increasingly intense interest from researchers from diverse fields. Human emotions can be recognized from facial expressions, speech, behavior (gesture/posture) or physiological signals. However, the first three methods can be ineffective since humans may involuntarily or deliberately conceal their real emotions (so-called social masking). The use of physiological signals can lead to more objective and reliable emotion recognition. Compared with peripheral neurophysiological signals, electroencephalogram (EEG) signals respond to fluctuations of affective states more sensitively and in real time and thus can provide useful features of emotional states. Therefore, various EEG-based emotion recognition techniques have been developed recently. In this paper, the emotion recognition methods based on multi-channel EEG signals as well as multi-modal physiological signals are reviewed. According to the standard pipeline for emotion recognition, we review different feature extraction (e.g., wavelet transform and nonlinear dynamics), feature reduction, and ML classifier design methods (e.g., k-nearest neighbor (KNN), naive Bayesian (NB), support vector machine (SVM) and random forest (RF)). Furthermore, the EEG rhythms that are highly correlated with emotions are analyzed and the correlation between different brain areas and emotions is discussed. Finally, we compare different ML and deep learning algorithms for emotion recognition and suggest several open problems and future research directions in this exciting and fast-growing area of AI.

Enhancing multivariate pattern analysis for magnetoencephalography through relevant sensor selection

AbstractWhile the popularity of multivariate pattern classification is growing rapidly in magnetoencephalography (MEG) data analysis, the analysis pipelines used by the neuroscience community are still missing some fundamental machine‐learning techniques and principles that would increase their effectiveness. Here, we show that MEG decoding accuracy improves significantly with the addition of feature selection methods to the analysis pipeline. We compare one unsupervised and two supervised feature reduction methods in the current study. Our results show that supervised feature selection methods like statistical dependency and mutual information improve decoding performance and attain higher session‐to‐session reliability compared to unsupervised dimensionality reduction methods like principal component analysis. Furthermore, we demonstrate that the selected sensors in the data related to a visual task at each time point are consistent with the pattern reflecting the sweep of information in the ventral visual pathway.

An ensemble feature reduction method for web-attack detection

There are a large number of features present in benchmark datasets that are used to test and evaluate intrusion detection systems. However, these high dimensional datasets require more computing resources and computation time. Identification of relevant and irrelevant features in high dimensional datasets plays a vital role in intrusion detection.This study proposes an ensemble feature reduction method to identify a reduced feature subset for the classification of web-attack. The ensemble method is based on information gain, correlation, gain ratio, chi-square, and ReliefF. Further, the system uses J48 classifier with a reduced feature subset for the classification of web-attack. The implemented system is tested on the CICIDS 2017 web-attack dataset which produces prominent results in terms of performance with reduced feature subset. Finally, the proposed method is compared with current state-of-the-art systems using J48 with 10-fold cross-validation.

Efficient Clustering of Emails Into Spam and Ham: The Foundational Study of a Comprehensive Unsupervised Framework

The spread and adoption of spam emails in malicious activities like information and identity theft, malware propagation, monetary and reputational damage etc. are on the rise with increased effectiveness and diversification. Without doubt these criminal acts endanger the privacy of many users and businesses’. Several research initiatives have taken place to address the issue with no complete solution until now; and we believe an intelligent and automated methodology should be the way forward to tackle the challenges. However, till date limited studies have been conducted on the applications of purely unsupervised frameworks and algorithms in tackling the problem. To explore and investigate the possibilities, we intend to propose an anti-spam framework that fully relies on unsupervised methodologies through a multi-algorithm clustering approach. This article presents an in-depth analysis on the methodologies of the first component of the framework, examining only the domain and header related information found in email headers. A novel method of feature reduction using an ensemble of ‘unsupervised’ feature selection algorithms has also been investigated in this study. In addition, a comprehensive novel dataset of 100,000 records of ham and spam emails has been developed and used as the data source. Key findings are summarized as follows: I) out of six different clustering algorithms used – Spectral and K-means demonstrated acceptable performance while OPTICS projected the optimum clustering with an average of 3.5% better efficiency than Spectral and K-means, validated through a range of validations processes II) The other three algorithms- BIRCH, HDBSCAN and K-modes, did not fare well enough. III) The average balanced accuracy for the optimum three algorithms has been found to be ≈94.91%, and IV) The proposed feature reduction framework achieved its goal with high confidence.

A Novel Data Instance Reduction Technique using Linear Feature Reduction

Representation of structurally significant data is indispensable to modern research. The need for dimensionality reduction finds its foray in varied genres viz-a-viz, Structural Bioinformatics, Machine Learning, Robotics, Artificial Intelligence, to name a few. The number of points required to effectively capture the essence of a structure is an intuitive decision. Feature reduction methods like Principal Component Analysis (PCA) have already been explored and proven to be an aid in classification and regression. In this work we present a novel approach that first performs PCA on a data set for reduction of features and then attempts to reduce the number of points itself to get rid of the points that have nothing or very little new to offer. The algorithm was tested on various kinds of data (points representing a spiral, protein coordinates, the Iris dataset prevalent in Machine Learning, face image) and the results agree with the quantitative tests applied. In each case, it turns out that a lot of data instances need not be stored to make any kind of decision. Matlab and R simulations were used to assess the structures with reduced data points. The time complexity of the algorithm is linear in the degrees of freedom of the data if the data is in a natural order.