Feature Ranking Techniques Research Articles

Health prediction of hydraulic cooling circuit using deep neural network with ensemble feature ranking technique

Health prediction of the hydraulic systems is of utmost importance as any breakdown may lead to severe losses. In the present manuscript, the emphasis is on developing an artificially intelligent model using a deep neural network to predict the working behaviour of the cooling circuit in the hydraulic system. Overall, four different models have been proposed and compared for their performance. Features are calculated from the pressure signals. The capabilities of XGBoost and ReliefF have been compared as a feature ranking technique and the implications of two different activation function “tanh” and “relu” have been analysed. Features shortlisted through XGBoost gives higher performance with “tanh” activation function. The result reveals that the deep neural network model can be effectively used to predict the health of hydraulic cooling circuit.

A machine learning algorithm to improve building performance modeling during design

Building design involves the optimization of factors affecting building performance such as building functions, comfort, safety, and energy. Building performance models (BPMs) help designers to evaluate and optimize such factors. However, the lack of design capabilities to validly describe human-building interactions for buildings under design may contribute to the development of inaccurate BPMs and the performance discrepancy between predictions and actual buildings. To address this challenge, a computational framework is proposed to increase the estimations performance of BPMs. The framework uses artificial neural networks (ANNs) to combine an existing BPM and context-aware design-specific data describing design-specific human-building interactions captured by using immersive virtual environments (IVEs). The framework produces an augmented BPM that can predict building performance taking human-building interactions specific to a new design into consideration. It incorporates a feature ranking technique allowing designers to assess impacts of contextual factors on human-building interactions. The paper focuses on providing details of theories, experiment and data collection designs, and algorithms behind the framework as a companion paper of [1].•A framework for combining contextual factors with building performance models to enhance their predictive performance.•Computation for determining impacts of contextual factors on human-building interaction.

Combining context-aware design-specific data and building performance models to improve building performance predictions during design

Abstract Building performance models (BPMs) such as building energy simulation models have been widely used in building design. Conventional BPMs may not be able to effectively address human- building interactions in new buildings still under design. The lack of such capability often contributes to the existence of building performance gaps, i.e., differences between predicted performance during design and actual performance of buildings. To improve the prediction accuracy of conventional BPMs, a computational framework is developed. It combines an existing BPM with context-aware design-specific data involving human-building interactions in new designs, using a machine learning approach. Immersive virtual environment (IVE) is used to capture data describing design-specific human-building interactions; and an artificial neural network (ANN) is used to combine data obtained from an existing BPM and an IVE to produce an augmented BPM. Additionally, the framework has the capability to rank influence of factors impacting human-building interactions using a feature ranking technique, which can help the design of future IVE experiments for better data collection. The framework is tested using an application of a single occupancy office. An IVE of the office is created to simulate key artificial light use events during design. The Hunt model is selected as an existing BPM. The actual use of artificial lighting in the office is observed for one month using sensors to validate the effectiveness of the framework. The results of the application have shown the potential of the framework in improving the prediction accuracy of the Hunt model evaluated against data obtained from the actual office. The results verify the important role of context-aware design-specific data in improving the prediction of human-building interactions during design. In addition, the feature ranking technique is effective in identifying influencing factors impacting human-building interactions. Limitations of this study and future work are also discussed.

Prediction of compressive strength and portland cement composition using cross-validation and feature ranking techniques

Prediction of compressive strength and concrete composition using machine learning models is an essential feature in civil engineering applications. In the present paper, a methodology is proposed for the prediction of compressive strength of concrete and portland cement composition using three experimental data sets. Ten-fold cross-validation procedure is applied to four machine learning models Isotonic regression, Artificial neural network, Support vector machine, and Random forest. Further comparison was made with feature ranking and without feature ranking to reduce the computational time and to achieve better prediction accuracy. The accuracy of machine learning models are analyzed with four parameters, Correlation coefficient, Kendall’s tau, Mean absolute error, and Root mean square error. It is observed that better prediction capability is achieved with the ten-fold cross-validation, since it gives a statistically unbiased result. Results obtain reveals a high correlation and less error between the experimental and predicted values for all the three experimental datasets consider.

A comparative study on feature selection for a risk prediction model for colorectal cancer

Background and objectiveRisk prediction models aim at identifying people at higher risk of developing a target disease. Feature selection is particularly important to improve the prediction model performance avoiding overfitting and to identify the leading cancer risk (and protective) factors. Assessing the stability of feature selection/ranking algorithms becomes an important issue when the aim is to analyze the features with more prediction power. MethodsThis work is focused on colorectal cancer, assessing several feature ranking algorithms in terms of performance for a set of risk prediction models (Neural Networks, Support Vector Machines (SVM), Logistic Regression, k-Nearest Neighbors and Boosted Trees). Additionally, their robustness is evaluated following a conventional approach with scalar stability metrics and a visual approach proposed in this work to study both similarity among feature ranking techniques as well as their individual stability. A comparative analysis is carried out between the most relevant features found out in this study and features provided by the experts according to the state-of-the-art knowledge. ResultsThe two best performance results in terms of Area Under the ROC Curve (AUC) are achieved with a SVM classifier using the top-41 features selected by the SVM wrapper approach (AUC=0.693) and Logistic Regression with the top-40 features selected by the Pearson (AUC=0.689). Experiments showed that performing feature selection contributes to classification performance with a 3.9% and 1.9% improvement in AUC for the SVM and Logistic Regression classifier, respectively, with respect to the results using the full feature set. The visual approach proposed in this work allows to see that the Neural Network-based wrapper ranking is the most unstable while the Random Forest is the most stable. ConclusionsThis study demonstrates that stability and model performance should be studied jointly as Random Forest turned out to be the most stable algorithm but outperformed by others in terms of model performance while SVM wrapper and the Pearson correlation coefficient are moderately stable while achieving good model performance.

Combining Multiple Feature-Ranking Techniques and Clustering of Variables for Feature Selection

Feature selection aims to eliminate redundant or irrelevant variables from input data to reduce computational cost, provide a better understanding of data and improve prediction accuracy. Majority of the existing filter methods utilize a single feature-ranking technique, which may overlook some important assumptions about the underlying regression function linking input variables with the output. In this paper, we propose a novel feature selection framework that combines clustering of variables with multiple feature-ranking techniques for selecting an optimal feature subset. Different feature-ranking methods typically result in selecting different subsets, as each method has its own assumption about the regression function linking input variables with the output. Therefore, we employ multiple feature-ranking methods having disjoint assumption about the regression function. The proposed approach has a feature ranking module to identify relevant features and a clustering module to eliminate redundant features. First, input variables are ranked using regression coefficients obtained by training $L1$ regularized Logistic Regression, Support Vector Machine and Random Forests models. Those features which are ranked lower than a certain threshold are filtered-out. The remaining features are grouped into clusters using an exemplar-based clustering algorithm, which identifies data-points that exemplify the data better, and associates each data-point with an exemplar. We use both linear correlation coefficients and information gain for measuring the association between a data-point and its corresponding exemplar. From each cluster the highest ranked feature is selected as a delegate, and all delegates from the three ranked lists are combined into the final feature set using union operation. Empirical results over a number of real-world data sets confirm the hypothesis that combining features selected using multiple heterogeneous methods results in a more robust feature set and improves prediction accuracy. As compared to other feature selection approaches evaluated, features selected using linear correlation-based multi-filter feature selection achieved the best classification accuracy with 98.7%, 100%, 92.3% and 100% for Ionosphere, Wisconsin Breast Cancer, Sonar and Wine data sets respectively.

Stability analysis of Feature Ranking Techniques in the presence of noise: A comparative study

Noisy data is one of the common problems associated with real-world data, and may affects the performance of the data models, consequent decisions and the performance of feature ranking techniques. In this paper, we show how stability performance can be changed if different feature ranking methods against attribute noise and class noise are used. We consider Kendall's Tau rank correlation and Spearman rank correlation to evaluate various feature ranking methods stability, and quantify the degree of agreement between ordered lists of features created by a filter on a clean dataset and its outputs on the same dataset corrupted with different combinations of the noise level. According to the results of Kendall and Spearman measures, Gini index (GI) and information gain (IG) have the best performances respectively. Nevertheless, both Kendall and Spearman measures results show that ReliefF (RF) is the most sensitive (the worst) performance.

Identification of hormone-binding proteins using a novel ensemble classifier

Hormone-binding proteins (HBPs) are important soluble carriers for growth hormones, and correct recognition of HBPs is crucial to understanding their functions. Therefore, we aimed to construct an efficient and reliable classifier to identify HBPs accurately. At first, 246 proteins were collected from UniProt database and considered as the objective benchmark dataset. We employed the 8000-dimensional feature extraction method based on tripeptide compositions to formulate protein samples. Subsequently, we alleviated the intricate feature set by utilizing ANOVA, a feature ranking technique, and acquired the optimal feature subset devoid of redundant information. Furthermore, we utilized three classification methods to process the selected tripeptide features, which generated three probability sequences. Finally, the three probability sequences were considered as new features, and addressed by the support vector machine to construct a prediction model. Results indicated that 90.6% of accuracy was achieved in five-fold cross validation, which was superior to that of other published methods.

Automated detection and classification of liver fibrosis stages using contourlet transform and nonlinear features

Background and objectiveLiver fibrosis is a type of chronic liver injury that is characterized by an excessive deposition of extracellular matrix protein. Early detection of liver fibrosis may prevent further growth toward liver cirrhosis and hepatocellular carcinoma. In the past, the only method to assess liver fibrosis was through biopsy, but this examination is invasive, expensive, prone to sampling errors, and may cause complications such as bleeding. Ultrasound-based elastography is a promising tool to measure tissue elasticity in real time; however, this technology requires an upgrade of the ultrasound system and software. In this study, a novel computer-aided diagnosis tool is proposed to automatically detect and classify the various stages of liver fibrosis based upon conventional B-mode ultrasound images. MethodsThe proposed method uses a 2D contourlet transform and a set of texture features that are efficiently extracted from the transformed image. Then, the combination of a kernel discriminant analysis (KDA)-based feature reduction technique and analysis of variance (ANOVA)-based feature ranking technique was used, and the images were then classified into various stages of liver fibrosis. ResultsOur 2D contourlet transform and texture feature analysis approach achieved a 91.46% accuracy using only four features input to the probabilistic neural network classifier, to classify the five stages of liver fibrosis. It also achieved a 92.16% sensitivity and 88.92% specificity for the same model. The evaluation was done on a database of 762 ultrasound images belonging to five different stages of liver fibrosis. ConclusionsThe findings suggest that the proposed method can be useful to automatically detect and classify liver fibrosis, which would greatly assist clinicians in making an accurate diagnosis.

Compressive Sampling and Feature Ranking Framework for Bearing Fault Classification With Vibration Signals

Failures of rolling element bearings are amongst the main causes of machines breakdowns. To prevent such breakdowns, bearing health monitoring is performed by collecting data from rotating machines, extracting features from the collected data, and applying a classifier to classify faults. To avoid the burden of much storage requirements and processing time of a tremendously large amount of vibration data, the present paper proposes a combined compressive sampling (CS)-based on multiple measurement vector (MMV) and feature ranking framework to learn optimally fewer features from a large amount of vibration data from which bearing health conditions can be classified. The CS-based on the MMV model is the first step in this framework and provides compressively sampled signals based on compressed sampling rates. In the second step, the search for the most important features of these compressively sampled signals is performed using the feature ranking and selection techniques. For that purpose, we have investigated the following: 1) two compressible representations of vibration signals that can be used within CS framework, namely, fast Fourier transform-based coefficients and thresholded Wavelet transform-based coefficients and 2) several feature ranking and selection techniques, namely, three similarity-based techniques, fisher score, Laplacian score, Relief-F; one correlation-based technique, Pearson correlation coefficients; and one independence test technique, Chi-Square (Chi-2) to select fewer features that can sufficiently represent the original vibration signals. These selected features, in combination with three of the popular classifiers–multinomial logistic regression classifier, artificial neural networks, and support vector machines, have been evaluated for the classification of bearing faults. Results show that the proposed framework achieves high classification accuracies with a limited amount of data using various combinations of methods, which outperform recently published results.

Sentiment analysis of feature ranking methods for classification accuracy

Text pre-processing and feature selection are important and critical steps in text mining. Text pre-processing of large volumes of datasets is a difficult task as unstructured raw data is converted into structured format. Traditional methods of processing and weighing took much time and were less accurate. To overcome this challenge, feature ranking techniques have been devised. A feature set from text preprocessing is fed as input for feature selection. Feature selection helps improve text classification accuracy. Of the three feature selection categories available, the filter category will be the focus. Five feature ranking methods namely: document frequency, standard deviation information gain, CHI-SQUARE, and weighted-log likelihood –ratio is analyzed.

A two stage grading approach for feature selection and classification of microarray data using Pareto based feature ranking techniques: A case study

High dimensional search space in microarray data with large number of genes and few dozen of samples increases the complexity of analysis of such databases. All the genes are not significant and hence informative genes are required to be extracted. So dimension reduction is necessary for this process. It is often found in literature that the ranking approaches are used for feature selection. Different ranking techniques may assign different rank to the same gene and the selection made based on these ranks may not be suitable for different problems. So use of one ranking technique may lead to rejection of some important genes and possibly selection of some insignificant genes. Such selection may degrade the performance of the classifier. To overcome this problem, here abi-objective ranked based Pareto front technique is proposed. In this technique using two ranked based technique the Pareto optimal solution is generated with a set of features. For the experimental work, 21 models based on 7 feature ranking strategies are considered. Eight different microarray data are taken to find the suitable ranking combination for the work. A grading method is used to rank the models and statistical test is performed to validate the findings.

A comprehensive data mining approach to estimate the rate of penetration: Application of neural network, rule based models and feature ranking

Rate of Penetration (ROP) estimation is one of the main factors in drilling optimization and minimizing the operation costs. However, ROP depends on many parameters which make its prediction a complex problem. In the presented study, a novel and reliable computational approach for prediction of ROP is proposed. Firstly, fscaret package in R environment was implemented to find out the importance and ranking of the inputs parameters. According to the feature ranking technique, weight on bit and mud weight had the highest impact on ROP based on their ranges within this dataset. Also, for developing further models Cubist method was applied to reduce the input vector from 13 to 6 and 4. Then, Random Forest (RF) and Monotone Multi-Layer Perceptron (MON-MLP) models were applied to predict ROP. The goodness of fit for all models were measured by RMSE and R2 in 10-fold cross validation scheme, and both models showed a reliable accuracy. In order to gain a deeper understanding of the relationships between input parameters and ROP, MON-MLP model with 6 inputs was used to check the effect of weight on bit, mud weight and viscosity. Finally, RF model with 4 variables was used to extract the most important rules from dataset as a transparent model.

Feature selection model based on clustering and ranking in pipeline for microarray data

Abstract Most of the available feature selection techniques in the literature are classifier bound. It means a group of features tied to the performance of a specific classifier as applied in wrapper and hybrid approach. Our objective in this study is to select a set of generic features not tied to any classifier based on the proposed framework. This framework uses attribute clustering and feature ranking techniques in pipeline in order to remove redundant features. On each uncovered cluster, signal-to-noise ratio, t -statistics and significance analysis of microarray are independently applied to select the top ranked features. Both filter and evolutionary wrapper approaches have been considered for feature selection and the data set with selected features are given to ensemble of predefined statistically different classifiers. The class labels of the test data are determined using majority voting technique. Moreover, with the aforesaid objectives, this paper focuses on obtaining a stable result out of various classification models. Further, a comparative analysis has been performed to study the classification accuracy and computational time of the current approach and evolutionary wrapper techniques. It gives a better insight into the features and further enhancing the classification accuracy with less computational time.

Data mining framework for breast lesion classification in shear wave ultrasound: A hybrid feature paradigm

Abstract Assessment of elasticity parameters of breast using ultrasound elastography (USE) provides exclusive information about the cancerous tissue. Shear wave elastography (SWE), a new USE imaging procedure is increasingly used for elasticity evaluation of breast lesions. SWE examination is gaining popularity in the characterization of benign and malignant breast lesions as it has high diagnostic performance accuracy. However, some degree of manual errors, such as probe compression or movement may cause inaccurate results. In addition, the systems cannot measure elasticity values in small lesions where the tissues do not vibrate enough. Thus, computer-aided methods suppress these technical or manual limitations of SWE during evaluation of breast lesions. Therefore, this paper proposes, a novel methodology for characterization of benign and malignant breast lesions using SWE. Original SWE image is subjected to three levels of Discrete wavelet transform (DWT) to obtain different coefficients. Second order statistics (Run Length Statistics) and Hu’s moments features are extracted from DWT coefficients. Extracted features are subjected to sequential forward selection (SFS) method to obtain the significant features and ranked using ReliefF feature ranking technique. Ranked features are fed to different classifiers for automated characterization of benign and malignant breast lesions. Our proposed technique achieved a significant accuracy of 93.59%, sensitivity of 90.41% and specificity of 96.39% using only three features. In addition, a unique integrated index named Shear Wave Breast Cancer Risk Index (sBCRI) is formulated for characterization of malignant and benign breast lesion using only two features. The proposed index, sBCRI, provides a single number which characterizes the malignant and benign cancer faster. This system can be employed as an ideal screening tool as it has high sensitivity and low false-positive rate. Hence, the women with benign lesions need not undergo unnecessary biopsies.

The impact of feature selection on maintainability prediction of service-oriented applications

Service-oriented development methodologies are very often considered for distributed system development. The quality of service-oriented computing can be best assessed by the use of software metrics that are considered to design the prediction model. Feature selection technique is a process of selecting a subset of features that may lead to build improved prediction models. Feature selection techniques can be broadly classified into two subclasses such as feature ranking and feature subset selection technique. In this study, eight different types of feature ranking and four different types of feature subset selection techniques have been considered for improving the performance of a prediction model focusing on maintainability criterion. The performance of these feature selection techniques is evaluated using support vector machine with different types of kernels over a case study, i.e., five different versions of eBay Web service. The performances are measured using accuracy and F-measure value. The results show that maintainability of the service-oriented computing paradigm can be predicted by using object-oriented metrics. The results also show that it is possible to find a small subset of object-oriented metrics which helps to predict maintainability with higher accuracy and also reduces the value of misclassification errors.

A comparative study of feature ranking techniques for epileptic seizure detection using wavelet transform

In this paper, an attempt is made to obtain optimal wavelet function and wavelet based Electroencephalograph (EEG) features for detection of epilepsy using appropriate feature ranking techniques. The EEG data includes normal, pre-ictal and ictal EEG signals. Initially, signals are decomposed using 16 discrete wavelets and the best basis wavelet is selected using Maximum Energy to Permutation Entropy ratio criterion. A range of statistical, fractal and entropy based features are calculated from selected wavelet coefficients. The performance of three different feature ranking techniques i.e. Fisher Score, ReliefF and Information Gain is investigated on computed features. Classification of the ranked features is performed by machine learning technique Least Square-Support Vector Machine. Features ranked through Fisher Score ranking technique show high discrimination ability and classified with high classification accuracy. Classification results ensure the suitability of proposed best basis wavelet based feature extraction methodology and Fisher Score ranking technique in epilepsy detection.

On Orthogonal Feature Extraction model with applications in medical prognosis

Microarray-based gene expression profiling for cancer prognosis can shed light on providing prognostic information and it has statistically become a highly reliable technique. From a computer scientist’s point of view, cancer prognosis can be termed as a classification problem. In microarray-based cancer prognosis problems, selection of critical attributes is extremely popular as a lot of features are irrelevant or redundant. Appropriate feature selection not only can filter irrelevant features but also improve prediction accuracy. Various models and methods have been developed to rank the importance of features. However, there is no dominant strategy because all methods have their own advantages and drawbacks. In this paper, we propose an Orthogonal Feature Extraction (OFE) model based on feature ranking techniques, which aims at improving cancer prediction accuracy. Our proposed model is compared with the most widely used feature extraction method: Principal Component Analysis (PCA), and two other existing feature extraction methods: Neighborhood Component Analysis (NCA) and Linear Discriminant Analysis (LDA) through 5-fold cross-validation. Numerical results indicated that OFE method can efficiently construct combinations of significant variables enabling computational complexity reduction and also provide recognizable better performance.

Novel ensemble techniques for classification of rolling element bearing faults

Rolling element bearings are the critical mechanical components in industrial applications. These rotary elements work continuously under different operating and environmental conditions. It leads to generation of various defects over the operating surfaces of bearing components. In this study, four machine learning techniques, two ensemble techniques, i.e., rotation forest and random subspace, and two well-established techniques, i.e., support vector machine and artificial neural network, are utilized for fault severity classification. Time domain, frequency domain and wavelet-based features are extracted from the raw vibration signals of rolling element bearings for the investigations. Four feature ranking techniques are employed to rank extracted features. Comparative study is carried out among the machine learning techniques with and without the ranking of features. Present study not only investigates various machine learning techniques but also examines the performance of various feature ranking techniques for fault severity classification of rolling element bearings. Results show that ensemble techniques have superior classification efficiency and take very less computational time for the analysis in comparison to support vector machine and artificial neural network.

Identification and functional assessment of novel gene sets towards better understanding of dysplasia associated oral carcinogenesis

Oral epithelial dysplasia (OED) often precedes oral cancer. Understanding the underlying complex biological aspects of dysplasia associated oral carcinogenesis using important gene sets is thus important. Computation assisted gene set identification through different feature ranking and visualization techniques was therefore attempted in this study. Result suggested that, weighted support vector machine (SVM) could be useful for feature ranking and SVM for attribute selection. Alteration in keratinization, cell–cell communication and peptidase activity was the major affected phenomena, while extracellular matrix dynamics was also found to be hampered. During best gene subset identification, set of six genes could classify normal (NOM) and oral squamous cell carcinoma (OSCC) conditions and two sets comprising four genes in each could classify NOM and dysplastic (DYS) conditions with 100% sensitivity and specificity. A gene set, comprising 32 genes showed best efficacy of 94.12% sensitivity, 99.40% specificity and 98.89% accuracy during classification of DYS and OSCC.