Adaptive Feature Selection Method Research Articles

VAM-Net: Vegetation-Attentive deep network for Multi-modal fusion of visible-light and vegetation-sensitive images

Multi-modal fusion of remote sensing images poses challenges because of the intricate imaging mechanisms and variations in radiation across different modalities. Specifically, the fusion of visible-light and vegetation-sensitive images encounters similar difficulties. Traditional methods have seldom considered the varied imaging mechanisms and radiation difference between modalities, resulting in discrepancies in the correspond features. To address the issue, we propose the VAM-Net (Vegetation-Attentive Multi-modal deep Network) combining a radiometric correction mechanism and a lightweight multi-modal adaptive feature selection method for fusing multi-modal images. First, the vegetation index (VDVI) is integrated into visible-light images to mitigate the radiometric differences between visible-light images and vegetation-sensitive images (e.g., infrared and red edge images). Then, a two-branch network incorporating attention mechanisms is designed to independently capture the texture features and select similar features cross two different modalities of images. Last, a new loss function is presented to ensure the learned features are suitable for multi-modal fusion. The VAM-Net is evaluated by visible-light and vegetation-sensitive images in three different areas, and the experimental results show that VAM-Net attains an average precision of 67.02%, and recall of 35.49%, and an average RMSE of 2.191px, demonstrating the accuracy and robustness of VAM-Net in multi-modal image fusion.

Adaptive Feature Selection With Augmented Attributes.

In many dynamic environment applications, with the evolution of data collection ways, the data attributes are incremental and the samples are stored with accumulated feature spaces gradually. For instance, in the neuroimaging-based diagnosis of neuropsychiatric disorders, with emerging of diverse testing ways, we get more brain image features over time. The accumulation of different types of features will unavoidably bring difficulties in manipulating the high-dimensional data. It is challenging to design an algorithm to select valuable features in this feature incremental scenario. To address this important but rarely studied problem, we propose a novel Adaptive Feature Selection method (AFS). It enables the reusability of the feature selection model trained on previous features and adapts it to fit the feature selection requirements on all features automatically. Besides, an ideal l0-norm sparse constraint for feature selection is imposed with a proposed effective solving strategy. We present the theoretical analyses about the generalization bound and convergence behavior. After tackling this problem in a one-shot case, we extend it to the multi-shot scenario. Plenty of experimental results demonstrate the effectiveness of reusing previous features and the superior of l0-norm constraint in various aspects, together with its effectiveness in discriminating schizophrenic patients from healthy controls.

Cross‐project defect prediction method based on genetic algorithm feature selection

AbstractWith the continuous development of Internet technology, the role of software in life is increasing, and software defect prediction (SDP) is a key means to ensure software reliability. SDP is to predict the modules that may have defects in advance based on the historical data of software projects, and its purpose is to maximize the use of testing resources. However, in the actual development process, the project that needs to be predicted is often a new project for which there is little or no historical data. Therefore, how to use the massive data of other related projects to build a cross‐project software defect prediction (CPDP) model has received extensive attention from scholars. However, due to the differences in data distribution and class imbalance between different projects, the performance of CPDP is greatly affected. Therefore, on the basis of CPDP, this article proposes a feature selection method based on genetic algorithm (genetic algorithm feature selection, GAFS). GAFS mainly includes two stages: feature selection and ensemble training. In the feature selection stage, this article proposes a global search adaptive feature selection method based on genetic algorithm, which uses the integrated training results of candidate feature subsets on target data to migrate the optimal feature subset. In the ensemble training phase, the EasyEnsemble method is used to alleviate the class imbalance problem, multiple naive Bayesian classifiers are constructed, and then the final model is constructed through ensemble learning. In this article, F1‐score and MCC are used as the test indicator, and comparative experiments are carried out on AEEEM and Promise. The results show that compared with the five comparison methods, GAFS can improve the average F1‐score and MCC much more. For example, GAFS can improve the average F1‐score value by 38.9%, 31.6%, 35.1%, 22.0%, and 31.6%, respectively. In most cases, it can effectively improve the performance of the model and achieve better prediction results.

Multi-Head Attention Network with Adaptive Feature Selection for RUL Predictions of Gradually Degrading Equipment

A multi-head-attention-network-based method is proposed for effective information extraction from multidimensional data to accurately predict the remaining useful life (RUL) of gradually degrading equipment. The multidimensional features of the desired equipment were evaluated using a comprehensive evaluation index, constructed of discrete coefficients, based on correlation, monotonicity, and robustness. For information extraction, the optimal feature subset, determined by the adaptive feature selection method, was input into the multi-head temporal convolution network–bidirectional long short-term memory (TCN-BILSTM) network. Each feature was individually mined to avoid the loss of information. The effectiveness of our proposed RUL prediction method was verified using the NASA IMS bearings dataset and C-MAPSS aeroengines dataset. The results indicate the superiority of our method for the RUL prediction of gradually degrading equipment compared to other mainstream machine learning methods.

Intelligent fault diagnosis for rail transit switch machine based on adaptive feature selection and improved LightGBM

The turnout switch machine is the critical equipment of the signal system, which has a significant influence on the efficiency and safety of train operation. However, most fault diagnosis technologies of the switch machine are difficult to distinguish samples with similar categories, which leads to the low diagnostic accuracy. Thus, a fault diagnosis method based on improved LightGBM is proposed to deal with the above problems. Time domain features and multi-scale permutation entropy are extracted to capture the weak fault. Moreover, an adaptive feature selection (AFS) method is presented to reduce redundant features. Especially an improved Focal Loss (IFL) function is established, which improves the ability to distinguish samples of similar features in a multi-classification model. The three-phase action current from the switch machine is utilized to testify to the proposed method and compare it with other methods. The experimental results show that the diagnosis accuracies of this method in the normal-reverse and reverse-normal conversion process reach 98.47 % and 96.09 %, respectively, which is well-suitable for practical application.

Hyperspectral Unmixing Using Higher-Order Graph Regularized NMF With Adaptive Feature Selection

Recently, graph learning methods have attracted much research attention, which uses first-order nearest-neighbor relation between pixels to construct adjacency graphs for capturing smooth abundance. However, the first-order nearest-neighbor information neglects the shared domain structure of pixels making the higher-order nearest-neighbor relation missing resulting in limited spatial structure learning. Additionally, the feature values in hyperspectral images vary greatly in different bands, tiny value contributions are easily neglected due to the statistical properties of the model, which hinders the learning efficiency of the approach. To address these shortcomings, a higher-order graph regularizer nonnegative matrix factorization with adaptive feature selection method is proposed. Specifically, we introduce pixel second-order nearest-neighbor relation in graph learning to capture the nearest-neighbor domains of pixels with missing connection in first-order neighborhoods and enhance the aggregation of homogeneous regions. Then, the first-order and second-order nearest-neighbor relation are combined to construct higher-order graph regularizer to better preserve the global spatial structure of the image. Additionally, adaptive weight is introduced in the data reconstruction to balance the effects of different feature values by adaptively selecting spectral features to enhance the contribution of tiny features. Finally, the objective function is designed by fusing the higher-order graph regularization, adaptive feature selection and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</sub> sparse regularizer into the nonnegative matrix factorization framework, and the optimization algorithm of the model is derived using a multiplicative update rule. We conducted extensive experiments on synthetic and real datasets, and the effectiveness and superiority of the proposed method were verified by comparing it with several state-of-the-art methods.

AFNFS: Adaptive fuzzy neighborhood-based feature selection with adaptive synthetic over-sampling for imbalanced data

The classification efficiency of majority classes for imbalanced data is so concerned in real-world applications. Almost fuzzy neighborhood radius still needs to be manually set and many entropy measures may ignore the boundary region of data, these limitations will result in the poor classification effect. To address these limitations, this paper designs a novel adaptive fuzzy neighborhood-based feature selection method for imbalanced data with adaptive synthetic over-sampling. First, the closeness is defined according to the variance distance between the samples of the minority class, the pair set of neighboring samples is designed, and then an improved adaptive synthetic over-sampling model is presented for constructing balanced decision systems consisting of the synthetic samples and original samples. Second, an adaptive fuzzy neighborhood radius is developed when using the data margins of all homogeneous and heterogeneous samples. Then the adaptive fuzzy neighborhood granule and upper and lower approximations are defined to construct a new FNRS model. Thus, approximate accuracy and roughness are presented to measure the uncertainty from the fuzzy and rough perspectives for imbalanced data. Third, by combining the roughness with adaptive fuzzy neighborhood entropy, adaptive fuzzy neighborhood joint entropy is constructed to evaluate the uncertainty in fuzzy neighborhood decision systems from two viewpoints of algebra and information. Then the reduced set and the significance of the feature are further developed. Finally, this improved adaptive synthetic over-sampling algorithm is designed to aim to build this balanced decision system, and an adaptive fuzzy neighborhood-based feature selection algorithm with the tolerance parameter is developed to achieve an optimal feature subset. Experiments on 26 imbalanced datasets demonstrate that the constructed algorithms compared to the other related algorithms are effective.

An adaptive and altruistic PSO-based deep feature selection method for Pneumonia detection from Chest X-rays

Pneumonia is one of the major reasons for child mortality especially in income-deprived regions of the world. Although it can be detected and treated with very less sophisticated instruments and medication, Pneumonia detection still remains a major concern in developing countries. Computer-aided based diagnosis (CAD) systems can be used in such countries due to their lower operating costs than professional medical experts. In this paper, we propose a CAD system for Pneumonia detection from Chest X-rays, using the concepts of deep learning and a meta-heuristic algorithm. We first extract deep features from the pre-trained ResNet50, fine-tuned on a target Pneumonia dataset. Then, we propose a feature selection technique based on particle swarm optimization (PSO), which is modified using a memory-based adaptation parameter, and enriched by incorporating an altruistic behavior into the agents. We name our feature selection method as adaptive and altruistic PSO (AAPSO). The proposed method successfully eliminates non-informative features obtained from the ResNet50 model, thereby improving the Pneumonia detection ability of the overall framework. Extensive experimentation and thorough analysis on a publicly available Pneumonia dataset establish the superiority of the proposed method over several other frameworks used for Pneumonia detection. Apart from Pneumonia detection, AAPSO is further evaluated on some standard UCI datasets, gene expression datasets for cancer prediction and a COVID-19 prediction dataset. The overall results are satisfactory, thereby confirming the usefulness of AAPSO in dealing with varied real-life problems. The supporting source codes of this work can be found at https://github.com/rishavpramanik/AAPSO.

Adaptive feature selection method with FF-FC-MIC for the detection of mutual faults in rotating machinery

In the coupling state of rotor unbalance fault and bearing defect fault for rotor system, the signals contain multiple fault components, and the fault diagnosis of the rotor system needs to contain comprehensive multidimensional feature quantities. However, irrelevant feature information in the multi-dimensional feature quantities increases the complexity of classification calculation and affects the efficiency and accuracy of diagnosis. In order to eliminate redundant and irrelevant features in the feature information, and achieve the goal of fewer diagnostic features and good diagnostic results, this paper proposes an adaptive feature selection based on the maximum information coefficient FF-FC-MIC (Feature-to-Feature and Feature-to-Category Maximum Information Coefficient) method. Firstly, the sparse representation algorithm is used to reconstruct the original signal to improve the signal-to-noise ratio, and the multi-dimensional feature quantity of the reconstructed signal is calculated; Secondly, calculate the correlation between features and features through MIC to obtain a feature set of weak correlation between features; thirdly, use MIC to calculate the correlation between features and signal categories to obtain a feature set with strong correlation between features and signals; Finally, the FF-FC-MIC feature selection method is used for feature adaptive selection and input into SVM to complete fault diagnosis. The method is analyzed by simulation signals and the real experiment signals. The results show that the method can effectively remove redundant and disclosed features in the coupling fault, reducing the characteristic dimension to reduce the fault classification time, and improve classification accuracy. Different experimental cases and various feature selection comparison methods further verify the accuracy and applicability of the proposed method.

An adaptive feature selection schema using improved technical indicators for predicting stock price movements

Accurate stock market forecasts can bring high returns for investors. There have been a growing number of studies employing machine learning technology to perform stock prediction tasks with the development of machine learning and artificial intelligence technologies. However, accurately predicting stock price trends still is an elusive goal, not only because the stock market is affected by policies, market environment, market sentiment, etc., but also because stock price data is inherently complex, noisy, and nonlinear. Many technical indicators have been used as input features to stock prediction models, but the quality of technical indicators has always been a neglected issue, thus the application of feature engineering in stock prediction tasks needs to be further expanded. Using 18 technical indicators as the original features, this paper presents improved technical indicators based on wavelet denoising and a novel two-stage adaptive feature selection method. Finally, the random forest model is used as the stock prediction model. Experiments show that in contrast to the original technical indicators, the improved technical indicators significantly enhance the performance of the model (e.g., F1 scores increased by 34.48% on the SSE Composite Index (SSEC) data set, 41.56% on the Hang Seng Index (HSI) data set, 34.48% on the Dow Jones Industrial Average (DJI) data set, 32.75% on the Standard & Poor's 500 Index (S&P 500) data set). The experimental results verify the importance of the quality of technical indicators in the task of stock prediction. Meanwhile, the results also demonstrate the effectiveness of the feature selection method, which can achieve higher prediction accuracy with fewer features. In addition, we established multiple data sets according to the size-varied time windows to study the influence of the size-varied time windows. The results show that properly increasing the size of the time window can exert a positive impact on the model. Finally, by utilizing our two-stage adaptive feature selection method, we remove redundant features, and achieve excellent results on data sets from four different stock markets (e.g., F1 scores reached 0.754 on the SSEC data set, 0.794 on the HSI data set, 0.789 on the DJI data set, 0.821 on the S&P 500 data set). Overall, this study experimentally verifies that improving feature quality can positively impact model performance, and that choosing an appropriate combination of input features can not only improve model performance, but reduces the negative impact of the curse of dimensionality as well.

An Adaptive Rank Aggregation-Based Ensemble Multi-Filter Feature Selection Method in Software Defect Prediction.

Feature selection is known to be an applicable solution to address the problem of high dimensionality in software defect prediction (SDP). However, choosing an appropriate filter feature selection (FFS) method that will generate and guarantee optimal features in SDP is an open research issue, known as the filter rank selection problem. As a solution, the combination of multiple filter methods can alleviate the filter rank selection problem. In this study, a novel adaptive rank aggregation-based ensemble multi-filter feature selection (AREMFFS) method is proposed to resolve high dimensionality and filter rank selection problems in SDP. Specifically, the proposed AREMFFS method is based on assessing and combining the strengths of individual FFS methods by aggregating multiple rank lists in the generation and subsequent selection of top-ranked features to be used in the SDP process. The efficacy of the proposed AREMFFS method is evaluated with decision tree (DT) and naïve Bayes (NB) models on defect datasets from different repositories with diverse defect granularities. Findings from the experimental results indicated the superiority of AREMFFS over other baseline FFS methods that were evaluated, existing rank aggregation based multi-filter FS methods, and variants of AREMFFS as developed in this study. That is, the proposed AREMFFS method not only had a superior effect on prediction performances of SDP models but also outperformed baseline FS methods and existing rank aggregation based multi-filter FS methods. Therefore, this study recommends the combination of multiple FFS methods to utilize the strength of respective FFS methods and take advantage of filter–filter relationships in selecting optimal features for SDP processes.

Automatic Selection of Relevant Attributes for Multi-Sensor Remote Sensing Analysis: A Case Study on Sea Ice Classification

It is of considerable benefit to combine information obtained from different satellite sensors to achieve advanced and improved characterization of sea ice conditions. However, it is also true that not all the information is relevant. It may be redundant, corrupted, or unnecessary for the given task, hence decreasing the performance of the algorithms. Therefore, it is crucial to select an optimal set of image attributes which provides the relevant information content to enhance the efficiency and accuracy of the image interpretation and retrieval of geophysical parameters. Comprehensive studies have been focused on the analysis of relevant features for sea ice analysis obtained from different sensors, especially synthetic aperture radar. However, the outcomes of these studies are mostly data and application-dependent and can, therefore, rarely be generalized. In this article, we employ a feature selection method based on graph Laplacians, which is fully automatic and easy to implement. The proposed approach assesses relevant information on a global and local level using two metrics and selects relevant features for different regions of an image according to their physical characteristics and observation conditions. In the recent study, we investigate the effectiveness of this approach for sea ice classification, using different multi-sensor data combinations. Experiments show the advantage of applying multi-sensor data sets and demonstrate that the attributes selected by our method result in high classification accuracies. We demonstrate that our approach automatically considers varying technical, sensor-specific, environmental, and sea ice conditions by employing flexible and adaptive feature selection method as a pre-processing step.

Rolling bearing remaining useful life prediction via weight tracking relevance vector machine

The application scenarios of rotating machinery are becoming increasingly complicated due to the rapid development of the manufacturing industry. The remaining useful life (RUL) prediction of rolling bearings has gradually been considered in many industry fields for ensuring the safety and reliability of whole systems. As an effective way to analyze data, the relevance vector machine (RVM) approach holds great potential for RUL prediction. However, the redundant features of rolling bearing vibration signals can easily lead to overfitting and low accuracy of the RVM model for RUL prediction. To conquer these issues, inspired by the idea of the boosting algorithm and ensemble learning, this paper proposes a new RVM model, called the weight-tracking relevance vector machine (WTRVM). Within the proposed WTRVM model, an adaptive sequential optimal feature selection method is designed to avoid overfitting by selecting the best features. The error between the prediction value of the RVM model and the true value is counted for the RVM model training and weight tracking. The most accurate model can be obtained when all selected features have been trained. Finally, the proposed WTRVM algorithm is experimentally demonstrated to be effective for the RUL prediction of rolling bearings.

Malware Detection Based on Multi-level and Dynamic Multi-feature Using Ensemble Learning at Hypervisor

As more and more applications migrate to clouds, the type and amount of malware attack against virtualized environments are increasing, which is a key factor that restricts the widespread deployment and application of cloud platforms. Traditional in-VM-based security software is not effective against malware attacks, as the security software itself becomes the target of malware attacks and can easily be tampered with or even subverted. In this paper, we propose a new malware detection method to improve virtual machine security performance and ensure the security of the entire cloud platform. This paper uses the virtual machine introspection(VMI) combined with the memory forensics analysis(MFA) technology to extract multiple types of dynamic features from the virtual machine memory, the hypervisor layer and the hardware layer. Furthermore, this paper proposes an adaptive feature selection method. By combining three different search strategies, three types of features are compared and analyzed from three aspects: effectiveness, system load and security. By adjusting the weight of each feature, it meets the detection requirements of different malware in the cloud environment as expected. Finally, the detection method improves the detection accuracy and generalization ability of the overall classifier using the AdaBoost ensemble learning method with Voting’s combination strategy. The experiment used a large number of real malicious samples, and achieved an accuracy of 0.999 (AUC), with a maximum performance overhead of 5.6%.

An Adaptive Regression Feature Selection Method for Datasets with Outliers

An Adaptive Regression Feature Selection Method for Datasets with Outliers

Mobile health framework based on adaptive feature selection of deep convolutional neural network and QoS optimisation for benign-malignant lung nodule classification

In order to explore the potential of Deep Learning (DL) methods in mobile health, we propose a novel framework combined enhanced DL methods and Quality of Service (QoS) optimisation for lung nodule classification. First, for classification-based DL methods, the methods of feature extraction and feature selection are widely used as the key steps in the classification of lung nodules. This paper proposes an adaptive feature selection method based on Deep Convolution Neural Network (DCNN). Based on the idea of transfer learning, we firstly use DCNN model pre-trained on ImageNet database to extract the features of multi-channel lung nodules images and then we use adaptive feature selection method extract sparse activation features. The experimental results show that the proposed method does improve the performance of benign and malignant lung nodule classification, which can achieve the classification accuracy of 89.30% and the AUC of 0.94.

Performance analysis of different classification algorithms using different feature selection methods on Parkinson's disease detection

BackgroundIn diagnosis of neurodegenerative diseases, the three-dimensional magnetic resonance imaging (3D-MRI) has been heavily researched. Parkinson's disease (PD) is one of the most common neurodegenerative disorders. New methodThe performances of five different classification approaches using five different attribute rankings each followed with an adaptive Fisher stopping criteria feature selection (FS) method are evaluated. To improve the performance of PD detection, a source fusion technique which combines the gray matter (GM) and white (WM) tissue maps and a decision fusion technique which combines the outputs of all classifiers using the correlation-based feature selection (CFS) method by majority voting are used. ResultsAmong the five FS methods, the CFS provides the highest results for all five classification algorithms and the SVM provides the best classification performances for all five different FS methods. The classification accuracy of 77.50% and 81.25% are obtained for the GM and WM tissues, respectively. However, the fusion of GM and WM datasets improves the classification accuracy of the proposed methodology up to 95.00%. Comparison with Existing MethodsAn f-contrast is used to generate 3D masks for GM and WM datasets and a fusion technique, combining the GM and WM datasets is used. Several classification algorithms using several FS methods are performed and a decision fusion technique is used. ConclusionsUsing the combination of the 3D masked GM and WM tissue maps and the fusion of the outputs of multiple classifiers with CFS method gives the classification accuracy of 95.00%.

Adaptive feature selection based on reconstruction residual and accurately located landmarks for expression-robust 3D face recognition

A novel adaptive feature selection based on reconstruction residual and accurately located landmarks for expression-robust 3D face recognition is proposed in this paper. Firstly, the novel facial coarse-to-fine landmarks localization method based on Active Shape Model and Gabor wavelets transformation is proposed to exactly and automatically locate facial landmarks in range image. Secondly, the multi-scale fusion of the pyramid local binary patterns (F-PLBP) based on the irregular segmentation associated with the located landmarks is proposed to extract the discriminative feature. Thirdly, a sparse representation-based classifier based on the adaptive feature selection (A-SRC) using the distribution of the reconstruction residual is presented to select the expression-robust feature and identify the faces. Finally, the experimental evaluation based on FRGC v2.0 indicates that the adaptive feature selection method using F-PLBP combined with the A-SRC can obtain the high recognition accuracy by performing the higher discriminative power to overcome the influence from the facial expression variations.

Adaptive feature selection using v-shaped binary particle swarm optimization.

Feature selection is an important preprocessing method in machine learning and data mining. This process can be used not only to reduce the amount of data to be analyzed but also to build models with stronger interpretability based on fewer features. Traditional feature selection methods evaluate the dependency and redundancy of features separately, which leads to a lack of measurement of their combined effect. Moreover, a greedy search considers only the optimization of the current round and thus cannot be a global search. To evaluate the combined effect of different subsets in the entire feature space, an adaptive feature selection method based on V-shaped binary particle swarm optimization is proposed. In this method, the fitness function is constructed using the correlation information entropy. Feature subsets are regarded as individuals in a population, and the feature space is searched using V-shaped binary particle swarm optimization. The above procedure overcomes the hard constraint on the number of features, enables the combined evaluation of each subset as a whole, and improves the search ability of conventional binary particle swarm optimization. The proposed algorithm is an adaptive method with respect to the number of feature subsets. The experimental results show the advantages of optimizing the feature subsets using the V-shaped transfer function and confirm the effectiveness and efficiency of the feature subsets obtained under different classifiers.

Adaptive multi-view feature selection for human motion retrieval

Human motion retrieval plays an important role in many motion data based applications. In the past, many researchers tended to use a single type of visual feature as data representation. Because different visual feature describes different aspects about motion data, and they have dissimilar discriminative power with respect to one particular class of human motion, it led to poor retrieval performance. Thus, it would be beneficial to combine multiple visual features together for motion data representation. In this article, we present an Adaptive Multi-view Feature Selection (AMFS) method for human motion retrieval. Specifically, we first use a local linear regression model to automatically learn multiple view-based Laplacian graphs for preserving the local geometric structure of motion data. Then, these graphs are combined together with a non-negative view-weight vector to exploit the complementary information between different features. Finally, in order to discard the redundant and irrelevant feature components from the original high-dimensional feature representation, we formulate the objective function of AMFS as a general trace ratio optimization problem, and design an effective algorithm to solve the corresponding optimization problem. Extensive experiments on two public human motion database, i.e., HDM05 and MSR Action3D, demonstrate the effectiveness of the proposed AMFS over the state-of-art methods for motion data retrieval. The scalability with large motion dataset, and insensitivity with the algorithm parameters, make our method can be widely used in real-world applications.