Feature Selection Task Research Articles

Addressing feature selection and extreme learning machine tuning by diversity-oriented social network search: an application for phishing websites detection

AbstractFeature selection and hyper-parameters optimization (tuning) are two of the most important and challenging tasks in machine learning. To achieve satisfying performance, every machine learning model has to be adjusted for a specific problem, as the efficient universal approach does not exist. In addition, most of the data sets contain irrelevant and redundant features that can even have a negative influence on the model’s performance. Machine learning can be applied almost everywhere; however, due to the high risks involved with the growing number of malicious, phishing websites on the world wide web, feature selection and tuning are in this research addressed for this particular problem. Notwithstanding that many metaheuristics have been devised for both feature selection and machine learning tuning challenges, there is still much space for improvements. Therefore, the research exhibited in this manuscript tries to improve phishing website detection by tuning extreme learning model that utilizes the most relevant subset of phishing websites data sets features. To accomplish this goal, a novel diversity-oriented social network search algorithm has been developed and incorporated into a two-level cooperative framework. The proposed algorithm has been compared to six other cutting-edge metaheuristics algorithms, that were also implemented in the framework and tested under the same experimental conditions. All metaheuristics have been employed in level 1 of the devised framework to perform the feature selection task. The best-obtained subset of features has then been used as the input to the framework level 2, where all algorithms perform tuning of extreme learning machine. Tuning is referring to the number of neurons in the hidden layers and weights and biases initialization. For evaluation purposes, three phishing websites data sets of different sizes and the number of classes, retrieved from UCI and Kaggle repositories, were employed and all methods are compared in terms of classification error, separately for layers 1 and 2 over several independent runs, and detailed metrics of the final outcomes (output of layer 2), including precision, recall, f1 score, receiver operating characteristics and precision–recall area under the curves. Furthermore, an additional experiment is also conducted, where only layer 2 of the proposed framework is used, to establish metaheuristics performance for extreme machine learning tuning with all features, which represents a large-scale NP-hard global optimization challenge. Finally, according to the results of statistical tests, final research findings suggest that the proposed diversity-oriented social network search metaheuristics on average obtains better achievements than competitors for both challenges and all data sets. Finally, the SHapley Additive exPlanations analysis of the best-performing model was applied to determine the most influential features.

GBDTMO: as new option for early-stage breast cancer detection and classification using machine learning

Breast cancer is the second leading cause of disease death in women, after lung and bronchus cancer. According to measurements, mammography misses breast cancer in 10% to 15% of cases for women aged 50 to 69 years. In the current study, we used the Wisconsin breast cancer dataset to develop a two-stage model for breast cancer diagnosis. The main goal of this study effort is to effectively carry out feature selection and classification tasks. Gradient Boosting Decision Tree-based Mayfly Optimisation (GBDTMO), an innovative and efficient breast cancer diagnostic machine learning system, is provided. In the second stage, we employ a Mayfly search to determine which subset of traits is the best. Two more well-known datasets on breast cancer, the ICCR and the Cancer Corpus, were also compared for classification accuracy. The accuracy of the suggested GBDTMO model was higher than that of the existing GBDT and Practical Federated Gradient Boosting Decision Tree (PFGBDT), which had accuracy values of 93.25% and 94.25%, respectively. Similarly, the recall, F-measure, and ROC area values were 98.52%, 97.52%, and 96.32%, respectively. Furthermore, it demonstrated a lower RMSE of 0.98 than the existing GBDT and PFGBDT.

Practical Markov Boundary Learning without Strong Assumptions

Theoretically, the Markov boundary (MB) is the optimal solution for feature selection. However, existing MB learning algorithms often fail to identify some critical features in real-world feature selection tasks, mainly because the strict assumptions of existing algorithms, on either data distribution, variable types, or correctness of criteria, cannot be satisfied in application scenarios. This paper takes further steps toward opening the door to real-world applications for MB. We contribute in particular to a practical MB learning strategy, which can maintain feasibility and effectiveness in real-world data where variables can be numerical or categorical with linear or nonlinear, pairwise or multivariate relationships. Specifically, the equivalence between MB and the minimal conditional covariance operator (CCO) is investigated, which inspires us to design the objective function based on the predictability evaluation of the mapping variables in a reproducing kernel Hilbert space. Based on this, a kernel MB learning algorithm is proposed, where nonlinear multivariate dependence could be considered without extra requirements on data distribution and variable types. Extensive experiments demonstrate the efficacy of these contributions.

Boosted local dimensional mutation and all-dimensional neighborhood slime mould algorithm for feature selection

The slime mould algorithm (SMA) is a population-based optimization algorithm that mimics the foraging behavior of slime moulds with a simple structure and few hyperparameters. However, SMA has some limitations, such as getting trapped in local optima when dealing with multimodal or combinatorial functions. To overcome these limitations and improve the algorithm's exploration and exploitation abilities, a local dimensional mutation strategy and an all-dimensional neighborhood search strategy for SMA, known as LASMA, were introduced. To evaluate the performance of LASMA, experiments were conducted on 30 benchmark functions from the CEC2014 competition, and the results were compared with up to 27 peers. The experimental results were then synthesized, and the Wilcoxon signed-rank test was used to evaluate the performance of LASMA. The results showed that LASMA outperformed other algorithms in terms of solution accuracy, stability, and convergence speed, with at least a 53.3% improvement in optimization performance on the 30 tested functions. Moreover, to demonstrate the applicability of LASMA to feature selection problems, a binary version of LASMA called bLASMA was developed and compared with eight binary classification algorithms on 18 datasets from the UCI repository. The experimental results showed that bLASMA not only had faster convergence speed and higher convergence accuracy in handling optimization problems but also performed well in feature selection applications. Thus, LASMA is a promising optimization tool for handling global and binary optimization problems, and its binary version, bLASMA, can be used for feature selection tasks. By addressing the limitations of SMA and improving the algorithm's exploration and exploitation abilities, LASMA provides a robust and effective solution for various optimization problems.

Adaptive cooperative coevolutionary differential evolution for parallel feature selection in high-dimensional datasets

In many fields, it is a common practice to collect large amounts of data characterized by a high number of features. These datasets are at the core of modern applications of supervised machine learning, where the goal is to create an automatic classifier for newly presented data. However, it is well known that the presence of irrelevant features in a dataset can make the learning phase harder and, most importantly, can lead to suboptimal classifiers. Consequently, it is becoming increasingly important to be able to select the right subset of features. Traditionally, optimization metaheuristics have been used with success in the task of feature selection. However, many of the approaches presented in the literature are not applicable to datasets with thousands of features because of the poor scalability of optimization algorithms. In this article, we address the problem using a cooperative coevolutionary approach based on differential evolution. In the proposed algorithm, parallelized for execution on shared-memory architectures, a suitable strategy for reducing the dimensionality of the search space and adjusting the population size during the optimization results in significant performance improvements. A numerical investigation on some high-dimensional and medium-dimensional datasets shows that, in most cases, the proposed approach can achieve higher classification performance than other state-of-the-art methods.

Multi-Target Markov Boundary Discovery: Theory, Algorithm, and Application.

Markov boundary (MB) has been widely studied in single-target scenarios. Relatively few works focus on the MB discovery for variable set due to the complex variable relationships, where an MB variable might contain predictive information about several targets. This paper investigates the multi-target MB discovery, aiming to distinguish the common MB variables (shared by multiple targets) and the target-specific MB variables (associated with single targets). Considering the multiplicity of MB, the relation between common MB variables and equivalent information is studied. We find that common MB variables are determined by equivalent information through different mechanisms, which is relevant to the existence of the target correlation. Based on the analysis of these mechanisms, we propose a multi-target MB discovery algorithm to identify these two types of variables, whose variant also achieves superiority and interpretability in feature selection tasks. Extensive experiments demonstrate the efficacy of these contributions.

Differential Evolution-Based Feature Selection: A Niching-Based Multiobjective Approach

Feature selection is to reduce both the dimensionality of data and the classification error rate (i.e., increase the classification accuracy) of a learning algorithm. The two objectives are often conflicting, thus a multiobjective feature selection method can obtain a set of nondominated feature subsets. Each solution in the set has a different size and a corresponding classification error rate. However, most existing feature selection algorithms have ignored that, for a given size, there can be different feature subsets with very similar or the same accuracy. This article introduces a niching-based multiobjective feature selection method that simultaneously minimizes the number of selected features and the classification error rate. The proposed method conceives to identify: 1) a set of feature subsets with good convergence and distribution and 2) multiple feature subsets choosing the same number of features with almost the same lowest classification error rate. The contributions of this article are threefold. First, a niching and global interaction mutation operator is proposed that can produce promising feature subsets. Second, a newly developed environmental selection mechanism allows equal informative feature subsets to be stored by relaxing the Pareto-dominance relationship. Finally, the proposed subset repairing mechanism can generate better feature subsets and further remove the redundant features. The proposed method is compared against seven multiobjective feature selection algorithms on 19 datasets, including both binary and multiclass classification tasks. The results show that the proposed method can evolve a rich and diverse set of nondominated solutions for different feature selection tasks, and their availability helps in understanding the relationships between features.

Incremental feature selection with fuzzy rough sets for dynamic data sets

Incremental mechanisms for feature selection in fuzzy rough sets are widely studied because they can effectively and efficiently complete feature selection tasks for dynamic data sets. However, the existing algorithms mainly focus on incremental mechanisms for dynamic data sets by increasing single dimensions of samples or features. In this study, we propose an incremental mechanism for feature selection with fuzzy rough sets where samples and features are added to dynamic data simultaneously. A relative discernibility relation is proposed to represent the samples and features of fuzzy rough sets in a unified manner. This relation provides a theoretical basis for the incremental analysis of dynamic data while simultaneously increasing the samples and features. By analyzing the changes in the relative discernibility relationship caused by the variations in samples and features, the feature and sample incremental mechanisms can be integrated well to form a unified method for simultaneously increasing the samples and features. Finally, an incremental algorithm is proposed for dynamic data sets based on the unified incremental process. The results obtained in numerical experiments showed that the algorithm can effectively deal with incremental feature selection by simultaneously adding features and samples.

Feature clustering-Assisted feature selection with differential evolution

Modern data collection technologies may produce thousands of or even more features in a single dataset. The high dimensionality of data poses a barrier to determining discriminating features due to the curse of dimensionality. Thanks to the global search ability, many population-based feature selection approaches have been proposed. However, very few studies pay attention on that a feature selection task has multiple optimal feature subsets. To search for multiple optimal feature subsets, we propose a feature clustering-assisted feature selection method. The proposed method employs the knowledge of correlation measures to group features. And, this correlation knowledge is embedded into the encoding method and the search process. A niching-based mutation operator is also used to explore the vicinity of a target individual. The aim is to find different feature subsets with very similar or the same classification performance. In addition, a modification operator is proposed aiming to increase the population diversity to improve the feature selection performance. The experiments on 16 datasets show that the proposed algorithm outperforms other popular feature selection methods in terms of classification accuracy and feature subset size.

A Tensor Method based on Enhanced Tensor Nuclear Norm and Hypergraph Laplacian Regularization for Pan-Cancer Omics Data Analysis.

As a powerful data representation technique, tensor robust principal component analysis (TRPCA) has been widely used for clustering and feature selection tasks. However, it ignores the significant difference in singular values of tensor data and the manifold information contained in different views, thereby causing serious degradation of conventional TRPCA performance. In this paper, a novel tensor method based on enhanced tensor nuclear norm and hypergraph Laplacian regularization (ETHLR) is developed to address the above problem. ETHLR can jointly learn the prior knowledge of singular values and high-order manifold structures in the unified tensor space and the view-specific feature spaces, respectively. Specifically, the enhanced tensor nuclear norm, namely, the weighted tensor Schatten p-norm, is used to shrink the singular values by fully considering the salient difference information of singular values and the complementary information embedded in the tensor space; the hypergraph Laplacian constraint helps encode high-order geometric structures among multiple samples in the nonlinear view-specific feature space. Furthermore, we employ inexact augmented Lagrange multipliers (ALM) to optimize the ETHLR method. Numerous experiments on pan-cancer omics data show that the superiority of ETHLR over several state-of-the-art competitors.

UDRN: Unified Dimensional Reduction Neural Network for feature selection and feature projection

Dimensional reduction (DR) maps high-dimensional data into a lower dimensions latent space with minimized defined optimization objectives. The two independent branches of DR are feature selection (FS) and feature projection (FP). FS focuses on selecting a critical subset of dimensions but risks destroying the data distribution (structure). On the other hand, FP combines all the input features into lower dimensions space, aiming to maintain the data structure, but lacks interpretability and sparsity. Moreover, FS and FP are traditionally incompatible categories and have not been unified into an amicable framework. Therefore, we consider that the ideal DR approach combines both FS and FP into a unified end-to-end manifold learning framework, simultaneously performing fundamental feature discovery while maintaining the intrinsic relationships between data samples in the latent space. This paper proposes a unified framework named Unified Dimensional Reduction Network (UDRN) to integrate FS and FP in an end-to-end way. Furthermore, a novel network framework is designed to implement FS and FP tasks separately using a stacked feature selection network and feature projection network. In addition, a stronger manifold assumption and a novel loss function are proposed. Furthermore, the loss function can leverage the priors of data augmentation to enhance the generalization ability of the proposed UDRN. Finally, comprehensive experimental results on four image and four biological datasets, including very high-dimensional data, demonstrate the advantages of DRN over existing methods (FS, FP, and FS&FP pipeline), especially in downstream tasks such as classification and visualization.

A novel firefly algorithm approach for efficient feature selection with COVID-19 dataset

Feature selection is one of the most important challenges in machine learning and data science. This process is usually performed in the data preprocessing phase, where the data is transformed to a proper format for further operations by machine learning algorithm. Many real-world datasets are highly dimensional with many irrelevant, even redundant features. These kinds of features do not improve classification accuracy and can even shrink down performance of a classifier. The goal of feature selection is to find optimal (or sub-optimal) subset of features that contain relevant information about the dataset from which machine learning algorithms can derive useful conclusions. In this manuscript, a novel version of firefly algorithm (FA) is proposed and adapted for feature selection challenge. Proposed method significantly improves performance of the basic FA, and also outperforms other state-of-the-art metaheuristics for both, benchmark bound-constrained and practical feature selection tasks. Method was first validated on standard unconstrained benchmarks and later it was applied for feature selection by using 21 standard University of California, Irvine (UCL) datasets. Moreover, presented approach was also tested for relatively novel COVID-19 dataset for predicting patients health, and one microcontroller microarray dataset. Results obtained in all practical simulations attest robustness and efficiency of proposed algorithm in terms of convergence, solutions’ quality and classification accuracy. More precisely, the proposed approach obtained the best classification accuracy on 13 out of 21 total datasets, significantly outperforming other competitor methods.

Disambiguation-based partial label feature selection via feature dependency and label consistency

Partial label learning refers to the issue that each training sample corresponds to a candidate label set containing only one valid label. Feature selection can be viewed as an effective pre-processing technology to improve the generalization performance of learning models, while the partial label feature selection task is challenging due to ambiguous labeling information. To this end, this paper utilizes granular ball computing and neighborhood rough sets to put forward a disambiguation-based partial label feature selection algorithm via feature dependency and label consistency. Firstly, the proposed algorithm performs an adaptive neighbor aggregation based on granular balls to disambiguate candidate labels. Adaptive neighbors are flexible to make more efficient learning performance. Then, considering the labeling confidence induced by disambiguation, the significance of each feature is evaluated by fusing the feature dependency from the neighborhood granularity and the label consistency among the nearest neighbors. Neighborhood rough sets directly handle continuous features that can reduce the adverse impact of data discretization on feature selection. The label consistency among adjacent samples is important for measuring feature discrimination. The fusion of feature dependency and label consistency can facilitate performance improvement. Finally, experiments conducted on eight controlled UCI and five real-world partial label datasets demonstrate that the proposed algorithm can improve the generalization performance of partial label learning and achieve superior performance compared to the partial label feature selection methods.

A modified binary version of aphid–ant mutualism for feature selection: a COVID-19 case study

AbstractThe speedy development of intelligent technologies and gadgets has led to a drastic increment of dimensions within the datasets in recent years. Dimension reduction algorithms, such as feature selection methods, are crucial to resolving this obstacle. Currently, metaheuristic algorithms have been extensively used in feature selection tasks due to their acceptable computational cost and performance. In this article, a binary-modified version of aphid–ant mutualism (AAM) called binary aphid–ant mutualism (BAAM) is introduced to solve the feature selection problems. Like AAM, in BAAM, the intensification and diversification mechanisms are modeled via the intercommunication of aphids with other colonies’ members, including aphids and ants. However, unlike AAM, the number of colonies’ members can change in each iteration based on the attraction power of their leaders. Moreover, the second- and third-best individuals can take the place of the ringleader and lead the pioneer colony. Also, to maintain the population diversity, prevent premature convergence, and facilitate information sharing between individuals of colonies including aphids and ants, a random cross-over operator is utilized in BAAM. The proposed BAAM is compared with five other feature selection algorithms using several evaluation metrics. Twelve medical and nine non-medical benchmark datasets with different numbers of features, instances, and classes from the University of California, Irvine and Arizona State University repositories are considered for all the experiments. Moreover, a coronavirus disease (COVID-19) dataset is used to validate the effectiveness of the BAAM in real-world applications. Based on the acquired outcomes, the proposed BAAM outperformed other comparative methods in terms of classification accuracy using various classifiers, including K nearest neighbor, kernel-based extreme learning machine, and multi-class support vector machine, choosing the most informative features, the best and mean fitness values and convergence speed in most cases. As an instance, in the COVID-19 dataset, BAAM achieved 96.53% average accuracy and selected the most informative feature subset.

DroidRL: Feature selection for android malware detection with reinforcement learning

Due to the completely open-source nature of Android, the exploitable vulnerability of malware attacks is increasing. Machine learning, leading to a great evolution in Android malware detection in recent years, is typically applied in the classification phase. Since the correlation between features is ignored in some traditional ranking-based feature selection algorithms, applying wrapper-based feature selection models is a topic worth investigating. Though considering the correlation between features, wrapper-based approaches are time-consuming for exploring all possible valid feature subsets when processing a large number of Android features. To reduce the computational expense of wrapper-based feature selection, a framework named DroidRL is proposed. The framework deploys DDQN algorithm to obtain a subset of features which can be used for effective malware classification. To select a valid subset of features over a larger range, the exploration-exploitation policy is applied in the model training phase. The recurrent neural network (RNN) is used as the decision network of DDQN to give the framework the ability to sequentially select features. Word embedding is applied for feature representation to enhance the framework’s ability to find the semantic relevance of features. The framework’s feature selection exhibits high performance without any human intervention and can be ported to other feature selection tasks with minor changes. The experiment results show a significant effect when using the Random Forest as DroidRL’s classifier, which reaches 95.6% accuracy with only 24 features selected.

A Tent Lévy Flying Sparrow Search Algorithm for Wrapper-Based Feature Selection: A COVID-19 Case Study

The “Curse of Dimensionality” induced by the rapid development of information science might have a negative impact when dealing with big datasets, and it also makes the problems of symmetry and asymmetry increasingly prominent. Feature selection (FS) can eliminate irrelevant information in big data and improve accuracy. As a recently proposed algorithm, the Sparrow Search Algorithm (SSA) shows its advantages in the FS tasks because of its superior performance. However, SSA is more subject to the population’s poor diversity and falls into a local optimum. Regarding this issue, we propose a variant of the SSA called the Tent Lévy Flying Sparrow Search Algorithm (TFSSA) to select the best subset of features in the wrapper-based method for classification purposes. After the performance results are evaluated on the CEC2020 test suite, TFSSA is used to select the best feature combination to maximize classification accuracy and simultaneously minimize the number of selected features. To evaluate the proposed TFSSA, we have conducted experiments on twenty-one datasets from the UCI repository to compare with nine algorithms in the literature. Nine metrics are used to evaluate and compare these algorithms’ performance properly. Furthermore, the method is also used on the coronavirus disease (COVID-19) dataset, and its classification accuracy and the average number of feature selections are 93.47% and 2.1, respectively, reaching the best. The experimental results and comparison in all datasets demonstrate the effectiveness of our new algorithm, TFSSA, compared with other wrapper-based algorithms.

Unsupervised Adaptive Feature Selection with Binary Hashing.

Unsupervised feature selection chooses a subset of discriminative features to reduce feature dimension under the unsupervised learning paradigm. Although lots of efforts have been made so far, existing solutions perform feature selection either without any label guidance or with only single pseudo label guidance. They may cause significant information loss and lead to semantic shortage of the selected features as many real-world data, such as images and videos are generally annotated with multiple labels. In this paper, we propose a new Unsupervised Adaptive Feature Selection with Binary Hashing (UAFS-BH) model, which learns binary hash codes as weakly-supervised multi-labels and simultaneously exploits the learned labels to guide feature selection. Specifically, in order to exploit the discriminative information under the unsupervised scenarios, the weakly-supervised multi-labels are learned automatically by specially imposing binary hash constraints on the spectral embedding process to guide the ultimate feature selection. The number of weakly-supervised multi-labels (the number of "1" in binary hash codes) is adaptively determined according to the specific data content. Further, to enhance the discriminative capability of binary labels, we model the intrinsic data structure by adaptively constructing the dynamic similarity graph. Finally, we extend UAFS-BH to multi-view setting as Multi-view Feature Selection with Binary Hashing (MVFS-BH) to handle the multi-view feature selection problem. An effective binary optimization method based on the Augmented Lagrangian Multiple (ALM) is derived to iteratively solve the formulated problem. Extensive experiments on widely tested benchmarks demonstrate the state-of-the-art performance of the proposed method on both single-view and multi-view feature selection tasks. For the purpose of reproducibility, we provide the source codes and testing datasets at https://github.com/shidan0122/UMFS.git..

A filter feature selection for high-dimensional data

In a classification problem, before building a prediction model, it is very important to identify informative features rather than using tens or thousands which may penalize some learning methods and increase the risk of over-fitting. To overcome these problems, the best solution is to use feature selection. In this article, we propose a new filter method for feature selection, by combining the Relief filter algorithm and the multi-criteria decision-making method called TOPSIS (Technique for Order Preference by Similarity to Ideal Solution), we modeled the feature selection task as a multi-criteria decision problem. Exploiting the Relief methodology, a decision matrix is computed and delivered to Technique for Order Preference by Similarity to Ideal Solution in order to rank the features. The proposed method ends up giving a ranking to the features from the best to the mediocre. To evaluate the performances of the suggested approach, a simulation study including a set of experiments and case studies was conducted on three synthetic dataset scenarios. Finally, the obtained results approve the effectiveness of our proposed filter to detect the best informative features.

Optimized cell type signatures revealed from single-cell data by combining principal feature analysis, mutual information, and machine learning

Machine learning techniques are excellent to analyze expression data from single cells. These techniques impact all fields ranging from cell annotation and clustering to signature identification. The presented framework evaluates gene selection sets how far they optimally separate defined phenotypes or cell groups. This innovation overcomes the present limitation to objectively and correctly identify a small gene set of high information content regarding separating phenotypes for which corresponding code scripts are provided. The small but meaningful subset of the original genes (or feature space) facilitates human interpretability of the differences of the phenotypes including those found by machine learning results and may even turn correlations between genes and phenotypes into a causal explanation. For the feature selection task, the principal feature analysis is utilized which reduces redundant information while selecting genes that carry the information for separating the phenotypes. In this context, the presented framework shows explainability of unsupervised learning as it reveals cell-type specific signatures. Apart from a Seurat preprocessing tool and the PFA script, the pipeline uses mutual information to balance accuracy and size of the gene set if desired. A validation part to evaluate the gene selection for their information content regarding the separation of the phenotypes is provided as well, binary and multiclass classification of 3 or 4 groups are studied. Results from different single-cell data are presented. In each, only about ten out of more than 30000 genes are identified as carrying the relevant information. The code is provided in a GitHub repository at https://github.com/AC-PHD/Seurat_PFA_pipeline.

Crow search algorithm with time varying flight length Strategies for feature selection

Feature Selection (FS) is an efficient technique use to get rid of irrelevant, redundant and noisy attributes in high dimensional datasets while increasing the efficacy of machine learning classification. The CSA is a modest and efficient metaheuristic algorithm which has been used to overcome several FS issues. The flight length (fl) parameter in CSA governs crows' search ability. In CSA, fl is set to a fixed value. As a result, the CSA is plagued by the problem of being hoodwinked in local minimum. This article suggests a remedy to this issue by bringing five new concepts of time dependent fl in CSA for feature selection methods including linearly decreasing flight length, sigmoid decreasing flight length, chaotic decreasing flight length, simulated annealing decreasing flight length, and logarithm decreasing flight length. The proposed approaches' performance is assessed using 13 standard UCI datasets. The simulation result portrays that the suggested feature selection approaches overtake the original CSA, with the chaotic-CSA approach beating the original CSA and the other four proposed approaches for the FS task.