High-dimensional Microarray Datasets Research Articles

Efficient and Intelligent Feature Selection via Maximum Conditional Mutual Information for Microarray Data

The challenge of analyzing microarray datasets is significantly compounded by the curse of dimensionality and the complexity of feature interactions. Addressing this, we propose a novel feature selection algorithm based on maximum conditional mutual information (MCMI) to identify a minimal feature subset that is maximally relevant and non-redundant. This algorithm leverages a greedy search strategy, prioritizing both feature quality and classification performance. Experimental results on high-dimensional microarray datasets demonstrate our algorithm’s superior ability to reduce dimensionality, eliminate redundancy, and enhance classification accuracy. Compared to existing filter feature selection methods, our approach exhibits higher adaptability and intelligence.

Optimal Feature Selection from High-dimensional Microarray Dataset Employing Hybrid IG-Jaya Model

Background: Feature selection (FS) is a crucial strategy for dimensionality reduction in data preprocessing since microarray data sets typically contain redundant and extraneous features that degrade the performance and complexity of classification models. Objective: The purpose of feature selection is to reduce the number of features from highdimensional cancer datasets and enhance classification accuracy. Methods: This research provides a wrapper-based hybrid model integrating information gain (IG) and Jaya algorithm (JA) for determining the optimum featured genes from high-dimensional microarray datasets. This paper's comprehensive study is divided into two segments: we employed the parameterless JA to identify the featured gene subsets in the first stage without filter methods. Various classifiers evaluate JA's performance, such as SVM, LDA, NB, and DT. In the second section, we introduce a hybrid IG-JA model. The IG is used as a filter to eliminate redundant and noisy features. The reduced feature subset is then given to the JA as a wrapper to improve the hybrid model's performance using the classifiers outlined above. Results: We used 13 benchmark microarray data sets from the public repository for experimental analysis. It is noteworthy to state that the hybrid IG-JA model performs better as compared to its counterparts. Conclusion: Tests and statistics show that the suggested model outperforms the standard feature selection method with JA and other existing models. Our proposed model is unable to provide the best accuracy compared to other existing approaches; however, it is quite steady and good. In the future, this work could be implemented with various filter methods and real-time data sets. A multi-filter approach with the Jaya algorithm will be used to check the efficiency of the proposed one. And it would be better to choose any other hybrid model (chaos-based) with Jaya to enhance the feature selection accuracy with a high dimensional dataset.

Stable Feature Selection using Improved Whale Optimization Algorithm for Microarray Datasets

A microarray is a collection of DNA sequences that reflect an organism’s whole gene set and are organized in a grid pattern for use in genetic testing. Microarray datasets are extremely high-dimensional and have a very small sample size, posing the challenges of insufficient data and high computational complexity. Identification of true biomarkers that are the most significant features (a very small subset of the complete feature set) is desired to solve these issues. This reduces over-fitting, and time complexity, and improves model generalization. Various feature selection algorithms are used for this biomarker identification. This research proposed a modification to the whale optimization algorithm (WOAm) for biomarker discovery, in which the fitness of each search agent is evaluated using the hinge loss function during the hunting for prey phase to determine the optimal search agent. Also compared the results of the proposed modified algorithm with the original whale optimization algorithm and also with contemporary algorithms like the marine predator algorithm and grey wolf optimization. All these algorithms are evaluated on six different high-dimensional microarray datasets. It has been observed that the proposed modification for the whale optimization algorithm has significantly improved the results of feature selection across all the datasets. Domain experts trust the resultant biomarker/ associated genes by the stability of the results obtained. The chosen feature set’s stability was also evaluated during the research work. According to the findings, our proposed WOAm has superior stability compared to other algorithms for the CNS, colon, Leukemia, and OSCC. datasets.

NNFSRR: Nearest Neighbor Feature Selection and Redundancy Removal Method for Nearest Neighbor Search in Microarray Gene Expression Data

INTRODUCTION: Gene expression data analysis is a critical aspect of disease prediction and classification, playing a pivotal role in the field of bioinformatics and biomedical research. High-dimensional gene expression datasets hold a wealth of information, but their effective utilization is hindered by the presence of irrelevant dimensions and noise. The challenge lies in extracting meaningful features from these datasets to enhance the accuracy of disease prediction and classification while maintaining computational efficiency.
 Feature selection is a crucial step in addressing these challenges, as it aims to identify and retain only the most informative characteristics from large high-dimensional microarray datasets. In the context of microarray gene expression data, characterized by its substantial dimensionality, selecting relevant features is essential for efficient nearest neighbor search, a fundamental component of various analytical tasks in bioinformatics and data mining.
 Existing feature selection methods in high-dimensional data often face issues related to the trade-off between search accuracy and computational efficiency. This paper introduces a novel approach, the Nearest Neighbor Feature Selection with Symmetrical Uncertainty-based Redundancy Removal (NNFSRR) method, designed to enhance the classification of microarray gene expression data through feature selection. The NNFSRR method focuses on reducing the dimensionality of the dataset by identifying and removing redundant features, allowing subsequent searches to operate solely on relevant dimensions.
 OBJECTIVES: The primary goal is to evaluate the NNFSRR method's effectiveness in improving nearest neighbor search in microarray gene expression datasets by reducing dimensionality. This method utilizes Symmetrical Uncertainty-based correlation between dimensions for feature selection and aims to enhance accuracy and efficiency compared to existing methods.
 METHODS: The NNFSRR method uses Symmetrical Uncertainty to identify and remove redundant features from microarray gene expression datasets. Reduced datasets are used for nearest neighbor search, improving accuracy and efficiency. Experiments are conducted using real-world datasets, and comparisons with existing methods are made based on search time and accuracy.
 RESULTS: The NNFSRR method demonstrates improved nearest neighbor search performance, outperforming basic brute force methods and existing feature selection techniques. Selected feature sets exhibit strong class associations while minimizing feature correlations, enhancing classification precision.
 CONCLUSION: In conclusion, the NNFSRR method presents a promising approach to address the challenges posed by high-dimensional gene expression data. It effectively reduces dimensionality, improves search accuracy, and enhances the efficiency of nearest neighbor search. Our experimental results demonstrate that this method outperforms existing techniques in terms of search time and accuracy, making it a valuable tool for applications in bioinformatics, data mining, pattern recognition, and biological information retrieval. The NNFSRR method holds the potential to advance our understanding of complex biological processes and support more accurate disease prediction and classification.

A weighted-sum chaotic sparrow search algorithm for interdisciplinary feature selection and data classification

In today’s data-driven digital culture, there is a critical demand for optimized solutions that essentially reduce operating expenses while attempting to increase productivity. The amount of memory and processing time that can be used to process enormous volumes of data are subject to a number of limitations. This would undoubtedly be more of a problem if a dataset contained redundant and uninteresting information. For instance, many datasets contain a number of non-informative features that primarily deceive a given classification algorithm. In order to tackle this, researchers have been developing a variety of feature selection (FS) techniques that aim to eliminate unnecessary information from the raw datasets before putting them in front of a machine learning (ML) algorithm. Meta-heuristic optimization algorithms are often a solid choice to solve NP-hard problems like FS. In this study, we present a wrapper FS technique based on the sparrow search algorithm (SSA), a type of meta-heuristic. SSA is a swarm intelligence (SI) method that stands out because of its quick convergence and improved stability. SSA does have some drawbacks, like lower swarm diversity and weak exploration ability in late iterations, like the majority of SI algorithms. So, using ten chaotic maps, we try to ameliorate SSA in three ways: (i) the initial swarm generation; (ii) the substitution of two random variables in SSA; and (iii) clamping the sparrows crossing the search range. As a result, we get CSSA, a chaotic form of SSA. Extensive comparisons show CSSA to be superior in terms of swarm diversity and convergence speed in solving various representative functions from the Institute of Electrical and Electronics Engineers (IEEE) Congress on Evolutionary Computation (CEC) benchmark set. Furthermore, experimental analysis of CSSA on eighteen interdisciplinary, multi-scale ML datasets from the University of California Irvine (UCI) data repository, as well as three high-dimensional microarray datasets, demonstrates that CSSA outperforms twelve state-of-the-art algorithms in a classification task based on FS discipline. Finally, a 5%-significance-level statistical post-hoc analysis based on Wilcoxon’s signed-rank test, Friedman’s rank test, and Nemenyi’s test confirms CSSA’s significance in terms of overall fitness, classification accuracy, selected feature size, computational time, convergence trace, and stability.

Bayesian weighted random forest for classification of high-dimensional genomics data

In this paper, a full Bayesian weighted probabilistic model is developed for random classification trees. The new model Bayesian Weighted Random Classification Forest (BWRCF) arises from the modification of the existing random classification forest in two ways. Firstly, the tree terminal node estimation procedure is replaced with a Bayesian estimation approach. Secondly, a new variable ranking procedure is developed and then hybridized with BWRCF to tackle the high-dimensionality issues. The performance of the proposed method is analyzed using simulated and real-life high-dimensional microarray datasets based on holdout accuracy and misclassification error rates. The results of the analyses showed that the proposed BWRCF is robust in terms of its ability to withstand moderate to large high-dimensionality scenarios. In addition, BWRCF also has improved predictive and efficiency abilities over selected competing methods.

Hybrid Filter and Genetic Algorithm-Based Feature Selection for Improving Cancer Classification in High-Dimensional Microarray Data

The advancements in intelligent systems have contributed tremendously to the fields of bioinformatics, health, and medicine. Intelligent classification and prediction techniques have been used in studying microarray datasets, which store information about the ways used to express the genes, to assist greatly in diagnosing chronic diseases, such as cancer in its earlier stage, which is important and challenging. However, the high-dimensionality and noisy nature of the microarray data lead to slow performance and low cancer classification accuracy while using machine learning techniques. In this paper, a hybrid filter-genetic feature selection approach has been proposed to solve the high-dimensional microarray datasets problem which ultimately enhances the performance of cancer classification precision. First, the filter feature selection methods including information gain, information gain ratio, and Chi-squared are applied in this study to select the most significant features of cancerous microarray datasets. Then, a genetic algorithm has been employed to further optimize and enhance the selected features in order to improve the proposed method’s capability for cancer classification. To test the proficiency of the proposed scheme, four cancerous microarray datasets were used in the study—this primarily included breast, lung, central nervous system, and brain cancer datasets. The experimental results show that the proposed hybrid filter-genetic feature selection approach achieved better performance of several common machine learning methods in terms of Accuracy, Recall, Precision, and F-measure.

Discrete equilibrium optimizer combined with simulated annealing for feature selection

This paper proposes a binary adaptation of the recently proposed meta-heuristic, Equilibrium Optimizer (EO), called Discrete EO (DEO), to solve binary optimization problems. A U-shaped transfer function is used to map the continuous values of EO into the binary domain. To further improve the exploitation capability of DEO, Simulated Annealing (SA) is used as a local search procedure and the combination is named as DEOSA. The proposed DEOSA algorithm is applied to 18 well-known UCI datasets and compared with a wide range of algorithms. The results are statistically validated using Wilcoxon rank-sum test and Friedman test. In order to test the scalability and robustness of DEOSA, it is additionally tested over seven high-dimensional Microarray datasets and 25 binary Knapsack problems. The results evidently demonstrate the superiority and merits of DEOSA when solving binary optimization problems.

Learning complex dependency structure of gene regulatory networks from high dimensional microarray data with Gaussian Bayesian networks

Reconstruction of Gene Regulatory Networks (GRNs) of gene expression data with Probabilistic Network Models (PNMs) is an open problem. Gene expression datasets consist of thousand of genes with relatively small sample sizes (i.e. are large-p-small-n). Moreover, dependencies of various orders coexist in the datasets. On the one hand transcription factor encoding genes act like hubs and regulate target genes, on the other hand target genes show local dependencies. In the field of Undirected Network Models (UNMs)—a subclass of PNMs—the Glasso algorithm has been proposed to deal with high dimensional microarray datasets forcing sparsity. To overcome the problem of the complex structure of interactions, modifications of the default Glasso algorithm have been developed that integrate the expected dependency structure in the UNMs beforehand. In this work we advocate the use of a simple score-based Hill Climbing algorithm (HC) that learns Gaussian Bayesian networks leaning on directed acyclic graphs. We compare HC with Glasso and variants in the UNM framework based on their capability to reconstruct GRNs from microarray data from the benchmarking synthetic dataset from the DREAM5 challenge and from real-world data from the Escherichia coli genome. We conclude that dependencies in complex data are learned best by the HC algorithm, presenting them most accurately and efficiently, simultaneously modelling strong local and weaker but significant global connections coexisting in the gene expression dataset. The HC algorithm adapts intrinsically to the complex dependency structure of the dataset, without forcing a specific structure in advance.

Attribute reduction with personalized information granularity of nearest mutual neighbors

Neighborhood-based attribute reduction plays a vital role in pattern recognition, for selecting a series of informative and relevant attributes from data sets. The increase in dimensionality and complexity of data along with superfluous and noisy attributes, results in significant and tricky issues when reasoning data. To address the ineffectiveness of classical attribute reduction within the neighborhood framework, a competent attribute reduction scheme based on a novel concept of nearest mutual neighbor-based personalized information granularity is proposed in this paper. This study comprises three phases. First, we intend to integrate the strategies of personalized radius and two-sided neighbors into the modeling of information granularity, and both personalized and mutual neighborhood rough sets are developed. In the second phase, the concept of the belief function from evidence theory is introduced to construct an adaptive neighborhood information granule with the guidance of the principle of justifiable granularity; in this sense, the search for a justifiable granularity is transformed into an optimization issue. In the third phase, a new neighborhood homogeneous evidence-based attribute significance evaluation is discussed to avoid the inadequacy of voting-based attribute measures. In addition, an enhanced forward greedy algorithm is proposed to address non-monotonic circumstances. For the experimental evaluation, fifteen benchmark data sets, involving six high-dimensional microarray data sets, are chosen to display the performance of the method and algorithm. The experimental findings reveal the superiority of our work from different perspectives.

Enriched Random Forest for High Dimensional Genomic Data.

Ensemble methods such as random forest works well on high-dimensional datasets. However, when the number of features is extremely large compared to the number of samples and the percentage of truly informative feature is very small, performance of traditional random forest decline significantly. To this end, we develop a novel approach that enhance the performance of traditional random forest by reducing the contribution of trees whose nodes are populated with less informative features. The proposed method selects eligible subsets at each node by weighted random sampling as opposed to simple random sampling in traditional random forest. We refer to this modified random forest algorithm as "Enriched Random Forest". Using several high-dimensional micro-array datasets, we evaluate the performance of our approach in both regression and classification settings. In addition, we also demonstrate the effectiveness of balanced leave-one-out cross-validation to reduce computational load and decrease sample size while computing feature weights. Overall, the results indicate that enriched random forest improves the prediction accuracy of traditional random forest, especially when relevant features are very few.

Binary Simulated Normal Distribution Optimizer for feature selection: Theory and application in COVID-19 datasets

Classification accuracy achieved by a machine learning technique depends on the feature set used in the learning process. However, it is often found that all the features extracted by some means for a particular task do not contribute to the classification process. Feature selection (FS) is an imperative and challenging pre-processing technique that helps to discard the unnecessary and irrelevant features while reducing the computational time and space requirement and increasing the classification accuracy. Generalized Normal Distribution Optimizer (GNDO), a recently proposed meta-heuristic algorithm, can be used to solve any optimization problem. In this paper, a hybrid version of GNDO with Simulated Annealing (SA) called Binary Simulated Normal Distribution Optimizer (BSNDO) is proposed which uses SA as a local search to achieve higher classification accuracy. The proposed method is evaluated on 18 well-known UCI datasets and compared with its predecessor as well as some popular FS methods. Moreover, this method is tested on high dimensional microarray datasets to prove its worth in real-life datasets. On top of that, it is also applied to a COVID-19 dataset for classification purposes. The obtained results prove the usefulness of BSNDO as a FS method. The source code of this work is publicly available at https://github.com/ahmed-shameem/Feature_selection.

AltWOA: Altruistic Whale Optimization Algorithm for feature selection on microarray datasets

The data-driven modern era has enabled the collection of large amounts of biomedical and clinical data. DNA microarray gene expression datasets have mainly gained significant attention to the research community owing to their ability to identify diseases through the “bio-markers” or specific alterations in the gene sequence that represent that particular disease (for example, different types of cancer). However, gene expression datasets are very high-dimensional, while only a few of those are “bio-markers”. Meta-heuristic-based feature selection effectively filters out only the relevant genes from a large set of attributes efficiently to reduce data storage and computation requirements. To this end, in this paper, we propose an Altruistic Whale Optimization Algorithm (AltWOA) for the feature selection problem in high-dimensional microarray data. AltWOA is an improvement on the basic Whale Optimization Algorithm. We embed the concept of altruism in the whale population to help efficient propagation of candidate solutions that can reach the global optima over the iterations. Evaluation of the proposed method on eight high dimensional microarray datasets reveals the superiority of AltWOA compared to popular and classical techniques in the literature on the same datasets both in terms of accuracy and the final number of features selected. The relevant codes for the proposed approach are available publicly at https://github.com/Rohit-Kundu/AltWOA.

A feature selection method for classification based on ensemble of penalized logistic models

Classification accuracy of any classifier can be enhanced by performing the classification on selected informative features. Features selection methods are generally approached for the purpose. Combining multiple models known as ensemble method emerged as prominent method for achieving classification accuracy. These techniques can be considered for variable selection for accuracy gain and interpretability of a classifier. we propose an ensemble of penalized logistic models (EPLM) for feature selection. EPLM employs Lasso, adaptive Lasso and elastic net for feature selection. EPLM is applied to high dimensional microarray data sets and simulated data sets. State-of-the-art classifiers, Lasso, Random Forest (RF), Support Vector Machine (SVM) and K- Nearest Neighbors (KNN) are employed for assessment of EPLM. Experimental comparisons reveal that significant improvement in classification accuracy is achieved for all the classifiers considered. The proposed method is also compared to the Ensemble of subset of KNN classifiers (ESKNN). In comparison to ESKNN EPLM achieves better performance accuracy for all the classifiers on simulated and microarray data sets. Moreover, it is also observed that EPLM has selected significantly smaller number of features as compared to the full feature set.

Dynamic relevance and interdependent feature selection for continuous data

Feature Selection (FS) techniques are employed in expert system for selecting relevant and eliminating redundant features. Information-theoretic based FS methods measure non-linear correlation and have gained popularity recently. However, these methods work with discrete data, leading to an additional overhead of discretization while dealing with continuous data. In order to resolve this, the paper proposes a FS method namely, Dynamic Relevance Interdependent Feature Selection (DRIFS) for continuous data which retains the essence of information-theoretic based methods. DRIFS incorporates two major aspects of FS that are ignored by many state of the art methods. First, the method updates the relevance of every candidate features after a new feature is selected and second, the method discriminates between redundant and interdependent features. The performance of DRIFS is compared with six other FS methods, CIFE, CMIM, DISR, ICAP, IGFS and MRMR. Ten real world datasets, which include seven high-dimensional microarray datasets are used for experimental validation. DRIFS performs better in terms of classification accuracy and the number of feature selected. Experimental results also show an improvement in classification accuracy after the application of DRIFS.

AIEOU: Automata-based improved equilibrium optimizer with U-shaped transfer function for feature selection

High dimensional data has become an essential challenge to address in Data Science and Machine Learning. Reducing the number of dimensions by excluding noisy, irrelevant, or correlated dimensions is often referred to as the feature selection (FS). The ultimate goal in FS is to identify an optimal set of dimensions (features) to realize better classification accuracy, decrease the computational time and optimize the memory requirements with the help of some methods. Recently, optimization algorithms have gained popularity in different fields because of their flexibility and effectiveness. Equilibrium optimizer (EO) is a physics-based meta-heuristic algorithm, which is inspired from a well-mixed dynamic mass balance on a control volume that has good exploration and exploitation capabilities. In this work, an improved version of EO is proposed with the inclusion of learning based automata to find proper values of its parameters and Adaptive β Hill Climbing (AβHC) to find a better equilibrium pool. The method is used as a feature selector, evaluated on 18 standard UCI datasets with the help of K-nearest neighbors (KNN) classifier, and compared with eight state-of-the-art methods including classical and hybrid meta-heuristic algorithms. Moreover, the proposed methods is applied on high dimensional Microarray datasets which generally contain a few samples but large number of features, and often lead to ‘curse of dimensionality’. The obtained results illustrate the supremacy of the proposed method over the other state-of-the-art methods mentioned in literature. The source code of this work is publicly available at https://github.com/ahmed-shameem/Feature_selection.

Improved salp swarm algorithm based on the levy flight for feature selection

The fields of data science and data mining are enduring high-dimensionality issues because of a high volume of data. Conventional machine learning techniques give disgruntled responses to high-dimensional datasets. Feature selection is used to get the appropriate information from the dataset to reduce the dimensionality of the data. The recently proposed Salp Swarm Algorithm (SSA) is a population-based meta-heuristic optimization technique inspired by the Sea Salps Swarming technique. SSA failed to converge initial random solutions to the global optimum owing to its complete dependency on the number of iterations for the process of exploration and exploitation. The proposed improved SSA (iSSA) aims to enhance the ability of Salps to explore divergent areas by randomly updating its location. Randomizing the Salps location via Levy flight enriches the exploitation potential of SSA resulting in it converging the model toward the global optima. The performance of the proposed iSSA is investigated using six different high-dimensional microarray datasets. While comparing the ability to converge, it is understood that the proposed model outperforms SSA providing 0.1033% more confidence in the selected features. The results of the simulation revealed that the iSSA can provide better competitive and significant results compared to SSA.

Stability Investigation of Improved Whale Optimization Algorithm in the Process of Feature Selection

The removal of irrelevant and insignificant features from the high-dimensional dataset is a necessary prerequisite for the exploration of information. Meta-heuristic optimization techniques have been widely used in the field of knowledge discovery over the last few years. The whale optimization algorithm (WOA) is a swarm-based metaheuristic technique that is often used in the field of dimensionality reduction. Among the various WOA-based feature selection techniques in the literature, not a single technology illuminates the stability issue of WOA. Stability is often identified as a sensitivity to the disruption of input data during the process of selecting significant features. In this study, a new feature selection model based on improved WOA (iWOA) is proposed to select significant features from a high-dimensional microarray dataset. The stability of the results obtained is evaluated with the existing stability index that satisfies all the required characteristics of the stability measure. In addition, the results of the proposed model are compared with other contemporary meta-heuristics techniques. The proposed iWOA proposes its identification as a well-stable feature selection technique according to the strength of the stability index agreement.

Cancer classification from microarray data for genomic disorder research using optimal discriminant independent component analysis and kernel extreme learning machine.

One of the challenging tasks in the medicinal field is genomic disorder investigation and its classification from the microarray dataset. The microarray dataset reorganization and its classification is more complex and expensive in the biomedical research area due to the larger number of features in the microarray dataset. In this paper, we construct a hybrid feature selection method such as t test, Fisher ration, and Bayesian logistic regression to select genes and that reduce the time cost. Based on the features, the top-ranked features are selected via the best hybrid rank method. Thereafter, the features are extracted using the modified firefly optimization-based discriminant independent component analysis (MF-DICA). Especially, the modified firefly optimization algorithm is capable of improving the search efficiency of DICA. From the high dimensional microarray dataset, MF-DICA is used to obtain the best features within the entire search space. The kernel extreme learning machine classifies the gene features depending upon the most relevant class. Experimentally, six datasets namely Leukemia dataset, Diffuse Larger B-cell Lymphomas, Lung cancer, Breast cancer, Prostate tumor, and Colon dataset are chosen to evaluate the performance of proposed approaches. Finally, the experimental data demonstrate that the proposed method is well suitable to classify the microarray data.

Feature Selection for High Dimensional Data Using Weighted K-Nearest Neighbors and Genetic Algorithm

Too many input features in applications may lead to over-fitting and reduce the performance of the learning algorithm. Moreover, in most cases, each feature containing different information content has different effects on the prediction target. Therefore, a feature selection method for calculating the importance of each feature, called WKNNGAFS, is proposed in this paper. In this method, the genetic algorithm (GA) is adopted to search the optimal weight vector, the value of the $i$ th component of which corresponds to the contribution degree of the $i$ th feature to the classification from a global perspective. Besides, weighted K-nearest neighbors algorithm (WKNN), which takes both the different contributions of nearest neighbors and the different classification ability of each feature into account, is used to determine the target label. To evaluate the effectiveness of the proposed method, nine existing feature selection methods are compared with it on 13 real datasets, including 6 high dimensional microarray datasets. Experimental results demonstrate the method is more effective and can improve classification performance.