Step In Machine Learning Research Articles

Effective Feature Selection Strategy for Supervised Classification based on an Improved Binary Aquila Optimization Algorithm

Feature Selection (FS) is considered a crucial step in machine learning and data mining tasks, which facilitates minimizing the direct consequence of redundant and irrelevant attributes on the model’s accuracy. Hence, the researchers developed various algorithms to choose the most appropriate features for improving the accuracy rate of the presented dataset. Nevertheless, these algorithms may fall into local optima problems when applied to substantial feature sizes’ datasets. In this paper, for handling the FS strategy through dimensionality-lessening while improving the classification accuracy, an effective Aquila Optimization (AO) algorithm is introduced. AO has stable exploration and exploitation capabilities. It is enhanced by integrating a random position amendment approach with Local Search (LS) strategy to avoid local optima and then cloned into a binary version called Improved Binary AO (IBAO). Additionally, k-Nearest Neighbor (k-NN) and Support Vector Machine (SVM) are quality estimators. On 18 multi-scale benchmarks, the IBAO algorithm is compared with the original BAO algorithm and twelve recent algorithms, such as Binary Artificial Bee Colony (BABC), Binary Bat Algorithm (BBA), Binary Particle Swarm Optimization (BPSO), Binary Whale Optimization Algorithm (BWOA), Binary Grey Wolf Optimization (BGWO), Binary Sailfish Optimizer (BSFO), Binary Henry Gas Solubility Optimization (BHGSO), Binary Harris Hawks Optimization (BHHO). According to Wilcoxon’s rank-sum test (α=0.05), the dominance and significant influence of IBAO are apparent on both small and large dimensional benchmarks and obtained classification accuracy up to 100% in some of these benchmarks integrated with a feature reduction size down to 92%.

Literature Review on Hybrid Evolutionary Approaches for Feature Selection

The efficiency and the effectiveness of a machine learning (ML) model are greatly influenced by feature selection (FS), a crucial preprocessing step in machine learning that seeks out the ideal set of characteristics with the maximum accuracy possible. Due to their dominance over traditional optimization techniques, researchers are concentrating on a variety of metaheuristic (or evolutionary) algorithms and trying to suggest cutting-edge hybrid techniques to handle FS issues. The use of hybrid metaheuristic approaches for FS has thus been the subject of numerous research works. The purpose of this paper is to critically assess the existing hybrid FS approaches and to give a thorough literature review on the hybridization of different metaheuristic/evolutionary strategies that have been employed for supporting FS. This article reviews pertinent documents on hybrid frameworks that were published in the period from 2009 to 2022 and offers a thorough analysis of the used techniques, classifiers, datasets, applications, assessment metrics, and schemes of hybridization. Additionally, new open research issues and challenges are identified to pinpoint the areas that have to be further explored for additional study.

Vehicle Detection and Recognition Approach in Multi-Scale Traffic Monitoring System via Graph-Based Data Optimization.

Over the past few years, significant investments in smart traffic monitoring systems have been made. The most important step in machine learning is detecting and recognizing objects relative to vehicles. Due to variations in vision and different lighting conditions, the recognition and tracking of vehicles under varying extreme conditions has become one of the most challenging tasks. To deal with this, our proposed system presents an adaptive method for robustly recognizing several existing automobiles in dense traffic settings. Additionally, this research presents a broad framework for effective on-road vehicle recognition and detection. Furthermore, the proposed system focuses on challenges typically noticed in analyzing traffic scenes captured by in-vehicle cameras, such as consistent extraction of features. First, we performed frame conversion, background subtraction, and object shape optimization as preprocessing steps. Next, two important features (energy and deep optical flow) were extracted. The incorporation of energy and dense optical flow features in distance-adaptive window areas and subsequent processing over the fused features resulted in a greater capacity for discrimination. Next, a graph-mining-based approach was applied to select optimal features. Finally, the artificial neural network was adopted for detection and classification. The experimental results show significant performance in two benchmark datasets, including the LISA and KITTI 7 databases. The LISA dataset achieved a mean recognition rate of 93.75% on the LDB1 and LDB2 databases, whereas KITTI attained 82.85% accuracy on separate training of ANN.

GRU-SVM Based Threat Detection in Cognitive Radio Network.

Cognitive radio networks are vulnerable to numerous threats during spectrum sensing. Different approaches can be used to lessen these attacks as the malicious users degrade the performance of the network. The cutting-edge technologies of machine learning and deep learning step into cognitive radio networks (CRN) to detect network problems. Several studies have been conducted utilising various deep learning and machine learning methods. However, only a small number of analyses have used gated recurrent units (GRU), and that too in software defined networks, but these are seldom used in CRN. In this paper, we used GRU in CRN to train and test the dataset of spectrum sensing results. One of the deep learning models with less complexity and more effectiveness for small datasets is GRU, the lightest variant of the LSTM. The support vector machine (SVM) classifier is employed in this study's output layer to distinguish between authorised users and malicious users in cognitive radio network. The novelty of this paper is the application of combined models of GRU and SVM in cognitive radio networks. A high testing accuracy of 82.45%, training accuracy of 80.99% and detection probability of 1 is achieved at 65 epochs in this proposed work.

Shortlisting machine learning-based stock trading recommendations using candlestick pattern recognition

Quantitative stock trading systems apply automated, data-driven models to invest in the stock markets. In the last two decades, the machine learning community has deeply explored the use of per-stock machine learning models to forecast next-day stocks’ directions and generate profitable trading signals. On the basis of the experience of discretionary traders, some promising attempts to enhance classifier performance by integrating the knowledge extracted from Japanese candlestick charts have been made. However, machine learning-based trading systems tend to generate an excessive number of false signals and do not necessarily consider the information provided by candlestick patterns in the appropriate manner. To alleviate these negative effects, this paper proposes to decouple the machine learning and pattern recognition steps so that the trading system can generate a reduced number of double-checked trading recommendations. Specifically, it proposes to selectively filter out the machine learning-based trading recommendations that are deemed as potentially unreliable according to the recognized graphical patterns. To this aim, it explores various alternatives to combine pattern recognition strategies with different machine learning models, including various shallow and deep supervised models and autoregressive techniques. The experiments, carried out on different market exchanges and under different conditions, demonstrate the effectiveness of the proposed approach in terms of return of investment and maximum drawdown of the trading system.

Feature Selection Based on a Sparse Neural-Network Layer With Normalizing Constraints.

Feature selection (FS) is an important step in machine learning since it has been shown to improve prediction accuracy while suppressing the curse of dimensionality of high-dimensional data. Neural networks have experienced tremendous success in solving many nonlinear learning problems. Here, we propose a new neural-network-based FS approach that introduces two constraints, the satisfaction of which leads to a sparse FS layer. We performed extensive experiments on synthetic and real-world data to evaluate the performance of our proposed FS method. In the experiments, we focus on high-dimensional, low-sample-size data since they represent the main challenge for FS. The results confirm that the proposed FS method based on a sparse neural-network layer with normalizing constraints (SNeL-FS) is able to select the important features and yields superior performance compared to other conventional FS methods.

Accurately modeling biased random walks on weighted networks using node2vec.

Accurately representing biological networks in a low-dimensional space, also known as network embedding, is a critical step in network-based machine learning and is carried out widely using node2vec, an unsupervised method based on biased random walks. However, while many networks, including functional gene interaction networks, are dense, weighted graphs, node2vec is fundamentally limited in its ability to use edge weights during the biased random walk generation process, thus under-using all the information in the network. Here, we present node2vec+, a natural extension of node2vec that accounts for edge weights when calculating walk biases and reduces to node2vec in the cases of unweighted graphs or unbiased walks. Using two synthetic datasets, we empirically show that node2vec+ is more robust to additive noise than node2vec in weighted graphs. Then, using genome-scale functional gene networks to solve a wide range of gene function and disease prediction tasks, we demonstrate the superior performance of node2vec+ over node2vec in the case of weighted graphs. Notably, due to the limited amount of training data in the gene classification tasks, graph neural networks such as GCN and GraphSAGE are outperformed by both node2vec and node2vec+. The data and code are available on GitHub at https://github.com/krishnanlab/node2vecplus_benchmarks. All additional data underlying this article are available on Zenodo at https://doi.org/10.5281/zenodo.7007164. Supplementary data are available at Bioinformatics online.

Hybrid Dipper Throated and Grey Wolf Optimization for Feature Selection Applied to Life Benchmark Datasets

Selecting the most relevant subset of features from a dataset is a vital step in data mining and machine learning. Each feature in a dataset has 2n possible subsets, making it challenging to select the optimum collection of features using typical methods. As a result, a new metaheuristics-based feature selection method based on the dipper-throated and grey-wolf optimization (DTO-GW) algorithms has been developed in this research. Instability can result when the selection of features is subject to metaheuristics, which can lead to a wide range of results. Thus, we adopted hybrid optimization in our method of optimizing, which allowed us to better balance exploration and harvesting chores more equitably. We propose utilizing the binary DTO-GW search approach we previously devised for selecting the optimal subset of attributes. In the proposed method, the number of features selected is minimized, while classification accuracy is increased. To test the proposed method’s performance against eleven other state-of-the-art approaches, eight datasets from the UCI repository were used, such as binary grey wolf search (bGWO), binary hybrid grey wolf, and particle swarm optimization (bGWO-PSO), bPSO, binary stochastic fractal search (bSFS), binary whale optimization algorithm (bWOA), binary modified grey wolf optimization (bMGWO), binary multiverse optimization (bMVO), binary bowerbird optimization (bSBO), binary hysteresis optimization (bHy), and binary hysteresis optimization (bHWO). The suggested method is superior and successful in handling the problem of feature selection, according to the results of the experiments.

JoMIC: A joint MI-based filter feature selection method

Feature selection (FS) is a common preprocessing step of machine learning that selects informative subset of features which fuels a model to perform better during prediction or classification. It helps in the design of an intelligent and expert system used in computer vision, image processing, gene expression data analysis, intrusion detection and natural language processing. In this paper, we introduce an effective filter method called Joint Mutual Information with Class relevance (JoMIC) using multivariate Joint Mutual Information (JMI) and Mutual Information (MI). Our method considers both JMI and MI of a non selected feature with selected ones w.r.t a given class to select a feature that is highly relevant to the class but non redundant to other selected features. We compare our method with seven other filter-based methods using the machine learning classifiers viz., Logistic Regression, Support Vector Machine, K-nearest Neighbor (KNN), Decision Tree, Random Forest, Naïve Bayes, and Stochastic Gradient Descent on various datasets. Experimental results reveal that our method yields better performance in terms of accuracy, Matthew’s Correlation Coefficient (MCC) and F1-score over 16 benchmark datasets, as compared to other competent methods. The superiority of our proposed method is that it uses an effective objective function that combines both JMI and MI to choose the relevant and non redundant features.

A Bridge to Machine Learning for Zero Computer Science Community: A Machine Learning Path with Linear Regression

Based on LinkedIn poll results, professionals from non-computer science backgrounds have developed an interest in data science and most intend to enter the field through Machine Learning, but do not know where to start (Webb et al., 2021) The aim of this study is to introduce an ideal systematic approach and workflow to get started with Machine Learning. Since the Linear Regression algorithm is the main algorithm used to introduce the concept of Machine Learning, it was chosen for this study to lay a fundamental platform. Using the proposed algorithm and appropriate python libraries, efficient methods of explanatory data analysis and procedural steps towards model training are also explained. Briefly, this study gives an insight on how to approach other Machine Learning algorithms as well. The first steps in Machine Learning involve data pre-processing. The dataset was chosen to meet data manipulation techniques suitable for people without a computer science background.

The Novel Method for Data Preprocessing CLI

Data preprocessing is the first step in machine learning to ensure data quality and extract useful information from datasets. Derived data after data processing is used for model training and has a direct impact on model efficiency. If there is no relevant and dispensable information in the dataset, it will be removed from the dataset to ensure data quality. Data pre-processing includes description of data, null value handling, categorical value coding, normalization, transformation, extraction and selection of various features.

Feature Selection Using Neighborhood based Entropy

Feature selection plays an important role as a preprocessing step for pattern recognition and machine learning. The goal of feature selection is to determine an optimal subset of relevant features out of a large number of features. The neighborhood discrimination index (NDI) is one of the newest and the most efficient measures to determine distinguishing ability of a feature subset. NDI is computed based on a neighborhood radius (E). Due to the significant impact of E on NDI, selecting an appropriate value of E for each data set might be challenging and very time-consuming. This paper proposes a new approach based on targEt PointS To computE neIgh- borhood relatioNs (EPSTEIN). At first, all the data points are sorted in the descending order of their density. Then, the highest density data points are selected as many as the number of classes. To determine the neighborhood relations, the circles centered on the target points are drawn and the points inside or on the circles are considered to be neighbors. In the next step, the significance of each feature is computed and a greedy algorithm selects appropriate features. The performance of the proposed approach is compared to both the commonest and newest methods of feature selection. The experimental results show that EPSTEIN could select more efficient subsets of features and improve the prediction accuracy of classifiers in comparison to the other state-of-the-art methods such as Correlation-based Feature Selection (CFS), Fast Correlation-Based Filter (FCBF), Heuris- tic Algorithm Based on Neighborhood Discrimination Index (HANDI), Ranking Based Feature Inclusion for Optimal Feature Subset (KNFI), Ranking Based Feature Elimination (KNFE) and Principal Component Analysis and Information Gain (PCA-IG).

Radiomics in bone pathology of the jaws.

To define which are and how the radiomics features of jawbone pathologies are extracted for diagnosis, predicting prognosis and therapeutic response. A comprehensive literature search was conducted using eight databases and gray literature. Two independent observers rated these articles according to exclusion and inclusion criteria. 23 papers were included to assess the radiomics features related to jawbone pathologies. Included studies were evaluated by using JBI Critical Appraisal Checklist for Analytical Cross-Sectional Studies. Agnostic features were mined from periapical, dental panoramic radiographs, cone beam CT, CT and MRI images of six different jawbone alterations. The most frequent features mined were texture-, shape- and intensity-based features. Only 13 studies described the machine learning step, and the best results were obtained with Support Vector Machine and random forest classifier. For osteoporosis diagnosis and classification, filtering, shape-based and Tamura texture features showed the best performance. For temporomandibular joint pathology, gray-level co-occurrence matrix (GLCM), gray level run length matrix (GLRLM), Gray Level Size Zone Matrix (GLSZM), first-order statistics analysis and shape-based analysis showed the best results. Considering odontogenic and non-odontogenic cysts and tumors, contourlet and SPHARM features, first-order statistical features, GLRLM, GLCM had better indexes. For odontogenic cysts and granulomas, first-order statistical analysis showed better classification results. GLCM was the most frequent feature, followed by first-order statistics, and GLRLM features. No study reported predicting response, prognosis or therapeutic response, but instead diseases diagnosis or classification. Although the lack of standardization in the radiomics workflow of the included studies, texture analysis showed potential to contribute to radiologists' reports, decreasing the subjectivity and leading to personalized healthcare.

Granularity Selection for Hierarchical Classification Based on Uncertainty Measure

Feature selection is an important preprocessing step for high-dimensional data mining and machine learning; it is viewed as the selection of the optimal granularity to describe the target concept in rough set theory. Currently, research on rough sets mainly focuses on granularity selection in flat classification scenarios, while organizing hundreds of labels for hierarchical classification (HC) can provide additional external information and achieve better performance in terms of both accuracy and efficiency. However, HC also faces the following problems: 1) the current measures’ failure to characterize the uncertainty in HC; 2) the inability to select the optimal granularity of the target concept in HC; and 3) no valid approach to select features in a decision system with hierarchical classification (HieDS). To address these problems, this article introduces HC into rough set theory and proposes an approach to granularity selection for HC. First, we introduce the knowledge distance to reflect the uncertainty of HC and define related important characteristic functions to describe a HieDS. Then, from the perspective of uncertainty, granularity selection for the target concept and feature selection are presented based on these characteristic functions. Finally, we experimentally realize granularity selection and demonstrate excellent performance of feature selection in a HieDS in terms of both feature selection and classification accuracy.

Fuzzy-based hunger games search algorithm for global optimization and feature selection using medical data

Feature selection (FS) is one of the basic data preprocessing steps in data mining and machine learning. It is used to reduce feature size and increase model generalization. In addition to minimizing feature dimensionality, it also enhances classification accuracy and reduces model complexity, which are essential in several applications. Traditional methods for feature selection often fail in the optimal global solution due to the large search space. Many hybrid techniques have been proposed depending on merging several search strategies which have been used individually as a solution to the FS problem. This study proposes a modified hunger games search algorithm (mHGS), for solving optimization and FS problems. The main advantages of the proposed mHGS are to resolve the following drawbacks that have been raised in the original HGS; (1) avoiding the local search, (2) solving the problem of premature convergence, and (3) balancing between the exploitation and exploration phases. The mHGS has been evaluated by using the IEEE Congress on Evolutionary Computation 2020 (CEC’20) for optimization test and ten medical and chemical datasets. The data have dimensions up to 20000 features or more. The results of the proposed algorithm have been compared to a variety of well-known optimization methods, including improved multi-operator differential evolution algorithm (IMODE), gravitational search algorithm, grey wolf optimization, Harris Hawks optimization, whale optimization algorithm, slime mould algorithm and hunger search games search. The experimental results suggest that the proposed mHGS can generate effective search results without increasing the computational cost and improving the convergence speed. It has also improved the SVM classification performance.

Machine learning-assisted upscaling analysis of reservoir rock core properties based on micro-computed tomography imagery

Optimum solutions for geologic modeling and reservoir simulation in industries such as oil and gas recovery and carbon capture and storage require accurate characterization of reservoir properties, which are often heterogeneous. In this study, high-quality micro-computed tomography (CT) images (1.475-μm/pixel resolution) of a sandstone core acquired from the Bell Creek oil field, USA, were used to provide nondestructive analysis of pore- and core-scale heterogeneity across measurement scales of 94–566 μm. In addition to characterizing the as-received sample, the core sample was flooded with brine to evaluate the capacity of the core sample to receive injected fluids. The micro-CT images were systematically segmented into pore spaces and grains via machine learning (ML) steps including image preprocessing, label creation using a traditional ML method based on limited manual image annotation, and finally U-Net segmentation. The segmented image stacks were reconstructed into digital cubes of various scales of voxel lengths. The 3D porosity values were calculated for all the digital cubes, and the fractal dimensions of the cubes were estimated using a box-counting method. The results showed that smaller cubes had greater heterogeneity and that the porosity values could be accurately estimated by fractal dimension and voxel lengths using ML models. For the core sample with brine flooding, the ratio of pores filled by brine to the total pore space was related to the porosity and could also be accurately estimated by porosity, fractal dimension, and voxel lengths using ML models. The results of this study demonstrate that the concept of fractal dimension can be a useful vector to perform upscaling analysis of sandstone rock heterogeneity from the pore to core scale and that fractal dimensions can be used to estimate porosity values and pore space-filling capacity across those scales.

Cognitive Workload Recognition Using EEG Signals and Machine Learning: A Review

Machine learning and its subfield deep learning techniques provide opportunities for the development of operator mental state monitoring, especially for cognitive workload recognition using electroencephalogram (EEG) signals. Although a variety of machine learning methods have been proposed for recognizing cognitive workload via EEG recently, there does not yet exist a review that covers in-depth the application of machine learning methods. To alleviate this gap, in this article, we survey cognitive workload and machine learning literature to identify the approaches and highlight the primary advances. To be specific, we first introduce the concepts of cognitive workload and machine learning. Then, we discuss the steps of classical machine learning for cognitive workload recognition from the following aspects, i.e., EEG data preprocessing, feature extraction and selection, classification method, and evaluation methods. Further, we review the commonly used deep learning methods for this domain. Finally, we expound on the open problem and future outlooks.

FewJoint: few-shot learning for joint dialogue understanding.

Few-shot learning (FSL) is one of the key future steps in machine learning and raises a lot of attention. In this paper, we focus on the FSL problem of dialogue understanding, which contains two closely related tasks: intent detection and slot filling. Dialogue understanding has been proven to benefit a lot from jointly learning the two sub-tasks. However, such joint learning becomes challenging in the few-shot scenarios: on the one hand, the sparsity of samples greatly magnifies the difficulty of modeling the connection between the two tasks; on the other hand, how to jointly learn multiple tasks in the few-shot setting is still less investigated. In response to this, we introduce FewJoint, the first FSL benchmark for joint dialogue understanding. FewJoint provides a new corpus with 59 different dialogue domains from real industrial API and a code platform to ease FSL experiment set-up, which are expected to advance the research of this field. Further, we find that insufficient performance of the few-shot setting often leads to noisy sharing between two sub-task and disturbs joint learning. To tackle this, we guide slot with explicit intent information and propose a novel trust gating mechanism that blocks low-confidence intent information to ensure high quality sharing. Besides, we introduce a Reptile-based meta-learning strategy to achieve better generalization in unseen few-shot domains. In the experiments, the proposed method brings significant improvements on two datasets and achieve new state-of-the-art performance.

Object based classification of a riparian environment using ultra-high resolution imagery, hierarchical landcover structures, and image texture

Land cover mapping is an important part of resource management, planning, and economic predictions. Improvements in remote sensing, machine learning, image processing, and object based image analysis (OBIA) has made the process of identifying land cover types increasingly faster and reliable but these advances have not been able to utilize all of the information encompassed within ultra-high (sub-meter) resolution imagery. There have been few known attempts to try and maximize this detailed information in high resolution imagery using advanced textural components. Hierarchical land classes are also rarely used as an attribute within the machine learning step of object-based image analysis. In this study we try to circumnavigate the inherent problems associated with high resolution imagery by combining well researched data transformations that aid the OBIA process with a seldom used texture transformation in Geographic Object Based Image Analyses (GEOBIA/OBIA) known as the Gabor Transform and the hierarchal organization of landscapes. We will observe the difference made in segmentation and classification accuracy of a random forest classifier when we fuse a Gabor transformed image to a Normalized Difference Vegetation Index (NDVI), high resolution multi-spectral imagery (RGB and NIR) and Light Detection and Ranging (LiDAR) derived canopy height model (CHM) within a riparian area in Southeast Iowa, United States. Additionally, we will observe the effects on classification accuracy when adding multi-scale land cover data to objects. Both, the addition of hierarchical information and Gabor textural information, could aid the GEOBIA process in delineating and classifying the same objects that human experts would delineate within this riparian landscape.

EGFAFS: A Novel Feature Selection Algorithm Based on Explosion Gravitation Field Algorithm.

Feature selection (FS) is a vital step in data mining and machine learning, especially for analyzing the data in high-dimensional feature space. Gene expression data usually consist of a few samples characterized by high-dimensional feature space. As a result, they are not suitable to be processed by simple methods, such as the filter-based method. In this study, we propose a novel feature selection algorithm based on the Explosion Gravitation Field Algorithm, called EGFAFS. To reduce the dimensions of the feature space to acceptable dimensions, we constructed a recommended feature pool by a series of Random Forests based on the Gini index. Furthermore, by paying more attention to the features in the recommended feature pool, we can find the best subset more efficiently. To verify the performance of EGFAFS for FS, we tested EGFAFS on eight gene expression datasets compared with four heuristic-based FS methods (GA, PSO, SA, and DE) and four other FS methods (Boruta, HSICLasso, DNN-FS, and EGSG). The results show that EGFAFS has better performance for FS on gene expression data in terms of evaluation metrics, having more than the other eight FS algorithms. The genes selected by EGFAGS play an essential role in the differential co-expression network and some biological functions further demonstrate the success of EGFAFS for solving FS problems on gene expression data.