Size Of Feature Space Research Articles

Comparative performance analysis of binary variants of FOX optimization algorithm with half-quadratic ensemble ranking method for thyroid cancer detection

Thyroid cancer is a life-threatening condition that arises from the cells of the thyroid gland located in the neck’s frontal region just below the adam’s apple. While it is not as prevalent as other types of cancer, it ranks prominently among the commonly observed cancers affecting the endocrine system. Machine learning has emerged as a valuable medical diagnostics tool specifically for detecting thyroid abnormalities. Feature selection is of vital importance in the field of machine learning as it serves to decrease the data dimensionality and concentrate on the most pertinent features. This process improves model performance, reduces training time, and enhances interpretability. This study examined binary variants of FOX-optimization algorithms for feature selection. The study employed eight transfer functions (S and V shape) to convert the FOX-optimization algorithms into their binary versions. The vision transformer-based pre-trained models (DeiT and Swin Transformer) are used for feature extraction. The extracted features are transformed using locally linear embedding, and binary FOX-optimization algorithms are applied for feature selection in conjunction with the Naïve Bayes classifier. The study utilized two datasets (ultrasound and histopathological) related to thyroid cancer images. The benchmarking is performed using the half-quadratic theory-based ensemble ranking technique. Two TOPSIS-based methods (H-TOPSIS and A-TOPSIS) are employed for initial model ranking, followed by an ensemble technique for final ranking. The problem is treated as multi-objective optimization task with accuracy, F2-score, AUC-ROC and feature space size as optimization goals. The binary FOX-optimization algorithm based on the V_1 transfer function achieved superior performance compared to other variants using both datasets as well as feature extraction techniques. The proposed framework comprised a Swin transformer to extract features, a Fox optimization algorithm with a V1 transfer function for feature selection, and a Naïve Bayes classifier and obtained the best performance for both datasets. The best model achieved an accuracy of 94.75%, an AUC-ROC value of 0.9848, an F2-Score of 0.9365, an inference time of 0.0353 seconds, and selected 5 features for the ultrasound dataset. For the histopathological dataset, the diagnosis model achieved an overall accuracy of 89.71%, an AUC-ROC score of 0.9329, an F2-Score of 0.8760, an inference time of 0.05141 seconds, and selected 12 features. The proposed model achieved results comparable to existing research with small features space.

Conditional Random Fields for Multiview Sequential Data Modeling.

Recently, multiview learning has been increasingly focused on machine learning. However, most existing multiview learning methods cannot directly deal with multiview sequential data, in which the inherent dynamical structure is often ignored. Especially, most traditional multiview machine learning methods assume that the items at different time slices within a sequence are independent of each other. In order to solve this problem, we propose a new multiview discriminant model based on conditional random fields (CRFs) to model multiview sequential data, called multiview CRF. It inherits the advantages of CRFs that build a relationship between items in each sequence. Moreover, by introducing specific features designed on the CRFs for multiview data, the multiview CRF not only considers the relationship among different views but also captures the correlation between the features from the same view. Particularly, some features can be reused or divided into different views to build an appropriate size of feature space. This helps to avoid underfitting problems caused by too small feature space or overfitting problems caused by too large feature space. In order to handle large-scale data, we use the stochastic gradient method to speed up our model. The experimental results on the text and video data illustrate the superiority of the proposed model.

High accuracy multilayer autoencoder trained classification method for diagnosis of Parkinson’s disease using vocal signals

Dysarthria and dysphonia are two commonly occurring disorders of speech in Parkinson’s disease patient, which are observed in approximately 90 % of the disease cases. It has been reported that these disorders give early sign of Parkinson’s disease. Hence, effective development of diagnostic tools for detecting early biomarkers can help in controlling the symptoms of disease. In this paper, we have proposed a classification model using Multi- layer Autoencoder for feature space reduction. Comparisons among 6 types of classification methods with various feature space sizes were conducted to find out the best performing classification algorithm and feature space size based on accuracy, specificity, and sensitivity. Mel Frequency Cepstral method was used to extract set of features from voice signals of both healthy group and unhealthy group. The model showed promising results in classification when combined with Multi-Layer Autoencoder and SVM classifier.

Modified Genetic Algorithm for Feature Selection and Hyper Parameter Optimization: Case of XGBoost in Spam Prediction

Recently, spam on online social networks has attracted attention in the research and business world. Twitter has become the preferred medium to spread spam content. Many research efforts attempted to encounter social networks spam. Twitter brought extra challenges represented by the feature space size, and imbalanced data distributions. Usually, the related research works focus on part of these main challenges or produce black-box models. In this paper, we propose a modified genetic algorithm for simultaneous dimensionality reduction and hyper parameter optimization over imbalanced datasets. The algorithm initialized an eXtreme Gradient Boosting classifier and reduced the features space of tweets dataset; to generate a spam prediction model. The model is validated using a 50 times repeated 10-fold stratified cross-validation, and analyzed using nonparametric statistical tests. The resulted prediction model attains on average 82.32% and 92.67% in terms of geometric mean and accuracy respectively, utilizing less than 10% of the total feature space. The empirical results show that the modified genetic algorithm outperforms <i>Chi</i>² and <i>PCA</i> feature selection methods. In addition, eXtreme Gradient Boosting outperforms many machine learning algorithms, including BERT-based deep learning model, in spam prediction. Furthermore, the proposed approach is applied to SMS spam modeling and compared to related works.

Deep mining of open source software bug repositories

Large scale software projects adopt bug tracking systems such as Bugzilla and Jira to manage the bugs’ fixes and store their information. Mining bug repositories is essential to automate some maintenance phase activities like bug triage. Recently, embedding and deep learning models have emerged as a breakthrough for text classification applications. This paper proposes utilizing word embedding and deep learning models for mining bug repositories, in order to identify severity classes of newly reported bugs. Embedding models were utilized to represent bug reports, due to their ability to acquire semantic relations among words and sentences. Five effective deep learning models (CNN, LSTM, GRU, hybrid CNN-LSTM/GRU) were trained to classify the bug reports to their severity classes. We attempted these models as each of them has different capabilities; CNN can extract key features of an input and reduce the feature space size. While, LSTM and GRU can represent sequential words (sentences) effectively. Experimental results on two open large-scale bug repositories, Eclipse and Mozilla, demonstrated that CNN model is the superior. However, it was found that the micro-average classification performance of the CNN could be achieved by some traditional classifiers, e.g. SVM and KNN, trained using the embedding vectors instead of the bag-of-words-model.

Machine learning topological phases in real space

We develop a supervised machine learning algorithm that is able to learn topological phases of finite condensed-matter systems from bulk data in real lattice space. The algorithm employs diagonalization in real space together with any supervised learning algorithm to learn topological phases through an eigenvector ensembling procedure. We combine our algorithm with decision trees and random forests to successfully recover topological phase diagrams of Su-Schrieffer-Heeger (SSH) models from bulk lattice data in real space and show how the Shannon information entropy of ensembles of lattice eigenvectors can be used to retrieve a signal detailing how topological information is distributed in the bulk. We further use insights obtained from these information entropy signatures to engineer global topological features from real-space lattice data that still carry most of the topological information in the lattice, while greatly diminishing the size of feature space, thus effectively amounting to a topological lattice compression. Finally, we explore the theoretical possibility of interpreting the information entropy topological signatures in terms of emergent information entropy wave functions, which lead us to Heisenberg and Hirschman uncertainty relations for topological phase transitions. The discovery of Shannon information entropy signals associated with topological phase transitions from the analysis of data from several thousand SSH systems illustrates how model explainability in machine learning can advance the research of exotic quantum materials with properties that may power future technological applications such as qubit engineering for quantum computing.

A Compressive Sensing Model for Speeding Up Text Classification.

Text classification plays an important role in various applications of big data by automatically classifying massive text documents. However, high dimensionality and sparsity of text features have presented a challenge to efficient classification. In this paper, we propose a compressive sensing- (CS-) based model to speed up text classification. Using CS to reduce the size of feature space, our model has a low time and space complexity while training a text classifier, and the restricted isometry property (RIP) of CS ensures that pairwise distances between text features can be well preserved in the process of dimensionality reduction. In particular, by structural random matrices (SRMs), CS is free from computation and memory limitations in the construction of random projections. Experimental results demonstrate that CS effectively accelerates the text classification while hardly causing any accuracy loss.

Machine Learning Approach for Answer Detection in Discussion Forums: An Application of Big Data Analytics

Nowadays, data are flooding into online web forums, and it is highly desirable to turn gigantic amount of data into actionable knowledge. Online web forums have become an integral part of the web and are main sources of knowledge. People use this platform to post their questions and get answers from other forum members. Usually, an initial post (question) gets more than one reply posts (answers) that make it difficult for a user to scan all of them for most relevant and quality answer. Thus, how to automatically extract the most relevant answer for a question within a thread is an important issue. In this research, we treat the task of answer extraction as classification problem. A reply post can be classified as relevant, partially relevant, or irrelevant to the initial post. To find the relevancy/similarity of a reply to the question, both lexical and nonlexical features are used. We proposed to use LinearSVC, a variant of support vector machine (SVM), for answer classification. Two selection techniques such as chi-square and univariate are employed to reduce the feature space size. The experimental results showed that LinearSVC classifier outperformed the other state-of-the-art classifiers in the context of classification accuracy for both Ubuntu and TripAdvisor (NYC) discussion forum datasets.

Analysis and prediction in sparse and high dimensional text data: The case of Dow Jones stock market

In this research, we proposed a text analysis system to predict stock market movements using news and social media data. It is a scalable prediction system for sparse and high dimensional feature sets. Using the developed system, we collected 12,560 articles from New York Times covering one year time period, and 2,854,333 tweets from Twitter covering 4 months time period. We analysed the collected data using entity extraction, sentiment analysis and topic modelling techniques. We applied our feature set creation and elastic net regression based training method . The analyses have been used to train different prediction models. Using these trained prediction models, we predicted stock market movements for Dow Jones Index and showed that the proposed method can make promising predictions. In different sets of experiments, highly accurate (up to 70.90% accuracy) predictions are made by the proposed approach. These predicted values also correlated (up to 0.2315 correlation coefficient value) with real Dow Jones Index values. Further, we report performance comparison results for various prediction models that we trained with different set of features to analyse the importance of time interval and feature space size. Our test results show that it is possible to make reasonable stock movement prediction by integrating news and related social media data, analysing them using named entity extraction, sentiment analysis and topic modelling techniques together with prediction models which use features that are created from these analysis results.

Locality-adapted kernel densities of term co-occurrences for location prediction of tweets

While geographical metadata referring to the originating locations of tweets provides valuable information to perform effective spatial analysis in social networks, scarcity of such geotagged tweets imposes limitations on their usability. In this work, we propose a content-based location prediction method for tweets by analyzing the geographical distribution of tweet texts using Kernel Density Estimation (KDE). The primary novelty of our work is to determine different settings of kernel functions for every term in tweets based on the location indicativeness of these terms. Our proposed method, which we call locality-adapted KDE, uses information-theoretic metrics and does not require any parameter tuning for these settings. As a further enhancement on the term-level distribution model, we describe an analysis of spatial point patterns in tweet texts in order to identify bigrams that exhibit significant deviation from the underlying unigram patterns. We present an expansion of feature space using the selected bigrams and show that it eventually yields further improvement in prediction accuracy of our locality-adapted KDE. We demonstrate that our expansion results in a limited increase in the size of feature space and it does not hinder online localization of tweets. The methods we propose rely purely on statistical approaches without requiring any language-specific setting. Experiments conducted on three tweet sets from different countries show that our proposed solution outperforms existing state-of-the-art techniques, yielding significantly more accurate predictions.

Extensive Experimental Evaluation of Self-Organizing Maps for Automatic Classification of a Multi-Class Multi-Label Corpus

This paper aims at bridging the gap between feature selection and feature space size by utilizing both square and non-square self-organizing maps under different configuration scenarios for classifying a multi-class multi-label corpus, the Reuters Mod Apte’ split. The selection of non-square maps is based on a heuristic process for finding the suitable size for the self-organizing map. Vector construction is based on a simple, yet effective procedure aiming at transforming the vectors from multi-label to uni-label. The proposed solution improves classification efficiency not only in terms of accuracy but also in computational resources needed and time for training. Extensive experiments were conducted, using different configurations regarding map and vector sizes, and training cycles, also utilizing context words, to assess their impact in the classifier’s performance. Furthermore, an intelligent algorithm for label selection is being proposed, aiming to show that the neighboring nodes on the map affect the selection of labels for a specific node. According to the experiments conducted, our approach achieves 10% increase in Macro-Average F1 scores, $30\times $ decrease in vector dimensionality, and approximately $34\times $ smaller maps when compared to the baseline scenario.

Incorporating known malware signatures to classify new malware variants in network traffic

SummaryContent‐based malware classification technique using n‐gram features required high computational overhead because of the size of feature space. This paper proposes the augmentation of domain knowledge in the form of known Snort malware signatures to machine learning techniques to reduce resources (in terms of the time to generate machine learning model and the memory usage to store generative model). Although current malware can be encrypted or mutated, these malware still exhibit prevalent contents or payloads as their predecessors. Using a dataset of traffic captured from a campus network, our approach is able to reduce initial generated million n‐gram features to only around 90000 features, which significantly reduces processing time to generate naive Bayes model by 95%. The generated model that has been trained by the most descriptive features (4‐gram Snort signatures with high information gain) produces lower false negative, about 2% compared with other models. Moreover, the proposed method is capable of detecting 10 new malware variants with 0% false negative. The findings from this paper can be the basis for improving malware classification based on content classification to detect known and new malware. Copyright © 2015 John Wiley &amp; Sons, Ltd.

Impact of feature selection on the accuracy and spatial uncertainty of per-field crop classification using Support Vector Machines

Crop mapping is one major component of agricultural resource monitoring using remote sensing. Yield or water demand modeling requires that both, the total surface that is cultivated and the accurate distribution of crops, respectively is known. Map quality is crucial and influences the model outputs. Although the use of multi-spectral time series data in crop mapping has been acknowledged, the potentially high dimensionality of the input data remains an issue. In this study Support Vector Machines (SVM) are used for crop classification in irrigated landscapes at the object-level. Input to the classifications is 71 multi-seasonal spectral and geostatistical features computed from RapidEye time series. The random forest (RF) feature importance score was used to select a subset of features that achieved optimal accuracies. The relationship between the hard result accuracy and the soft output from the SVM is investigated by employing two measures of uncertainty, the maximum a posteriori probability and the alpha quadratic entropy. Specifically the effect of feature selection on map uncertainty is investigated by looking at the soft outputs of the SVM, in addition to classical accuracy metrics. Overall the SVMs applied to the reduced feature subspaces that were composed of the most informative multi-seasonal features led to a clear increase in classification accuracy up to 4.3%, and to a significant decline in thematic uncertainty. SVM was shown to be affected by feature space size and could benefit from RF-based feature selection. Uncertainty measures from SVM are an informative source of information on the spatial distribution of error in the crop maps.

Feature elimination based random subspace ensembles learning for ECG arrhythmia diagnosis

Various methods for ensembles selection and classifier combination have been designed to optimize the performance of ensembles of classifiers. However, use of large number of features in training data can affect the classification performance of machine learning algorithms. The objective of this paper is to represent a novel feature elimination (FE) based ensembles learning method which is an extension to an existing machine learning environment. Here the standard 12 lead ECG signal recordings data have been used in order to diagnose arrhythmia by classifying it into normal and abnormal subjects. The advantage of the proposed approach is that it reduces the size of feature space by way of using various feature elimination methods. The decisions obtained from these methods have been coalesced to form a fused data. Thus the idea behind this work is to discover a reduced feature space so that a classifier built using this tiny data set would perform no worse than a classifier built from the original data set. Random subspace based ensembles classifier is used with PART tree as base classifier. The proposed approach has been implemented and evaluated on the UCI ECG signal data. Here, the classification performance has been evaluated using measures such as mean absolute error, root mean squared error, relative absolute error, F-measure, classification accuracy, receiver operating characteristics and area under curve. In this way, the proposed novel approach has provided an attractive performance in terms of overall classification accuracy of 91.11 % on unseen test data set. From this work, it is shown that this approach performs well on the ensembles size of 15 and 20.

Prediction of Thermophilic Protein with Pseudo Amino Acid Composition: An Approach from Combined Feature Selection and Reduction

Prediction of thermophilic and mesophilic protein plays a crucial role in both biochemistry and bioengineering. In this study, a different mode of pseudo amino acid composition (PseAAC) was proposed to formulate the protein samples by integrating the amino acid composition, the physic chemical features, as well as the composition transition and distribution features, where each of the protein samples was represented by a numerical vector through the sequence-based approach. Using the support vector machine algorithm, an accurate and reliable classifier was constructed to predict the thermophilic and mesophilic proteins. Moreover, three feature reduction algorithms were obtained for locating the most vital features and reducing the size of feature space. Among the three feature reduction algorithms, the genetic algorithm performed best. Finally, with the reduced features extracted from the genetic algorithm, it was observed that for the selected dataset the new classifier achieved a high accuracy of 95.93% with the Matthews correlation coefficient of 0.9187.

Gas chimney detection based on improving the performance of combined multilayer perceptron and support vector classifier

Abstract. Seismic object detection is a relatively new field in which 3-D bodies are visualized and spatial relationships between objects of different origins are studied in order to extract geologic information. In this paper, we propose a method for finding an optimal classifier with the help of a statistical feature ranking technique and combining different classifiers. The method, which has general applicability, is demonstrated here on a gas chimney detection problem. First, we evaluate a set of input seismic attributes extracted at locations labeled by a human expert using regularized discriminant analysis (RDA). In order to find the RDA score for each seismic attribute, forward and backward search strategies are used. Subsequently, two non-linear classifiers: multilayer perceptron (MLP) and support vector classifier (SVC) are run on the ranked seismic attributes. Finally, to capitalize on the intrinsic differences between both classifiers, the MLP and SVC results are combined using logical rules of maximum, minimum and mean. The proposed method optimizes the ranked feature space size and yields the lowest classification error in the final combined result. We will show that the logical minimum reveals gas chimneys that exhibit both the softness of MLP and the resolution of SVC classifiers.

Language morphology offset: Text classification on a Croatian–English parallel corpus

We investigate how, and to what extent, morphological complexity of the language influences text classification using support vector machines (SVM). The Croatian–English parallel corpus provides the basis for direct comparison of two languages of radically different morphological complexity. We quantified, compared, and statistically tested the effects of morphological normalisation on SVM classifier performance based on a series of parallel experiments on both languages, carried over a large scale of different feature subset sizes obtained by different feature selection methods, and applying different levels of morphological normalisation. We also quantified the trade-off between feature space size and performance for different levels of morphological normalisation, and compared the results for both languages. Our experiments have shown that the improvements in SVM classifier performance is statistically significant; they are greater for small and medium number of features, especially for Croatian, whereas for large number of features the improvements are rather small and may be negligible in practice for both languages.

Genetic engineering of hierarchical fuzzy regional representations for handwritten character recognition

This paper presents a genetic programming based approach for optimizing the feature extraction step of a handwritten character recognizer. This recognizer uses a simple multilayer perceptron as a classifier and operates on a hierarchical feature space of orientation, curvature, and center of mass primitives. The nodes of the hierarchy represent rectangular sub-regions of their parent node, the tree root corresponding to the character's bounding box. Within each sub-region, a variable number of fuzzy features are extracted. Genetic programming is used to simultaneously learn the best hierarchy and the best combination of fuzzy features. Moreover, the fuzzy features are not predetermined, they are inferred from the evolution process which runs a two-objective selection operator. The first objective maximizes the recognition rate, and the second minimizes the feature space size. Results on Unipen data show that, using this approach, robust representations could be obtained that out-performed comparable human designed hierarchical fuzzy regional representations.