Dimensional Feature Space Research Articles

Experimental quantum kernel trick with nuclear spins in a solid

The kernel trick allows us to employ high-dimensional feature space for a machine learning task without explicitly storing features. Recently, the idea of utilizing quantum systems for computing kernel functions using interference has been demonstrated experimentally. However, the dimension of feature spaces in those experiments have been smaller than the number of data, which makes them lose their computational advantage over explicit method. Here we show the first experimental demonstration of a quantum kernel machine that achieves a scheme where the dimension of feature space greatly exceeds the number of data using 1H nuclear spins in solid. The use of NMR allows us to obtain the kernel values with single-shot experiment. We employ engineered dynamics correlating 25 spins which is equivalent to using a feature space with a dimension over 1015. This work presents a quantum machine learning using one of the largest quantum systems to date.

VIF-Regression Screening Ultrahigh Dimensional Feature Space

Iterative Sure Independent Screening (ISIS) was proposed for the problem of variable selection with ultrahigh dimensional feature space. Unfortunately, the ISIS method transforms the dimensionality of features from ultrahigh to ultra-low and may result in un-reliable inference when the number of important variables particularly is greater than the screening threshold. The proposed method has transformed the ultrahigh dimensionality of features to high dimension space in order to remedy of losing some information by ISIS method. The proposed method is compared with ISIS method by using real data and simulation. The results show this method is more efficient and more reliable than ISIS method.

A supervised clustering MCMC methodology for large categorical feature spaces.

There is a well-established tradition within the statistics literature that explores different techniques for reducing the dimensionality of large feature spaces. The problem is central to machine learning and it has been largely explored under the unsupervised learning paradigm. We introduce a supervised clustering methodology that capitalizes on a Metropolis Hastings algorithm to optimize the partition structure of a large categorical feature space tailored towards minimizing the test error of a learning algorithm. This is a general methodology that can be applied to any supervised learning problem with a large categorical feature space. We show the benefits of the algorithm by applying this methodology to the problem of risk adjustment in competitive health insurance markets. We use a large claims data set that records ICD-10 codes, a large categorical feature space. We aim at improving risk adjustment by clustering diagnostic codes into risk groups suitable for health expenditure prediction. We test the performance of our methodology against common alternatives using panel data from a representative sample of twenty three million citizens in Colombian Healthcare System. Our results outperform common alternatives and suggest that it has potential to improve risk adjustment.

Classification of Toddler Nutritional Status Based on Antrophometric Index and Feature Discrimination using Support Vector Machine Hyperparameter Tuning

Nutritional status is the study of food and is related to health. Nutritional status is a benchmark to assess the health development of toddler. The nutritional status of toddler is assessed according to three index, such as body weight to age (BW / A), body height to age (BH / A), body weight to body height (BW / BH). The issue of nutrition is still a major factor in the growth and development of toddler in Indonesia. Public Health Center (Puskesmas) and Integrated Healthcare Center (Posyandu) as public health services work together to control the growth and development of toddler in Indonesia. To help control the growth and development of toddler, we proposed a research to classify the nutritional status of toddler based on anthropometric index. The nutritional status of toddler dataset was formed into a classification model using SVM Hyperparameter Tuning. SVM is a machine learning which the classification model used a hypothesis space in the form of linear functions in a high dimensional feature space. Adjustment of the hyperparameter was involved to reach a model that can optimally solve machine learning problems. We implemented feature selection using Fisher's Discriminant Ratio as a preprocessing stage, which the most important features were body weight (BB) and height (BH). The experimental results showed the classification model using SVM on training and testing data with a ratio of 70:30 reached accuracy of 84%, while SVM Hyperparameter Tuning with parameter of Cost = 100 parameters, Gamma = 0.01, Kernel = RBF reached accuracy of 97%. They represented a significant accuracy difference of 13%.

An Enhanced Gated Recurrent Unit with Auto-Encoder for Solving Text Classification Problems

Classification has become an important task for automatically categorizing documents based on their respective group. The purpose of classification is to assign the pre-specified group or class to an instance based on the observed features related to that instance. For accurate text classification, feature selection techniques are normally used to identify important features and to remove irrelevant, undesired and noisy features for minimizing the dimensionality of feature space. Therefore, in this research, a new model namely Encoder Simplified GRU (ES-GRU) is proposed to reduce dimension of data using an auto-encoder (AE). Gated Recurrent Unit (GRU) is a deep learning algorithm that contains update gate and reset gate, which is considered as one of the most efficient text classification technique, specifically on sequential datasets. Accordingly, the reset gate is replaced with an update gate in order to reduce the redundancy and complexity in the standard GRU. The proposed model has been evaluated on five benchmark text datasets and compared with six baseline well-known text classification approaches, which includes standard GRU, AE, Long Short-Term Memory, Convolutional Neural Network, Support Vector Machine, and Naïve Bayes. Based on various types of performance evaluation parameters, a considerable amount of improvement has been observed in the performance of the proposed model as compared to state-of-the-art approaches.

Gene selection using hybrid dragonfly black hole algorithm: A case study on RNA-seq COVID-19 data

This paper introduces a new hybrid approach (DBH) for solving gene selection problem that incorporates the strengths of two existing metaheuristics: binary dragonfly algorithm (BDF) and binary black hole algorithm (BBHA). This hybridization aims to identify a limited and stable set of discriminative genes without sacrificing classification accuracy, whereas most current methods have encountered challenges in extracting disease-related information from a vast amount of redundant genes. The proposed approach first applies the minimum redundancy maximum relevancy (MRMR) filter method to reduce the dimensionality of feature space and then utilizes the suggested hybrid DBH algorithm to determine a smaller set of significant genes. The proposed approach was evaluated on eight benchmark gene expression datasets, and then, was compared against the latest state-of-art techniques to demonstrate algorithm efficiency. The comparative study shows that the proposed approach achieves a significant improvement as compared with existing methods in terms of classification accuracy and the number of selected genes. Moreover, the performance of the suggested method was examined on real RNA-Seq coronavirus-related gene expression data of asthmatic patients for selecting the most significant genes in order to improve the discriminative accuracy of angiotensin-converting enzyme 2 (ACE2). ACE2, as a coronavirus receptor, is a biomarker that helps to classify infected patients from uninfected in order to identify subgroups at risk for COVID-19. The result denotes that the suggested MRMR-DBH approach represents a very promising framework for finding a new combination of most discriminative genes with high classification accuracy.

Advanced Classification Techniques for Improving Networks’ Intrusion Detection System Efficiency

This research aims to enhance the accuracy and speed of the intrusion detection process by using the feature selection method to reduce the feature space dimensions that eliminate irrelevant features. Further, we employed ensemble learning in the UNSW-NB15 dataset, by using a classifier of the Stacking method, to prevent the intrusion detection system (IDS) from becoming archaic, to adjust it with a modern attack resistance feature, and to make it less costly. We used logistic regression as a meta-classifier and combined random forests, sequential minimal optimization (SMO), and naïve Bayes methods. Our approach allowed us to achieve 97.88% accuracy in intrusion detection.

Efficient Image Inpainting Using Dimensional Space Reduction via Adaptive Patch-based Concept and Rank Lowering Technique

In this paper, we present a new algorithm for image inpainting using low dimensional feature space. In our method, projecting a low dimensional space from the original space is accomplished firstly using SVD, which is named low rank component, and then the missing pixels are filled in the new space. Finally, the original image is inpainted so that adaptive patch size is considered by quad-tree based on the previous step. In our algorithm, the missing pixels in the target region are estimated twice, one in low dimension feature space and another in the original space. It is noticeable that both processes estimate the unknown pixels using patch-based idea and rank lowering concept. Experimental results of this algorithm show better consistency in comparison with state-of-the-art methods.

Feature selection using hybrid poor and rich optimization algorithm for text classification

In order to reduce the high dimensional feature space in the text classification, feature selection plays a significant role. The dimension reduction of feature space reduces the computation cost and improves the text classification system accuracy. Hence, the identification of an optimal combination of features is an essential task in text classification. In this paper, the proposed work introduces a novel hybrid feature selection method based on binary poor and rich optimization algorithm (HBPRO) to obtain the appropriate subset of optimal features. The optimal feature subset which is selected by our proposed work is evaluated using Nave Bayes classifier with two popular benchmark text corpus datasets. The experimental results confirm that the proposed feature selection scheme (HBPRO) produces higher accuracy with a reduced number of features when compared with other feature selection techniques.

Investigation of low dimensional feature spaces for automatic sleep staging

Background: Automatic sleep stage classification depends crucially on the selection of a good set of descriptive features. However, the selection of a feature set with an appropriate low computational cost without compromising classification performance is still a challenge. This study attempts to represent sleep EEG patterns using a minimum number of features, without significant performance loss. Methods: Three feature selection algorithms were applied to a high dimensional feature space comprising 84 features. These methods were based on a bootstrapping approach guided by Gini ranking and mutual information between the features. The algorithms were tested on three scalp electroencephalography (EEG) and one ear-EEG datasets. The relationship between the information carried by different features was investigated using mutual information and illustrated by a graphical clustering tool. Results: The minimum number of features that can represent the whole feature set without performance loss was found to range between 5 and 11 for different datasets. In ear-EEG, 7 features based on Continuous Wavelet Transform (CWT) resulted in similar performance as the whole set whereas in two scalp EEG datasets, the difference between minimal CWT set and the whole set was statistically significant (0.008 and 0.017 difference in average kappa). Features were divided into groups according to the type of information they carry. The group containing relative power features was identified as the most informative feature group in sleep stage classification, whereas the group containing non-linear features was found to be the least informative. Conclusions: This study showed that EEG sleep staging can be performed based on a low dimensional feature space without significant decrease in sleep staging performance. This is especially important in the case of wearable devices like ear-EEG where low computational complexity is needed. The division of the feature space into groups of features, and the analysis of the distribution of feature groups for different feature set sizes, is helpful in the selection of an appropriate feature set.

Optimal multi-kernel local fisher discriminant analysis for feature dimensionality reduction and fault diagnosis

To achieve a more desirable fault diagnosis accuracy by applying multi-domain features of vibration signals, it is significative and challenging to refine the most representative and intrinsic feature components from the original high dimensional feature space. A novel dimensionality reduction method for fault diagnosis is proposed based on local Fisher discriminant analysis (LFDA) which takes both label information and local geometric structure of the high dimensional features into consideration. Multi-kernel trick is introduced into the LFDA to improve its performance in dealing with the nonlinearity of mapping high dimensional feature space into a lower one. To obtain an optimal diagnosis accuracy by the reduced features of low dimensionality, binary particle swarm optimization (BPSO) algorithm is utilized to search for the most appropriate parameters of kernels and K-nearest neighbor (kNN) recognition model. Samples with labels are used to train the optimal multi-kernel LFDA and kNN (OMKLFDA-kNN) fault diagnosis model to obtain the optimal transformation matrix. Consequently, the trained fault diagnosis model implements the recognition of machinery health condition with the most representative feature space of vibration signals. A bearing fault diagnosis experiment is conducted to verify the effectiveness of proposed diagnostic approach. Performance comparison with some other methods are investigated, and the improvement for fault diagnosis of the proposed method are confirmed in different aspects.

Symbolic Sequence Classification in the Fractal Space

Sequence classification has a range of applications and attracted a lot of attentions. Different from feature vectors, symbolic sequences have no explicit features. Due to this limitation, even with sophisticated feature selection techniques, the dimension of potential feature space could be very high, making classification methods hard to capture the nature of sequences. In this paper, we propose a novel scheme that first constructs a new lower dimensional representation space for symbolic sequences. Next, we carry out learning in this newly generated space rather than on the sequences. The first step is implemented with a chaos game representation, which converts a long sequence into a graphical form by applying an iterated function system on the input. For this reason, this new target space is referred as “fractal space” in this paper. The second step consists of carrying out sequence comparison and quantitative analysis with an alignment-free measure based on the holistic features from the sequences. This scheme is highly flexible and could permit high-performance implementation. The experimental results demonstrate the effectiveness and efficiency of our proposed approach.

SVM Model to Predict the Water Quality Based on Physicochemical Parameters

Analysis of water quality is a very important and challenging task in the management of water bodies and requires immediate attention as it adversely affects the health of living beings. Three parameters namely, pH, Total Dissolved Solids (TDS), and Turbidity were used for data analysis. In this study for mapping of training samples from input space to higher dimensional feature space, LibSVM, (a library of SVM) was used with the use of two kernel function types Radial Basis Function and Polynomial function. For performing the experiment, the three parameter combinations (C, d, ϒ) were evaluated based upon the kernel by taking various range values to obtain the best type of kernel functions through a 10-fold cross-validation process. After performing all experiments, a comparative analysis was done to evaluate the best parameter combination (C, d, ϒ) and the values of performance measures. The result shows that the optimum model developed using LibSVM with the use of Polynomial Kernel function which gives an accuracy of 99.434% in predicting water quality.

A deep neural network classifier for P300 BCI speller based on Cohen’s class time-frequency distribution

This paper presents a new method of predicting the P300 component of an electroencephalography (EEG)signal to recognize the characters in a P300 brain-computer interface (BCI) speller accurately. This method consistsof a deep learning model and the nonlinear time-frequency features. It is believed that the combination of the deepmodel network and extracting the nonlinear features of the EEG led this research to a better prediction of the P300and, therefore, character recognition. Cohen's class distribution is used in order to extract the nonlinear features of theEEG. Evaluating all of the kernels, Butterworth found to be more informative and it produced better results. Basedon the differences observed between time-frequency responses of target and nontarget signals, specific subbands areselected to extract seven features. A deep-structured neural network, namely stacked sparse autoencoders, is appliedfor BCI character recognition. This deep network reduces the dimension of feature space by extracting unsupervisedfeatures. Then, the features are fed to a Softmax classifier. Afterward, the whole network passes a fine-tuning phase by asupervised backpropagation algorithm. For evaluating the work, Dataset II of BCI Competition III is utilized. Based onthe results, this approach would improve the accuracy in both P300 detection and character recognition. This researchresults in 82.7% and 93.5% accuracy for P300 classification and character recognition, respectively.

Feature Selection for Text Classification Using Machine Learning Approaches

In the present scenario, millions of internet users are contributing a huge amount of data in the form of unstructured text documents. In text classification, the high dimensional feature space, noise and irrelevant information of unstructured text documents are reducing the accuracy of text classifier. The feature selection scheme is adopted to address the high dimensional feature space problem of text classification. In this proposed research, a feature selection method based on the term frequency distribution measure is deployed. We have used the Naive Bayes and SVM classifiers with two benchmark datasets (WebKB and BBC). The experimental outcome confirms that the proposed feature selection method has a better classification accuracy when compared with other feature selection techniques.

Pavement crack detection and classification based on fusion feature of LBP and PCA with SVM

ABSTRACT A new crack detection approach based on local binary patterns (LBP) with support vector machine (SVM) was proposed in this paper. The propsed algorithm can extract the LBP feature from each frame of the video taken from the road. Then, the dimension of the LBP feature spaces can be reduced by Principal Component Analysis(PCA). The simplified samples are trained to be decided the type of crack using Support Vector Machine(SVM). In order to reflect the directional imformation in detail, the LBP processed image is devided into nine sub-blocks. In this paper, driving tests were performed 10 times and 12,000 image data were applied to the proposed algorithm. The average accuracy of the proposed algorithm with sub-blocks is 91.91%, which is about 6.6% higher than the algorithm without sub-blocks. The LBP-PCA with SVM applying sub-blocks reflects the directional information of the crack so that it has high accuracy of 89.41% and 88.24%, especially in transverse and longitudinal cracks. In the performance analysis of different crack classifiers, the F-Measure, which considered balance between the precision and the recall, of alligator cracks classifier was the highest at 0.7601 and hence crack detection performance is higher than others.

Evaluating the phase dynamics of coupled oscillators via time-variant topological features.

By characterizing the phase dynamics in coupled oscillators, we gain insights into the fundamental phenomena of complex systems. The collective dynamics in oscillatory systems are often described by order parameters, which are insufficient for identifying more specific behaviors. To improve this situation, we propose a topological approach that constructs the quantitative features describing the phase evolution of oscillators. Here, the phase data are mapped into a high-dimensional space at each time, and the topological features describing the shape of the data are subsequently extracted from the mapped points. These features are extended to time-variant topological features by adding the evolution time as an extra dimension in the topological feature space. The time-variant features provide crucial insights into the evolution of phase dynamics. Combining these features with the kernel method, we characterize the multiclustered synchronized dynamics during the early evolution stages. Finally, we demonstrate that our method can qualitatively explain chimera states. The experimental results confirmed the superiority of our method over those based on order parameters, especially when the available data are limited to the early-stage dynamics.

Performance improvement of extreme multi-label classification using K-way tree construction with parallel clustering algorithm

eXtreme Multi-Label Classification (XMLC) is the particular case of Multi-Label Classification, which deals with an extremely high number of labels. The main goal is to learn extreme classifier which extracts the subset of relevant labels from extremely large label space. In an extreme environment, one of the big issues is to deal with an extreme number of features, labels and instances which affects the performance of the classifier. The high dimensional feature space and label space makes existing approaches intractable in terms of data scalability, data sparsity, training and prediction cost. The appropriate input representation technique can be used to maintain the interdependency among labels and correlation between feature space and label space. The proposed approach called “K-way Tree based eXtreme Multi-Label Classifier (KTXMLC)“ works on tree based classifier to maintain correlations using feature-label input representation technique and node partitioning using a clustering algorithm. KTXMLC constructs multi-way multiple trees using a parallel clustering algorithm, which leads to fast computational cost. KTXMLC outperforms over the existing tree based classifier in terms of ranking based measures on six datasets named Delicious, Mediamill, Eurlex-4K, Wiki10-31K, AmazonCat-13K, Delicious-200K.

Cross-view kernel collaborative representation classification for person re-identification

Currently, person re-identification (re-ID) has been applied in many public security applications. Yet owing to the big visual appearance changes of the same identity under different views, re-ID still faces many challenges. To reduce the intra-person discrepancy, extracting more power feature representations from pedestrian images is a reasonable solution. We propose a cross-view kernel collaborative representation based classification (CV-KCRC) method for person re-ID in our work. Our method aims to find more robust and discriminative feature representations that embody cross-view information to enhance the identification capability of features. We map the image features into a high dimensional feature space first and then use view-specific projection matrices to project the high dimensional features into a common low dimensional subspace. We expect that in the shared subspace the codings of same person from different views have the highest similarity and better performance can be achieved. Experiments on seven commonly used datasets reveal that our algorithm outperforms many state-of-the-art algorithms.

Abstract PS5-13: Holistic artificial intelligence-driven predictor in HER2-positive (HER2+) early breast cancer (BC) treated with neoadjuvant lapatinib and trastuzumab without chemotherapy: A correlative analysis from SOLTI-1114 PAMELA

Abstract Background: Positive partial response was observed in patients with primary HER2+ early BC with dual HER2 blockade that were not treated with chemotherapy. In this context and beyond, the low (partial) response rate of chemotherapy-free treatment strategies creates the necessity for patient stratification prior to treatment selection. Here, we tackle the challenging task of evaluating the ability of clinical, gene expression and histopathology data to predict response following dual HER2 blockade without chemotherapy. Our aim is through artificial intelligence to automatically decipher the complementarity of clinical, genomic and histopathology data through an evidence-driven approach towards a low dimensional holistic signature that determines outcomes and could be subsequently used as a clinical biomarker for treatment patient inclusion. Methods: PAMELA (Lancet Oncology 2017) was a prospective study in HER2+ BC designed to evaluate the ability of the PAM50 HER2-enriched intrinsic subtype to predict pCR following 18-weeks of neoadjuvant lapatinib and trastuzumab (and hormonal therapy if hormone receptor-positive [HR+]). Clinical-pathological variables (15) were included such as tumor cellularity, tumor-infiltrating lymphocytes (TILs), the expression of BC-related genes/signatures (567) along with histopathological data from pre-treatment samples. Imaging information was obtained from H/E slides, through an unsupervised deep learning approach using an attention network. The semantic segmentation was used to derive at the patch level image and shape characteristics resulted on a pathomics-derived feature vector of (300) variables. An integrative approach that harnessed clinical, genomics and pathomics data into a unified prediction framework was used. Patients were divided into a training set 80% and a testing set 20% with proportions of pCR and non-pCR corresponding to the ones observed. A 100-fold Cross-validation (CV) was performed on the training. Linear and non-linear robust feature selection were used to recover a low dimensional holistic signature along with an ensemble learning approach to select the top 5 machine learning/artificial intelligence methods for prognosis. Results: From the high dimensional feature (882) space, a low dimensional holistic signature of 8 predictive variables was automatically retrieved through. The signature consisted of 4 genomics variables (expression levels of ERBB2, ESR1, Luminal A signature and Risk of Relapse score), 2 clinical-pathological variables (histologic grade and ER-status) and 2 imaging variables (mean Short Run Low Gray Level Emphasis of the gray level run length matrix and the mean absolute deviation). To ensure the robustness and generalizability of the results, we present results averaged over 100 splits into training and test. On all cases the same holistic signature was uses and the same prediction methods/principles. The proposed AI-driven prognosis mechanism reached 75% balanced accuracy, 69% precision, 65% sensitivity, 86% specificity and 0.84 AUC demonstrating the relevance of the approach. It was observed a successful classification of 86% for the non-pCR and 65% for the pCR cases. Ablation studies were performed to determine the relevance of the different categories of variables. Genomics variables were the most informative since their removal led to the highest decrease of the metrics (11% in average). Conclusion: The proposed method has great potentials for an effective and clinically meaningful implementation of pre-selecting patients that will not achieve a pCR after neoadjuvant dual HER2 blockade. Besides, the generality of the method used here makes it transposable to any type of cancer or therapy. Citation Format: Enzo Batistella, Laia Paré, Mihir Sahasrabudhe, Tomás Pascual, Maria Vakalopoulou, Patricia Villagrasa, Eric Deutsch, Núria Chic, Guillermo Villacampa, Paolo Nuciforo, Javier Cortes, Antonio Llombart-Cussac, Nikos Paragios, Aleix Prat. Holistic artificial intelligence-driven predictor in HER2-positive (HER2+) early breast cancer (BC) treated with neoadjuvant lapatinib and trastuzumab without chemotherapy: A correlative analysis from SOLTI-1114 PAMELA [abstract]. In: Proceedings of the 2020 San Antonio Breast Cancer Virtual Symposium; 2020 Dec 8-11; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2021;81(4 Suppl):Abstract nr PS5-13.