High-dimensional Feature Space Research Articles

Near-optimal quantum kernel principal component analysis

Abstract Kernel principal component analysis (kernel PCA) is a nonlinear dimensionality reduction technique that employs kernel functions to map data into a high-dimensional feature space, thereby extending the applicability of linear PCA to nonlinear data and facilitating the extraction of informative principal components. However, kernel PCA necessitates the manipulation of large-scale matrices, leading to high computational complexity and posing challenges for efficient implementation in big data environments. Quantum computing has recently been integrated with kernel methods in machine learning, enabling effective analysis of input data within intractable feature spaces. Although existing quantum kernel PCA proposals promise exponential speedups, they impose stringent requirements on quantum hardware that are challenging to fulfill. In this work, we propose a quantum algorithm for kernel PCA by establishing a connection between quantum kernel methods and block encoding, thereby diagonalizing the centralized kernel matrix on a quantum computer. The query complexity is logarithmic with respect to the size of the data vector, $D$, and linear with respect to the size of the dataset. An exponential speedup could be achieved when the dataset consists of a few high-dimensional vectors, wherein the dataset size is polynomial in $\log(D)$, with $D$ being significantly large. In contrast to existing work, our algorithm enhances the efficiency of quantum kernel PCA and reduces the requirements for quantum hardware. Furthermore, we have also demonstrated that the algorithm based on block encoding matches the lower bound of query complexity, indicating that our algorithm is nearly optimal. Our research has laid down new pathways for developing quantum machine learning algorithms aimed at addressing tangible real-world problems and demonstrating quantum advantages within machine learning.

Kernel machine tests of association using extrinsic and intrinsic cluster evaluation metrics

Modeling the network topology of the human brain within the mesoscale has become an increasing focus within the neuroscientific community due to its variation across diverse cognitive processes, in the presence of neuropsychiatric disease or injury, and over the lifespan. Much research has been done on the creation of algorithms to detect these mesoscopic structures, called communities or modules, but less has been done to conduct inference on these structures. The literature on analysis of these community detection algorithms has focused on comparing them within the same subject. These approaches, however, either do not accomodate a more general association between community structure and an outcome or cannot accommodate additional covariates that may confound the association of interest. We propose a semiparametric kernel machine regression model for either a continuous or binary outcome, where covariate effects are modeled parametrically and brain connectivity measures are measured nonparametrically. By incorporating notions of similarity between network community structures into a kernel distance function, the high-dimensional feature space of brain networks, defined on input pairs, can be generalized to non-linear spaces, allowing for a wider class of distance-based algorithms. We evaluate our proposed methodology on both simulated and real datasets.

Performance of uncertainty-based active learning for efficient approximation of black-box functions in materials science.

Obtaining a fine approximation of a black-box function is important for understanding and evaluating innovative materials. Active learning aims to improve the approximation of black-box functions with fewer training data. In this study, we investigate whether active learning based on uncertainty sampling enables the efficient approximation of black-box functions in regression tasks using various material databases. In cases where the inputs are provided uniformly and defined in a relatively low-dimensional space, the liquidus surfaces of the ternary systems are the focus. The results show that uncertainty-based active learning can produce a better black-box function with higher prediction accuracy than that by random sampling. Furthermore, in cases in which the inputs are distributed discretely and unbalanced in a high-dimensional feature space, datasets extracted from materials databases for inorganic materials, small molecules, and polymers are addressed, and uncertainty-based active learning is occasionally inefficient. Based on the dependency on the material descriptors, active learning tends to produce a better black-box functions than random sampling when the dimensions of the descriptor are small. The results indicate that active learning is occasionally inefficient in obtaining a better black-box function in materials science.

Elliptic geometry-based kernel matrix for improved biological sequence classification

Protein sequence classification plays a pivotal role in bioinformatics as it enables the comprehension of protein functions and their involvement in diverse biological processes. While numerous machine learning models have been proposed to tackle this challenge, traditional approaches face limitations in capturing the intricate relationships and hierarchical structures inherent in genomic sequences. These limitations stem from operating within high-dimensional non-Euclidean spaces. To address this issue, we introduce the application of the elliptic geometry-based approach for protein sequence classification. First, we transform the problem in elliptic geometry and integrate it with the Gaussian kernel to map the problem into the Mercer kernel. The Gaussian-Elliptic approach allows for the implicit mapping of data into a higher-dimensional feature space, enabling the capture of complex nonlinear relationships. This feature becomes particularly advantageous when dealing with hierarchical or tree-like structures commonly encountered in biological sequences. Experimental results highlight the effectiveness of the proposed model in protein sequence classification, showcasing the advantages of utilizing elliptic geometry in bioinformatics analyses. It outperforms state-of-the-art methods by achieving 76% and 84% accuracies for DNA and Protein datasets, respectively. Furthermore, we provide theoretical justifications for the proposed model. This study contributes to the burgeoning field of geometric deep learning, offering insights into the potential applications of elliptic representations in the analysis of biological data.

Mcadet: A feature selection method for fine-resolution single-cell RNA-seq data based on multiple correspondence analysis and community detection.

Single-cell RNA sequencing (scRNA-seq) data analysis faces numerous challenges, including high sparsity, a high-dimensional feature space, and biological noise. These challenges hinder downstream analysis, necessitating the use of feature selection methods to identify informative genes, and reduce data dimensionality. However, existing methods for selecting highly variable genes (HVGs) exhibit limited overlap and inconsistent clustering performance across benchmark datasets. Moreover, these methods often struggle to accurately select HVGs from fine-resolution scRNA-seq datasets and minority cell types, which are more difficult to distinguish, raising concerns about the reliability of their results. To overcome these limitations, we propose a novel feature selection framework for scRNA-seq data called Mcadet. Mcadet integrates Multiple Correspondence Analysis (MCA), graph-based community detection, and a novel statistical testing approach. To assess the effectiveness of Mcadet, we conducted extensive evaluations using both simulated and real-world data, employing unbiased metrics for comparison. Our results demonstrate the superior performance of Mcadet in the selection of HVGs in scenarios involving fine-resolution scRNA-seq datasets and datasets containing minority cell populations. Overall, we demonstrate that Mcadet enhances the reliability of selected HVGs, although the impact of HVG selection on various downstream analyses varies and needs to be further investigated.

The Application of a Random Forest Classifier to ToF-SIMS Imaging Data.

Time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging is a potent analytical tool that provides spatially resolved chemical information on surfaces at the microscale. However, the hyperspectral nature of ToF-SIMS datasets can be challenging to analyze and interpret. Both supervised and unsupervised machine learning (ML) approaches are increasingly useful to help analyze ToF-SIMS data. Random Forest (RF) has emerged as a robust and powerful algorithm for processing mass spectrometry data. This machine learning approach offers several advantages, including accommodating nonlinear relationships, robustness to outliers in the data, managing the high-dimensional feature space, and mitigating the risk of overfitting. The application of RF to ToF-SIMS imaging facilitates the classification of complex chemical compositions and the identification of features contributing to these classifications. This tutorial aims to assist nonexperts in either machine learning or ToF-SIMS to apply Random Forest to complex ToF-SIMS datasets.

Neuronal parts list and wiring diagram for a visual system

A catalogue of neuronal cell types has often been called a ‘parts list’ of the brain1, and regarded as a prerequisite for understanding brain function2,3. In the optic lobe of Drosophila, rules of connectivity between cell types have already proven to be essential for understanding fly vision4,5. Here we analyse the fly connectome to complete the list of cell types intrinsic to the optic lobe, as well as the rules governing their connectivity. Most new cell types contain 10 to 100 cells, and integrate information over medium distances in the visual field. Some existing type families (Tm, Li, and LPi)6–10 at least double in number of types. A new serpentine medulla (Sm) interneuron family contains more types than any other. Three families of cross-neuropil types are revealed. The consistency of types is demonstrated by analysing the distances in high-dimensional feature space, and is further validated by algorithms that select small subsets of discriminative features. We use connectivity to hypothesize about the functional roles of cell types in motion, object and colour vision. Connectivity with ‘boundary types’ that straddle the optic lobe and central brain is also quantified. We showcase the advantages of connectomic cell typing: complete and unbiased sampling, a rich array of features based on connectivity and reduction of the connectome to a substantially simpler wiring diagram of cell types, with immediate relevance for brain function and development.

Low-Light Image Enhancement via Dual Information-Based Networks

Recently, deep-learning-based low-light image enhancement (LLIE) methods have made great progress. Benefiting from elaborately designed model architectures, these methods enjoy considerable performance gains. However, the generalizability of these methods may be weak, and they may suffer from the overfitting risk in the case of insufficient data as a result. At the same time, their complex model design brings serious computational burdens. To further improve performance, we exploit dual information, including spatial and channel (contextual) information, in the high-dimensional feature space. Specifically, we introduce customized spatial and channel blocks according to the feature difference of different layers. In shallow layers, the feature resolution is close to that of the original input image, and the spatial information is well preserved. Therefore, the spatial restoration block is designed for leveraging such precise spatial information to achieve better spatial restoration, e.g., revealing the textures and suppressing the noise in the dark. In deep layers, the features contain abundant contextual information, which is distributed in various channels. Hence, the channel interaction block is incorporated for better feature interaction, resulting in stronger model representation capability. Combining the U-Net-like model with the customized spatial and channel blocks makes up our method, which effectively utilizes dual information for image enhancement. Through extensive experiments, we demonstrate that our method, despite its simplicity of design, can provide advanced or competitive performance compared to some state-of-the-art deep learning- based methods.

Specific Emitter Identification Algorithm Based on Time–Frequency Sequence Multimodal Feature Fusion Network

Specific emitter identification is a challenge in the field of radar signal processing. Its aims to extract individual fingerprint features of the signal. However, early works are all designed using either signal or time–frequency image and heavily rely on the calculation of hand-crafted features or complex interactions in high-dimensional feature space. This paper introduces the time–frequency multimodal feature fusion network, a novel architecture based on multimodal feature interaction. Specifically, we designed a time–frequency signal feature encoding module, a wvd image feature encoding module, and a multimodal feature fusion module. Additionally, we propose a feature point filtering mechanism named FMM for signal embedding. Our algorithm demonstrates high performance in comparison with the state-of-the-art mainstream identification methods. The results indicate that our algorithm outperforms others, achieving the highest accuracy, precision, recall, and F1-score, surpassing the second-best by 9.3%, 8.2%, 9.2%, and 9%. Notably, the visual results show that the proposed method aligns with the signal generation mechanism, effectively capturing the distinctive fingerprint features of radar data. This paper establishes a foundational architecture for the subsequent multimodal research in SEI tasks.

Machine-Learning Predictions of Critical Temperatures from Chemical Compositions of Superconductors.

In the quest for advanced superconducting materials, the accurate prediction of critical temperatures (Tc) poses a formidable challenge, largely due to the complex interdependencies between superconducting properties and the chemical and structural characteristics of a given material. To address this challenges, we have developed a machine-learning framework that aims to elucidate these complicated and hitherto poorly understood structure-property and property-property relationships. This study introduces a novel machine-learning-based workflow, termed the Gradient Boosted Feature Selection (GBFS), which has been tailored to predict Tc for superconductors by employing a distributed gradient-boosting framework. This approach integrates exploratory data analyses, statistical evaluations, and multicollinearity reduction techniques to select highly relevant features from a high-dimensional feature space, derived solely from the chemical composition of materials. Our methodology was rigorously tested on a data set comprising approximately 16,400 chemical compounds with around 12,000 unique chemical compositions. The GBFS workflow enabled the development of a classification model that distinguishes compositions likely to exhibit Tc values greater than 10 K. This model achieved a weighted average F1-score of 0.912, an AUC-ROC of 0.986, and an average precision score of 0.919. Additionally, the GBFS workflow underpinned a regression model that predicted Tc values with an R2 of 0.945, an MAE of 3.54 K, and an RMSE of 6.57 K on a test set obtained via random splitting. Further exploration was conducted through out-of-sample Tc predictions, particularly those exceeding the liquid nitrogen temperature, and out-of-distribution predictions for (Ca1-xLax)FeAs2 based on varying lanthanum content. The outcome of our study underscores the significance of systematic feature analysis and selection in enhancing predictive model performance, offering various advantages over models that rely primarily on algorithmic complexity. This research not only advances the field of superconductivity but also sets a precedent for the application of machine learning in materials science.

Model-free detection and quantitative assessment of micro short circuits in lithium-ion battery packs based on incremental capacity and unsupervised clustering

Timely diagnosis of micro short circuit (MSC) faults is crucial for ensuring the safe operation of lithium-ion battery energy storage systems. Existing diagnostic methods face limitations such as high dependency on battery models, difficulty in determining accurate diagnostic thresholds, or low computational efficiency. This work presents a model-free approach for the detection and quantitative assessment of MSCs in lithium-ion battery packs, with incremental capacity (IC) and unsupervised clustering. First, the IC is extracted from charging voltage data to effectively characterize MSC faults in lithium-ion batteries. Next, principal component analysis is used to map the high-dimensional feature space onto a two-dimensional plane to facilitate fault detection and result visualization. Then, an unsupervised clustering algorithm is employed for anomaly detection to identify MSC cells within the battery pack. For the detected MSC cells, a method based on the maximum charging voltage difference between adjacent cycles is designed to estimate the MSC resistance, quantitatively assessing the severity and evolution stage of the MSC. Experimental results show that the accuracy of MSC detection is 99.17 % and the minimum relative error of short-circuit resistance estimation is 1.20 %, which demonstrates the effectiveness and feasibility of the proposed method.

ECF-Net: Enhanced, Channel-Based, Multi-Scale Feature Fusion Network for COVID-19 Image Segmentation

Accurate segmentation of COVID-19 lesion regions in lung CT images aids physicians in analyzing and diagnosing patients’ conditions. However, the varying morphology and blurred contours of these regions make this task complex and challenging. Existing methods utilizing Transformer architecture lack attention to local features, leading to the loss of detailed information in tiny lesion regions. To address these issues, we propose a multi-scale feature fusion network, ECF-Net, based on channel enhancement. Specifically, we leverage the learning capabilities of both CNN and Transformer architectures to design parallel channel extraction blocks in three different ways, effectively capturing diverse lesion features. Additionally, to minimize irrelevant information in the high-dimensional feature space and focus the network on useful and critical information, we develop adaptive feature generation blocks. Lastly, a bidirectional pyramid-structured feature fusion approach is introduced to integrate features at different levels, enhancing the diversity of feature representations and improving segmentation accuracy for lesions of various scales. The proposed method is tested on four COVID-19 datasets, demonstrating mIoU values of 84.36%, 87.15%, 83.73%, and 75.58%, respectively, outperforming several current state-of-the-art methods and exhibiting excellent segmentation performance. These findings provide robust technical support for medical image segmentation in clinical practice.

HEnsem_DTIs: A heterogeneous ensemble learning model for drug-target interactions prediction

Drug discovery is the process by which a drug is discovered. Drug-target interactions prediction is a major part of drug discovery. Unfortunately, producing new drugs is time-consuming and expensive; Because it requires a lot of human and laboratory resources. Recently, predictions have been made using computational methods to solve these problems and prevent blindly examining all interactions. Various experiences using computational methods show that no single algorithm can be suitable for all applications; Hence, ensemble learning is expressed. Although various ensemble methods have been proposed, it is still not easy to find a suitable ensemble method for a particular dataset. In general, the existing algorithms in aggregation and combination method are selected manually based on experience. Reinforcement learning can be one way to meet this challenge. High-dimensional feature space and class imbalance are among the challenges of drug-target interactions prediction. This paper proposes HEnsem_DTIs, a heterogeneous ensemble model, for predicting drug-target interactions using dimensionality reduction and concepts of recommender systems to address these challenges. HEnsem_DTIs is configured with reinforcement learning. Dimensionality reduction is applied to handle the challenge of high-dimensional feature space and recommender systems to improve under-sampling and solve the class imbalance challenge. Six datasets are used to evaluate the proposed model; Results of the evaluation on datasets show that HEnsem_DTIs works better than other models in this field. Results of evaluation of the proposed model on the first dataset using 10-fold cross-validation experiments show the amount of sensitivity 0.896, specificity 0.954, GM 0.924, AUC 0.930 and AUPR 0.935.

Distributed monitoring of nonlinear plant-wide processes based on GA-regularized kernel canonical correlation analysis

Fault detection and diagnosis is important for ensuring process safety and is gaining increasing attention in the system safety field. A regularized kernel canonical correlation analysis (RKCCA) approach is proposed for monitoring nonlinear plantwide processes. For each local unit, genetic algorithm (GA)-based regularization is performed to determine the communication variables from neighboring units, which preserves the maximum correlations and eliminates the irrelevant variables. Then variables from a local unit and the communication variables are mapped into high-dimensional feature spaces, and the feature space of the local unit is decomposed into three orthogonal subspaces, namely the residual subspace, the inner subspace, and the outer-related subspace. Monitoring statistics to identify both the process status and the characteristic of a detected fault are constructed. The proposed RKCCA-based monitoring method considers both the information of a local unit and the beneficial information of related units to facilitate fault detection, thereby exhibiting superior performance to some state-of-the-arts methods. Applications on the Tennessee Eastman benchmark process and an industrial tail gas treatment process demonstrate the superiority of RKCCA monitoring.

An improved equilibrium optimization algorithm for feature selection problem in network intrusion detection

In this paper, an enhanced equilibrium optimization (EO) version named Levy-opposition-equilibrium optimization (LOEO) is proposed to select effective features in network intrusion detection systems (IDSs). The opposition-based learning (OBL) approach is applied by this algorithm to improve the diversity of the population. Also, the Levy flight method is utilized to escape local optima. Then, the binary rendition of the algorithm called BLOEO is employed to feature selection in IDSs. One of the main challenges in IDSs is the high-dimensional feature space, with many irrelevant or redundant features. The BLOEO algorithm is designed to intelligently select the most informative subset of features. The empirical findings on NSL-KDD, UNSW-NB15, and CIC-IDS2017 datasets demonstrate the effectiveness of the BLOEO algorithm. This algorithm has an acceptable ability to effectively reduce the number of data features, maintaining a high intrusion detection accuracy of over 95%. Specifically, on the UNSW-NB15 dataset, BLOEO selected only 10.8 features on average, achieving an accuracy of 97.6% and a precision of 100%.

A portable affective computing system for identifying mate preference

Recognizing an individual’s preference state for potential romantic partners based on electroencephalogram (EEG) signals holds significant practical value in enhancing matchmaking success rates and preventing romance fraud. Despite some progress has been made in this field, challenges such as high-dimensional feature space and channel redundancy limited the technology’s practical application. The aim of this study is to explore the most discriminative EEG features and channels, in order to enhance the recognition performance of the system, while maximizing the portable and practical value of EEG-based systems for recognizing romantic attraction. To achieve this goal, we first conducted an interesting simulated dating experiment to collect the necessary data. Next, EEG features were extracted from various dimensions, including band power and asymmetry index features. Then, we introduced a novel method for EEG feature and channel selection that combines the sequential forward selection (SFS) algorithm with the frequency-based feature subset integration (FFSI) algorithm. Finally, we used the random forest classifier (RFC) to determine a person's preference state for potential romantic partners. Experimental results indicate that the optimal feature subset, selected using the SFS-FFSI method, attained an average classification accuracy of 88.42%. Notably, these features were predominantly sourced from asymmetry index features of electrodes situated in the frontal, parietal, and occipital lobes.

AN AUTOMATED NEW APPROACH IN FAST TEXT CLASSIFICATION: A CASE STUDY FOR KURDISH TEXT

With the rapid development of internet technology, text classification has become a vital part of obtaining quick and accurate data. Traditional machine learning methods often suffer from poor performance and high-dimensional feature spaces, which reduce their accuracy. In this paper, the FastText model is proposed as the first-ever classifier on Kurdish text and the results are compared with traditional machine learning methods to show the effects on Kurdish Text. For evaluating the model four datasets Kurdish News Dataset Headlines (KNDH), Medical Kurdish Dataset (MKD), Kurdish-Emotional-Dataset (KMD-77000), and KurdiSent are utilized and compared the results with the traditional machine learning algorithms such as: Random Forest (RF), k-nearest Neighbor (k-NN), Logistic Regression (LR), Multinomial Naïve Bayes (MNB), Support Vector Machine (SVM), Decision Tree (DT), Stochastic Gradient Descent (SGD), as well as the deep learning model Bidirectional Encoder Representations from Transformers (BERT). The outcomes indicate that the FastText model achieved the highest performance with 89% for each precision, recall, F1-score, and 89.10% accuracy for the KNDH dataset. Moreover, when the KMD dataset is utilized the FatText model obtained outperforms all others by approximately 2%. In addition, the comparative analysis showed that FastText is superior when Kurdisent is considered with precision, recall, F1-score, and accuracy by 81.32, 81.83, 81.57, and 81.4 respectively. In addition, when MKD is implemented, the FastText model obtained the highest performance with a precision of 93.32%, recall of 93.36, F1-score of 93.34, and accuracy of 93.1%.

Hybrid genetic optimization for quantum feature map design

Kernel methods are an import class of techniques in machine learning. To be effective, good feature maps are crucial for mapping non-linearly separable input data into a higher dimensional (feature) space, thus allowing the data to be linearly separable in feature space. Previous work has shown that quantum feature map design can be automated for a given dataset using NSGA-II, a genetic algorithm, while both minimizing circuit size and maximizing classification accuracy. However, the evaluation of the accuracy achieved by a candidate feature map is costly. In this work, we demonstrate the suitability of kernel-target alignment as a substitute for accuracy in genetic algorithm-based quantum feature map design. Kernel-target alignment is faster to evaluate than accuracy and does not require some data points to be reserved for its evaluation. To further accelerate the evaluation of genetic fitness, we provide a method to approximate kernel-target alignment. To improve kernel-target alignment and root mean squared error, the final trainable parameters of the generated circuits are further trained using COBYLA to determine whether a hybrid approach applying conventional circuit parameter training can easily complement the genetic structure optimization approach. A total of eight new approaches are compared to the original across nine varied binary classification problems from the UCI machine learning repository, showing that kernel-target alignment and its approximation produce feature map circuits enabling comparable accuracy to the previous work but with larger margins on training data (in excess of 20% larger) that improve further with circuit parameter training.

An optimization framework with dimensionality reduction using Markov Chain Monte Carlo and genetic algorithms for groundwater potential assessment

Limited samples and high-dimensional feature spaces often hinder the accuracy of machine learning (ML) models in regional groundwater potential assessment (GPA). This study proposes a novel framework, the GPA with Dimensionality Optimization (GPADO), that optimizes feature dimension reduction to enhance prediction performance. Taking the Jianghan Basin as an example, data on nine continuous variables and five categorical variables influencing the region's GPA were gathered, expanding the feature set to 37 through One-hot encoding for categorical variables. Three scenarios were devised to assess prediction outcomes following various dimensionality reduction approaches. Comparative analysis revealed that a hybrid dimension reduction method, incorporating both continuous and categorical variables, yielded the highest validation set accuracy. Consequently, genetic algorithm and Markov Chain Monte Carlo methods were employed to determine the optimal solution and uncertainties associated with four unknown parameters: the chosen dimension reduction method for continuous and categorical variables, and the number of dimensions retained. Results indicated that utilizing singular value decomposition to reduce categorical variables to three dimensions, coupled with principal component analysis reducing continuous variables to three dimensions, produced the highest model validation accuracy of 0.834 within the GPADO framework. This optimal configuration facilitated automated ML training, resulting in a final validation set accuracy of 0.851 and a test set accuracy of 0.836. The resulting model provided a more precise spatial distribution of groundwater potential and demonstrated the GPADO framework's effectiveness in improving GPA accuracy, particularly in data-scarce regions. The GPADO framework offers a valuable approach for enhancing GPA studies.

SVR-based method for fixed effects interval-valued panel models

. This article studies a class of fixed effects interval-valued panel models in which the response variables are intervals and the explanatory variables are single data. We propose an SVR-based estimation method to estimate unknown parameters and smoothing functions. The proposed method utilizes the kernel trick to map the original data (also called input data) to a high-dimensional feature space, which is more likely to achieve linearity than in the original space. Moreover, the proposed method is resistant to the influence of outliers. Monte Carlo experiments validate that the proposed models have good fitting accuracy and prediction performance, while the empirical application shows that the proposed models work efficiently on finite samples.