Nonlinear Feature Space Research Articles

Research on Quantum Computing Acceleration of Support Vector Machines in Multi-dimensional Nonlinear Feature Spaces

The current development of Support Vector Machines (SVM) has reached a bottleneck, with issues such as long training times and weak interpretability when dealing with large-scale, multi-dimensional data. This paper introduces the concept of Quantum Support Vector Machines (QSVM) and achieves efficient solutions through quantum algorithms such as the HHL algorithm. The research designs a computational architecture that combines classical and quantum computing, utilizing the Pauli decomposition of Hermitian matrices to simulate the quantum simulation of Hamiltonian quantities and implementing the quantum simulation of unitary operators. Based on the conclusions, a complete quantum linear solver circuit is designed, achieving exponential growth in computational complexity. Experiments using the Iris dataset demonstrate the excellent classification performance of QSVM, which outperform classical SVM in classification accuracy, computational time complexity, and memory requirements. More efficient quantum mapping algorithms and quantum circuit optimization methods are implemented, providing new ideas and methods for the application of SVM to large-scale datasets.

Spectroscopic Phenological Characterization of Mangrove Communities

Spaceborne spectroscopic imaging offers the potential to improve our understanding of biodiversity and ecosystem services, particularly for challenging and rich environments like mangroves. Understanding the signals present in large volumes of high-dimensional spectroscopic observations of vegetation communities requires the characterization of seasonal phenology and response to environmental conditions. This analysis leverages both spectroscopic and phenological information to characterize vegetation communities in the Sundarban riverine mangrove forest of the Ganges–Brahmaputra delta. Parallel analyses of surface reflectance spectra from NASA’s EMIT imaging spectrometer and MODIS vegetation abundance time series (2000–2022) reveal the spectroscopic and phenological diversity of the Sundarban mangrove communities. A comparison of spectral and temporal feature spaces rendered with low-order principal components and 3D embeddings from Uniform Manifold Approximation and Projection (UMAP) reveals similar structures with multiple spectral and temporal endmembers and multiple internal amplitude continua for both EMIT reflectance and MODIS Enhanced Vegetation Index (EVI) phenology. The spectral and temporal feature spaces of the Sundarban represent independent observations sharing a common structure that is driven by the physical processes controlling tree canopy spectral properties and their temporal evolution. Spectral and phenological endmembers reside at the peripheries of the mangrove forest with multiple outward gradients in amplitude of reflectance and phenology within the forest. Longitudinal gradients of both phenology and reflectance amplitude coincide with LiDAR-derived gradients in tree canopy height and sub-canopy ground elevation, suggesting the influence of surface hydrology and sediment deposition. RGB composite maps of both linear (PC) and nonlinear (UMAP) 3D feature spaces reveal a strong contrast between the phenological and spectroscopic diversity of the eastern Sundarban and the less diverse western Sundarban.

TaBE: Decoupling spatial and spectral processing with Taylor’s unfolding method in the beamspace domain for multi-channel speech enhancement

In recent years, significant advancements have been made in neural beamforming, leveraging spectral and spatial cues to enhance their performance in multi-channel speech enhancement. When the frame-wise processing mechanism is required, there exists a trade-off between performance and algorithmic delay for existing all-neural beamformers. However, from the perspective of multi-source information fusion, the network is often encapsulated into a black box to entangle and fuse the spatial and spectral features into a non-linear feature space, which hinders our understanding of how they work collaboratively for target speech extraction. In this regard, this paper proposes to decouple the spatial and spectral domain processing inspired by Taylor’s approximation theory. Specifically, we reformulate the time-variant beamforming defined in the spatial domain into the adaptive weighting and mixing of different beam components in the beamspace domain. This reformulation enables us to model the recovery of target speech as a weighted sum operation in the beamspace domain, where each beam component is associated with an introduced unknown term for residual interference cancellation. By virtue of Taylor’s series expansion, the recovery process can be decomposed into the superimposition of the 0th-order non-derivative and high-order derivative terms, where the former acts as spatial filtering in the spatial domain, and the latter serves as a residual interference canceller in the spectral domain. We conduct extensive experiments on the spatialized LibriSpeech and L3DAS Challenge datasets. Experimental results show that, compared with existing advanced approaches, the proposed method not only achieves competitive performance in terms of multiple objective metrics but also provides feasible guidance in multi-channel speech enhancement pipeline design.

Adaptive Kernel Graph Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is an efficient method for feature learning in the field of machine learning and data mining. To investigate the nonlinear characteristics of datasets, kernel-method-based NMF (KNMF) and its graph-regularized extensions have received much attention from various researchers due to their promising performance. However, the graph similarity matrix of the existing methods is often predefined in the original space of data and kept unchanged during the matrix-factorization procedure, which leads to non-optimal graphs. To address these problems, we propose a kernel-graph-learning-based, nonlinear, nonnegative matrix-factorization method in this paper, termed adaptive kernel graph nonnegative matrix factorization (AKGNMF). In order to automatically capture the manifold structure of the data on the nonlinear feature space, AKGNMF learned an adaptive similarity graph. We formulated a unified objective function, in which global similarity graph learning is optimized jointly with the matrix decomposition process. A local graph Laplacian is further imposed on the learned feature subspace representation. The proposed method relies on both the factorization that respects geometric structure and the mapped high-dimensional subspace feature representations. In addition, an efficient iterative solution was derived to update all variables in the resultant objective problem in turn. Experiments on the synthetic dataset visually demonstrate the ability of AKGNMF to separate the nonlinear dataset with high clustering accuracy. Experiments on real-world datasets verified the effectiveness of AKGNMF in three aspects, including clustering performance, parameter sensitivity and convergence. Comprehensive experimental findings indicate that, compared with various classic methods and the state-of-the-art methods, the proposed AKGNMF algorithm demonstrated effectiveness and superiority.

A Tensor Method based on Enhanced Tensor Nuclear Norm and Hypergraph Laplacian Regularization for Pan-Cancer Omics Data Analysis.

As a powerful data representation technique, tensor robust principal component analysis (TRPCA) has been widely used for clustering and feature selection tasks. However, it ignores the significant difference in singular values of tensor data and the manifold information contained in different views, thereby causing serious degradation of conventional TRPCA performance. In this paper, a novel tensor method based on enhanced tensor nuclear norm and hypergraph Laplacian regularization (ETHLR) is developed to address the above problem. ETHLR can jointly learn the prior knowledge of singular values and high-order manifold structures in the unified tensor space and the view-specific feature spaces, respectively. Specifically, the enhanced tensor nuclear norm, namely, the weighted tensor Schatten p-norm, is used to shrink the singular values by fully considering the salient difference information of singular values and the complementary information embedded in the tensor space; the hypergraph Laplacian constraint helps encode high-order geometric structures among multiple samples in the nonlinear view-specific feature space. Furthermore, we employ inexact augmented Lagrange multipliers (ALM) to optimize the ETHLR method. Numerous experiments on pan-cancer omics data show that the superiority of ETHLR over several state-of-the-art competitors.

Classification via Structure-Preserved Hypergraph Convolution Network for Hyperspectral Image

Graph convolutional network (GCN) as a combination of deep learning and graph learning has gained increasing attention in hyperspectral image (HSI) classification. However, most GCN methods consider the simple point-to-point structure between two pixels rather than the high-order structure of multiple pixels, which is contradict with the real feature distribution of ground object. And the nonlinear property of HSI also brings challenge for precise structural representation in GCN. To tackle these problems, this work proposes a structure preserved hypergraph convolution network (SPHGCN). It first builds a multiple neighborhood reconstruction (MNR) model to reveal the essential resemblance of multiple pixels in nonlinear spectral feature space. With the high-order structure, SPHGCN designs the hypergraph convolution operation for irregular feature aggregation among similar pixels from different regions, which achieves more discriminative features from multiple pixel nodes. Meanwhile, a structure preservation layer is built to optimize the distribution of convolutional features under the guidance of high-order structure. Moreover, SPHGCN integrates local regular convolution and irregular hypergraph convolution to learn the structured semantic feature of HSI. This strategy breaks the boundary restriction in traditional convolution and aggregates semantic feature across different image patches. Experiments on three HSI data sets indicate that SPHGCN outperforms a few state-of-the-art methods for HSI classification.

A Robust Computer-Aided Automated Brain Tumor Diagnosis Approach Using PSO-ReliefF Optimized Gaussian and Non-Linear Feature Space.

Brain tumors are among the deadliest diseases in the modern world. This study proposes an optimized machine-learning approach for the detection and identification of the type of brain tumor (glioma, meningioma, or pituitary tumor) in brain images recorded using magnetic resonance imaging (MRI). The Gaussian features of the image are extracted using speed-up robust features (SURF), whereas its non-linear features are obtained using KAZE, owing to their high performance against rotation, scaling, and noise problems. To retrieve local-level information, all brain MRI images are segmented into an 8 × 8 pixel grid. To enhance the accuracy and reduce the computational time, the variance-based k-means clustering and PSO-ReliefF algorithms are employed to eliminate the redundant features of the brain MRI images. Finally, the performance of the proposed hybrid optimized feature vector is evaluated using various machine learning classifiers. An accuracy of 96.30% is obtained with 169 features using a support vector machine (SVM). Furthermore, the computational time is also reduced to 1 min compared to the non-optimized features used for training of the SVM. The findings are also compared with previous research, demonstrating that the suggested approach might assist physicians and doctors in the timely detection of brain tumors.

An adaptive subspace data-driven method for nonlinear dynamic systems

Quality-related data-driven fault detection methods have been universally recognized by researchers in industry and academia. However, the detection performance of the common data-driven method will degrade if it cannot keep up with the dynamic shifts of the production process. In this paper, an adaptive kernel principal component regression (AKPCR) method is thus proposed to process the nonlinear feature and the time-varying property in the system. The approximate Hilbert space approach is provided to update the projection matrix based on the innovation information from the new data for the nonlinear high-dimensional feature space. Besides, the recursive post-decomposition technique is created from the updated projection matrix to better monitor the quality state. Compared with the fixed threshold, an adaptive one is designed in this study to detect the fault encountered in the time-varying system, which can decrease false alarms in fault detection. Finally, the Tennessee Eastman case (TEC) and the waste water treatment system (WWTS) are leveraged to illustrate the reliability of the proposed approach.

GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability.

Computational visual encoding models play a key role in understanding the stimulus–response characteristics of neuronal populations in the brain visual cortex. However, building such models typically faces challenges in the effective construction of non-linear feature spaces to fit the neuronal responses. In this work, we propose the GaborNet visual encoding (GaborNet-VE) model, a novel end-to-end encoding model for the visual ventral stream. This model comprises a Gabor convolutional layer, two regular convolutional layers, and a fully connected layer. The key design principle for the GaborNet-VE model is to replace regular convolutional kernels in the first convolutional layer with Gabor kernels with learnable parameters. One GaborNet-VE model efficiently and simultaneously encodes all voxels in one region of interest of functional magnetic resonance imaging data. The experimental results show that the proposed model achieves state-of-the-art prediction performance for the primary visual cortex. Moreover, the visualizations demonstrate the regularity of the region of interest fitting to the visual features and the estimated receptive fields. These results suggest that the lightweight region-based GaborNet-VE model based on combining handcrafted and deep learning features exhibits good expressiveness and biological interpretability.

Fracture identification in tight reservoirs by multiple kernel Fisher discriminant analysis using conventional logs

Kernel Fisher discriminant analysis (KFD) can map well-log data into a nonlinear feature space to make a linear nonseparable problem of fracture identification into a linear separable one. Commonly, KFD uses one kernel. However, the prediction capacity of KFD based on one kernel is limited to some extent, especially for a complex classification problem, such as fracture identification in tight sandstone reservoirs. To alleviate this problem, we have used a multiple kernel Fisher discriminant analysis (MKFD) method to recognize fracture zones. MKFD uses multiscaled Gaussian kernel functions instead of a single kernel to realize optimal nonlinear mapping. To assess the effectiveness of MKFD in fracture identification for complex reservoirs, we chose a data set from tight sandstone reservoirs in China to implement comparison experiments. In the experiments, we used the MKFD with 20 Gaussian kernels to map the original well logs into nonlinear feature spaces so that we could obtain appropriate features for fracture identification. The comparison results demonstrated that the accuracy of fracture identification by MKFD improved about 13.4% over KFD and that MKFD also outperformed KFD in the blind well test, although the improvement of the generalization ability of MKFD was not very obvious. Overall, MKFD can provide an accurate means for the identification of fracture zones in tight reservoirs. We also evaluate the problems for fracture identification by MKFD.

Tunable Adaptive Target Detection With Kernels in Colocated MIMO Radar

We introduce a framework for exploring adaptive target detection in colocated MIMO radar in non-linear feature space by exploiting the theory of kernel. The kernel theory inspires us to replace the inner products of test statistics with that of nonlinear mapped data in the feature space or that of their kernel tricks to achieve better detection performance. We apply this framework to the problem of tunable adaptive target detection in colocated MIMO radar, according to the principle of the generalized likelihood ratio (GLR), Rao and Wald (RaW) tests, and propose several new detectors under two new tunable detector forms, namely kernel tunable GLR-based and kernel tunable Raw-based detectors. The proposed tunable detectors include most of the prior detectors in colocated MIMO radar as special cases. One capability of the proposed detectors is that their robustness or selectivity (RoS) to steering vector mismatch (SVM) can be tuned flexibility through an RoS tuning parameter. In addition, we are able to incorporate the prior distribution (PD) of the disturbance covariance matrix, if available, through a PD tuning parameter. Therefore, the proposed detectors can be tuned based on the tuning RoS parameter to achieve robust or selective performance in the presence of SVM as well as to switch between the Bayesian or non-Bayesian based detectors through the PD parameter. For practical situations, we show that the proposed detectors possess CFAR property against disturbance covariance matrix by resorting to the invariance principle. Extensive Monte Carlo simulation results are provided to indicate that the proposed detectors have better detection performance than their counterparts in both single-target and multi-target scenarios.

An Optimal Monitoring Model of Desertification in Naiman Banner Based on Feature Space Utilizing Landsat8 Oli Image

Current feature space models of desertification were almost linear, which ignored the complicated and non-linear relationships among variables for monitoring desertification. Fully considering the influencing factors of the desertification process in Naiman Banner, four sensitive indices including MSAVI, NDVI, TGSI, and Albedo have been selected to construct five feature spaces. Then, the precisions of different feature space models for monitoring desertification information (including non-linear and linear models) have been compared and analyzed. The non-linear Albedo-MSAVI feature space model for Naiman Banner has higher efficiency with the overall precision of 90.1%, while that of Albedo-TGSI had the worst precision with 0.69. Overall, the feature space model (non-linear) of Albedo-MSAVI has the highest applicability for monitoring the desertification information in Naiman Banner.

A novel gas turbine fault detection and identification strategy based on hybrid dimensionality reduction and uncertain rule-based fuzzy logic

Persistent health monitoring of a gas turbine engine preserves the operational efficiency and lifetime of its components. An online monitoring system should meet requirements such as high accuracy, reliability against measurement uncertainty, and ease of implementation. Factors such as high dimensions of data, variety of deteriorations, and similarity of fault patterns are also challenging to the Fault Detection and Identification (FDI) process. In this paper, a new gas turbine health assessment strategy based on hybrid dimensionality reduction and Interval Type-2 Fuzzy Logic (IT2FL) is developed, and it is demonstrated that the proposed pre-processing method not only reduces the computational burden, but also preserves the part of information that is necessary for maintaining the reliability of the FDI procedure. In this approach, raw data are projected to a nonlinear self-organized feature space, and optimum feature subsets are selected using multi-objective optimization. Pre-processed data are then utilized for training a bank of IT2FL Systems (IT2FLSs), which are developed to estimate the gas turbine’s health status at various ambient and working conditions. The proficiency of the proposed approach is compared to other feature extraction methods, and it is indicated that the Self-Organizing Map (SOM) based FDI system outstands in terms of incipient fault diagnosis, reliability, and robustness against measurement uncertainty.

The extension of the largest generalized-eigenvalue based distance metric D_ij (γ_1 ) in arbitrary feature spaces to classify composite data points.

Analyzing patterns in data points embedded in linear and non-linear feature spaces is considered as one of the common research problems among different research areas, for example: data mining, machine learning, pattern recognition, and multivariate analysis. In this paper, data points are heterogeneous sets of biosequences (composite data points). A composite data point is a set of ordinary data points (e.g., set of feature vectors). We theoretically extend the derivation of the largest generalized eigenvalue-based distance metric Dij (γ1) in any linear and non-linear feature spaces. We prove that Dij (γ1) is a metric under any linear and non-linear feature transformation function. We show the sufficiency and efficiency of using the decision rule (i.e., mean of Dij (γ1)) in classification of heterogeneous sets of biosequences compared with the decision rules minΞi and medianΞi. We analyze the impact of linear and non-linear transformation functions on classifying/clustering collections of heterogeneous sets of biosequences. The impact of the length of a sequence in a heterogeneous sequence-set generated by simulation on the classification and clustering results in linear and non-linear feature spaces is empirically shown in this paper. We propose a new concept: the limiting dispersion map of the existing clusters in heterogeneous sets of biosequences embedded in linear and nonlinear feature spaces, which is based on the limiting distribution of nucleotide compositions estimated from real data sets. Finally, the empirical conclusions and the scientific evidences are deduced from the experiments to support the theoretical side stated in this paper.

Multiple Kernel Based Remote Sensing Vegetation Classifier with Levy Optimized Subspace

Vegetation classification in remote sensing (RS) applications details a rich source of information on land use/land cover decision making. To classify the mixed land-cover area of diverse categories through RS imagery, robust classification methods and their techniques act as a substratum for thematic interpretation. Though many classification techniques have been raised for the study of remote sensing images, support vector machine (SVM) has received huge attention. In order to handle non-linear separable high dimensional feature space, optimized subspace-based classifier acts as a bedrock for improved accuracy as accuracy is still a primary concern to design efficient classifiers for nonlinear separable feature space. In this work, a classifier Enhanced Entropy based Multiple Kernel Support Vector Machine with Levy optimized subspace has experimented for vegetation classification. Since SVMs are a typically linear methods, they can be easily derived into non-linear decision criteria by substituting the inner products with kernel functions. The results indicate that the proposed method is getting better accuracy than the existing Methods.

A structured perspective of volumes on active learning

We approximate the version space which covers all feasible classification hypotheses into a structured geometric hypersphere against agnostic distribution. We present a structured perspective that divides the available active learning (AL) sampling approaches into two kinds of strategies: Outer Volume Sampling and Inner Volume Sampling. For the outer volume, it is represented by a circumscribed hypersphere which would exclude any outlier (non-promising) hypothesis from the version space globally. While for the inner volume, it is represented by many inscribed hyperspheres, which cover most of hypotheses within the outer volume. To enhance the performance of AL, we aggregate the two kinds of volumes to eliminate their sampling biases via finding the optimal inscribed hyperspheres in the enclosing space of outer volume. We then propose a Volume-based Model for the AL sampling without any distribution assumption. To generalize our theoretical model, in a non-linear feature space, spanned by kernel, we use sequential optimization to globally optimize the original space to a sparse space by halving the size of the kernel space. Then, the expectation maximization (EM) model which returns the local center helps us to find a local representation. To describe this process, we propose an easy-to-implement algorithm called Volume-based AL (VAL). Empirical evaluation on a various set of structured clustering and unstructured handwritten digit data sets have demonstrated that, employing our proposed model can accelerate the decline of the prediction error rate with fewer sampling number compared with the other algorithms.

Bi-adapting kernel learning for unsupervised domain adaptation

Unsupervised domain adaptation aims to use labeled instances from a source domain to train a good learning model, which can classify unlabeled instances from a target domain as accurate as possible. The biggest challenge is that datasets from the source and target domains have different distributions, thus the general classification model trained on the source domain can not perform well on the target domain data. The classic methods solve this problem mainly by narrowing the distance between the source and target domains. Those methods, however, is not optimal since the nonlinear feature space may not match the kernel-based learning machine. In this paper, we design a new method called bi-adapt kernel learning (BAKL) to learn a domain-invariant kernel by transferring the source and target domains to each other simultaneously. Specifically, we derive the new source and target domain kernel matrix according to the Mercer’s theorem. The domain-invariant kernel machines are then constructed by minimizing the approximation error between the newly generated kernel matrices and the ground truth source domain kernel matrices. Experiments on benchmark tasks of text and object recognition demonstrate that it significantly improves classification accuracy compared to the state-of-art methods.

Sparse kernel subspace method for classifying and representing patterns from data with complex structure

The subspace method based on principal component analysis (PCA) is a useful tool for classifying and representing patterns. In order to learn the dynamical nonlinear structure of data, there is currently much discussion about kernel subspace methods based on kernel PCA. By using a nonlinear transformation, we can efficiently capture the nonlinear structure of data, which leads to effective subspace construction for data having a dynamical nonlinear structure. However, the performance of a kernel subspace method becomes dramatically worse in the presence of noisy features, because the method is based on a fully dense loading matrix in the constructed subspace. This implies that the subspace for each class is constructed by using not only crucial features but also noisy features, which significantly disturb the PCA procedures. To resolve this issue, we propose a sparse kernel subspace method based on sparse PCA. In order to construct subspaces based only on crucial features, we incorporate sparsity into the eigenvector estimation procedures in the nonlinear feature space. We then construct the subspaces of nonlinear features based on the sparse eigenvectors, and classify the unknown vector by projecting it into the constructed subspace. The proposed method can effectively not only capture nonlinear structures of data but also estimate eigenvectors even in the presence of noisy features. The effectiveness of the proposed method leads to the high-precision of the classification. Monte Carlo simulations and a real data analysis show that the proposed sparse kernel subspace method provides effective results for classification.

A Novel Sketch-Based Three-Dimensional Shape Retrieval Method Using Multi-View Convolutional Neural Network

Retrieving 3D models by adopting hand-drawn sketches to be the input has turned out to be a popular study topic. Most current methods are based on manually selected features and the best view produced for 3D model calculations. However, there are many problems with these methods such as distortion. For the purpose of dealing with such issues, this paper proposes a novel feature representation method to select the projection view and adapt the maxout network to the extended Siamese network architecture. In addition, the strategy is able to handle the over-fitting issue of convolutional neural networks (CNN) and mitigate the discrepancies between the 3D shape domain and the sketch. A pre-trained AlexNet was used to sketch the extract features. For 3D shapes, multiple 2D views were compiled into compact feature vectors using pre-trained multi-view CNNs. Then the Siamese convolutional neural networks were learnt for transforming the two domains’ original characteristics into nonlinear feature space, which mitigated the domain discrepancy and kept the discriminations. Two large data sets were used for experiments, and the experimental results show that the method is superior to the prior art methods in accuracy.

A Novel Integrated SVM for Fault Diagnosis Using KPCA and GA

Fault diagnosis has been more and more significant in modern factories to ensure the proper functionality of the manufacturing process and to improve product quality and efficiency. A novel integrated SVM model is developed in this paper. Since KPCA can implicitly mapped the data into a higher nonlinear feature space, it is applied in our model to extract key information of original data sets for further training of SVMs. SVM is the main part to detect and identify different kinds of faults, and GA is utilized to optimize the parameters of SVM to avoid over fitting or under fitting problems caused by wrong parameters. Compared with other fault diagnosis methods applied on Tennessee Eastman (TE) process benchmark, we can safely conclude that the proposed model can achieve good predictive accuracy in a relatively short time based on our experiment results.