Spectral Kernel Research Articles

Dynamic functional connections analysis with spectral learning for brain disorder detection

Dynamic functional connections (dFCs), can reveal neural activities, which provides an insightful way of mining the temporal patterns within the human brain and further detecting brain disorders. However, most existing studies focus on the dFCs estimation to identify brain disorders by shallow temporal features and methods, which cannot capture the inherent temporal patterns of dFCs effectively. To address this problem, this study proposes a novel method, named dynamic functional connections analysis with spectral learning (dCSL), to explore inherently temporal patterns of dFCs and further detect the brain disorders. Concretely, dCSL includes two components, dFCs estimation module and dFCs analysis module. In the former, dFCs are estimated via the sliding window technique. In the latter, the spectral kernel mapping is first constructed by combining the Fourier transform with the non-stationary kernel. Subsequently, the spectral kernel mapping is stacked into a deep kernel network to explore higher-order temporal patterns of dFCs through spectral learning. The proposed dCSL, sharing the benefits of deep architecture and non-stationary kernel, can not only calculate the long-range relationship but also explore the higher-order temporal patterns of dFCs. To evaluate the proposed method, a set of brain disorder classification tasks are conducted on several public datasets. As a result, the proposed dCSL achieves 5% accuracy improvement compared with the widely used approaches for analyzing sequence data, 1.3% accuracy improvement compared with the state-of-the-art methods for dFCs. In addition, the discriminative brain regions are explored in the ASD detection task. The findings in this study are consistent with the clinical performance in ASD.

Scheme using a finite‐volume method for local horizontal derivatives in the spectral model: The non‐hydrostatic case

AbstractIn a typical non‐hydrostatic spectral dynamic numerical weather prediction (NWP) kernel, all forecast variables are transformed between grid point and spectral spaces to compute their gradients and solve the implicit problem. This kernel requires numerous spectral transformations, which depend heavily on extensive global communication and significantly hinder parallel computing efficiency. This paper introduces an innovative non‐hydrostatic spectral kernel that incorporates a finite‐volume method within the spectral framework. We have developed a horizontal divergence (D)‐based structure equation, allowing direct computation of most prognostic variables and their horizontal gradients at grid point space. By doing so, the need for spectral transformations is substantially decreased. Our experiments demonstrate that this new approach reduces the cost of spectral transformations by up to 40%, enhancing the overall model efficiency by 15%–22%. Additionally, a series of tests confirmed the accuracy and stability of this new solver.

Convolutional Spectral Kernel Learning with Generalization Guarantees (Abstract Reprint)

Kernel methods are powerful tools to capture nonlinear patterns behind given data but often lead to poor performance on complicated tasks compared to convolutional neural networks. The reason is that kernel methods are still shallow and fully connected models, failing to reveal hierarchical features and local interdependencies. In this paper, to acquire hierarchical and local knowledge, we incorporate kernel methods with deep architectures and convolutional operators in a spectral kernel learning framework. Based on the inverse Fourier transform and Rademacher complexity theory, we provide the generalization error bounds for the proposed model and prove that under suitable initialization, deeper networks lead to tighter error bounds. Inspired by theoretical findings, we finally completed the convolutional spectral kernel network (CSKN) with two additional regularizers and an initialization strategy. Extensive ablation results validate the effectiveness of non-stationary spectral kernel, multiple layers, additional regularizers, and the convolutional filters, which coincide with our theoretical findings. We further devise a VGG-type 8-layers CSKN, and it outperforms the existing kernel-based networks and popular CNN models on the medium-sized image classification tasks.

Band-by-band spectral radiative kernels based on the ERA5 reanalysis

Radiative kernel is a widely adopted method for diagnosing radiation variability and climate feedback. However, most of the existing radiative kernels are broadband flux kernels and lack the spectral information. Motivated by the growing interest in the spectral changes of the Earth radiation budget, we generate a new set of band-by-band radiative kernels based on the fifth generation European Center for Medium-Range Weather Forecasts (ERA5) reanalysis, which can be used for analyzing the spectrally decomposed changes in the top of atmosphere, surface and atmospheric radiation. The radiative sensitivity quantified by the ERA5 band-by-band kernel is compared to another spectral kernel and validated in a spectral radiation closure test. The use and benefits of the new ERA5 kernels are then demonstrated in an analysis of spectral feedbacks of an ensemble of global climate models (GCMs).

Spectral kernel sorting based on high-risk visual features associated with mycotoxin contamination reduces aflatoxin and fumonisin contamination in maize from Ghana

Rapid single kernel analysis could enable physical sorting to remove mycotoxins from bulk grains. The purpose of this study was to use visual characteristics previously associated with aflatoxin and fumonisin contamination of maize kernels to calibrate a multi-spectral sorter and then sort mycotoxin contaminated lots. A total of 76 corn samples were collected from poultry farmers in the Dorma-Ahenkro area, Ghana. Paired 400-g subsamples were used for bulk analysis and single kernel sorting. Individual kernels were selected from contaminated samples by 2 levels of stratification: visible high-risk kernels (n = 1000) and visible low risk kernels (n = 1000). High-risk kernels had one or more of 3 features: fluorescence under UV (366 nm) light, mold, or brokenness. Kernels were used to calibrate a multi-spectral sorter (individual wavelengths from 470 to 1070 nm) to remove high-risk kernels. Then, kernel samples were sorted and the reject and accept streams individually ground and tested for aflatoxin and fumonisin contamination using ELISA. Bulk sample levels ranged between 0.78 and 67 ppb aflatoxin and <2.5 × 10−3 – 5.7 ppm fumonisin. Classification algorithms to reject visible high-risk spectra were 63% sensitive and 90% specific. After sorting, samples showed a significant aflatoxin reduction (p < 0.001, 73/76 samples reduced, mean reduction 31 ppb, range −9.7 – 67 ppb); all samples showed a significant fumonisin reduction (p < 0.001, mean reduction 1.9 ppm, range 9.3 × 10−2 – 6.1 ppm). From the accepted stream, 61/76 samples tested <15 ppb aflatoxin, significantly more than the 40/76 prior to sorting (p < 0.001); all accepted samples tested <2 ppm fumonisin concentration compared to only 2/76 prior to sorting. From sorting, average mass rejected was 12% (range 1.2%–36%), and that rejected mass contained an average of 46% of the total aflatoxin (range 4.3%–97%) and 88% of the total fumonisin (range 10%–84%). Visual characteristics associated with mycotoxin contamination can inform classification models which can enable sorting contaminated maize to reduce aflatoxin and fumonisin contamination.

Gaussian Processes on Graphs Via Spectral Kernel Learning

We propose a graph spectrum-based Gaussian process for prediction of signals defined on nodes of the graph. The model is designed to capture various graph signal structures through a highly adaptive kernel that incorporates a flexible polynomial function in the graph spectral domain. Unlike most existing approaches, we propose to learn such a spectral kernel, where the polynomial setup enables learning without the need for eigen-decomposition of the graph Laplacian. In addition, this kernel has the interpretability of graph filtering achieved by a bespoke maximum likelihood learning algorithm that enforces the positivity of the spectrum. We demonstrate the interpretability of the model in synthetic experiments from which we show the various ground truth spectral filters can be accurately recovered, and the adaptability translates to superior performances in the prediction of real-world graph data of various characteristics.

Convolutional spectral kernel learning with generalization guarantees

Kernel methods are powerful tools to capture nonlinear patterns behind given data but often lead to poor performance on complicated tasks compared to convolutional neural networks. The reason is that kernel methods are still shallow and fully connected models, failing to reveal hierarchical features and local interdependencies. In this paper, to acquire hierarchical and local knowledge, we incorporate kernel methods with deep architectures and convolutional operators in a spectral kernel learning framework. Based on the inverse Fourier transform and Rademacher complexity theory, we provide the generalization error bounds for the proposed model and prove that under suitable initialization, deeper networks lead to tighter error bounds. Inspired by theoretical findings, we finally completed the convolutional spectral kernel network (CSKN) with two additional regularizers and an initialization strategy. Extensive ablation results validate the effectiveness of non-stationary spectral kernel, multiple layers, additional regularizers, and the convolutional filters, which coincide with our theoretical findings. We further devise a VGG-type 8-layers CSKN, and it outperforms the existing kernel-based networks and popular CNN models on the medium-sized image classification tasks.

Learning “best” kernels from data in Gaussian process regression. With application to aerodynamics

This paper introduces algorithms to select/design kernels in Gaussian process regression/kriging surrogate modeling techniques. We adopt the setting of kernel method solutions in ad hoc functional spaces, namely Reproducing Kernel Hilbert Spaces (RKHS), to solve the problem of approximating a regular target function given observations of it, i.e. supervised learning. A first class of algorithms is kernel flow, which was introduced in the context of classification in machine learning. It can be seen as a cross-validation procedure whereby a “best” kernel is selected such that the loss of accuracy incurred by removing some part of the dataset (typically half of it) is minimized. A second class of algorithms is called spectral kernel ridge regression, and aims at selecting a “best” kernel such that the norm of the function to be approximated is minimal in the associated RKHS. Within Mercer's theorem framework, we obtain an explicit construction of that “best” kernel in terms of the main features of the target function. Both approaches of learning kernels from data are illustrated by numerical examples on synthetic test functions, and on a classical test case in turbulence modeling validation for transonic flows about a two-dimensional airfoil.

Optimal Superpixel Kernel-Based Kernel Low-Rank and Sparsity Representation for Brain Tumour Segmentation.

Given the need for quantitative measurement and 3D visualisation of brain tumours, more and more attention has been paid to the automatic segmentation of tumour regions from brain tumour magnetic resonance (MR) images. In view of the uneven grey distribution of MR images and the fuzzy boundaries of brain tumours, a representation model based on the joint constraints of kernel low-rank and sparsity (KLRR-SR) is proposed to mine the characteristics and structural prior knowledge of brain tumour image in the spectral kernel space. In addition, the optimal kernel based on superpixel uniform regions and multikernel learning (MKL) is constructed to improve the accuracy of the pairwise similarity measurement of pixels in the kernel space. By introducing the optimal kernel into KLRR-SR, the coefficient matrix can be solved, which allows brain tumour segmentation results to conform with the spatial information of the image. The experimental results demonstrate that the segmentation accuracy of the proposed method is superior to several existing methods under different indicators and that the sparsity constraint for the coefficient matrix in the kernel space, which is integrated into the kernel low-rank model, has certain effects in preserving the local structure and details of brain tumours.

Hyperspectral Classification Based on Coupling Multiscale Super-Pixels and Spatial Spectral Features

Hyperspectral remotely sensed images contain not only the spatial information of ground objects, but also their rich spectral information. Effectively improve the classification accuracy of hyperspectral images (HSIs), the purpose is to accurately grasp the current land resource utilization information, which is of great significance to the formulation and implementation of future land and space planning. Existing research on the classification of hyperspectral remotely sensed images uses a single-scale superpixel method for image segmentation. The optimal number of superpixels cannot be determined, image details may get missed, and a single kernel matrix cannot represent the information from multiple features, which results in reduced classification accuracy. Therefore, this study intends to use the superpixel segmentation method to perform multiscale superpixel segmentation on the principal components of HSIs at multiple scales, and to couple the multiscale superpixel spatial spectral kernel (SSK) with the original spectral kernel to form a synthetic kernel using weights for HSI classification. The three HSIs of Pavia University, Pavia Center, and Washington DC Mall are used as the experimental data to test and analyze this method. The experimental results show that the effective classification accuracies for the three datasets obtained by this method are 5.28%, 5.90%, and 7.71% higher than the five comparison methods, at best. The results prove that this method can effectively solve the problem of imprecise image feature extraction and an unknown number of initial superpixels and can significantly improve the classification accuracy of HSIs.

Lightweight Heterogeneous Kernel Convolution for Hyperspectral Image Classification With Noisy Labels

Convolutional neural networks (CNNs) have exhibited commendable performance in the hyperspectral images (HSIs) classification task with manually annotated limited available training data for supervision. The accurate classification of pixel-wise land covers using traditional CNNs is often hampered by the presence of wrong (noisy) labels in the training data and can easily be overfitted to the label noises. However, training on noisy labeled data inevitably suffers from performance degradation since CNNs tend to overfit the label noises. To overcome this problem, we propose a lightweight heterogeneous kernel convolution ( <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HetConv3D</monospace> ) for HSI classification with noisy labels, where <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HetConv3D</monospace> uses two different types of convolutional kernels, i.e., spectral and spatial domains, and fuses them to produce the final feature maps that are less prompted to the noises and also reduces the computation time. The experiments are conducted using three well-known HSI datasets, i.e., Kennedy Space Center (KSC), Salinas Scene (SA), and University of Pavia (UP), and results are compared with traditional supervised classification methods, including support vector machine (SVM), random forest (RF), CNN3D, ContextNet, MS3DNet, lightweight dual-channel residual network (DCRN), and HetConv3DNet. The superior performance exhibited by the proposed model <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HetConv3D-HSI</monospace> confirms the importance of learning a fusion of spatial and spectral kernel features. The source code will be made available publicly at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/purbayankar/HetConv3DNet</uri> .

Regulation-based probabilistic substance quality index and automated geo-spatial modeling for water quality assessment

Natural environments are recognized as complex heterogeneous structures thus requiring numerous multi-scale observations to yield a comprehensive description. To monitor the current state and identify negative impacts of human activity, fast and precise instruments are in urgent need. This work provides an automated approach to the assessment of spatial variability of water quality using guideline values on the example of 1526 water samples comprising 21 parameters at 448 unique locations across the New Moscow region (Russia). We apply multi-task Gaussian process regression (GPR) to model the measured water properties across the territory, considering not only the spatial but inter-parameter correlations. GPR is enhanced with a Spectral Mixture Kernel to facilitate a hyper-parameter selection and optimization. We use a 5-fold cross-validation scheme along with R^2-score to validate the results and select the best model for simultaneous prediction of water properties across the area. Finally, we develop a novel Probabilistic Substance Quality Index (PSQI) that combines probabilistic model predictions with the regulatory standards on the example of the epidemiological rules and hygienic regulations established in Russia. Moreover, we provide an interactive map of experimental results at 100 m2 resolution. The proposed approach contributes significantly to the development of flexible tools in environment quality monitoring, being scalable to different standard systems, number of observation points, and region of interest. It has a strong potential for adaption to environmental and policy changes and non-unified assessment conditions, and may be integrated into support-decision systems for the rapid estimation of water quality spatial distribution.

Hyperspectral image classification based on composite kernel relevance vector machine

This paper presents a composite kernel Relevance Vector Machine(RVM) algorithm, for enhanced classification accuracy of hyperspectral images. This paper constructs three forms of composite kernels based on properties of kernels. The spatial feature is extracted using multi-scale morphological method from the image after principal components transform. The final classification is achieved by composite kernel RVM classifier. The proposed approach is tested in experiments on AVIRIS data. Compared with spectral kernel RVM, the OA and Kappa coefficient of composite kernel RVM increased obviously. However, the training time dose not increased. Meanwhile, composite kernel RVM has ability to get high accuracy with relative small training set. The proposed method has practical use in hyperspectral imagery classification.

A Discriminative Multi-Output Gaussian Processes Scheme for Brain Electrical Activity Analysis

The study of brain electrical activity (BEA) from different cognitive conditions has attracted a lot of interest in the last decade due to the high number of possible applications that could be generated from it. In this work, a discriminative framework for BEA via electroencephalography (EEG) is proposed based on multi-output Gaussian Processes (MOGPs) with a specialized spectral kernel. First, a signal segmentation stage is executed, and the channels from the EEG are used as the model outputs. Then, a novel covariance function within the MOGP known as the multispectral mixture kernel (MOSM) allows us to find and quantify the relationships between different channels. Several MOGPs are trained from different conditions grouped in bi-class problems, and the discrimination is performed based on the likelihood score of the test signals against all the models. Finally, the mean likelihood is computed to predict the correspondence of new inputs with each class’s existing models. Results show that this framework allows us to model the EEG signals adequately using generative models and allows analyzing the relationships between channels of the EEG for a particular condition. At the same time, the set of trained MOGPs is well suited to discriminate new input data.

Testing-optimal kernel choice in HAR inference

The paper investigates the optimal kernel choice in heteroskedasticity and autocorrelation robust tests based on the fixed-b asymptotics. In parallel with the optimality of the quadratic spectral kernel under the asymptotic mean squared error criterion of the point estimator of the long run variance as considered in Andrews (1991) , we show that the optimality of the quadratic spectral kernel continues to hold under the testing-oriented criterion of Sun, Philips and Jin (2008) which takes a weighted average of the probabilities of type I and type II errors of the fixed-b asymptotic test.

Spectral-Spatial Weighted Kernel Manifold Embedded Distribution Alignment for Remote Sensing Image Classification.

Feature distortions of data are a typical problem in remote sensing image classification, especially in the area of transfer learning. In addition, many transfer learning-based methods only focus on spectral information and fail to utilize spatial information of remote sensing images. To tackle these problems, we propose spectral-spatial weighted kernel manifold embedded distribution alignment (SSWK-MEDA) for remote sensing image classification. The proposed method applies a novel spatial information filter to effectively use similarity between nearby sample pixels and avoid the influence of nonsample pixels. Then, a complex kernel combining spatial kernel and spectral kernel with different weights is constructed to adaptively balance the relative importance of spectral and spatial information of the remote sensing image. Finally, we utilize the geometric structure of features in manifold space to solve the problem of feature distortions of remote sensing data in transfer learning scenarios. SSWK-MEDA provides a novel approach for the combination of transfer learning and remote sensing image characteristics. Extensive experiments have demonstrated that the proposed method is more effective than several state-of-the-art methods.

AN EXPERIMENTAL STUDY OF FACE RECOGNITION METHOD

The increased use of face recognition techniques leads to the development of improved methods with higher accuracy and efficiency. Currently, there are various face recognition techniques based on different algorithm. In this study, a new method of face recognition is proposed based on the idea of wavelet operators for creating spectral graph wavelet transformation. The proposed idea relies on the spectral graph wavelet kernel procedure. In this proposed method, feature extraction is based on transformation into SGWT by means of spatial domain. For recognition purpose, the feature vectors are used for computation of selected training samples which makes the classification. The decomposition of face image is done using the SGWT. The system identifies the test image by calculating the Euclidean distance. Finally, the study conducted an experiment using the ORL face database. The result states that the recognition accuracy is higher in the proposed system which can be further improved using the number of training images. Overall, the result shows that the proposed method has good performance in terms of accuracy of the face recognition

Automated Spectral Kernel Learning

The generalization performance of kernel methods is largely determined by the kernel, but spectral representations of stationary kernels are both input-independent and output-independent, which limits their applications on complicated tasks. In this paper, we propose an efficient learning framework that incorporates the process of finding suitable kernels and model training. Using non-stationary spectral kernels and backpropagation w.r.t. the objective, we obtain favorable spectral representations that depends on both inputs and outputs. Further, based on Rademacher complexity, we derive data-dependent generalization error bounds, where we investigate the effect of those factors and introduce regularization terms to improve the performance. Extensive experimental results validate the effectiveness of the proposed algorithm and coincide with our theoretical findings.

Spectral and matrix factorization methods for consistent community detection in multi-layer networks

We consider the problem of estimating a consensus community structure by combining information from multiple layers of a multi-layer network using methods based on the spectral clustering or a low-rank matrix factorization. As a general theme, these “intermediate fusion” methods involve obtaining a low column rank matrix by optimizing an objective function and then using the columns of the matrix for clustering. However, the theoretical properties of these methods remain largely unexplored. In the absence of statistical guarantees on the objective functions, it is difficult to determine if the algorithms optimizing the objectives will return good community structures. We investigate the consistency properties of the global optimizer of some of these objective functions under the multi-layer stochastic blockmodel. For this purpose, we derive several new asymptotic results showing consistency of the intermediate fusion techniques along with the spectral clustering of mean adjacency matrix under a high dimensional setup, where the number of nodes, the number of layers and the number of communities of the multi-layer graph grow. Our numerical study shows that the intermediate fusion techniques outperform late fusion methods, namely spectral clustering on aggregate spectral kernel and module allegiance matrix in sparse networks, while they outperform the spectral clustering of mean adjacency matrix in multi-layer networks that contain layers with both homophilic and heterophilic communities.

Local Binary Patterns and Superpixel-Based Multiple Kernels for Hyperspectral Image Classification

The superpixel-based multiple kernels model uses the average value of all pixels within superpixel as the spatial feature, which results in inaccurate extraction of edge pixels. To solve this problem, a local binary patterns and superpixel-based multiple kernels method is proposed for hyperspectral image (HSI) classification. First, the original HSI is segmented into multiple superpixels by using the entropy rate superpixel segmentation algorithm. On the HSI with superpixel index, the spectral kernel is second obtained by combining the spectral feature map with the radial basis kernel (RBF). By introducing local binary pattern (LBP) and weighted average filtering into RBF, the spatial kernels are obtained within and among superpixels. Finally, the combined kernel containing the abovementioned three kernels is inputted into the support vector machine classifier to generate a classification map. The experimental procedure in this article uses LBP to extract the information in superpixels, which effectively prevents the loss of edge features in superpixels. The experimental results show that the proposed method is superior to the state-of-the-art classifiers for HSI classification.