Choice Of Kernel Research Articles

Modeling the Multiwavelength Variability of Mrk 335 Using Gaussian Processes

Abstract The optical and UV variability of the majority of active galactic nuclei may be related to the reprocessing of rapidly changing X-ray emission from a more compact region near the central black hole. Such a reprocessing model would be characterized by lags between X-ray and optical/UV emission due to differences in light travel time. Observationally, however, such lag features have been difficult to detect due to gaps in the lightcurves introduced through factors such as source visibility or limited telescope time. In this work, Gaussian process regression is employed to interpolate the gaps in the Swift X-ray and UV lightcurves of the narrow-line Seyfert 1 galaxy Mrk 335. In a simulation study of five commonly employed analytic Gaussian process kernels, we conclude that the Matern 1 2 and rational quadratic kernels yield the most well-specified models for the X-ray and UVW2 bands of Mrk 335. In analyzing the structure functions of the Gaussian process lightcurves, we obtain a broken power law with a break point at 125 days in the UVW2 band. In the X-ray band, the structure function of the Gaussian process lightcurve is consistent with a power law in the case of the rational quadratic kernel while a broken power law with a break point at 66 days is obtained from the Matern 1 2 kernel. The subsequent cross-correlation analysis is consistent with previous studies and furthermore shows tentative evidence for a broad X-ray-UV lag feature of up to 30 days in the lag-frequency spectrum where the significance of the lag depends on the choice of Gaussian process kernel.

Large- d behavior of the Feynman amplitudes for a just-renormalizable tensorial group field theory

This paper aims at giving a novel approach to investigate the behavior of the renormalization group flow for tensorial group field theories to all orders of the perturbation theory. From an appropriate choice of the kinetic kernel, we build an infinite family of just-renormalizable models, for tensor fields with arbitrary rank $d$. Investigating the large $ d$ limit, we show that the self-energy melonic amplitude is decomposed as a product of loop-vertex functions, depending only on dimensionless mass. The corresponding melonic amplitudes may be mapped as trees in the so-called Hubbard-Stratonivich representation, and we show that only trees with edges of different colors survive in the large $d$-limit. These two key features allow us to resum the perturbative expansion for self-energy, providing an explicit expression for arbitrary external momenta in terms of Lambert function. Finally, inserting this resumed solution into the Callan-Symanzik equations, and taking into account the strong relation between two and four-point functions arising from melonic Ward-Takahashi identities, we then deduce an explicit expression for relevant and marginal $\beta$-functions, valid to all orders of the perturbative expansion. By investigating the solutions of the resulting flow, we conclude about the non-existence of any fixed point in the investigated region of the full phase space.

A kernel-based measure for conditional mean dependence

A novel metric, called kernel-based conditional mean dependence (KCMD), is proposed to measure and test the departure from conditional mean independence between a response variable Y and a predictor variable X, based on the reproducing kernel embedding and the Hilbert-Schmidt norm of a tensor operator. The KCMD has several appealing merits. It equals zero if and only if the conditional mean of Y given X is independent of X, i.e. E(Y|X)=E(Y) almost surely, provided that the employed kernel is characteristic; it can be used to detect all kinds of conditional mean dependence with an appropriate choice of kernel; it has a simple expectation form and allows an unbiased empirical estimator. A class of test statistics based on the estimated KCMD is constructed, and a wild bootstrap test procedure to the conditional mean independence is presented. The limit distributions of the test statistics and the bootstrapped statistics under null hypothesis, fixed alternative hypothesis and local alternative hypothesis are given respectively, and a data-driven procedure to choose a suitable kernel is suggested. Simulation studies indicate that the tests based on the KCMD have close powers to the tests based on martingale difference divergence in monotone dependence, but excel in the cases of nonlinear relationships or the moment restriction on X is violated. Two real data examples are presented for the illustration of the proposed method.

A particle flow filter for high-dimensional system applications.

A novel particle filter proposed recently, the particle flow filter (PFF), avoids the long‐existing weight degeneracy problem in particle filters and, therefore, has great potential to be applied in high‐dimensional systems. The PFF adopts the idea of a particle flow, which sequentially pushes the particles from the prior to the posterior distribution, without changing the weight of each particle. The essence of the PFF is that it assumes the particle flow is embedded in a reproducing kernel Hilbert space, so that a practical solution for the particle flow is obtained. The particle flow is independent of the choice of kernel in the limit of an infinite number of particles. Given a finite number of particles, we have found that a scalar kernel fails in high‐dimensional and sparsely observed settings. A new matrix‐valued kernel is proposed that prevents the collapse of the marginal distribution of observed variables in a high‐dimensional system. The performance of the PFF is tested and compared with a well‐tuned local ensemble transform Kalman filter (LETKF) using the 1,000‐dimensional Lorenz 96 model. It is shown that the PFF is comparable to the LETKF for linear observations, except that explicit covariance inflation is not necessary for the PFF. For nonlinear observations, the PFF outperforms LETKF and is able to capture the multimodal likelihood behavior, demonstrating that the PFF is a viable path to fully nonlinear geophysical data assimilation.

A novel mathematical model of heterogeneous cell proliferation.

We present a novel mathematical model of heterogeneous cell proliferation where the total population consists of a subpopulation of slow-proliferating cells and a subpopulation of fast-proliferating cells. The model incorporates two cellular processes, asymmetric cell division and induced switching between proliferative states, which are important determinants for the heterogeneity of a cell population. As motivation for our model we provide experimental data that illustrate the induced-switching process. Our model consists of a system of two coupled delay differential equations with distributed time delays and the cell densities as functions of time. The distributed delays are bounded and allow for the choice of delay kernel. We analyse the model and prove the nonnegativity and boundedness of solutions, the existence and uniqueness of solutions, and the local stability characteristics of the equilibrium points. We find that the parameters for induced switching are bifurcation parameters and therefore determine the long-term behaviour of the model. Numerical simulations illustrate and support the theoretical findings, and demonstrate the primary importance of transient dynamics for understanding the evolution of many experimental cell populations.

Emotion Analysis on Text Using Multiple Kernel Gaussian...

The ability to discern human emotions is critical for making chatbox behave like humans. Gaussian Process (GP) is a non-parametric Bayesian modeling and can be used to predict the presence of either a single emotion (single-task GP) or multiple emotions (multi-task GP) in natural language text. Employing multiple kernels in GP can enhance the performance of the emotion analysis tasks. The particular choice of kernel functions determines the properties such as smoothness, length scales, sharpness, and amplitude, drawn from the GP prior. Using a specific kernel may be a source of bias and can be avoided by using different kernels together. The default kernel used with GP is a Radial Basis Function (RBF). It is infinitely differentiable; GP with this function has mean square derivatives of all orders and is thus very smooth. The sharpness which occurs in the midst of the smoothness can be detected using the exponential kernel. The multi-layer perceptron kernel has greater generalization for each training example and is good for extrapolation. Our experiments show that, for learning the presence of a single emotion in a natural language sentence (single-task), multiple kernel GP with the sum of RBF and multi-layer perceptron kernels performs better than single kernel GP. Likewise, for learning the presence of several different emotions in a sentence (multi-task), multiple kernel GP with the sum of RBF, exponential and multi-layer perceptron kernels performs better than single kernel GP. Multiple Kernel Gaussian Process also outperforms Convolutional Neural Network (CNN).

Multikernel Clustering via Non-Negative Matrix Factorization Tailored Graph Tensor Over Distributed Networks

Next-generation wireless networks are witnessing an increasing number of clustering applications, and produce a large amount of non-linear and unlabeled data. In some degree, single kernel methods face the challenging problem of kernel choice. To overcome this problem for non-linear data clustering, multiple kernel graph-based clustering (MKGC) has attracted intense attention in recent years. However, existing MKGC methods suffer from two common problems: (1) they mainly aim to learn a consensus kernel from multiple candidate kernels, slight affinity graph learning, such that cannot fully exploit the underlying graph structure of non-linear data; (2) they disregard the high-order correlations between all base kernels, which cannot fully capture the consistent and complementary information of all kernels. In this paper, we propose a novel non-negative matrix factorization (NMF) tailored graph tensor MKGC method for non-linear data clustering, namely TMKGC. Specifically, TMKGC integrates NMF and graph learning together in kernel space so as to learn multiple candidate affinity graphs. Afterwards, the high-order structure information of all candidate graphs is captured in a 3-order tensor kernel space by introducing tensor singular value decomposition based tensor nuclear norm, such that an optimal affinity graph can be obtained subsequently. Based on the alternating direction method of multipliers, the effective local and distributed solvers are elaborated to solve the proposed objective function. Extensive experiments have demonstrated the superiority of TMKGC compared to the state-of-the-art MKGC methods.

Spatial HAC Estimator: Analysis of Convergence of European Regions

This paper applies a nonparametric heteroskedasticity and autocorrelation consistent (HAC) estimator of error terms in the context of the spatial autoregressive model of GDP per capita convergence of European regions at NUTS 2 level. By introducing the spatial dimension, it looks at how the equilibrium distribution of GDP per capita of EU regions evolves both in time and space dimensions. Results demonstrate that the global spatial spillovers of growth rates make an important contribution to the process of convergence by reinforcing the economic growth of neighboring regions. Results are even more pronounced when the convergence in wage per worker is considered. The choice of kernel functions does not significantly affect the estimation of the variance-covariance matrix, while the choice of the bandwidth parameter is quite important. Finally, results are sensitive to the weighting matrix specification, and further research is needed to give a more rigorous justification for the selection of the weighting matrix.

Spectrophotometric Online Detection of Drinking Water Disinfectant: A Machine Learning Approach.

The spectra fingerprint of drinking water from a water treatment plant (WTP) is characterised by a number of light-absorbing substances, including organic, nitrate, disinfectant, and particle or turbidity. Detection of disinfectant (monochloramine) can be better achieved by separating its spectra from the combined spectra. In this paper, two major focuses are (i) the separation of monochloramine spectra from the combined spectra and (ii) assessment of the application of the machine learning algorithm in real-time detection of monochloramine. The support vector regression (SVR) model was developed using multi-wavelength ultraviolet-visible (UV-Vis) absorbance spectra and online amperometric monochloramine residual measurement data. The performance of the SVR model was evaluated by using four different kernel functions. Results show that (i) particles or turbidity in water have a significant effect on UV-Vis spectral measurement and improved modelling accuracy is achieved by using particle compensated spectra; (ii) modelling performance is further improved by compensating the spectra for natural organic matter (NOM) and nitrate (NO3) and (iii) the choice of kernel functions greatly affected the SVR performance, especially the radial basis function (RBF) appears to be the highest performing kernel function. The outcomes of this research suggest that disinfectant residual (monochloramine) can be measured in real time using the SVR algorithm with a precision level of ± 0.1 mg L−1.

Bridging deep and multiple kernel learning: A review

Kernel methods and deep learning are two of the most currently remarkable machine learning techniques that have achieved great success in many applications. Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the reproducing kernel Hilbert space (RKHS) while making the computation tractable by leveraging the kernel trick. It is commonly agreed that the success of kernel methods is very much dependent on the choice of kernel. Multiple kernel learning (MKL) is one possible scheme that performs kernel combination and selection for a variety of learning tasks, such as classification, clustering, and dimensionality reduction. Deep learning models project input data through several layers of nonlinearity and learn different levels of abstraction. The composition of multiple layers of nonlinear functions can approximate a rich set of naturally occurring input-output dependencies. To bridge kernel methods and deep learning, deep kernel learning has been proven to be an effective method to learn complex feature representations by combining the nonparametric flexibility of kernel methods with the structural properties of deep learning. This article presents a comprehensive overview of the state-of-the-art approaches that bridge the MKL and deep learning techniques. Specifically, we systematically review the typical hybrid models, training techniques, and their theoretical and practical benefits, followed by remaining challenges and future directions. We hope that our perspectives and discussions serve as valuable references for new practitioners and theoreticians seeking to innovate in the applications of the approaches incorporating the advantages of both paradigms and exploring new synergies.

Testing-optimal kernel choice in HAR inference

The paper investigates the optimal kernel choice in heteroskedasticity and autocorrelation robust tests based on the fixed-b asymptotics. In parallel with the optimality of the quadratic spectral kernel under the asymptotic mean squared error criterion of the point estimator of the long run variance as considered in Andrews (1991) , we show that the optimality of the quadratic spectral kernel continues to hold under the testing-oriented criterion of Sun, Philips and Jin (2008) which takes a weighted average of the probabilities of type I and type II errors of the fixed-b asymptotic test.

Multiple Kernel Driven Clustering With Locally Consistent and Selfish Graph in Industrial IoT

In the cognitive computing of intelligent industrial Internet of Things, clustering is a fundamental machine learning problem to exploit the latent data relationships. To overcome the challenge of kernel choice for nonlinear clustering tasks, multiple kernel clustering (MKC) has attracted intensive attention. However, existing graph-based MKC methods mainly aim to learn a consensus kernel as well as an affinity graph from multiple candidate kernels, which cannot fully exploit the latent graph information. In this article, we propose a novel pure graph-based MKC method. Specifically, a new graph model is proposed to preserve the local manifold structure of the data in kernel space so as to learn multiple candidate graphs. Afterward, the latent consistency and selfishness of these candidate graphs are fully considered. Furthermore, a graph connectivity constraint is introduced to avoid requiring any postprocessing clustering step. Comprehensive experimental results demonstrate the superiority of our method.

Spectral backtests of forecast distributions with application to risk management

We study a class of backtests for forecast distributions in which the test statistic depends on a spectral transformation that weights exceedance events by a function of the modeled probability level. The weighting scheme is specified by a kernel measure which makes explicit the user’s priorities for model performance. The class of spectral backtests includes tests of unconditional coverage and tests of conditional coverage. We show how the class embeds a wide variety of backtests in the existing literature, and further propose novel variants which are easily implemented, well-sized and have good power. In an empirical application, we backtest forecast distributions for the overnight P&L of ten bank trading portfolios. For some portfolios, test results depend materially on the choice of kernel.

Gaussian process estimation of transition redshift

This paper aims to put constraints on the transition redshift zt, which determines the onset of cosmic acceleration, in cosmological-model independent frameworks. In order to do that, we use the non-parametric Gaussian Process method with H(z) and SNe Ia data. The deceleration parameter reconstruction from H(z) data yields zt=0.59+0.12−0.11. The reconstruction from SNe Ia data assumes spatial flatness and yields zt=0.683+0.11−0.082. These results were found with a Gaussian kernel and we show that they are consistent with two other kernel choices.

Multilevel Sequential Importance Sampling for Rare Event Estimation

The estimation of the probability of rare events is an important task in reliability and risk assessment. We consider failure events that are expressed in terms of a limit state function, which depends on the solution of a partial differential equation (PDE). Since numerical evaluations of PDEs are computationally expensive, estimating such probabilities of failure by Monte Carlo sampling is intractable. More efficient sampling methods from reliability analysis, such as Subset Simulation, are popular, but can still be impracticable if the PDE evaluations are very costly. In this article, we develop a novel, highly efficient estimator for probabilities of rare events. Our method is based on a Sequential Importance sampler using discretizations of PDE-based limit state functions with different accuracies. A twofold adaptive algorithm ensures that we obtain an estimate based on the desired discretization accuracy. In contrast to the Multilevel Subset Simulation estimator of [Ullmann, Papaioannou 2015; SIAM/ASA J. Uncertain. Quantif. 3(1):922-953], our estimator overcomes the nestedness problem. Of particular importance in Sequential Importance sampling algorithms is the correct choice of the MCMC kernel. Instead of the popular adaptive conditional sampling method, we propose a new algorithm that uses independent proposals from an adaptively constructed von Mises-Fischer-Nakagami distribution. The proposed algorithm is applied to test problems in 1D and 2D space, respectively, and is compared to the Multilevel Subset Simulation estimator and to single-level versions of Sequential Importance Sampling and Subset Simulation.

Off‐road ground robot path energy cost prediction through probabilistic spatial mapping

AbstractEnergy is an important factor in planning for ground robots, particularly in off‐road environments where the terrain significantly impacts energy usage. Unfortunately, energy costs in these environments are variable and uncertain, making planning difficult. In this paper, we present a method, based on Gaussian process regression (GPR) and known vehicle modeling information, for predicting future path energy costs. The method uses data, collected by a robot operating in a 3D environment with varying terrains, to build a map from inputs (including position, heading, slope, and satellite imagery) to power consumption. The energy costs of future paths are predicted through the summation of power predictions. Importantly, correlations in those predictions are considered to avoid overconfidence. Experimental cross‐validation results demonstrate improved accuracy of path energy cost predictions against a baseline approach, as well as the effect of Gaussian process inputs and kernel choice. Additionally, we show how vehicle modeling can aid in predicting energy costs, particularly when data on the environment is sparse.

Embedding Distortion Analysis in Wavelet-domain Watermarking

Imperceptibility and robustness are two complementary fundamental requirements of any watermarking algorithm. Low-strength watermarking yields high imperceptibility, but exhibits poor robustness. High-strength watermarking schemes achieve good robustness but often infuse distortions resulting in poor visual quality in host images. This article analyses the embedding distortion for wavelet-based watermarking schemes. We derive the relationship between distortion, measured in mean square error (MSE), and the watermark embedding modification and propose the linear proportionality between MSE and the sum of energy of the selected wavelet coefficients for watermark embedding modification. The initial proposition assumes the orthonormality of discrete wavelet transform. It is further extended for non-orthonormal wavelet kernels using a weighting parameter that follows the energy conservation theorems in wavelet frames. The proposed analysis is verified by experimental results for both non-blind and blind watermarking schemes. Such a model is useful to find the optimum input parameters, including the wavelet kernel, coefficient selection, and subband choices for wavelet domain image watermarking.

Nonlinear Structural Vector Autoregressive Models with Application to Directed Brain Networks.

Structural equation models (SEMs) and vector autoregressive models (VARMs) are two broad families of approaches that have been shown useful in effective brain connectivity studies. While VARMs postulate that a given region of interest in the brain is directionally connected to another one by virtue of time-lagged influences, SEMs assert that directed dependencies arise due to instantaneous effects, and may even be adopted when nodal measurements are not necessarily multivariate time series. To unify these complementary perspectives, linear structural vector autoregressive models (SVARMs) that leverage both instantaneous and time-lagged nodal data have recently been put forth. Albeit simple and tractable, linear SVARMs are quite limited since they are incapable of modeling nonlinear dependencies between neuronal time series. To this end, the overarching goal of the present paper is to considerably broaden the span of linear SVARMs by capturing nonlinearities through kernels, which have recently emerged as a powerful nonlinear modeling framework in canonical machine learning tasks, e.g., regression, classification, and dimensionality reduction. The merits of kernel-based methods are extended here to the task of learning the effective brain connectivity, and an efficient regularized estimator is put forth to leverage the edge sparsity inherent to real-world complex networks. Judicious kernel choice from a preselected dictionary of kernels is also addressed using a data-driven approach. Numerical tests on ECoG data captured through a study on epileptic seizures demonstrate that it is possible to unveil previously unknown directed links between brain regions of interest.

Discrete Two-Dimensional Fourier Transform in Polar Coordinates Part I: Theory and Operational Rules

The theory of the continuous two-dimensional (2D) Fourier transform in polar coordinates has been recently developed but no discrete counterpart exists to date. In this paper, we propose and evaluate the theory of the 2D discrete Fourier transform (DFT) in polar coordinates. This discrete theory is shown to arise from discretization schemes that have been previously employed with the 1D DFT and the discrete Hankel transform (DHT). The proposed transform possesses orthogonality properties, which leads to invertibility of the transform. In the first part of this two-part paper, the theory of the actual manipulated quantities is shown, including the standard set of shift, modulation, multiplication, and convolution rules. Parseval and modified Parseval relationships are shown, depending on which choice of kernel is used. Similar to its continuous counterpart, the 2D DFT in polar coordinates is shown to consist of a 1D DFT, DHT and 1D inverse DFT.

Deep learning-enabled accurate normalization of reconstruction kernel effects on emphysema quantification in low-dose CT

Lung densitometry is being frequently adopted in CT-based emphysema quantification, yet known to be affected by the choice of reconstruction kernel. This study presents a two-step deep learning architecture that enables accurate normalization of reconstruction kernel effects on emphysema quantification in low-dose CT. Deep learning is used to convert a CT image of a sharp kernel to that of a standard kernel with restoration of truncation artifacts and smoothing-free pixel size normalization. We selected 353 scans reconstructed by both standard and sharp kernels from four different CT scanners from the United States National Lung Screening Trial program database. A truncation artifact correction model was constructed with a combination of histogram extrapolation and a deep learning model trained with truncated and non-truncated image sets. Then, we performed frequency domain zero-padding to normalize reconstruction field of view effects while preventing image smoothing effects. The kernel normalization model has a U-Net based architecture trained for each CT scanner dataset. Three lung density measurements including relative lung area under 950 HU (RA950), lower 15th percentile threshold (perc15), and mean lung density were obtained in the datasets from standard, sharp, and normalized kernels. The effect of kernel normalization was evaluated with pair-wise differences in lung density metrics. The mean of pair-wise differences in RA950 between standard and sharp kernel reconstructions was reduced from 10.75% to −0.07% using kernel normalization. The difference for perc15 decreased from −31.03 HU to −0.30 HU after kernel normalization. Our study demonstrated the feasibility of applying deep learning techniques for normalizing CT kernel effects, thereby reducing the kernel-induced variability in lung density measurements. The deep learning model could increase the accuracy of emphysema quantification, thereby allowing reliable surveillance of emphysema in lung cancer screening even when follow-up CT scans are acquired with different reconstruction kernels.