Higher-order Kernels Research Articles

Unsteady aerodynamics modeling using Volterra series expansed by basis function

Unsteady aerodynamics modeling must accurately describe nonlinear aerodynamic characteristics in addition to unsteady aerodynamic characteristics. The Volterra series has attracted increasing attention as a powerful tool for nonlinear system modeling. It is essential to incorporate the influence of the second-order Volterra kernel or higher-order kernels to build a nonlinear unsteady aerodynamics model. The main difficulty in the identification of higher-order kernels is that the number of parameters to be identified increases exponentially with the order of a kernel. This paper expands the Volterra kernels with the four-order B-spline wavelet on the interval as the basis function, converts the problem into the solution of low-dimensional equations, and obtains a stable solution. A nonlinear unsteady aerodynamics model is built by identifying the second-order and third-order kernels of the lift, drag, and pitching moment coefficients of the NACA0012 airfoil. Then the model is verified at different reduced frequencies using CFD.

Optimal rates for coefficient-based regularized regression

In this paper, we consider the coefficient-based regularized kernel regression. In form, the algorithm minimizes a least-square loss functional adding a coefficient-based ℓ2-penalty term over a linear span of features generated by a kernel function. We study the asymptotic behavior of the algorithm under the framework of learning theory. Compared with the classical kernel ridge regression (KRR), the algorithm under consideration does not require the kernel to be positive semi-definite and hence provides a simple paradigm for designing indefinite kernel methods. Another important feature of this algorithm is that, it can improve the saturation effect suffered by KRR. In fact, this algorithm can be viewed as a variant of KRR based on high-order kernels, which provides an effective way to improve the saturation effect. In this paper, we establish a nice convergence analysis by means of a novel integral operator approach. Then we obtain the optimal rates for the algorithm in a mini-max sense. We thus demonstrate that the performance of the concerned algorithm is comparable with that of KRR, and even outperforms it when the regression function has higher regularities.

Learning epistatic interactions from sequence-activity data to predict enantioselectivity.

Enzymes with a high selectivity are desirable for improving economics of chemical synthesis of enantiopure compounds. To improve enzyme selectivity mutations are often introduced near the catalytic active site. In this compact environment epistatic interactions between residues, where contributions to selectivity are non-additive, play a significant role in determining the degree of selectivity. Using support vector machine regression models we map mutations to the experimentally characterised enantioselectivities for a set of 136 variants of the epoxide hydrolase from the fungus Aspergillus niger (AnEH). We investigate whether the influence a mutation has on enzyme selectivity can be accurately predicted through linear models, and whether prediction accuracy can be improved using higher-order counterparts. Comparing linear and polynomial degree = 2 models, mean Pearson coefficients (r) from [Formula: see text]-fold cross-validation increase from 0.84 to 0.91 respectively. Equivalent models tested on interaction-minimised sequences achieve values of [Formula: see text] and [Formula: see text]. As expected, testing on a simulated control data set with no interactions results in no significant improvements from higher-order models. Additional experimentally derived AnEH mutants are tested with linear and polynomial degree = 2 models, with values increasing from [Formula: see text] to [Formula: see text] respectively. The study demonstrates that linear models perform well, however the representation of epistatic interactions in predictive models improves identification of selectivity-enhancing mutations. The improvement is attributed to higher-order kernel functions that represent epistatic interactions between residues.

Paley–Wiener Characterization of Kernels for Finite-Rate-of-Innovation Sampling

Exact reconstruction of finite-rate-of-innovation signals can be achieved by employing customized sampling kernels that satisfy certain frequency-domain properties. We impose compact support in time as an additional constraint. Considering frequency-domain reconstruction, we derive conditions for admissible sampling kernels and corresponding sampling rates. Our constructive kernel design methodology is based on the Paley–Wiener theorem for compactly supported functions. The new kernels satisfy generalized Strang-Fix conditions and have specific polynomial-modulated-exponential-reproducing properties. Unlike exponential splines, which have a support that is directly proportional to the number of exponentials they can generate, the proposed kernels have a support that is independent of that number. To analyze noise robustness, we consider a special member of the class that has a sum-of-modulated splines (SMS) form in the time domain and optimize its parameters to minimize the noise variance. The sum-of-sincs (SoS) kernel reported in the literature is an instance of this construction. In noise robustness analysis, SMS kernels show improvement in mean-squared error (MSE) compared with the state-of-the-art alternatives. In continuous-time noise, the improvement in MSE is about 2 dB for low signal-to-noise ratio (SNR) and 7 dB for high SNR. In the case of discrete white Gaussian noise, the MSE is lower by as much as 25 dB by using a higher-order SMS kernel compared with the SoS kernel for SNRs in the range of 10–15 dB.

Local orthogonal polynomial expansion for density estimation

ABSTRACTA local orthogonal polynomial expansion (LOrPE) of the empirical density function is proposed as a novel method to estimate the underlying density. The estimate is constructed by matching localised expectation values of orthogonal polynomials to the values observed in the sample. LOrPE is related to several existing methods, and generalises straightforwardly to multivariate settings. By manner of construction, it is similar to local likelihood density estimation (LLDE). In the limit of small bandwidths, LOrPE functions as kernel density estimation (KDE) with high-order (effective) kernels inherently free of boundary bias, a natural consequence of kernel reshaping to accommodate endpoints. Consistency and faster asymptotic convergence rates follow. In the limit of large bandwidths LOrPE is equivalent to orthogonal series density estimation (OSDE) with Legendre polynomials, thereby inheriting its consistency. We compare the performance of LOrPE to KDE, LLDE, and OSDE, in a number of simulation studies. In terms of mean integrated squared error, the results suggest that with a proper balance of the two tuning parameters, bandwidth and degree, LOrPE generally outperforms these competitors when estimating densities with sharply truncated supports.

Higher order kernel density estimation on the circle

A new class of pth-order kernels corresponding to new moments on the circle is introduced. We propose two methods for constructing higher-order kernel density estimators, and we derive theoretical and empirical results for the kernel density estimators.

A Method for Computing Moments of Quadratic Forms Involving Wrapped Random Variables

This paper provides an analytical method for computing moments of quadratic forms containing independent wrapped random variables and observed complex phase data. Higher-order moments are obtained using block matrix representations (wherein each moment is defined in terms of a structured kernel matrix) and by noting the relationship between the expectation of a wrapped variable and the characteristic function of the associated linear variable. Higher-order kernel matrices are found to exhibit a “level in which the matrix is invariant to index exchanges between Kronecker block levels. This symmetry, a block matrix relative of higher-order tensor symmetry, is developed and leveraged to reduce the number of terms that must be computed in the moment algorithm. Results for wrapped normal variables are validated using Monte Carlo integration, whose moment estimates are shown to converge (with increasing ensemble size) to values obtained by the method presented here.

A New Family of Regularized Kernels for the Harmonic Oscillator

In this paper, a new two-parameter family of regularized kernels is introduced, suitable for applying high-order time stepping to N-body systems. These high-order kernels are derived by truncating a Taylor expansion of the non-regularized kernel about $(r^2+\epsilon^2)$, generating a sequence of increasingly more accurate kernels. This paper proves the validity of this two-parameter family of regularized kernels, constructs error estimates, and illustrates the benefits of using high-order kernels through numerical experiments.

A reduced-rank approach for implementing higher-order Volterra filters

The use of Volterra filters in practical applications is often limited by their high computational burden. To cope with this problem, many strategies for implementing Volterra filters with reduced complexity have been proposed in the open literature. Some of these strategies are based on reduced-rank approaches obtained by defining a matrix of filter coefficients and applying the singular value decomposition to such a matrix. Then, discarding the smaller singular values, effective reduced-complexity Volterra implementations can be obtained. The application of this type of approach to higher-order Volterra filters (considering orders greater than 2) is however not straightforward, which is especially due to some difficulties encountered in the definition of higher-order coefficient matrices. In this context, the present paper is devoted to the development of a novel reduced-rank approach for implementing higher-order Volterra filters. Such an approach is based on a new form of Volterra kernel implementation that allows decomposing higher-order kernels into structures composed only of second-order kernels. Then, applying the singular value decomposition to the coefficient matrices of these second-order kernels, effective implementations for higher-order Volterra filters can be obtained. Simulation results are presented aiming to assess the effectiveness of the proposed approach.

Acceleration of Perturbation-Based Electric Field Integral Equations Using Fast Fourier Transform

In this communication, the computation of the perturbation-based electric field integral equation of the form ${R^{n-1},~n = 0, 1, 2, \ldots ,}$ is accelerated by using fast Fourier transform (FFT) technique. As an effective solution of the low-frequency problem, the perturbation method employs the Taylor expansion of the scalar Green’s function in free space. However, multiple impedance matrices have to be solved at different frequency orders, and the computational cost becomes extremely high, especially for large-scale problems. Since the perturbed kernels still satisfy Toeplitz property on the uniform Cartesian grid, the FFT based on Lagrange interpolation can be well incorporated to accelerate the multiple matrix vector products. Because of the nonsingularity property of high-order kernels when $n\geq 1$ , we do not need to do any near field amendment. Finally, the efficiency of the proposed method is validated in an iterative solver with numerical examples.

A fourth-order elliptic Riemann type problem in R 3 $\mathbb{R}^{3}$

This article is concerned with a fourth-order elliptic equation i.e., $(\Delta^{2}-\kappa^{2}\Delta)[u]=0$ ( $\kappa>0$ ) coupled by Riemann boundary value conditions in Clifford analysis. In the framework of a Clifford algebra $\operatorname{Cl}(V_{3,3})$ , we obtain factorizations of the fourth-order elliptic equation and construct the explicit expressions of higher-order kernel functions. Some integral representation formulas and properties of the null solution of the fourth-order elliptic equations in Clifford analysis are presented. Based on these integral representation formulas, the boundary behavior of some singular integral operators, and the Clifford analytic approach, we prove that the fourth-order elliptic Riemann type problem in $\mathbb{R}^{3}$ is solvable. The explicit representation formula of the solution is also established.

Accurate estimators of correlation functions in Fourier space

Efficient estimators of Fourier-space statistics for large number of objects rely on Fast Fourier Transforms (FFTs), which are affected by aliasing from unresolved small scale modes due to the finite FFT grid. Aliasing takes the form of a sum over images, each of them corresponding to the Fourier content displaced by increasing multiples of the sampling frequency of the grid. These spurious contributions limit the accuracy in the estimation of Fourier-space statistics, and are typically ameliorated by simultaneously increasing grid size and discarding high-frequency modes. This results in inefficient estimates for e.g. the power spectrum when desired systematic biases are well under per-cent level. We show that using interlaced grids removes odd images, which include the dominant contribution to aliasing. In addition, we discuss the choice of interpolation kernel used to define density perturbations on the FFT grid and demonstrate that using higher-order interpolation kernels than the standard Cloud in Cell algorithm results in significant reduction of the remaining images. We show that combining fourth-order interpolation with interlacing gives very accurate Fourier amplitudes and phases of density perturbations. This results in power spectrum and bispectrum estimates that have systematic biases below 0.01% all the way to the Nyquist frequency of the grid, thus maximizing the use of unbiased Fourier coefficients for a given grid size and greatly reducing systematics for applications to large cosmological datasets.

On testing sphericity and identity of a covariance matrix with large dimensions

Tests for certain covariance structures, including sphericity, are presented when the data may be high-dimensional but not necessarily normal. The tests are formulated as functions of location-invariant estimators defined as U-statistics of higher order kernels. Under a few mild assumptions, the limit distributions of the tests are shown to be normal. The accuracy of the tests is demonstrated by simulations.

Higher-order probabilistic sensitivity calculations using the multicomplex score function method

The score function method used to compute first order probabilistic sensitivities is extended in this work to arbitrary-order derivatives included mixed partial derivatives through the use of multicomplex mathematics. Multicomplex mathematics provides an effective and convenient numerical means to compute the high-order kernel functions with respect to natural parameters or moments (mean and standard deviation) obviating the need to analytically determine the kernel functions. Using these numerical kernel functions, high-order derivatives of the response moments or the probability-of-failure with respect to the parameters of the input distributions can be obtained. Numerical results indicate that the high-order probabilistic sensitivities converge with respect to the number of samples at the same rate as standard Monte Carlo estimates. Implementation of multicomplex mathematics is facilitated through the use of the Cauchy–Riemann matrices; therefore, the extension of common engineering probability distributions to matrix form is presented.

An FPGA-Based High-Performance Neural Ensemble Spiking Activity Simulator Utilizing Generalized Volterra Kernel and Complexity Analysis

Neural information is represented and transmitted among neuronal units by a series of all-or-none “neural codes”. During the process of neural prosthesis design, generally, a large amount of “neural codes” need to be captured and analyzed, which brings about an important discipline, known as neuroinformatics. However, in neuroinformatics study, this coding process, also termed as “spiking activity”, is not straightforward for prediction. It is owing to the high nonlinearity and dynamic property involved in generation of the neuronal spikes. In this paper, a novel generalized Volterra kernel-based neural spiking activity simulator is introduced for prediction of the neural codes in mammalian hippocampal region. High-performance VLSI architecture is established for the simulator based on high-order Volterra kernels involving cross-terms. The effectiveness and efficiency of the simulator are proven in experimental settings. This simulator has the potential to serve as a core functional unit in future hippocampal cognitive neural prosthesis.

High-order neural networks and kernel methods for peptide-MHC binding prediction.

Effective computational methods for peptide-protein binding prediction can greatly help clinical peptide vaccine search and design. However, previous computational methods fail to capture key nonlinear high-order dependencies between different amino acid positions. As a result, they often produce low-quality rankings of strong binding peptides. To solve this problem, we propose nonlinear high-order machine learning methods including high-order neural networks (HONNs) with possible deep extensions and high-order kernel support vector machines to predict major histocompatibility complex-peptide binding. The proposed high-order methods improve quality of binding predictions over other prediction methods. With the proposed methods, a significant gain of up to 25-40% is observed on the benchmark and reference peptide datasets and tasks. In addition, for the first time, our experiments show that pre-training with high-order semi-restricted Boltzmann machines significantly improves the performance of feed-forward HONNs. Moreover, our experiments show that the proposed shallow HONN outperform the popular pre-trained deep neural network on most tasks, which demonstrates the effectiveness of modelling high-order feature interactions for predicting major histocompatibility complex-peptide binding. There is no associated distributable software. renqiang@nec-labs.com or mark.gerstein@yale.edu Supplementary data are available at Bioinformatics online.

Making a Non-Parametric Density Estimator More Attractive, and More Accurate, by Data Perturbation

Summary Motivated by both the shortcomings of high order density estimators, and the increasingly large data sets in many areas of modern science, we introduce new high order, non-parametric density estimators that are guaranteed to be positive and do not have highly oscillatory tails. Our approach is based on data perturbation, e.g. by tilting or data sharpening. It leads to new estimators that are more accurate than conventional kernel techniques that use positive kernels, but which nevertheless enjoy the positivity property, and are far less ‘wiggly’ than high order kernel estimators. We investigate performance by theoretical analysis and in a numerical study.

Gains from joint cross validation bandwidth selection for derivatives of conditional multidimensional densities

This paper studies bandwidth selection for kernel estimation of derivatives of multidimensional conditional densities, a non-parametric realm unexplored in the literature. This paper extends Baird [Cross validation bandwidth selection for derivatives of multidimensional densities. RAND Working Paper series, WR-1060; 2014] in its examination of conditional multivariate densities, derives and presents criteria for arbitrary kernel order and density dimension, shows consistency of the estimators, and investigates a minimization criterion which jointly estimates numerator and denominator bandwidths. I conduct a Monte Carlo simulation study for various orders of kernels in the Gaussian family and compare the new cross validation criterion with those implied by Baird [Cross validation bandwidth selection for derivatives of multidimensional densities. RAND Working Paper series, WR-1060; 2014]. The paper finds that higher order kernels become increasingly important as the dimension of the distribution increases. I find that the cross validation criterion developed in this paper that jointly estimates the derivative of the joint density (numerator) and the marginal density (denominator) does orders of magnitude better than criteria that estimate the bandwidths separately. I further find that using the infinite order Dirichlet kernel tends to have the best results.

Support vector analysis of large-scale data based on kernels with iteratively increasing order

This study presents an efficient approach for large-scale data training. To deal with the rapid growth of training complexity for big data analysis, a novel mechanism, which utilizes fast kernel ridge regression (Fast KRR) and ridge support vector machines (Ridge SVMs), is proposed in this study. Firstly, Fast KRR based on low-order intrinsic-space computation is developed. Preliminary support vectors are located by using Fast KRR. Subsequently, the system iteratively removes indiscriminant data until a Ridge SVM with a high-order kernel can accommodate the data size and generate a hyperplane. To speed up the removal of indiscriminant data, quick intrinsic-matrix rebuilding is devised in the iteration. Experiments on three databases were carried out for evaluating the proposed method. Moreover, different percentages of data removal were examined in the test. The results show that the performance is enhanced by as high as 78---152 folds. Besides, the mechanisms still maintain the accuracy. These findings thereby demonstrate the effectiveness of the proposed idea.

A continued fraction resummation form of bath relaxation effect in the spin-boson model.

In the spin-boson model, a continued fraction form is proposed to systematically resum high-order quantum kinetic expansion (QKE) rate kernels, accounting for the bath relaxation effect beyond the second-order perturbation. In particular, the analytical expression of the sixth-order QKE rate kernel is derived for resummation. With higher-order correction terms systematically extracted from higher-order rate kernels, the resummed quantum kinetic expansion approach in the continued fraction form extends the Pade approximation and can fully recover the exact quantum dynamics as the expansion order increases.