Reproducing Kernel Hilbert Space Framework Research Articles

Convergence analysis of kernel conjugate gradient for functional linear regression

In this paper, we discuss the convergence analysis of the conjugate gradient-based algorithm for the functional linear model in the reproducing kernel Hilbert space framework, utilizing early stopping results in regularization against over-fitting. We establish the convergence rates depending on the regularity condition of the slope function and the decay rate of the eigenvalues of the operator composition of covariance and kernel operator. Our convergence rates match the minimax rate available from the literature.

Koopman-Theoretic Modeling of Quasiperiodically Driven Systems: Example of Signalized Traffic Corridor

This article presents a novel approach to analyzing quasiperiodically driven dynamical systems. It presents a holistic data-driven framework for reconstructing such a system and gaining insights into its dynamical behavior. A quasiperiodically driven dynamics has two components: 1) the quasiperiodic driving source with generating frequencies and 2) the driven nonlinear dynamics. The driving dynamics are reconstructed by accurately computing a large number of Koopman eigenfrequencies, using techniques based on the theory of reproducing kernel Hilbert space (RKHS) and from ergodic theory. Unlike Fourier analysis and dynamic mode decomposition (DMD), the proposed framework can analyze systems/signals with nondominant or absent periodic components. The driven system is next reconstructed by continuing to use the RKHS framework. As a case study, we investigate data collected from a corridor of nine consecutive signalized traffic intersections. The proposed framework provides insights into generating frequencies and associated modes, accurately reconstructs the queue lengths at the signalized intersections, and makes stable long-term forecasts. The unique contributions of this article are spread across multiple domains. It includes the novel modeling framework for quasiperiodically driven systems, a computationally efficient approach able to handle a large amount of data, utilization of true Koopman eigenfrequencies, and an illustration of the proposed approach on a corridor of signalized traffic intersections.

Covariance estimation error of incomplete functional data under RKHS framework

In recent years, functional data analysis (FDA) and reproducing kernel Hilbert space (RKHS) are frequently encountered in various applications. However, employing the RKHS framework to study the covariance of incomplete functional data is rare. In this paper, we consider the global estimation error of the covariance function obtained by fragment data. Our theorem is built on the connection between functional data and RKHS, by using the covariance function as the reproducing kernel. We take the mean square integrability of functional data into consideration, and ease the previous restrictions of covariance function and observation area. Simulation results show the veracity of our theoretical finding by comparing the error of two kinds of functional data in covariance estimation. The existing N resolution patched (N-rp) method to estimate covariance in a local observation area has been improved, resulting in a considerable reduction in computing costs.

RKHS Representations for Augmented Quaternion Random Signals: Application to Detection Problems

The reproducing kernel Hilbert space (RKHS) methodology has shown to be a suitable tool for the resolution of a wide range of problems in statistical signal processing both in the real and complex domains. It relies on the idea of transforming the original functional data into an infinite series representation by projection onto an specific RKHS, which usually simplifies the statistical treatment without any loss of efficiency. Moreover, the advantages of quaternion algebra over real-valued three and four-dimensional vector algebra in the modelling of multidimensional data have been proven useful in much relatively recent research. This paper accordingly proposes a generic RKHS framework for the statistical analysis of augmented quaternion random vectors, which provide a complete description of their second order characteristics. It will allow us to exploit the full advantages of the RKHS theory in widely linear processing applications, such as signal detection. In particular, we address the detection of a quaternion signal disturbed by additive Gaussian noise and the discrimination between two quaternion Gaussian signals in continuous time.

Asynchronous functional linear regression models for longitudinal data in reproducing kernel Hilbert space.

Motivated by the analysis of longitudinal neuroimaging studies, we study the longitudinal functional linear regression model under asynchronous data setting for modeling the association between clinical outcomes and functional (or imaging) covariates. In the asynchronous data setting, both covariates and responses may be measured at irregular and mismatched time points, posing methodological challenges to existing statistical methods. We develop a kernel weighted loss function with roughness penalty to obtain the functional estimator and derive its representer theorem. The rate of convergence, a Bahadur representation, and the asymptotic pointwise distribution of the functional estimator are obtained under the reproducing kernel Hilbert space framework. We propose a penalized likelihood ratio test to test the nullity of the functional coefficient, derive its asymptotic distribution under the null hypothesis, and investigate the separation rate under the alternative hypotheses. Simulation studies are conducted to examine the finite-sample performance of the proposed procedure. We apply the proposed methods to the analysis of multitype data obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study, which reveals significant association between 21 regional brain volume density curves and the cognitive function. Data used in preparation of this paper were obtained from the ADNI database (adni.loni.usc.edu).

State-Space Kernelized Closed-Loop Identification of Nonlinear Systems

In this paper, we propose a non-parametric state-space identification approach for open-loop and closed-loop discrete-time nonlinear systems with multiple inputs and multiple outputs. Employing a least squares support vector machine (LS-SVM) approach in a reproducing kernel Hilbert space framework, a nonlinear auto-regressive model with exogenous terms is identified to provide a non-parametric estimate of the innovation noise sequence. Subsequently, this estimate is used to obtain a compatible non-parametric estimate of the state sequence in an unknown basis using kernel canonical correlation analysis. Finally, the estimate of the state sequence is used together with the estimated innovation noise sequence to find a non-parametric state-space model, again using a LS-SVM approach. The performance of the approach is analyzed in a simulation study with a nonlinear system operating both in open loop and closed loop. The identification approach can be viewed as a nonlinear counterpart of consistent subspace identification techniques for linear time-invariant systems operating in closed loop.

A reproducing kernel Hilbert space framework for inverse scattering problems within the Born approximation

In this work we develop a new reproducing kernel Hilbert space (RKHS) framework for inverse scattering problems using the Born approximation. We assume we have backscattered data of a field that is dependent on an unknown scattering potential. Our goal is to reconstruct or image this scattering potential. Assuming the scattering potential lies in a RKHS, we find that the imaging equation can be rewritten as the inner product of the desired unknown function with the adjoint of the forward operator applied to the kernel of the imaging operator. We therefore may choose the kernel of the imaging operator such that the adjoint applied to this kernel is precisely the reproducing kernel of the Hilbert space the reflectivity function lies in. In this way we are able to obtain an alternative definition of an ideal image. We will demonstrate this theory using synthetic aperture radar imaging as an example, though there are other applicable imaging modalities i.e. inverse diffraction and diffraction tomography [ 1 , 6 ]. We choose SAR as it was the motivating application for this work. We will compare the RKHS ideal imaging technique to the standard microlocal analytic ideal image from backprojection theory. Note this method requires a variation of the standard SAR data model with the assumption of a full two dimensional data collection surface as opposed to a one dimensional flight path, however we are able to perform imaging with a single frequency and avoid the approximations made in the backprojection imaging operator derivation.

Analysis of regularized least squares for functional linear regression model

In this paper, we study and analyze the regularized least squares for functional linear regression model. The approach is to use the reproducing kernel Hilbert space framework and the integral operators. We show with a more general and realistic assumption on the reproducing kernel and input data statistics that the rate of excess prediction risk by the regularized least squares is minimax optimal.

Nonparametric inference in generalized functional linear models

We propose a roughness regularization approach in making nonparametric inference for generalized functional linear models. In a reproducing kernel Hilbert space framework, we construct asymptotically valid confidence intervals for regression mean, prediction intervals for future response and various statistical procedures for hypothesis testing. In particular, one procedure for testing global behaviors of the slope function is adaptive to the smoothness of the slope function and to the structure of the predictors. As a by-product, a new type of Wilks phenomenon [Ann. Math. Stat. 9 (1938) 60-62; Ann. Statist. 29 (2001) 153-193] is discovered when testing the functional linear models. Despite the generality, our inference procedures are easy to implement. Numerical examples are provided to demonstrate the empirical advantages over the competing methods. A collection of technical tools such as integro-differential equation techniques [Trans. Amer. Math. Soc. (1927) 29 755-800; Trans. Amer. Math. Soc. (1928) 30 453-471; Trans. Amer. Math. Soc. (1930) 32 860-868], Stein's method [Ann. Statist. 41 (2013) 2786-2819] [Stein, Approximate Computation of Expectations (1986) IMS] and functional Bahadur representation [Ann. Statist. 41 (2013) 2608-2638] are employed in this paper.

A reproducing kernel Hilbert space constructive approximation for a special class of integral equations

A reproducing kernel Hilbert space approach is proposed to study a class of integral equations with Toeplitz and Hankel kernel functions. The existence property and approximate representations of the solutions are given by constructing appropriate auxiliary operators and positive definite matrices within a reproducing kernel Hilbert space framework. Moreover, conditions for the boundedness and uniqueness of the solution are also obtained.

Quadratic Fourier Transforms

In this paper we shall examine the quadratic Fourier transform which is introduced by the generalized quadratic function for one order parameter in the ordinary Fourier transform. This will be done by analyzing corresponding six subcases of the quadratic Fourier transform within a reproducing kernel Hilbert spaces framework.

Fractional Functions and their Representations

For arbitrary non-identically zero functions f , we will introduce some natural fractional functions f1 having f as denominators and we shall consider their representations f1 by appropriate numerator functions within a reproducing kernel Hilbert spaces framework. That is, in the present work we would like to introduce very general fractional functions (e.g., having the possibility of admitting zeros in their denominators) by means of the theory of reproducing kernels.

Image and Video Colorization Using Vector-Valued Reproducing Kernel Hilbert Spaces

Motivated by the setting of reproducing kernel Hilbert space (RKHS) and its extensions considered in machine learning, we propose an RKHS framework for image and video colorization. We review and study RKHS especially in vectorial cases and provide various extensions for colorization problems. Theory as well as a practical algorithm is proposed with a number of numerical experiments.

A Reproducing Kernel Hilbert Space Framework for Spike Train Signal Processing

This letter presents a general framework based on reproducing kernel Hilbert spaces (RKHS) to mathematically describe and manipulate spike trains. The main idea is the definition of inner products to allow spike train signal processing from basic principles while incorporating their statistical description as point processes. Moreover, because many inner products can be formulated, a particular definition can be crafted to best fit an application. These ideas are illustrated by the definition of a number of spike train inner products. To further elicit the advantages of the RKHS framework, a family of these inner products, the cross-intensity (CI) kernels, is analyzed in detail. This inner product family encapsulates the statistical description from the conditional intensity functions of spike trains. The problem of their estimation is also addressed. The simplest of the spike train kernels in this family provide an interesting perspective to others' work, as will be demonstrated in terms of spike train distance measures. Finally, as an application example, the RKHS framework is used to derive a clustering algorithm for spike trains from simple principles.

LM-g splines

As an extension of the notion of an L-g spline, three mathematical structures called LM-g splines of types I, II, and III are introduced. Each is defined in terms of two differential operators L = D n + ∑ j=0 n−1 a j(t)D j and M = ∑ i=0 m b i(t)D i, where n ⩾ m ges; 0, n > 0, D = d dt the coefficients a j , j = 0,…, n − 1, and b i , i = 0,…, m, are sufficiently smooth; and b m is bounded away from zero on [0, T]. Each of the above types of splines is the solution of an optimization problem more general than the one used in the definition of the L-g spline and hence it is recognized as an entity which is distinct from and more general mathematically than the L-g spline. The LM-g splines introduced here reduce to an L-g spline in the special case in which m = 0 and b 0 = constant ≠ 0. After the existence and uniqueness conditions, characterization, and best approximation properties for the proposed splines are obtained in an appropriate reproducing kernel Hilbert space framework, their usefulness in extending the range of applicability of spline theory to problems in estimation, optimal control, and digital signal processing are indicated. Also, as an extension of recent results in the generalized spline literature, state variable models for the LM-g splines introduced here are exhibited, based on which existing least squares algorithms can be used for the recursive calculation of these splines from the data.