Kernel Representation Research Articles

Multiple kernel clustering with late fusion consensus local graph preserving

Multiple kernel clustering (MKC) methods aim at integrating an optimal partition from a set of precalculated kernel matrices. Though achieving success in various applications, we observe that existing MKC methods: (i) lack of representation flexibility; and (ii) do not considerably preserve the locality structure in partition space. These issues may adversely affect the learning procedure of MKC, leading to unsatisfying clustering performance. In this paper, we propose a late fusion MKC method with local graph refinement to address the aforementioned issues. Different from existing MKC mechanisms, our method unifies the traditional weighted multiple kernel k-means, kernel partition, and graph construction into a single optimization procedure. The local graph is utilized to preserve the locality information in partition space and therefore all of the counterparts can be boosted for mutual clustering improvements. By this way, our approach enhances the local graph structure in partition space and enjoys more flexible kernel representations, leading to significant clustering improvements. Moreover, a three-step alternate algorithm is developed to solve the resultant optimization problem with proved convergence. Extensive experiments are conducted on several multiple kernel benchmark datasets to compare the proposed algorithm with the state-of-the-art ones, and the results well demonstrate its effectiveness and superiority.

Component-Level Fault Detection for Suspension System of Maglev Trains Based on Autocorrelation Length and Stable Kernel Representation

At present, in the suspension system of maglev train, the self-diagnosis of most of the components can be realized employing direct sensor measurement, however, there are still some components that cannot be directly measured by sensors or model-based methods, such as the power components in the suspension control box, electromagnetics and related connectors. And although the detection method based on Stable Kernel Representation (SKR) can detect the fault, the length of the data used for SKR affects the speed and results of the detection. To obtain better detection results with the shortest possible data, this paper studies a component-level fault detection method based on autocorrelation length and SKR. This method uses the autocorrelation length to determine the length of the data used to identify the SKR, and then applies the SKR to build a residual generator and set the residual threshold. The experimental results show that the studied method can detect faults of the suspension control box in real-time and effectively, and the simulation results show that the studied method can detect faults of the electromagnet in real-time and effectively. In addition, compared with the traditional method, the method in this paper can obtain better results with less data.

Modified kernel MLAA using autoencoder for PET-enabled dual-energy CT.

Combined use of PET and dual-energy CT provides complementary information for multi-parametric imaging. PET-enabled dual-energy CT combines a low-energy X-ray CT image with a high-energy γ-ray CT (GCT) image reconstructed from time-of-flight PET emission data to enable dual-energy CT material decomposition on a PET/CT scanner. The maximum-likelihood attenuation and activity (MLAA) algorithm has been used for GCT reconstruction but suffers from noise. Kernel MLAA exploits an X-ray CT image prior through the kernel framework to guide GCT reconstruction and has demonstrated substantial improvements in noise suppression. However, similar to other kernel methods for image reconstruction, the existing kernel MLAA uses image intensity-based features to construct the kernel representation, which is not always robust and may lead to suboptimal reconstruction with artefacts. In this paper, we propose a modified kernel method by using an autoencoder convolutional neural network (CNN) to extract an intrinsic feature set from the X-ray CT image prior. A computer simulation study was conducted to compare the autoencoder CNN-derived feature representation with raw image patches for evaluation of kernel MLAA for GCT image reconstruction and dual-energy multi-material decomposition. The results show that the autoencoder kernel MLAA method can achieve a significant image quality improvement for GCT and material decomposition as compared to the existing kernel MLAA algorithm. A weakness of the proposed method is its potential over-smoothness in a bone region, indicating the importance of further optimization in future work.This article is part of the theme issue ‘Synergistic tomographic image reconstruction: part 2’.

Data-driven design of fault detection and isolation method for distributed homogeneous systems

Driven by the increasing needs in large-scale industrial processes for maintaining stable operation using process data, a data-driven fault detection and isolation (FDI) method for distributed homogeneous systems is proposed in this paper. To this end, the original distributed homogeneous system is first decomposed into several separate subsystems using invertible transformations of the state, input, and output variables. On this basis, the fault detection method according to the one-step identification of the stable kernel representation (SKR) is developed for the transformed subsystems. It is followed by the fault estimation for the transformed subsystems along with implementation on fault isolation for the original distributed system. Experiments comparing the proposed FDI method and the centralized FD method are performed using a numerical example. In this specific example, the proposed FDI method has a computational complexity of 2.32×107, while the centralized FD method has a computational complexity of 9.31×107. This shows the effectiveness and advantages of the proposed method.

Real Time Analysis Based on Reproducing Kernel Henderson Filters

Recently, reproducing kernel Hilbert spaces have been introduced to provide a common approach for studying several nonparametric estimators used for smoothing functional time series data (Dagum and Bianconcini, 2006 and 2008). The reproducing kernel representation is based on the derivation of the density function (i.e. a second order kernel) embedded on the linear filter. This is the starting point for deriving higher order kernels, which are obtained from the product of the density and its orthonormal polynomials. This paper focuses on the Henderson filter, for which two density functions and corresponding hierarchies have been derived. The properties of the Henderson reproducing kernels are analyzed when the filters are adapted at the end of the sample period. The optimality criterion satisfied as well as the influence of the kernel order and bandwidth parameter are studied.

Probabilistic Semi-Supervised Learning via Sparse Graph Structure Learning.

We present a probabilistic semi-supervised learning (SSL) framework based on sparse graph structure learning. Different from existing SSL methods with either a predefined weighted graph heuristically constructed from the input data or a learned graph based on the locally linear embedding assumption, the proposed SSL model is capable of learning a sparse weighted graph from the unlabeled high-dimensional data and a small amount of labeled data, as well as dealing with the noise of the input data. Our representation of the weighted graph is indirectly derived from a unified model of density estimation and pairwise distance preservation in terms of various distance measurements, where latent embeddings are assumed to be random variables following an unknown density function to be learned, and pairwise distances are then calculated as the expectations over the density for the model robustness to the data noise. Moreover, the labeled data based on the same distance representations are leveraged to guide the estimated density for better class separation and sparse graph structure learning. A simple inference approach for the embeddings of unlabeled data based on point estimation and kernel representation is presented. Extensive experiments on various data sets show promising results in the setting of SSL compared with many existing methods and significant improvements on small amounts of labeled data.

Modelling of rectangular Kirchhoff plate oscillations under unsteady elastodiffusive perturbations

We study unsteady elastic-diffusion vibrations of a rectangular isotropic Kirchhoff plate. The coupled elastic-diffusion multicomponent continuum model is used to formulate the problem. We are using the d’Alembert variational principle to obtain from the model the equations of longitudinal and transverse vibrations of a rectangular isotropic elastodiffusive Kirchhoff plate. The problem solution is sought in integral form. The kernels of integral representations are the Green’s functions. For their determination, the Laplace transform and expansion in double Fourier series are used. As an example, we consider the bending of a simply supported plate. The effect of bending on the diffusion processes in the plate is investigated.

PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text

Scene text detection and recognition have been well explored in the past few years. Despite the progress, efficient and accurate end-to-end spotting of arbitrarily-shaped text remains challenging. In this work, we propose an end-to-end text spotting framework, termed PAN++, which can efficiently detect and recognize text of arbitrary shapes in natural scenes. PAN++ is based on the kernel representation that reformulates a text line as a text kernel (central region) surrounded by peripheral pixels. By systematically comparing with existing scene text representations, we show that our kernel representation can not only describe arbitrarily-shaped text but also well distinguish adjacent text. Moreover, as a pixel-based representation, the kernel representation can be predicted by a single fully convolutional network, which is very friendly to real-time applications. Taking the advantages of the kernel representation, we design a series of components as follows: 1) a computationally efficient feature enhancement network composed of stacked Feature Pyramid Enhancement Modules (FPEMs); 2) a lightweight detection head cooperating with Pixel Aggregation (PA); and 3) an efficient attention-based recognition head with Masked RoI. Benefiting from the kernel representation and the tailored components, our method achieves high inference speed while maintaining competitive accuracy. Extensive experiments show the superiority of our method. For example, the proposed PAN++ achieves an end-to-end text spotting F-measure of 64.9 at 29.2 FPS on the Total-Text dataset, which significantly outperforms the previous best method. Code will be available at: git.io/PAN.

KERNEL OPTIMIZATION FOR LOW-RANK MULTIFIDELITY ALGORITHMS

One of the major challenges for low-rank multi-fidelity (MF) approaches is the assumption that low-fidelity (LF) and high-fidelity (HF) models admit "similar" low-rank kernel representations. Low-rank MF methods have traditionally attempted to exploit low-rank representations of linear kernels, which are kernel functions of the form $K(u,v) = v^T u$ for vectors $u$ and $v$. However, such linear kernels may not be able to capture low-rank behavior, and they may admit LF and HF kernels that are not similar. Such a situation renders a naive approach to low-rank MF procedures ineffective. In this paper, we propose a novel approach for the selection of a near-optimal kernel function for use in low-rank MF methods. The proposed framework is a two-step strategy wherein: (1) hyperparameters of a library of kernel functions are optimized, and (2) a particular combination of the optimized kernels is selected, through either a convex mixture (Additive Kernels) or through a data-driven optimization (Adaptive Kernels). The two resulting methods for this generalized framework both utilize only the available inexpensive low-fidelity data and thus no evaluation of high-fidelity simulation model is needed until a kernel is chosen. These proposed approaches are tested on five non-trivial problems including multi-fidelity surrogate modeling for one- and two-species molecular systems, gravitational many-body problem, associating polymer networks, plasmonic nano-particle arrays, and an incompressible flow in channels with stenosis. The results for these numerical experiments demonstrate the numerical stability efficiency of both proposed kernel function selection procedures, as well as high accuracy of their resultant predictive models for estimation of quantities of interest. Comparisons against standard linear kernel procedures also demonstrate increased accuracy of the optimized kernel approaches.

Traffic Signal Prediction on Transportation Networks Using Spatio-Temporal Correlations on Graphs

Multivariate time series forecasting poses challenges as the variables are intertwined in time and space, like in the case of traffic signals. Defining signals on graphs relaxes such complexities by representing the evolution of signals over a space using relevant graph kernels such as the heat diffusion kernel. However, this kernel alone does not fully capture the actual dynamics of the data as it only relies on the graph structure. The gap can be filled by combining the graph kernel representation with data-driven models that utilize historical data. This paper proposes a traffic propagation model that merges multiple heat diffusion kernels into a data-driven prediction model to forecast traffic signals. We optimize the model parameters using Bayesian inference to minimize the prediction errors and, consequently, determine the mixing ratio of the two approaches. Such mixing ratio strongly depends on training data size and data anomalies, which typically correspond to the peak hours for traffic data. The proposed model demonstrates prediction accuracy comparable to that of the state-of-the-art deep neural networks with lower computational effort. It notably achieves excellent performance for long-term prediction through the inheritance of periodicity modeling in data-driven models.

A Random Fourier Feature Method for Emulating Computer Models With Gradient Information

Computer models with gradient information are increasingly used in engineering and science. The gradient-enhanced Gaussian process emulator can be used for emulating such models. Because the size of the covariance matrix increases proportionally with the dimension of inputs and the sample size, it is computationally challenging to fit such an emulator for large datasets. We propose a random Fourier feature method to mitigate this difficulty. The key idea of the proposed method is to employ random Fourier features to obtain an easily computable, low-dimensional feature representation for shift-invariant kernels involving gradients. The effectiveness of the proposed method is illustrated by several examples.

Multiscale Representation Learning of Graph Data With Node Affinity

Graph neural networks have emerged as a popular and powerful tool for learning hierarchical representation of graph data. In complement to graph convolution operators, graph pooling is crucial for extracting hierarchical representation of data in graph neural networks. However, most recent graph pooling methods still fail to efficiently exploit the geometry of graph data. In this paper, we propose a novel graph pooling strategy that leverages node affinity to improve the hierarchical representation learning of graph data. Node affinity is computed by harmonizing the kernel representation of topology information and node features. In particular, a structure-aware kernel representation is introduced to explicitly exploit advanced topological information for efficient graph pooling without eigendecomposition of the graph Laplacian. Similarities of node signals are evaluated using the Gaussian radial basis function (RBF) in an adaptive way. Experimental results demonstrate that the proposed graph pooling strategy is able to achieve state-of-the-art performance on a collection of public graph classification benchmark datasets.

Convolution algebra for extended Feller convolution

We apply the recently introduced framework of admissible homomorphisms in the form of a convolution algebra of $$\mathbb{C}^2$$ -valued admissible homomorphisms to handle two-dimensional uni-directional homogeneous stochastic kernels. The algebra product is a non-commutative extension of the Feller convolution needed for an adequate operator representation of such kernels: a pair of homogeneous transition functions uni-directionally intertwined by the extended Chapman–Kolmogorov equation is a convolution empathy; the associated Fokker–Planck equations are re-written as an implicit Cauchy equation expressed in terms of admissible homomorphisms. The conditions of solvability of such implicit evolution equations follow from the consideration of generators of a convolution empathy.

On Potential Functions Associated with Eigenfunctions of the Discrete Sturm–Liouville Operator

Potential functions associated with eigenfunctions of the discrete Sturm–Liouville operator are studied in a loaded space. On the basis of representations of kernels with respect to the corresponding orthogonal polynomials, an estimate for the growth of potential functions is obtained.

Kernel‐based low‐rank tensorized multiview spectral clustering

Multiview spectral clustering aims to separate data into different clusters efficiently by the use of multiview information. Many studies learn the affinity matrix from the original high-dimensional data, whose noise goes against the clustering results. Besides, some methods based on self-representation subspace clustering have a high time complexity. In this paper, we propose a simple, yet effective, and efficient method named Kernel-based Low-rank Tensorized Multiview Spectral Clustering (KLTMSC) to address these issues. Instead of using the original data to get the affinity matrix, KLTMSC learns the affinity matrix from kernel representation of the high-dimensional data to reduce the noisy information. Furthermore, to be robust to noise, the low-rank tensor is learned in the process of exploring the high-order correlations between data. Experiments on real-world data sets show that our method not only yields better results but also is quite time-saving compared with other state-of-the-art models.

Boolean Valued Representation of Random Sets and Markov Kernels with Application to Large Deviations

We establish a connection between random set theory and Boolean valued analysis by showing that random Borel sets, random Borel functions, and Markov kernels are respectively represented by Borel sets, Borel functions, and Borel probability measures in a Boolean valued model. This enables a Boolean valued transfer principle to obtain random set analogues of available theorems. As an application, we establish a Boolean valued transfer principle for large deviations theory, which allows for the systematic interpretation of results in large deviations theory as versions for Markov kernels. By means of this method, we prove versions of Varadhan and Bryc theorems, and a conditional version of Cramér theorem.

Data-driven design of fault-tolerant control systems based on recursive stable image representation

Driven by the increasing needs in the process industry for designing fault-tolerant control systems based on process data and maintaining its performance during online operation, data-driven realization and design of feed-forward fault-tolerant control systems with embedded residual generation are studied. To this end, a parameterized and data-driven realization form of stable image representation in a feedback control loop is derived first. On this basis, a feed-forward controller design scheme is proposed for fault-tolerant purpose. It is followed by a recursive approach for online tuning the feed-forward controller in case of malfunctions, which is triggered by a fault detection scheme based on the so-called stable kernel representation. The effectiveness of the proposed fault-tolerant method is demonstrated by a case study on the laboratory three-tank system.

Pseudo-differential operator in the framework of linear canonical transform domain

The main goal of this paper is to study properties of the linear canonical transform (LCT) on Schwartz-type space [Formula: see text]. The symbol class [Formula: see text] is introduced. The pseudo-differential operator (p.d.o.) involving LCT is defined and also some of its properties including boundedness are investigated in Sobolev-type space. Kernel and integral representation of p.d.o. are obtained. Some applications of LCT to generalized partial differential equations and canonical convolution integral equation have been solved.

Kernel representation of long-wave dynamics on a uniform slope.

Long-wave propagation on a uniformly sloping beach is formulated as a transient-response problem, with initially stationary water subjected to an incident wave. Both the water surface elevation and the horizontal flow velocity on the slope can be represented as convolutions of the rate of displacement of the water surface at the toe of the slope with a singular kernel function of time and space. The kernel, which is typically expressed in the form of an infinite series, accommodates the dynamic processes of long waves, such as shoaling, reflection, and multiple reflections over the slope and yields exact solutions of the linear shallow water equations for any smooth incident wave. The kernel convolution can be implemented numerically by using double exponential formulas to avoid the kernel singularity. The kernel formulation can be extended readily to nonlinear dynamics via the hodograph transform, which in turn enables the instantaneous prediction of nonlinear wave properties and of the occurrence of wave breaking in the near-shore area. This general description of long-wave dynamics provides new insights into the long-studied problem.

Multi-scale structural kernel representation for object detection

Existing high-performance object detection methods greatly benefit from the powerful representation ability of deep convolutional neural networks (CNNs). Recent researches show that integration of high-order statistics remarkably improves the representation ability of deep CNNs. However, high-order statistics for object detection lie in two challenges. Firstly, previous methods insert high-order statistics into deep CNNs as global representations, which lose spatial information of inputs, and so are not applicable to object detection. Furthermore, high-order statistics have special structures, which should be considered for proper use of high-order statistics. To overcome above challenges, this paper proposes a Multi-scale Structural Kernel Representation (MSKR) for improving performance of object detection. Our MSKR is developed based on the polynomial kernel approximation, which does not only draw into high-order statistics but also preserve the spatial information of input. To consider geometry structures of high-order representations, a feature power normalization method is introduced before computation of kernel representation. Comparing with the most commonly used first-order statistics in existing CNN-based detectors, our MSKR can generate more discriminative representations, and so be flexibly integrated into deep CNNs for improving performance of object detection. By adopting the proposed MSKR to existing object detection methods (i.e., Faster R-CNN, FPN, Mask R-CNN and RetinaNet), it achieves clear improvement on three widely used benchmarks, while obtaining very competitive performance with state-of-the-art methods.