Choice Of Kernel Function Research Articles

Traveling wave solution for a nonlocal Belousov-Zhabotinskii reaction-diffusion system

We consider a Belousov-Zhabotinskii reaction-diffusion system with nonlocal effects and study the existence of traveling wave solutions. By constructing appropriate super- and sub-solutions and using Schauder’s fixed point theorem, we show that there is a critical speed c * > 0 such that when the wave speed c > c *, there exists a traveling wave solution connecting (0, 0) to a positive steady-state, while there is no traveling wave solution when c < c *. Moreover, we also examine a special case where ϕ 1(x) is the Dirac function, and demonstrate the existence of the traveling wave solution connecting the equilibria (0, 0) and (1, 1) for c > c *, whereas the traveling wave solution does not exist when c < c *. Finally, the long-time behavior of the solution is investigated through numerical simulation and theoretical analysis, and it is found that the choice of kernel functions and the setting of initial value conditions play a crucial role.

Kernel Masked Image Modeling Through the Lens of Theoretical Understanding.

Masked image modeling (MIM) has been considered as the state-of-the-art (SOTA) self-supervised learning (SSL) technique in terms of visual pretraining. The impressive generalization ability of MIM also paves the way for the remarkable success of large-scale vision foundation models. In this article, we further discuss the validity and advantages of implementing MIM techniques in the reproducing kernel Hilbert spaces (RKHSs) and we associate the analysis with a novel MIM method named R-MIM (short for RKHS-MIM). Through the careful construction of an augmentation graph and by using spectral decomposition techniques, we establish a systematic theoretical understanding between the proposed R-MIM's generalization ability and the choice of kernel function used during training. Specifically, we reach a conclusion that both of the local Lipschitz constant of the resultant R-MIM model and the corresponding expected pretraining error can have a strong composite effect on bounding downstream task error, depending on the kernel options. We demonstrate that under mild mathematical assumptions, R-MIM method is guaranteed to return a lower bound on downstream tasks in comparison to vanilla MIM techniques, such as masked autoencoder (MAE) and SimMIM. Empirical justification well corroborates our theoretical hypothesis and analysis in showing the superior generalization of the proposed R-MIM and the theoretical link to kernel choices. The code is available at: https://github.com/yurui-q/R-MIM.

A framework on the right kernel and Lattice verses scale of data for better classification in Self organising map

The classification of huge complex datasets to extract the information needed for making informed decisions has gained a lot of importance in today's scenario due to the outspurt of data. With an increase in dataset size, the means of obtaining the desired information via data clustering techniques also became complex. This paper illustrates a framework for selecting the best combination of the Kernel function and the Lattice structure for better classification based on the size of the data. The performance of the SOM majorly depends on the choice of Kernel Function, Lattice Structures, and the size of the datasets. The types of kernel functions deployed in this study are Gaussian, Exponential, and Laplacian Kernel along with the lattice Structures such as hexagonal, Rectangular, and Square which result in nine combinations respectively. These nine combinations of the Kernel Function and the Lattice Structure have been tested on datasets ranging from small to large size using the performance metrics. A frequency table has been arrived at based on the classification of the dataset size, combinations of Kernel functions, and Lattice Structures. The results of this study infer that the combination of Gaussian Kernel -Square Lattice performs better for small-size datasets wherein the Exponential and Laplacian Kernel functions along with Hexagonal Lattice perform better for mid-size datasets, while the Laplacian Kernel – Square Lattice perform better for large size datasets.

Memory response on a piezoelectric rod under three-phase-lag model

ABSTRACT In this article, the concept of the memory-dependent derivative has been utilized to build up the heat equation to investigate the piezoelectric effect on a finite rod. The governing equations are written under the generalized theory of thermoelasticity based on the three-phase-lag model and solved in the Laplace transform domain. The numerical inverse Laplace transformation technique has been adapted to obtain the solution in the space–time domain. The effect of the different choices of kernel functions, time delay and comparison with other models are discussed and represented graphically. It is noted that the field variables attain the maximum amplitudes for the highest values of the time delay for a fixed kernel.

Machine learning modeling of power delivery networks with varying decoupling capacitors

&lt;p&gt;This paper presents modeling of power delivery network (PDN) impedance with varying decoupling capacitor placements using machine learning techniques. The use of multilayer perceptron artificial neural networks (ANN) and gaussian process regression (GPR) techniques are explored, and the effects of the hyperparameters such as the number of hidden neurons in the ANN, and the choice of kernel functions in the GPR are investigated. The best performing networks in each case are selected and compared in terms of accuracy using test data consisting of PDN impedance responses that were never encountered during training. Results show that the GPR models were significantly more accurate than the ANN models, with an average mean absolute error of 5.23 mΩ compared to 11.33 mΩ for the ANN.&lt;/p&gt;

Maximum Entropy Approach to Massive Graph Spectrum Learning with Applications

We propose an alternative maximum entropy approach to learning the spectra of massive graphs. In contrast to state-of-the-art Lanczos algorithm for spectral density estimation and applications thereof, our approach does not require kernel smoothing. As the choice of kernel function and associated bandwidth heavily affect the resulting output, our approach mitigates these issues. Furthermore, we prove that kernel smoothing biases the moments of the spectral density. Our approach can be seen as an information-theoretically optimal approach to learning a smooth graph spectral density, which fully respects moment information. The proposed method has a computational cost linear in the number of edges, and hence can be applied even to large networks with millions of nodes. We showcase the approach on problems of graph similarity learning and counting cluster number in the graph, where the proposed method outperforms existing iterative spectral approaches on both synthetic and real-world graphs.

Research on Application of Support Vector Machine in Intrusion Detection

There are some problems such as the high false negative rate and false positive rate in intrusion detection technology. The paper researches on the technology of intrusion detection based on support vector machine. The forming process of SVM classifier is analyzed. The choice of kernel function is discussed. The experimental results show that the SVM-based intrusion detection system with the appropriate kernel function has high detection-accuracy and it can effectively predict and classify the network attacks.

Physical Layer Security: Detection of Active Eavesdropping Attacks by Support Vector Machines

This article presents a framework for converting wireless signals into structured datasets, which can be fed into machine learning algorithms for the detection of active eavesdropping attacks at the physical layer. More specifically, a wireless communication system, which consists of an access point (AP), $K$ legitimate users and an active eavesdropper, is considered. To detect the eavesdropper who breaks into the system during the authentication phase, we first build structured datasets based on different features and then apply sophisticated support vector machine (SVM) classifiers to those structured datasets. To be more specific, we first process the signals received by the AP and then define a pair of statistical features based on the post-processing of the signals. By arranging for the AP to simulate the entire process of transmission and the process of constructing features, we form the so-called artificial training data (ATD). By training SVM classifiers on the ATD, we classify the received signals associated with eavesdropping attacks and non-attacks, thereby detecting the presence of the eavesdropper. Two SVM classifiers are considered, including a classic twin-class SVM (TC-SVM) and a single-class SVM (SC-SVM). While the TC-SVM is preferred in the case of having perfect channel state information (CSI) of all channels , the SC-SVM is preferred in the realistic scenario when we have only the CSI of legitimate users . We also evaluate the accuracy of the trained models depending on the choice of kernel functions, the choice of features and on the eavesdropper’s power. Our numerical results show that careful parameter-tuning is required for exceeding an eavesdropper detection probability of 95%

Latent likelihood ratio tests for assessing spatial kernels in epidemic models

One of the most important issues in the critical assessment of spatio-temporal stochastic models for epidemics is the selection of the transmission kernel used to represent the relationship between infectious challenge and spatial separation of infected and susceptible hosts. As the design of control strategies is often based on an assessment of the distance over which transmission can realistically occur and estimation of this distance is very sensitive to the choice of kernel function, it is important that models used to inform control strategies can be scrutinised in the light of observation in order to elicit possible evidence against the selected kernel function. While a range of approaches to model criticism is in existence, the field remains one in which the need for further research is recognised. In this paper, building on earlier contributions by the authors, we introduce a new approach to assessing the validity of spatial kernels—the latent likelihood ratio tests—which use likelihood-based discrepancy variables that can be used to compare the fit of competing models, and compare the capacity of this approach to detect model mis-specification with that of tests based on the use of infection-link residuals. We demonstrate that the new approach can be used to formulate tests with greater power than infection-link residuals to detect kernel mis-specification particularly when the degree of mis-specification is modest. This new tests avoid the use of a fully Bayesian approach which may introduce undesirable complications related to computational complexity and prior sensitivity.

On the Choice of Kernel Function in Nonlocal Wave Propagation

It is a challenge to choose the appropriate kernel function in nonlocal problems. We tackle this challenge from the aspect of nonlocal wave propagation and study the dispersion relation at the analytical level. The kernel function enters the formulation as an input. Any effort to narrow down this function family is valuable. Dispersion relations of the nonlocal governing operators are identified. Using a Taylor expansion, a selection criterion is devised to determine the kernel function that provides the best approximation to the classical (linear) dispersion relation. The criterion is based on selecting the smallest coefficient in magnitude of the dominant term in the Taylor expansion after the constant term. The governing operators are constructed using functional calculus, which provides the explicit expression of the eigenvalues of the operators. The ability to express eigenvalues explicitly allows us to obtain dispersion relations at the analytical level, thereby isolating the effect of discretization on the dispersion relation. With the presence of expressions of eigenvalues of the governing operator, the analysis is clear and accessible. The choices made to obtain the best approximation to the classical dispersion relation become completely transparent. We find that the truncated Gaussian family is the most effective compared to power and rational function families.

Distributed learning automata-based scheme for classification using novel pursuit scheme

Learning Automata (LA) is a popular decision making mechanism to “determine the optimal action out of a set of allowable actions” (Agache and Oommen, IEEE Trans Syst Man Cybern-Part B Cybern 2002(6): 738–749, 2002). The distinguishing characteristic of automata-based learning is that the search for the optimising parameter vector is conducted in the space of probability distributions defined over the parameter space, rather than in the parameter space itself (Thathachar and Sastry, IEEE Trans Syst Man Cybern-Part B Cybern 32(6): 711–722, 2002). Recently, Goodwin and Yazidi pioneered the use of Ant Colony Optimisation (ACO) for solving classification problems (Goodwin and Yazidi 2016). In this paper, we propose a novel classifier based on the theory of LA. The classification problem is formulated as a deterministic optimization problem involving a team of LA that operate collectively to optimize an objective function. Although many LA algorithms have been devised in the literature, those LA schemes are not able to solve deterministic optimization problems as they suppose that the environment is stochastic. In this paper, we develop a novel pursuit LA which can be seen as the counterpart of the family of pursuit LA developed for stochastic environments (Agache and Oommen, IEEE Trans Syst Man Cybern Part B Cybern 32(6): 738–749, 2002). While classical pursuit LA are able to pursue the action with the highest reward estimate, our pursuit LA rather pursues the collection of actions that yield the highest performance. The theoretical analysis of the pursuit scheme does not follow classical LA proofs and can pave the way towards more schemes where LA can be applied to solve deterministic optimization problems. When applied to classification, the essence of our scheme is to search for a separator in the feature space by imposing a LA based random walk in a grid system. To each node in the gird we attach an LA, whose actions are the choice of the edges forming the separator. The walk is self-enclosing, i.e., a new random walk is started whenever the walker returns to starting node forming a closed classification path yielding a multiedged polygon. In our approach, the different LA attached at the different nodes search for a polygon that best encircles and separates each class. Based on the obtained polygons, we perform classification by labelling items encircled by a polygon as part of a class using ray casting function. Seen from a methodological perspective, PolyPursuit-LA has appealing properties compared to SVM. In fact, unlike PolyPursuit-LA, the SVM performance is dependent on the right choice of kernel function (e.g. Linear Kernel, Gaussian Kernel)— which is considered a “black art”. PolyPursuit-LA can find arbitrarily complex separators in the feature space. Experimental results from both synthetic and real-life data show that our scheme is able to perfectly separate both simple and complex patterns outperforming existing classifiers, including polynomial and linear SVM, without the need of any “kernel trick”. We believe that the results are impressive given the simplicity of PolyPursuit-LA compared to other approaches such as SVM.

Face recognition using nonnegative matrix factorization with fractional power inner product kernel

Kernel nonnegative matrix factorization (KNMF) algorithms have been widely used to extract features for face recognition. The choice of kernel function is vital to facial feature extraction. The polynomial kernel function has been commonly used in KNMF. The power of the polynomial kernel is required to be a positive integer, thereby ensuring that the kernel generates a positive semi-definite matrix. In this paper, we investigate a new type of inner-product kernel that has a fractional power. The new kernel offers us flexibility in data representation as the power can be any positive real number. Based on the fractional power inner-product kernel, we present a novel KNMF algorithm called fractional power inner-product KNMF (FPKNMF). The FPKNMF algorithm is theoretically and experimentally validated to be convergent. The experimental results confirm that our algorithm exhibits a performance superior to the state-of-the-art methods in terms of facial representation and recognition accuracy.

Adaptive Image Mismatch Removal With Vector Field Interpolation Based on Improved Regularization and Gaussian Kernel Function

When the regularized kernel methods are utilized in the mismatch removal problem, the regularization coefficient and the choice of kernel function will seriously affect the performance of the methods. In this paper, we propose a method that combines an improved regularization and an adaptive Gaussian kernel function to interpolate the vector fields so as to overcome the issue. We formulated the problem as a modified maximum a posterior estimation of a Bayesian model. In this model, a two-order term of the regularization coefficient is introduced into the regularized risk function in order that the coefficient can be adaptively estimated in the expectation–maximization algorithm. In addition, an adaptive Gaussian kernel function also is imposed to construct the regularization, in which the width of the kernel function is adaptively determined by the diagonal length of the maximum enveloping rectangle of the sample set. Our experimental results verified that our method was robust to large outlier percentages and was slightly superior to some state-of-the-art methods in precision-recall tradeoff and efficiency. The evidence that the performance of our method was insensitive to the remaining inner parameters verified its good self-adaptability. Finally, airborne image pairs were used to demonstrate that our method can establish the feature correspondences even under a discontinuous vector field scene. In addition, we found that our method can obtain higher precision given a residual threshold for special applications such as robust epipolar geometry estimation in computer vision and photogrammetry.

Kernel fusion based extreme learning machine for cross-location activity recognition

Fixed placements of inertial sensors have been utilized by previous human activity recognition algorithms to train the classifier. However, the distribution of sensor data is seriously affected by the sensor placement. The performance will be degraded when the model trained on one placement is used in others. In order to tackle this problem, a fast and robust human activity recognition model called TransM-RKELM (Transfer learning mixed and reduced kernel Extreme Learning Machine) is proposed in this paper; It uses a kernel fusion method to reduce the influence by the choice of kernel function and the reduced kernel is utilized to reduce the computational cost. After realizing initial activity recognition model by mixed and reduced kernel extreme learning model (M-RKELM), in the online phase M-RKELM is utilized to classify the activity and adapt the model to new locations based on high confident recognition results in real time. Experimental results show that the proposed model can adapt the classifier to new sensor locations quickly and obtain good recognition performance.

A Novel multiple kernel-based dictionary learning for distributive and collective sparse representation based classifiers

In recent years, sparse representation theory has attracted the attention of many researchers in the signal processing, pattern recognition and computer vision communities. The choice of dictionary matrix plays a key role in the sparse representation based methods. It can be a pre-defined dictionary or can be learned via an optimization procedure. Furthermore, the dictionary learning process can be extended to a non-linear setting using an appropriate kernel function in order to handle non-linear structured data. In this framework, the choice of kernel function is also a key step. Multiple kernel learning is an appealing strategy for dealing with this problem. In this paper, within the framework of kernel sparse representation based classification, we propose an iterative algorithm for coincident learning of the dictionary matrix and multiple kernel function. The weighted sum of a set of basis functions is considered as the multiple kernel function where the weights are optimized such that the reconstruction error of the sparse coded data is minimized. In our proposed algorithm, the sparse coding, dictionary learning and multiple kernel learning processes are performed in three steps. The optimization process is performed considering two different structures namely distributive and collective for the sparse representation based classifier. Our experimental results show that the proposed algorithm outperforms the other existing sparse coding based approaches. These results also confirm that the collective setting leads to better results when the number of training examples is limited. On the other hand, the distributive setting is more appropriate when there are enough training samples.

Exploring different kernel functions for kernel-based clustering

Kernel methods are ones that, by replacing the inner product with positive definite function, implicitly perform a non-linear mapping of input data into a high dimensional feature space. Kernel functions are used to make this mapping in higher dimension redundant. These kernel functions play an important role in classification. The kernel-based clustering methods are found to be superior in accuracy to the conventional ones. The choice of kernel function is neither easy nor trivial. Various types of kernel based clustering methods have been studied so far by many researchers, where Gaussian kernel, in particular, has been found to be useful. In this study, we present a comprehensive comparative analysis of kernel based hybrid c-means clustering using different kernel functions. We have incorporated Mercer kernel functions positive definite kernels as well as conditionally positive definite kernel functions. Various synthetic datasets and real-life datasets are used for analysis. Experiments results show that there exist other robust kernel functions which hold like Gaussian kernel.

Multiple kernel boosting framework based on information measure for classification

The performance of kernel-based method, such as support vector machine (SVM), is greatly affected by the choice of kernel function. Multiple kernel learning (MKL) is a promising family of machine learning algorithms and has attracted many attentions in recent years. MKL combines multiple sub-kernels to seek better results compared to single kernel learning. In order to improve the efficiency of SVM and MKL, in this paper, the Kullback–Leibler kernel function is derived to develop SVM. The proposed method employs an improved ensemble learning framework, named KLMKB, which applies Adaboost to learning multiple kernel-based classifier. In the experiment for hyperspectral remote sensing image classification, we employ feature selected through Optional Index Factor (OIF) to classify the satellite image. We extensively examine the performance of our approach in comparison to some relevant and state-of-the-art algorithms on a number of benchmark classification data sets and hyperspectral remote sensing image data set. Experimental results show that our method has a stable behavior and a noticeable accuracy for different data set.

On enhancement of Incompressible SPH method for simulation of violent sloshing flows

The paper presents two schemes for enhancement of Incompressible SPH (Smoothed Particle Hydrodynamics)-based methods in simulation of violent sloshing flows, and in particular, sloshing induced impact pressures. The enhanced schemes include a Higher order Laplacian and an Error-Compensating Source of Poisson pressure equation, abbreviated as HL and ECS, respectively. These two schemes correspond to those derived within the framework of MPS (Moving Particle Semi-implicit) method and are proposed for the first time within the framework of SPH. The enhancing effects of HL and ECS schemes are shown by simulating violent sloshing flows induced by sway excitations and rotational ones. The significance of dynamically adjusted coefficients in the ECS-related schemes is highlighted by considering a previously applied scheme comprising of a constant coefficient and a corresponding newly proposed one which incorporates a dynamic coefficient. Concise insights are presented on appropriate choice of kernel function.

Evolutionary combination of kernels for nonlinear feature transformation

The performance of kernel-based feature transformation methods depends on the choice of kernel function and its parameters. In addition, most of these methods do not consider the classification information and error for the mapping features. In this paper, we propose to determine a kernel function for kernel principal components analysis (KPCA) and kernel linear discriminant analysis (KLDA), considering the classification information. To this end, we combine the conventional kernel functions using genetic algorithm and genetic programming in linear and non-linear forms, respectively. We use the classification error and the mutual information between features and classes in the kernel feature space as evolutionary fitness functions. The proposed methods are evaluated on the basis of the University of California Irvine (UCI) datasets and Aurora2 speech database. We evaluate the methods using clustering validity indices and classification accuracy. The experimental results demonstrate that KPCA using a nonlinear combination of kernels based on genetic programming and the classification error fitness function outperforms conventional KPCA using Gaussian kernel and also KPCA using linear combination of kernels.

High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso

The goal of supervised feature selection is to find a subset of input features that are responsible for predicting output values. The least absolute shrinkage and selection operator (Lasso) allows computationally efficient feature selection based on linear dependency between input features and output values. In this letter, we consider a feature-wise kernelized Lasso for capturing nonlinear input-output dependency. We first show that with particular choices of kernel functions, nonredundant features with strong statistical dependence on output values can be found in terms of kernel-based independence measures such as the Hilbert-Schmidt independence criterion. We then show that the globally optimal solution can be efficiently computed; this makes the approach scalable to high-dimensional problems. The effectiveness of the proposed method is demonstrated through feature selection experiments for classification and regression with thousands of features.