Class Of Kernels Research Articles

Fast and Exact Leave-One-Out Analysis of Large-Margin Classifiers

Motivated by the Golub–Heath–Wahba formula for ridge regression, we first present a new leave-one-out lemma for the kernel support vector machines (SVM) and related large-margin classifiers. We then use the lemma to design a novel and efficient algorithm, named “magicsvm,” for training the kernel SVM and related large-margin classifiers and computing the exact leave-one-out cross-validation error. By “magicsvm,” the computational cost of leave-one-out analysis is of the same order of fitting a single SVM on the training data. We show that “magicsvm” is much faster than the state-of-the-art SVM solvers based on extensive simulations and benchmark examples. The same idea is also used to boost the computation speed of the V-fold cross-validation of the kernel classifiers.

Characterizations of Herglotz\u2013Nevanlinna Functions Using Positive Semi-Definite Functions and the Nevanlinna Kernel in Several Variables

In this paper, we give several characterizations of Herglotz–Nevanlinna functions in terms of a specific type of positive semi-definite functions called Poisson-type functions. This allows us to propose a multidimensional analogue of the classical Nevanlinna kernel and a definition of generalized Nevanlinna functions in several variables. Furthermore, a characterization of the symmetric extension of a Herglotz–Nevanlinna function is also given. The subclass of Loewner functions is discussed as well, along with an interpretation of the main result in terms of holomorphic functions on the unit polydisk with non-negative real part.

Kernel density estimation in mixture models with known mixture proportions.

In this article, we consider the density estimation for data with a mixture structure, where the component densities are assumed unknown, but for each observation, the probabilities of its membership to the subpopulations are known or estimable from other resources. Data of this kind arise from practice and have wide applications. Motivated from the classical kernel density estimation method for a single population, we propose a weighted kernel density estimation method to estimate the component density functions nonparametrically. Within the framework of the EM algorithm, we derive an algorithm that computes our proposed estimates effectively. Via extensive simulation studies, we demonstrate that our methods outperform the existing methods in most occasions. We further compare our methods with existing methods by real data examples.

Nonlinear fractional distributed Halanay inequality and application to neural network systems

The standard first order distributed Halanay inequality is generalized in more than one direction. We prove a fractional nonlinear version of this inequality for a large class of kernels which are not necessarily exponentially decaying to zero. This result is used to prove Mittag-Leffler stability of a Hopfiled neural network system with not necessarily globally Lipschitz continuous activation functions. Two classes of important admissible kernels and an example are provided to illustrate our findings.

Learning 2D Gabor filters by infinite kernel learning regression

Gabor functions have wide-spread applications both in analyzing the visual cortex of mammalians and in designing machine vision algorithms. It is known that the receptive field of neurons of V1 layer in the visual cortex can be accurately modeled by Gabor functions. In addition, Gabor functions are extensively used for feature extraction in machine vision tasks. In this paper, we prove that Gabor functions are translation-invariant positive-definite kernels and show that the problem of image representation with Gabor functions can be formulated as infinite kernel learning regression. Specifically, we use the stabilized infinite kernel learning regression algorithm that has already been introduced for learning translation-invariant positive-definite kernels and has enough flexibility and generality to embrace the class of Gabor kernels. The algorithm yields a representation of the image as a support vector expansion with a compound kernel that is a finite mixture of Gabor functions. The problem with this representation is that all Gabor functions are present at all support vector pixels. Using LASSO, we propose a method for sparse representation of an image with Gabor functions in which each Gabor function is positioned at a very sparse set of pixels. As a practical application, we introduce a novel method for learning a dataset-specific set of Gabor filters that can be used subsequently for feature extraction. Our experiments on CMU-PIE and Extended Yale B datasets show that use of the learned Gabor filters significantly improves the recognition accuracy of a recently introduced face recognition algorithm.

RESEARCH ON CROP INFORMATION EXTRACTION OF AGRICULTURAL UAV IMAGES BASED ON BLIND IMAGE DEBLURRING TECHNOLOGY AND SVM

The blurring of crop images acquired by agricultural Unmanned Aerial Vehicle (UAV) due to sudden inputs by operators, atmospheric disturbance, and many other factors will eventually affect the subsequent crop identification, information extraction, and yield estimation. Aiming at the above problems, the new proposed combined deblurring algorithm based on the re-weighted graph total variation (RGTV) and L0-regularized prior, and the other two representative deblurring algorithms were applied to restore blurry crop images acquired during UAV flight, respectively. The restoration performance was measured by subjective vision, and objective evaluation indexes. The crop shape-related and texture-related feature parameters were then extracted, the Support Vector Machine (SVM) classifier with four common kernel functions was implemented for crop classification to realize the purpose of crop information extraction. The deblurring results showed that the proposed algorithm performed better in suppressing the ringing effect and preserving the image fine details, and retained higher objective evaluation indexes than the other two deblurring algorithms. The comparative analysis of different classification kernel functions showed that the Polynomial kernel function with an average recognition rate of 94.83% was most suitable for crop classification and recognition. The research will help in further popularization of crop monitoring based on UAV low-altitude remote sensing.

Stability for a retarded impulsive Cohen–Grossberg BAM neural network system

ABSTRACT In this paper, an impulsive Cohen-Grossberg bidirectional associative neural network with both time-varying and distributed delays is examined. Novel sufficient conditions for deriving stability with a desired rate, including the exponential one, are obtained. We consider a large class of admissible kernels encompassing the existing ones. Our findings cover the existing stability results in the literature. Finally, a numerical example is given for the validation of the theoretical outcomes.

Towards understanding the power of quantum kernels in the NISQ era

A key problem in the field of quantum computing is understanding whether quantum machine learning (QML) models implemented on noisy intermediate-scale quantum (NISQ) machines can achieve quantum advantages. Recently, Huang et al. [Nat Commun 12, 2631] partially answered this question by the lens of quantum kernel learning. Namely, they exhibited that quantum kernels can learn specific datasets with lower generalization error over the optimal classical kernel methods. However, most of their results are established on the ideal setting and ignore the caveats of near-term quantum machines. To this end, a crucial open question is: does the power of quantum kernels still hold under the NISQ setting? In this study, we fill this knowledge gap by exploiting the power of quantum kernels when the quantum system noise and sample error are considered. Concretely, we first prove that the advantage of quantum kernels is vanished for large size of datasets, few number of measurements, and large system noise. With the aim of preserving the superiority of quantum kernels in the NISQ era, we further devise an effective method via indefinite kernel learning. Numerical simulations accord with our theoretical results. Our work provides theoretical guidance of exploring advanced quantum kernels to attain quantum advantages on NISQ devices.

Path-integral approach to the thermodynamics of bosons with memory: Partition function and specific heat

For a system of bosons that interact through a class of general memory kernels, a recurrence relation for the partition function is derived within the path-integral formalism. This approach provides a generalization to previously known treatments in the literature of harmonically coupled systems of identical particles. As an example the result is applied to the specific heat of a simplified model of an open quantum system of bosons, harmonically coupled to a reservoir of distinguishable fictitious masses.

Enhancing the Scalability of Multi-FPGA Stencil Computations via Highly Optimized HDL Components

Stencil-based algorithms are a relevant class of computational kernels in high-performance systems, as they appear in a plethora of fields, from image processing to seismic simulations, from numerical methods to physical modeling. Among the various incarnations of stencil-based computations,Iterative Stencil Loops (ISLs)andConvolutional Neural Networks (CNNs)represent two well-known examples of kernels belonging to the stencil class. Indeed, ISLs apply the same stencil several times until convergence, while CNN layers leverage stencils to extract features from an image. The computationally intensive essence of ISLs, CNNs, and in general stencil-based workloads, requires solutions able to produce efficient implementations in terms of throughput and power efficiency. In this context, FPGAs are ideal candidates for such workloads, as they allow design architectures tailored to the stencil regular computational pattern. Moreover, the ever-growing need for performance enhancement leads FPGA-based architectures to scale to multiple devices to benefit from a distributed acceleration. For this reason, we propose a library of HDL components to effectively compute ISLs and CNNs inference on FPGA, along with a scalable multi-FPGA architecture, based on custom PCB interconnects. Our solution eases the design flow and guarantees both scalability and performance competitive with state-of-the-art works.

Towards a model-independent reconstruction approach for late-time Hubble data

Gaussian processes offers a convenient way to perform nonparametric reconstructions of observational data assuming only a kernel which describes the covariance between neighbouring points in a data set. We approach the ambiguity in the choice of kernel in Gaussian processes with two methods — (a) approximate Bayesian computation with sequential Monte Carlo sampling and (b) genetic algorithm — and use the overall resulting method to reconstruct the cosmic chronometers and supernovae type Ia data sets.The results have shown that the Matérn( ν = 5/2 ) kernel emerges on top of the two-hyperparameter family of kernels for both cosmological data sets. On the other hand, we use the genetic algorithm in order to select a most naturally-fit kernel among a competitive pool made up of a ten-hyperparameters class of kernels. Imposing a Bayesian information criterion-inspired measure of the fitness, the results have shown that a hybrid of the Radial Basis Function and the Matérn( ν = 5/2 ) kernel best represented both data sets. The kernel selection problem is not totally closed and may benefit from further analysis using other strategies to resolve an optimal kernel for a particular data set.

Dynamics of partially mitigated multi-phasic epidemics at low susceptible depletion: phases of COVID-19 control in Italy as case study

To mitigate the harmful effects of the COVID-19 pandemic, world countries have resorted – though with different timing and intensities – to a range of interventions. These interventions and their relaxation have shaped the epidemic into a multi-phase form, namely an early invasion phase often followed by a lockdown phase, whose unlocking triggered a second epidemic wave, and so on. In this article, we provide a kinematic description of an epidemic whose time course is subdivided by mitigation interventions into a sequence of phases, on the assumption that interventions are effective enough to prevent the susceptible proportion to largely depart from 100% (or from any other relevant level). By applying this hypothesis to a general SIR epidemic model with age-since-infection and piece-wise constant contact and recovery rates, we supply a unified treatment of this multi-phase epidemic showing how the different phases unfold over time. Subsequently, by exploiting a wide class of infectiousness and recovery kernels allowing reducibility (either to ordinary or delayed differential equations), we investigate in depth a low-dimensional case allowing a non-trivial full analytical treatment also of the transient dynamics connecting the different phases of the epidemic. Finally, we illustrate our theoretical results by a fit to the overall Italian COVID-19 epidemic since March 2020 till February 2021 i.e., before the mass vaccination campaign. This show the abilities of the proposed model in effectively describing the entire course of an observed multi-phasic epidemic with a minimal set of data and parameters, and in providing useful insight on a number of aspects including e.g., the inertial phenomena surrounding the switch between different phases.

A predictive intelligence system of credit scoring based on deep multiple kernel learning

Banks face the task of improving the accuracy in predicting the behavior of individuals who utilize credit cards, as issuing cards to an appropriate applicant is considered an important matter. Credit card debt that is overdue by at least six months has seen a rapid increase in the past decade in the Chinese credit card industry. The problem of delinquency in credit cards not only affects the development of credit cards but it also influences the sustainability of banks. The use of credit risk assessment is critical in providing a solid foundation upon which the credit card issuer can appropriately approve an applicant for a credit card. In previous studies, a variety of machine learning methods have been proposed to assess credit risk. However, conventional methods are viewed as shallow models and are not good at representing compositional features. Thus, this study applies a deep multiple kernel classifier as a state-of-the-art technique, which is proficient in coping with deep structure and complex data in credit risk assessment. It will support decision-makers issuing credit cards in China appropriately. The results indicate that deep multiple kernel classifier outperforms conventional and ensemble models. Credit card departments with better risk management can avoid possible bad debt, hence benefiting banks’ operations. The applications of predictive intelligence enhance the prediction of human behavior in the credit card industry.

Statistical Model Based Breast Tumor Classification in Contourlet Transform Domain

Determination of breast tumors from B-Mode Ultrasound (US) image is a perplexing one. Researches employing statistical modeling such as Nakagami, Normal Inverse Gaussian (NIG) distributed parametric images in this classification task have already explored but experimentation of those statistical models on contourlet transformed coefficient image in breast tumor classification task has not reported yet. The proposed method is established by considering 250 clinical cases from a publicly available database. In this database each clinical case exists as *.bmp format. In the preprocessing step firstly, the ultrasound B-Mode image is binarized to detect the lesion contour. Then contourlet transformation is employed. These contourlet sub band coefficients are shown to be modeled effectively by Nakagami and NIG distributions. These Nakagami and NIG parametric images are obtained by estimating the parameters of those prior statistical distributions locally. Few shape and statistical features are chosen according to their effectiveness on those parametric images. The benign and malignant breast tumors are classified utilizing these features with different classifiers such as the support vector machine, k-nearest neighbors, fitted binary classification decision tree, binary Gaussian kernel classification model, linear classification models for binary learning with high-dimensional etc. It is observed that classification performance of NIG statistical model based parametric version of contourlet coefficient images gained better accuracy than those of Nakagami statistical model.
 GUB JOURNAL OF SCIENCE AND ENGINEERING, Vol 7, Dec 2020 P 21-26

Frontal Cortex Neuron Type Classification with Deep Learning and Recurrence Plot

One of the goals of neural decoding in neuroscience is to create Brain-Computer Interfaces (BCI) that use nerve signals. In this context, we are interested in the activity of nerve cells. It is possible to classify nerve cells as excitatory or inhibitors by evaluating individual extra-cellular measurements taken from the frontal cortex of rats. Classification of neurons with only spike timing values has not been studied before, with deep learning, without knowing all of the wave properties and the intercellular interactions. In this study, inter-spike interval values of individual neuronal spike sequences were converted into recurrence plot images to analyze as point processing, image features were extracted using the pre-trained AlexNet with CNN deep learning method, and frontal cortex nerve cell type classification was made. Kernel classification, SVM, Naive Bayes, Ensemble, decision trees classification methods were used. The accuracy, sensitivity and specificity evaluate the proposed methods. A success of more than 81% has been achieved. Thus, the cell type is defined automatically. It has been observed that the ISI properties of spike trains can carry out information on cell type and thus neural network activity. Under these circumstances, these values are significant and important for neuroscientists.

The Vectorial Kernel Method for Walks with Longer Steps

Asinowski, Bacher, Banderier and Gittenberger [1] recently developed the vectorial kernel method – a powerful extension of the classical kernel method useable for paths that obey constraints that can be described by finite automata, e.g. avoid a fixed pattern, avoid several patterns at once, stay in a horizontal strip and many others more. However, they only considered walks with steps of length one. In this paper we will generalize their results to walks with longer steps. We will also give some applications of this extension and prove a conjecture about the asymptotic behavior of the expected number of ascents in Schröder paths.

Intelligent Aspect based Model for Efficient Sentiment Analysis of User Reviews

Mining online reviews to extract opinion targets and words are substantial tasks in fine-grained opinion mining. The main aim is to mine sensible multi-grain aspects and opinion words from unlabeled reviews. In this paper, Combined Aspect based Sentiment Model (CASM) is propounded to cooperatively mine multi-grain features and opinions. CASM deals with aspects, opinions, sentiment polarity and granularity concurrently. Support Vector Machine-Radial Basis Function kernel (SVM-RBF) classifier is applied to improve CASM by splitting aspects and opinion words. CASM-SVM-RBF deals with two different kinds of aspects and opinions: general and particular aspects, overall opinions and aspect-precise opinions. Candidates with improved confidence are mined as opinion targets or words. Index Terms:Aspects, Opinions, Sentiment, Support Vector Machine, Radial Basis Function, Support Vector Machine (SVM), Radial Basis Function (RBF)

Specific radar emitter identification based on two stage multiple kernel extreme learning machine

To make full use of the discriminative information containing in the whole ambiguity function (AF) plane, a novel two stage multiple kernel extreme learning machine (TSMKELM) method for specific radar emitter identification is proposed. Firstly, the AF plane is segmented into the non-overlapping Doppler shift stripes and each stripe is encoded as a kernel. Next, the discrimination of these stripes is evaluated and sorted according to the kernel discriminant ratio (KDR) criterion, which is in line with the large margin principle of KELM. Then, only the stripes with large KDRs are kept and the combined kernel is calculated by directly using the normalized KDRs as combination weights. At last, the KELM classifier is employed to fulfil the individual identification task. The proposed algorithm, named as KDR-TSMKELM, solves the kernel combination weights and kernel classifier parameter separately, bringing much efficiency in practice. Experiments on two real radar datasets validate the proposed algorithm.

The equivalent parameter conditions for constructing multiple integral half-discrete Hilbert-type inequalities with a class of nonhomogeneous kernels and their applications

Abstract In this paper, we establish equivalent parameter conditions for the validity of multiple integral half-discrete Hilbert-type inequalities with the nonhomogeneous kernel G ( n λ 1 ∥ x ∥ m , ρ λ 2 ) G\left({n}^{{\lambda }_{1}}\parallel x{\parallel }_{m,\rho }^{{\lambda }_{2}}\hspace{-0.16em}) ( λ 1 λ 2 > 0 {\lambda }_{1}{\lambda }_{2}\gt 0 ) and obtain best constant factors of the inequalities in specific cases. In addition, we also discuss their applications in operator theory.

Fast multivariate empirical cumulative distribution function with connection to kernel density estimation

The problem of computing empirical cumulative distribution functions (ECDF) efficiently on large, multivariate datasets, is revisited. Computing an ECDF at one evaluation point requires O(N) operations on a dataset composed of N data points. Therefore, a direct evaluation of ECDFs at N evaluation points requires a quadratic O(N2) operations, which is prohibitive for large-scale problems. Two fast and exact methods are proposed and compared. The first one is based on fast summation in lexicographical order, with a O(Nlog⁡N) complexity and requires the evaluation points to lie on a regular grid. The second one is based on the divide-and-conquer principle, with a O(Nlog⁡(N)(d−1)∨1) complexity and requires the evaluation points to coincide with the input points. The two fast algorithms are described and detailed in the general d-dimensional case, and numerical experiments validate their speed and accuracy. Secondly, a direct connection between cumulative distribution functions and kernel density estimation (KDE) is established for a large class of kernels. This connection paves the way for fast exact algorithms for multivariate kernel density estimation and kernel regression. Numerical tests with the Laplacian kernel validate the speed and accuracy of the proposed algorithms. A broad range of large-scale multivariate density estimation, cumulative distribution estimation, survival function estimation and regression problems can benefit from the proposed numerical methods.