Kernel-based Methods Research Articles

A kernel-based method for fractional integro-differential equations with a weakly singular kernel in multi-dimensional complex domains

We present an efficient and practical numerical method for solving fractional integro-differential equations with a weakly singular kernel in complex two and three-dimensional domains. The reproducing kernel pseudo-spectral method is used to discretize the multidimensional regular and irregular spatial domains, while time discretization is done using a third-order backward differentiation formula. We have investigated the unconditional stability of the utilized temporal discretization. Finally, we have used the proposed method to solve the examples presented in the articles and compared the results. The obtained results show the accuracy and ability of the presented method.

Evaluating multivariate time-series clustering using simulated ecological momentary assessment data

During an Ecological Momentary Assessment (EMA) study, through repeated digital questionnaires, we have the opportunity to collect multiple multivariate time-series (MTS) data for all participants. Although, it is common that individual data is analyzed per participant, the richness of such dataset poses the question of whether meaningful groups of individuals could be uncovered to better understand the underlying processes on an individual and a group level. Such grouping could be obtained by clustering. Therefore, this paper examines the performance of various clustering approaches for grouping individuals based on the similarity of their raw time-series data patterns. Clustering is an unsupervised task, where the true underlying groups are not usually available, making the result difficult to evaluate. Therefore, in the current paper, simulated irregular time-series data, resembling EMA, are used to validate the performance of several methods under different clustering-related choices, such as the distance metric. Data are generated with a varying number of clusters, total number of individuals and time-points as well as number of variables and proportions of noisy variables, while their time-series represent well-shaped patterns, typically observed in emotional behavior. After applying clustering to all simulated datasets, clustering performance was first assessed by comparing the true and predicted labels, while the impact of the different datasets’ parameters was also examined. Because ground truth labels are not always available, or do not even exist, in real-world scenarios, clustering evaluation through distance-based and distance-free measures was further investigated. Overall, all clustering methods (e.g. k-means, Hierarchical clustering, Fuzzy k-medoids) proved reliable in different configurations, revealing the true number of clusters. Moreover, kernel-based methods appeared more efficient when highly noisy variables are involved, becoming more promising for real-world data. As a second part, an illustration of two specific simulated scenarios (datasets) is provided, showing, in more detail, all different analysis steps before drawing a conclusion about the choice of the optimal number of clusters.

Students’ Multiple Perceptions of English Teaching Model Reform in Colleges and Universities under Cognitive Coupling Networks

Abstract On the basis of linear multiple regression methods such as PLSR and ICR, this paper introduces the kernel function to kernelize these linear multiple statistical regression methods, derives the KPLSR and KICR algorithms, and gives the specific implementation process. Then kernel-based multiple regression methods are applied to analyze the English teaching mode using a cognitive coupling network. According to the students’ learning data under the English teaching mode of cognitive coupling network, then the students’ learning situation and other precise calculations and statistics, so as to truly reflect the student’s learning status, presenting the students’ learning process in need of an urgent solution. Finally, the correlation between students’ multi-sensory learning styles and English performance is analyzed, and the results show that visual, auditory, kinesthetic, experiential, and independent styles all show a significant positive correlation to the total English performance, with correlation coefficients of 0.416, 0.158, 0.349, 0.339, and 0.327, respectively, among which the correlation coefficient of the visual type of perception is the largest.

Nonparametric augmented probability weighting with sparsity

Nonresponse frequently arises in practice, and simply ignoring it may lead to erroneous inference. Besides, the number of collected covariates may increase as the sample size in modern statistics, so parametric imputation or propensity score weighting usually leads to estimation inefficiency and introduces a large variability without consideration of sparsity. In this paper, we propose a nonparametric imputation method with sparsity to estimate the finite population mean, where an efficient kernel-based method in the reproducing kernel Hilbet space is employed for estimation and sparse learning. Moreover, an augmented inverse probability weighting framework is adopted to achieve a central limit theorem for the proposed estimator under regularity conditions. The performance of the proposed method is also supported by several simulated examples and one real-life analysis.

Enhancing Alzheimer Disease Segmentation through Adaptively Regularized Weighted Kernel-Based Clustering

Image segmentation is important in image analysis because it helps to locate objects and boundaries within a picture. This study offers Adaptively Regularized Weighted Kernel-Based Clustering (ARWKC), a unique segmentation technique built exclusively for recovering brain tissue from medical pictures. The proposed approach incorporates adaptive regularization and weighted kernel-based clustering techniques to increase the accuracy and resilience of brain tissue segmentation. The picture is initially preprocessed with the ARWKC method to improve its quality and eliminate any noise or artifacts. The adaptive regularization method is then utilized to effectively deal with the visual variation of brain tissue in clinical images. This adaptive regularization contributes to more accurate and consistent segmentation outcomes. The weighted kernel-based clustering method is then used to find and group pixels with comparable properties, with a focus on brain tissue areas. This clustering approach employs a weighted kernel function that takes into account both geographical closeness and pixel intensities, allowing the algorithm to capture local picture features and improve segmentation accuracy. Extensive experiments were conducted on a collection of medical images to evaluate the efficacy of the ARWKC algorithm. The well-known k-means clustering method, often used in image segmentation applications, was utilized as a benchmark for comparison. In terms of accuracy and resilience for brain tissue segmentation, the experimental findings showed that the ARWKC method surpasses the k-means clustering approach.

On kernel design for regularized non-causal system identification

Through one decade’s development, the kernel-based regularization method (KRM) has become a complement to the classical maximum likelihood/prediction error method and an emerging new system identification paradigm. One recent example is its application in the non-causal system identification, and the key issue lies in the design and analysis of kernels for non-causal systems. In this paper, we develop systematic ways to deal with this issue. In particular, we first introduce the guidelines for kernel design and then extend the system theoretic framework to design the so-called non-causal simulation-induced (NCSI) kernel, and we also study its structural properties, including stability and semiseparability. Finally, we consider some special cases of the NCSI kernel and show their advantage over the existing kernels through numerical simulations.

Simple yet efficient downscaling of land surface temperatures by suitably integrating kernel- and fusion-based methods

Kernel-based and fusion-based methods have been widely used to downscale satellite-derived land surface temperatures (LSTs) for obtaining LSTs with high spatiotemporal resolution. The integration of these two methods can potentially provide a more effective method for LST downscaling. However, a downscaling algorithm that appropriately integrates the advantages of these two methods in accuracy and stability, yet is characterized by adequate simplicity, effectiveness, and global applicability, is still lacking. To address this issue, here we proposed the Simple and Effective Downscaling (SED) algorithm, which integrates kernel- and fusion-based methods suitably and uses a single adjacent Landsat image to downscale MODIS LSTs by statistical regression. The SED was tested in 50 different study areas worldwide, and its performance was compared with four benchmark downscaling algorithms, including a kernel-based algorithm (the RFT21), a fusion-based algorithm (the FSDAF), and the simple average (the Hybrid 1) and weighted average (the Hybrid 2) of the RFT21 and the FSDAF. Our evaluations show that the SED performs better in almost all study areas and has finer global applicability when compared with these four benchmark algorithms. The mean RMSE of the SED is 1.29 K over these 50 study regions, with an accuracy improvement of over 13 % relative to the best benchmark algorithm. Particularly, the SED exhibits higher accuracies in tropical and temperate regions, over urban areas and savannas, and when using Landsat images closer to MODIS LSTs. We believe that the SED can contribute to the generation of high-quality and high spatiotemporal resolution LSTs over global lands.

Gradient COBRA: A kernel-based consensual aggregation for regression

In this article, we introduce a kernel-based consensual aggregation method for regression problems. We aim to flexibly combine individual regression estimators r1, ..., rM using a weighted average where the weights are defined based on predicted features given by all the basic estimators and some kernel function. This work extends the context of Biau et al. (2016) to a more general kernel-based framework. We show that this more general configuration also inherits the consistency of the basic consistent estimators, and the same convergence rate as in the classical method is achieved. Moreover, an optimization method based on gradient descent algorithm is proposed to efficiently and rapidly estimate the key parameter of the strategy. Various of numerical experiments carried out on several simulated and real datasets are also provided to illustrate the efficiency and accuracy of the method with the introduction of gradient descent algorithm and smoother kernel functions respectively. Moreover, a domain adaptation-like property of the method is also illustrated on a physics data provided by Commissariat à l'Énergie Atomique (CEA).

How to Find a Bitcoin Mixer: A Dual Ensemble Model for Bitcoin Mixing Service Detection

Bitcoin is the first decentralized peer-to-peer cryptocurrency that has gained popularity by providing users with transaction anonymity. With the development of Bitcoin and the higher privacy requirements of users, mixing services have emerged to enhance Bitcoin anonymity by obfuscating the flow of funds. However, they are also widely used for illegal activities due to its strong anonymity, especially for money laundering. Therefore, detecting mixing services has great significance for Bitcoin anti-money laundering. In this paper, we propose a novel detection scheme to identify the addresses belonging to Bitcoin mixing services. Specifically, we first construct the Bitcoin mixing dataset, which summarizes a total of 26 features to describe the transaction behavior of addresses. Next, we design a new classification model, called the Dual Ensemble Classification Model. The model combines the advantages of multiple models based on different algorithms and obtains better classification performance. In order to detect more complex mixing patterns, we also extract transaction sub-graphs from the established Bitcoin address-transaction network. The sub-graphs are then classified using a kernel-based graph classification method, which is embedded in the model. Comprehensive experiments on three datasets demonstrate the effectiveness of our scheme, and the proposed model has a detection accuracy of 99.84% for the Bitcoin mixing service.

NeuralBO: A black-box optimization algorithm using deep neural networks

Bayesian Optimization (BO) is an effective approach for the global optimization of black-box functions when function evaluations are expensive. Most prior works use Gaussian processes to model the black-box function, however, the use of kernels in Gaussian processes leads to two problems: first, the kernel-based methods scale poorly with the number of data points and second, kernel methods are usually not effective on complex structured high dimensional data due to curse of dimensionality. Therefore, we propose a novel black-box optimization algorithm where the black-box function is modeled using a neural network. Our algorithm does not need a Bayesian neural network to estimate predictive uncertainty and is therefore computationally favorable. We analyze the theoretical behavior of our algorithm in terms of regret bound using advances in NTK theory showing its efficient convergence. We perform experiments with both synthetic and real-world optimization tasks and show that our algorithm is more sample efficient compared to existing methods.

Transfer Learning with Kernel Methods

Transfer learning refers to the process of adapting a model trained on a source task to a target task. While kernel methods are conceptually and computationally simple models that are competitive on a variety of tasks, it has been unclear how to develop scalable kernel-based transfer learning methods across general source and target tasks with possibly differing label dimensions. In this work, we propose a transfer learning framework for kernel methods by projecting and translating the source model to the target task. We demonstrate the effectiveness of our framework in applications to image classification and virtual drug screening. For both applications, we identify simple scaling laws that characterize the performance of transfer-learned kernels as a function of the number of target examples. We explain this phenomenon in a simplified linear setting, where we are able to derive the exact scaling laws.

CURVATURE BASED CHARACTERIZATION OF RADIAL BASIS FUNCTIONS: APPLICATION TO INTERPOLATION

Choosing the scale or shape parameter of radial basis functions (RBFs) is a well-documented but still an open problem in kernel-based methods. It is common to tune it according to the applications, and it plays a crucial role both for the accuracy and stability of the method. In this paper, we first devise a direct relation between the shape parameter of RBFs and their curvature at each point. This leads to characterizing RBFs to scalable and unscalable ones. We prove that all scalable RBFs lie in the -class which means that their curvature at the point xj is proportional to, where cj is the corresponding spatially variable shape parameter at xj. Some of the most commonly used RBFs are then characterized and classified accordingly to their curvature. Then, the fundamental theory of plane curves helps us recover univariate functions from scattered data, by enforcing the exact and approximate solutions have the same curvature at the point where they meet. This leads to introducing curvature-based scaled RBFs with shape parameters depending on the function values and approximate curvature values of the function to be approximated. Several numerical experiments are devoted to show that the method performs better than the standard fixed-scale basis and some other shape parameter selection methods.

Investigation into facial expression recognition methods: a review

Facial expression recognition (FER) is a rapidly emerging topic in computer vision that has gotten a lot of interest because of its numerous applications in fields including psychology, sociology, human-computer interaction (HCI), and security. FER seeks to recognise and analyse human facial expressions in order to determine emotions and other mental states. Several strategies, including feature-based, kernel-based, and deep learning-based methods, have been developed and implemented in FER in recent years. FER’s major goal is to extract and identify the most discriminating elements that accurately represent the emotions expressed by facial expressions. The literature reviewed in this field shows that deep learning-based methods have outperformed traditional feature-based and kernel-based methods in terms of accuracy and robustness in recognizing facial expressions. However, these deep learning-based methods also pose several challenges, such as the need for large labeled-data-sets, robustness to different facial poses and illumination conditions, and generalization to unseen data. Despite these challenges, the field of FER is expected to continue growing, and future research will likely focus on addressing these challenges and improving the accuracy and robustness of FER systems.

Geostatistical analysis of groundwater levels in a mining area with three active mines

AbstractMining activities can significantly impact groundwater reservoirs in their vicinity. Different approaches have been employed, with varying success, to investigate the spatial variability of groundwater levels in mining areas. Typical problems include the small sample size, the non-Gaussian distribution of the data, and the clustering of sample locations near the mines. These conditions complicate the estimation of spatial dependence. Under sparse and irregular sampling conditions, stochastic methods, which can provide estimates of prediction uncertainty, are preferable to deterministic ones. This research focuses on the comparison of two stochastic methods, stochastic local interactions (SLI) and universal Kriging (UK), using water level data from 72 locations around three mines in Northern Greece. UK is a well-known, variogram-based geostatistical method, while SLI is a computationally efficient kernel-based method that can cope with large spatial datasets. The non-Gaussian distribution of the data is handled by means of a flexible, data-driven Gaussian anamorphosis method that uses kernel functions. The spatial prediction performance of both methods is assessed based on cross-validation. UK performs better than SLI, due to the fact that the former incorporates a linear trend function. On the other hand, a comparison of the two methods using data from a single mine that contains only 28 measurement locations shows that SLI performs slightly better than UK. The prediction uncertainties for both methods are also estimated and compared. The results suggest that SLI can provide better estimates than classical geostatistical methods for small sample sizes that do not allow reliable estimation of the variogram model.

Performance of advanced machine learning algorithms overlogistic regression in predicting hospital readmissions: A meta-analysis

ObjectivesMachine learning algorithms are being increasingly used for predicting hospital readmissions. This meta-analysis evaluated the performance of logistic regression (LR) and machine learning (ML) models for the prediction of 30-day hospital readmission among patients in the US. MethodsElectronic databases (i.e., Medline, PubMed, and Embase) were searched from January 2015 to December 2019. Only studies in the English language were included. Two reviewers performed studies screening, quality appraisal, and data collection. The quality of the studies was assessed using the Quality in Prognosis Studies (QUIPS) tool. Model performance was evaluated using the Area Under the Curve (AUC). A random-effects meta-analysis was performed using STATA 16. ResultsNine studies were included based on the selection criteria. The most common ML techniques were tree-based methods such as boosting and random forest. Most of the studies had a low risk of bias (8/9). The AUC was greater with ML to predict 30-day all-cause hospital readmission compared with LR [Mean Difference (MD): 0.03; 95% Confidence Interval (CI) 0.01–0.05]. Subgroup analyses found that deep-learning methods had a better performance compared with LR (MD 0.06; 95% CI, 0.04–0.09), followed by neural networks (MD: 0.03; 95% CI, 0.03–0.03), while the AUCs of the tree-based (MD: 0.02; 95% CI -0.00-0.04) and kernel-based (MD: 0.02; 95% CI 0.02 (−0.13–0.16) methods were no different compared to LR. More than half of the studies evaluated heart failure-related rehospitalization (N = 5). For the readmission prediction among heart failure patients, ML performed better compared with LR, with a mean difference in AUC of 0.04 (95% CI, 0.01–0.07). The leave-one-out sensitivity analysis confirmed the robustness of the findings. ConclusionMultiple ML methods were used to predict 30-day all-cause hospital readmission. Performance varied across the ML methods, with deep-learning methods showing the best performance over the LR.

Learning with Asymmetric Kernels: Least Squares and Feature Interpretation.

Asymmetric kernels naturally exist in real life, e.g., for conditional probability and directed graphs. However, most of the existing kernel-based learning methods require kernels to be symmetric, which prevents the use of asymmetric kernels. This paper addresses the asymmetric kernel-based learning in the framework of the least squares support vector machine named AsK-LS, resulting in the first classification method that can utilize asymmetric kernels directly. We will show that AsK-LS can learn with asymmetric features, namely source and target features, while the kernel trick remains applicable, i.e., the source and target features exist but are not necessarily known. Besides, the computational burden of AsK-LS is as cheap as dealing with symmetric kernels. Experimental results on various tasks, including Corel, PASCAL VOC, Satellite, directed graphs, and UCI database, all show that in the case asymmetric information is crucial, the proposed AsK-LS can learn with asymmetric kernels and performs much better than the existing kernel methods that rely on symmetrization to accommodate asymmetric kernels.

High-dimensional approximation with kernel-based multilevel methods on sparse grids

Moderately high-dimensional approximation problems can successfully be solved by combining univariate approximation processes using an intelligent combination technique. While this has so far predominantly been done with either polynomials or splines, we suggest to employ a multilevel kernel-based approximation scheme. In contrast to those schemes built upon polynomials and splines, this new method is capable of combining arbitrary low-dimensional domains instead of just intervals and arbitrarily distributed points in these low-dimensional domains. We introduce the method and analyse its convergence in the so-called isotropic and anisotropic cases.

Static and adaptive subspace information fusion for indefinite heterogeneous proximity data

Heterogeneous data is common in many real-world machine learning applications, such as healthcare, market analysis, environmental sciences, and social media analysis. In these domains, data is often represented in different modalities and, most of the time, in non-vectorial formats, like text, images, and video. Traditional machine learning algorithms are often limited in their ability to effectively analyze and learn from such diverse data types. In this paper, we propose two approaches for such heterogeneous data analysis: static and adaptive subspace kernel fusion. The first approach is a kernel-based method extracting the essential parts of the subspace of each input modality and creating one single fused representation of the data. The second approach utilizes an adaptation step by integrating the weighting of spectral properties into the fusion process in order to improve the data’s representation with respect to a given classification task. Our proposed methods are evaluated on several multi-modal, heterogeneous data sets and demonstrate significant performance improvement compared to other methods in the field. Our results highlight the importance of fusing the underlying subspace information of heterogeneous data for achieving superior performance in machine learning tasks.

Optimal design of thermal cycles for experimental processing of advanced TRIP-assisted galvanized steels using support vector regression and kernel-based gradient evolution method

Optimal design of thermal cycles for experimental processing of advanced TRIP-assisted galvanized steels using support vector regression and kernel-based gradient evolution method

A novel algorithm for classification using a low rank approximation of kernel-based support vector machines with applications

Support vector machines (SVMs), as a powerful technique for classification, are becoming increasingly popular in a wide range of applications. This is simply due to their robustness against several types of model assumptions violations and outliers. The Kernel-based SVM are very useful to capture non-linear patterns in the data, and for classification. However, this kernel-based method could become computationally very challenging because it increases the required time to train data. This increase in computational time is mainly due to the appearance of the kernel in solving the quadratic optimization problem (QOP). In order to tackle this computational complexity, we propose a novel method based on the low-rank approximation, by adapting a truncated Mercer series to the kernels. The quadratic optimization problem in the structure of kernel-based SVM will then be replaced with a much simpler optimization problem. In the proposed approach, the required time for the vector computations and matrix decompositions will be much faster such that these changes lead to efficiently resolve the QOP and ultimately increase efficiency in classification. We finally present some numerical illustrations based on the ROC curves and other classification performance benchmarks considered in this paper to assess the performance of the proposed low-rank approximation to the kernel in SVM structure. The results suggest considerable efficiency improvement has been observed in classification with significant reduction in computational time required to train and forecast the stock market index (S&P 500 index) and promoter recognition in DNA sequences.