Data-dependent Kernel Function Research Articles

Visual Process Monitoring by Data-Dependent Kernel Discriminant Analysis with t-Distributed Similarities.

Visual process monitoring would provide more directly appreciable and more easily comprehensible information about the process operating status as well as clear depictions of the occurrence path of faults; however, as a more challenging task, it has been sporadically discussed in the research literature on conventional process monitoring. In this paper, the Data-Dependent Kernel Discriminant Analysis (D2K-DA) model is proposed. A special data-dependent kernel function is constructed and learned from the measured data, so that the low-dimensional visualizations are guaranteed, combined with intraclass compactness, interclass separability, local geometry preservation, and global geometry preservation. The new optimization is innovatively designed by exploiting both discriminative information and t-distributed geometric similarities. On the construction of novel indexes for visualization, experiments of visual monitoring tasks on simulated and real-life industrial processes illustrate the merits of the proposed method.

Kernel Based Data-Adaptive Support Vector Machines for Multi-Class Classification

Imbalanced data exist in many classification problems. The classification of imbalanced data has remarkable challenges in machine learning. The support vector machine (SVM) and its variants are popularly used in machine learning among different classifiers thanks to their flexibility and interpretability. However, the performance of SVMs is impacted when the data are imbalanced, which is a typical data structure in the multi-category classification problem. In this paper, we employ the data-adaptive SVM with scaled kernel functions to classify instances for a multi-class population. We propose a multi-class data-dependent kernel function for the SVM by considering class imbalance and the spatial association among instances so that the classification accuracy is enhanced. Simulation studies demonstrate the superb performance of the proposed method, and a real multi-class prostate cancer image dataset is employed as an illustration. Not only does the proposed method outperform the competitor methods in terms of the commonly used accuracy measures such as the F-score and G-means, but also successfully detects more than 60% of instances from the rare class in the real data, while the competitors can only detect less than 20% of the rare class instances. The proposed method will benefit other scientific research fields, such as multiple region boundary detection.

Generalized Semisupervised Self-Optimizing Kernel Model for Quality-Related Industrial Process Monitoring

The process monitoring performance of traditional kernel-based modeling methods strongly depends on the appropriate selection of the kernel function, which solely aims at solving the nonlinearity and totally ignores the relationships between the process variables and the quality variables. In this article, a supervised self-optimizing kernel model is first proposed. The unique special kernel space with data-dependent kernel function is designed and learned from the modeling data, so that the authentic relationships between the process variables and quality variables can be discovered and extracted combined with dimension reduction on the process variables. To be more general than the commonly used semisupervised assumptions, the generalized semisupervised self-optimizing kernel model is successively proposed to make full use of the collected information in the modeling data with irregular measurements of quality variables. Both of the proposed models only contain one compact step of optimization with scalable parameters. The feasibility and efficiency of the proposed algorithms are demonstrated and evaluated through the industrial case studies.

Local and Nonlocal Steering Kernel Weighted Total Variation Model for Image Denoising

To eliminate heavy noise and retain more scene details, we propose a structure-oriented total variation (TV) model based on data dependent kernel function and TV criterion for image denoising application. The innovative model introduces the weights produced from the local and nonlocal symmetry features involved in the image itself to pick more precise solutions in the TV denoising process. As a result, the proposed local and nonlocal steering kernel weighted TV model yields excellent noise suppression and structure-preserving performance. The experimental results verify the validity of the proposed model in objective quantitative indices and subjective visual appearance.

Development of Self-Learning Kernel Regression Models for Virtual Sensors on Nonlinear Processes

The prediction accuracy of the traditional kernel with data-driven regression methods strongly depends on the appropriate selection of the kernel function and aims at solving the nonlinearity of the input space. In this paper, a self-learning kernel regression model is proposed. A special kernel space from the measured data is learned and designed, so that combined with dimension reductions on the input variables, the regression behavior between the projected input variables and the output variable is found. The model is posed as a semidefinite programming problem with the objective function to find the maximum variance between the learned manifolds. The kernel is data dependent and can be generated online whenever a new data point is available. The effectiveness of the proposed algorithm is demonstrated through the case studies on a simple nonlinear system and a real semiconductor process. Note to Practitioners—Virtual sensing of quality/key variables is critical for the control and optimization of industrial processes. Traditional data-driven kernel regression methods cannot achieve guaranteed high prediction accuracy. This paper aims to develop a high-accuracy virtual sensor based on the self-learning kernel regression (SLKR) model, which simultaneously constructs the relationship between the input and the output data as well as represents the data in a low-dimensional manifold. For practical implementation, the following are necessary to: 1) select a historical data set with abundant information of the process; 2) preprocess industrial data well and eliminate irregular entries; 3) establish a self-learning regression kernel based on local manifold structure and regression relationship; 4) approximate the explicit data-dependent kernel function and determine suitable parameter; and 5) extract regression coefficients from the learned kernel matrix.

Electric-Parameter-Based Inversion of Dynamometer Card Using Hybrid Modeling for Beam Pumping System

Conventional fault diagnosis and production calculation of an oil well can be conducted with the surface dynamometer cards, which are obtained by load sensor installed on the horse head. This method to measure the dynamometer cards is limited by the sensor maintenance and calibration, battery replacement, and safety hazards for staff. As the basic parameter of the oil extraction industry, electric parameters have the advantages of low cost and high efficiency. So the inversions of dynamometer card with electric parameters are attracting more and more attention. In order to solve the problem of insufficient data and consider the real-time performance in the actual oil extraction process, this paper proposes a novel hybrid model which consists of two parts: the mechanism model of polished rod load and the suspension displacement calculated with the space vector equations of motor and a data-dependent kernel online sequential extreme learning machine (DDKOS-ELM) model proposed to correct the output error of the mechanism model, which improves the kernel function selection and makes it real-time. Thus, the highlights of this paper can be summed up in two points: (1) under the circumstance of the bottom dead point detection without sensors, the mechanism modeling of the polished rod load and suspension displacement has been carried out from the perspective of mathematical model of AC motor; (2) a novel data-driven model based on data-dependent kernel online sequential extreme learning machine (DDKOS-ELM) has been proposed to improve the kernel functions selection. The coefficients in the data-dependent kernel function are optimized with improved free search algorithm (IFSA). The proposed hybrid model has been applied to a normal working oil well and the prediction results show better accuracy than the pure data-driven model and mechanism model.

Learning a data-dependent kernel function for KPCA-based nonlinear process monitoring

Kernel principal component analysis (KPCA)-based process monitoring methods have recently shown to be very effective for monitoring nonlinear processes. However, their performances largely depend on the kernel function and currently there is no general rule for kernel selection. Existing methods simply choose the kernel function empirically or experimentally from a given set of candidates. This paper proposes a kernel function learning method for KPCA to learn a kernel function tailored to specific data and explores its potential for KPCA-based process monitoring. Motivated by the manifold learning method maximum variance unfolding (MVU), we obtain the kernel function by optimizing over a family of data-dependent kernels such that the nonlinear structure in input data is unfolded in the kernel feature space and gets more likely to be linear there. Using the optimized kernel, the nonlinear principal components of KPCA which are linear principal components in the kernel feature space can effectively capture the variation in data, and thus the data under normal operating conditions can be more precisely modeled by KPCA for process monitoring. Simulation results on an simple nonlinear system and the benchmark Tennessee Eastman (TE) demonstrate that the optimized kernel functions lead to significant improvement in the performance over the popular Gaussian kernels when used in the KPCA-based process monitoring.

Optimizing the data-dependent kernel under a unified kernel optimization framework

The kernel functions play a central role in kernel methods, accordingly over the years the optimization of kernel functions has been a promising research area. Ideally Fisher discriminant criteria can be used as an objective function to optimize the kernel function to augment the margin between different classes. Unfortunately, Fisher criteria are optimal only in the case that all the classes are generated from underlying multivariate normal distributions of common covariance matrix but different means and each class is expressed by a single cluster. Due to the assumptions, Fisher criteria obviously are not a suitable choice as a kernel optimization rule in some applications such as the multimodally distributed data. In order to solve this problem, recently many improved discriminant criteria (DC) have been also developed. Therefore, to apply these discriminant criteria to kernel optimization, in this paper based on a data-dependent kernel function we propose a unified kernel optimization framework, which can use any discriminant criteria formulated in a pairwise manner as the objective functions. Under the kernel optimization framework, to employ different discriminant criteria, one has to only change the corresponding affinity matrices without having to resort to any complex derivations in feature space. Experimental results based on some benchmark data demonstrate the efficiency of our method.