Multi-kernel Method Research Articles

A deep kernel method for lithofacies identification using conventional well logs

How to fit a properly nonlinear classification model from conventional well logs to lithofacies is a key problem for machine learning methods. Kernel methods (e.g., KFD, SVM, MSVM) are effective attempts to solve this issue due to abilities of handling nonlinear features by kernel functions. Deep mining of log features indicating lithofacies still needs to be improved for kernel methods. Hence, this work employs deep neural networks to enhance the kernel principal component analysis (KPCA) method and proposes a deep kernel method (DKM) for lithofacies identification using well logs. DKM includes a feature extractor and a classifier. The feature extractor consists of a series of KPCA models arranged according to residual network structure. A gradient-free optimization method is introduced to automatically optimize parameters and structure in DKM, which can avoid complex tuning of parameters in models. To test the validation of the proposed DKM for lithofacies identification, an open-sourced dataset with seven conventional logs (GR, CAL, AC, DEN, CNL, LLD, and LLS) and lithofacies labels from the Daniudi Gas Field in China is used. There are eight lithofacies, namely clastic rocks (pebbly, coarse, medium, and fine sandstone, siltstone, mudstone), coal, and carbonate rocks. The comparisons between DKM and three commonly used kernel methods (KFD, SVM, MSVM) show that (1) DKM (85.7%) outperforms SVM (77%), KFD (79.5%), and MSVM (82.8%) in accuracy of lithofacies identification; (2) DKM is about twice faster than the multi-kernel method (MSVM) with good accuracy. The blind well test in Well D13 indicates that compared with the other three methods DKM improves about 24% in accuracy, 35% in precision, 41% in recall, and 40% in F1 score, respectively. In general, DKM is an effective method for complex lithofacies identification. This work also discussed the optimal structure and classifier for DKM. Experimental results show that (m1,m2,0) is the optimal model structure and linear SVM is the optimal classifier. (m1,m2,0) means there are m1 KPCAs, and then m2 residual units. A workflow to determine an optimal classifier in DKM for lithofacies identification is proposed, too.

Communication-Efficient Randomized Algorithm for Multi-Kernel Online Federated Learning.

Online federated learning (OFL) is a promising framework to learn a sequence of global functions from distributed sequential data at local devices. In this framework, we first introduce a single kernel-based OFL (termed S-KOFL) by incorporating random-feature (RF) approximation, online gradient descent (OGD), and federated averaging (FedAvg). As manifested in the centralized counterpart, an extension to multi-kernel method is necessary. Harnessing the extension principle in the centralized method, we construct a vanilla multi-kernel algorithm (termed vM-KOFL) and prove its asymptotic optimality. However, it is not practical as the communication overhead grows linearly with the size of a kernel dictionary. Moreover, this problem cannot be addressed via the existing communication-efficient techniques (e.g., quantization and sparsification) in the conventional federated learning. Our major contribution is to propose a novel randomized algorithm (named eM-KOFL), which exhibits similar performance to vM-KOFL while maintaining low communication cost. We theoretically prove that eM-KOFL achieves an optimal sublinear regret bound. Mimicking the key concept of eM-KOFL in an efficient way, we propose a more practical pM-KOFL having the same communication overhead as S-KOFL. Via numerical tests with real datasets, we demonstrate that pM-KOFL yields the almost same performance as vM-KOFL (or eM-KOFL) on various online learning tasks.

Indian classical musical instrument classification using Timbral features

SummaryMusical instrument classification becomes effective when the music signal arrives with profound characteristics. This urged the researchers to develop an automatic system of recognizing the music signals and classify the instruments interplayed through the music. Thus, this paper proposes a model for the Indian music classification system using the optimization‐based stacked autoencoder. The significance of this research is based on the proposed Cuckoo‐dragonfly optimization (CuDro)‐based stacked autoencoder, where the proposed CuDro optimization trains the stacked autoencoder for acquiring accurate classification results. The proposed CuDro technique is the combination of the standard Cuckoo search (CS) and the Dragonfly algorithm (DA) that renders optimal weights for training the stacked autoencoder (SAE). Moreover, the musical instrument classification using the proposed CuDro‐based stack autoencoder is based on the compact features, such as Timbral features and proposed FrMkMFCC features, which further add value to this research. The Timbral features like Spectral flux, spectral kurtosis (SK), Spectral skewness, Spectral pitch similarity, Roughness, In harmonicity are added in the research for efficient musical instrument classification. The proposed FrMkMFCC feature is the integration of the Fractional Fourier transforms and Multi kernel method, and Mel Frequency Cepstral Coefficient (MFCC) features. The analysis using the developed classification methodology confirms that the proposed method acquired the maximum accuracy of 96.16%, the sensitivity of 86.86%, and specificity of 92.85%, respectively.

A Multi-Kernel Mode Using a Local Binary Pattern and Random Patch Convolution for Hyperspectral Image Classification

With the development of deep learning technology, more and more scholars have applied it to hyperspectral image (HSI) classification to improve classification accuracy. However, these deep-learning methods not only take a lot of time in the pre-training phase, but also have relatively limited classification performance when there are fewer labeled samples. In order to improve classification performance while reducing costs, this article proposes a multikernel method based on a local binary pattern and random patches (LBPRP-MK), which integrates a local binary pattern (LBP) and deep learning into a multiple-kernel framework. First, we use LBP and hierarchical convolutional neural networks to extract local textural features and multilayer convolutional features, respectively. The convolution kernel for the convolution operation is obtained from the original image using a random strategy without training. Then, we input local textural features, multilayer convolutional features, and spectral features obtained from the original image into the radial basis function to obtain three kernel functions. Finally, the three kernel functions are merged into a multikernel function according to their optimal weights under the composite kernel strategy. This multikernel function is used as the input for the support vector machine to obtain the classification result map. Experiments show that compared with other HSI classification methods, the proposed method achieves better classification performance on three HSI datasets.

A multi-kernel method of measuring adaptive similarity for spectral clustering

Accurate representation of similarities between data points is an important determinant in the success of many clustering approaches. Previous studies have shown that kernel methods are effective in this representation. However, using multi-kernels to merge probabilistic neighborhoods in data still needs further research. In this paper, a multi-kernel method of measuring adaptive similarity for spectral clustering is proposed. Kernels with more accurate adaptive similarity measure on the data are assigned bigger weights and an optimum combined kernel that truly reflects the internal structure of the data points is obtained. The proposed method ascribes adaptive and optimal neighbors to each data point based on the local structure using the combined kernels. The combined similarity measure and data clustering are learnt simultaneously to obtain optimal clustering results. We rank constraint the Laplacian matrix of the data similarity matrix to ensure that the connected components in the similarity matrix are exactly equal to the cluster number. The presented technique is significant in the sense that it is able to search the underlying similarity relationships amongst data points and is robust to complex data. Compared with other state-of-the-art spectral clustering methods, our proposed method achieves better performance in terms of NMI and accuracy in experiments performed on both synthetic and real datasets.

Robust Multikernel Maximum Correntropy Filters

The multikernel adaptive filters based on the minimum mean square error (MMSE) criterion have been proposed to improve the performance of the kernel least mean square (KLMS), efficiently. However, these multikernel methods suffer from large computational burden as well as instability in impulsive noises. To address these issues, a novel multikernel method is proposed by replacing the trace operation of a matrix with the inner product of two vectors, thus leading to higher computational efficiency, significantly. Based on the maximum correntropy criterion, the multikernel maximum correntropy (MKMC) algorithm is therefore proposed. To further reduce the complexity, an online vector quantization strategy is presented for MKMC to generate the quantized MKMC (QMKMC) algorithm. Monte Carlo simulations on different nonlinear examples validate the superiorities of the proposed two algorithms.

Multi-objective artificial immune algorithm for fuzzy clustering based on multiple kernels

This paper presents a multi-objective artificial immune algorithm for fuzzy clustering based on multiple kernels (MAFC). MAFC extends the classical Fuzzy C-Means (FCM) algorithm and improves some of its important limitations, such as vulnerability to local optima convergence, which can lead to poor clustering quality. MAFC unifies multi-kernel learning and multi-objective optimization in a joint clustering framework, which preserves the geometric information of the dataset. The multi-kernel method maps data from the feature space to kernel space by using kernel functions. Additionally, the introduction of multi-objective optimization helps to optimize between-cluster separation and within-cluster compactness simultaneously via two different clustering validity criteria. These properties help the proposed algorithm to avoid becoming stuck at local optima. Furthermore, this paper utilizes an artificial immune algorithm to address the multi-objective clustering problem and acquire a Pareto optimal solution set. The solution set is obtained through the process of antibody population initialization, clone proliferation, non-uniform mutation and uniformity maintaining strategy, which avoids the problems of degradation and prematurity which can occur with conventional genetic algorithms. Finally, we choose the best solution from the Pareto optimal solution set. We use a semi-supervised method to achieve the final clustering results. We compare our method against state-of-the-art methods from the literature by performing experiments with both UCI datasets and face datasets. The results suggest that MAFC is significantly more efficient for clustering and has a wider scope of application.

Integrating the Local Property and Topological Structure in the Minimum Spanning Tree Brain Functional Network for Classification of Early Mild Cognitive Impairment.

Abnormalities in the brain connectivity in patients with neurodegenerative diseases, such as early mild cognitive impairment (EMCI), have been widely reported. Current research shows that the combination of multiple features of the threshold connectivity network can improve the classification accuracy of diseases. However, in the construction of the threshold connectivity network, the selection of the threshold is very important, and an unreasonable setting can seriously affect the final classification results. Recent neuroscience research suggests that the minimum spanning tree (MST) brain functional network is helpful, as it avoids the methodological biases while comparing networks. In this paper, by employing the multikernel method, we propose a framework to integrate the multiple properties of the MST brain functional network for improving the classification performance. Initially, the Kruskal algorithm was used to construct an unbiased MST brain functional network. Subsequently, the vector kernel and graph kernel were used to quantify the two different complementary properties of the network, such as the local connectivity property and the topological property. Finally, the multikernel support vector machine (SVM) was adopted to combine the two different kernels for EMCI classification. We tested the performance of our proposed method for Alzheimer's Disease Neuroimaging Initiative (ANDI) datasets. The results showed that our method achieved a significant performance improvement, with the classification accuracy of 85%. The abnormal brain regions included the right hippocampus, left parahippocampal gyrus, left posterior cingulate gyrus, middle temporal gyrus, and other regions that are known to be important in the EMCI. Our results suggested that, combining the multiple features of the MST brain functional connectivity offered a better classification performance in the EMCI.

A quality-driven stability analysis framework based on state fluctuation space model for manufacturing process

Industrial manufacturing processes often show multiple operating modes, where different modes present different regularities, so real-time monitor and analyzing the quality state stability is an important way to ensure product quality. This paper proposes a state-driven fluctuation space model for quality stability analysis for multimode manufacturing process. First, the whole process is divided into many sub-processes and the multimode formation mechanism is analyzed to form the stability analysis framework. Then each single-mode quality state fluctuation space model is built based on multi-kernel support vector data description method to determine the max effective fluctuation border of the process state. For the current process state, the deep neural network (DNN) is adopted to extract process state features automatically and recognize the mode type. Thus appropriate quality stable fluctuation space model is selected to monitor and analyze the process stability state. Finally, a case study is performed to evaluate the feasibility of proposed stability analysis method, and the result reveals that the method shows good effect for analyzing the process stability in manufacturing process.

Kernel-Based Adaptive Online Reconstruction of Coverage Maps With Side Information

In this paper, we address the problem of reconstructing coverage maps from path-loss measurements in cellular networks. We propose and evaluate two kernel-based adaptive online algorithms as an alternative to typical offline methods. The proposed algorithms are application-tailored extensions of powerful iterative methods such as the adaptive projected subgradient method and a state-of-the-art adaptive multikernel method. Assuming that the moving trajectories of users are available, it is shown how side information can be incorporated in the algorithms to improve their convergence performance and the quality of the estimation. The complexity is significantly reduced by imposing sparsity-awareness in the sense that the algorithms exploit the compressibility of the measurement data to reduce the amount of data which is saved and processed. Finally, we present extensive simulations based on realistic data to show that our algorithms provide fast, robust estimates of coverage maps in real-world scenarios. Envisioned applications include path-loss prediction along trajectories of mobile users as a building block for anticipatory buffering or traffic offloading.

Multi-channel differencing adaptive noise cancellation with multi-kernel method

Although a various of existing techniques are able to improve the performance of detection of the weak interesting signal, how to adaptively and efficiently attenuate the intricate noises especially in the case of no available reference noise signal is still the bottleneck to be overcome. According to the characteristics of sonar arrays, a multi-channel differencing method is presented to provide the prerequisite reference noise. However, the ingredient of obtained reference noise is too complicated to be used to effectively reduce the interference noise only using the classical linear cancellation methods. Hence, a novel adaptive noise cancellation method based on the multi-kernel normalized least-mean-square algorithm consisting of weighted linear and Gaussian kernel functions is proposed, which allows to simultaneously consider the cancellation of linear and nonlinear components in the reference noise. The simulation results demonstrate that the output signal-to-noise ratio (SNR) of the novel multi-kernel adaptive filtering method outperforms the conventional linear normalized least-mean-square method and the mono-kernel normalized least-mean-square method using the realistic noise data measured in the lake experiment.

Learning Similarity With Multikernel Method

In the field of machine learning, it is a key issue to learn and represent similarity. This paper focuses on the problem of learning similarity with a multikernel method. Motivated by geometric intuition and computability, similarity between patterns is proposed to be measured by their included angle in a kernel-induced Hilbert space. Having noticed that the cosine of such an included angle can be represented by a normalized kernel, it can be said that the task of learning similarity is equivalent to learning an appropriate normalized kernel. In addition, an error bound is also established for learning similarity with the multikernel method. Based on this bound, a boosting-style algorithm is developed. The preliminary experiments validate the effectiveness of the algorithm for learning similarity.