Multiple Kernel Research Articles

Multi-dimensional deep dense residual networks and multiple kernel learning for hyperspectral image classification

To address the issues of inadequate feature expression capacity and poor adaptability of feature fusion in traditional hyperspectral image classification methods, a new approach to classification utilizing a multi-dimensional deep dense residual network and multiple kernel learning is suggested. First, we construct a deep dense residual structure to obtain the spectral, spatial and spectral-spatial features of the image, respectively. Then, the multiple kernel learning is employed to map the three extracted features into the high-dimensional space to achieve the adaptive fusion between features. Finally, multiple kernel support vector machine is adopted to accurately classify the multiple kernel fused features. By randomly selecting 10%, 1% and 1% of samples in each ground object category as training samples, our proposed method demonstrated an enhanced overall classification accuracy of 99.58%, 99.52% and 99.66% according to the experimental results conducted on three typical hyperspectral images. Comparing with some multiple kernel learning based methods, our method can obtain higher classification accuracy. So our method has better recognition ability for hyperspectral image features, and has strong adaptability and robustness.

Bayesian hybrid-kernel machine-learning-assisted sensitivity analysis and sensitivity-relevant inverse modeling for groundwater DNAPL contamination

Accurate source characterization and transport parameter estimation is important when seeking to predict the spatiotemporal distribution of dense non-aqueous phase liquid (DNAPL) contaminants in groundwater. However, this is a complex multimodal search problem prone to equifinality and premature convergence, which leads to considerable error. To address this, a sensitivity-relevant dynamic swarm intelligence (SRD-SI) algorithm embedded in a homotopy-variation mechanism is proposed in the present study to rationally balance the inversion processes of sensitivity-varied source characteristics and DNAPL transport parameters. In this approach, global optima are progressively approached in conjunction with the homotopy variation of the search space. Furthermore, to avoid computationally expensive numerical model repetition during the sensitivity analysis and inverse iterations, a Bayesian-based optimization framework that combines multiple kernel functions in a kernel extreme learning machine (KELM) model is designed considering the complex site conditions and statistical characteristics of the input variables, thus creating the Bayesian hybrid KELM (BHK-ELM) model for the reliable surrogate modeling of numerical DNAPL-transport simulations. The results show that the BHK-ELM model recognizes and effectively reconstructs the complex input − output mapping of the numerical model by increasing the determination coefficient R2 to 0.9988 while improving the computational efficiency approximately 4500-fold. Because source characteristics and boundary conditions are far more sensitive than transport parameters to the contaminant distribution, conventional inverse modeling methods struggle to accurately identify these transport parameters. In contrast, the proposed inverse modeling system combining sensitivity analysis, swarm intelligence, and homotopy variation is more stable and provides significantly more accurate estimations for all unknown variables. Compared with the traditional SI algorithm, the homotopy-variation SRD-SI reduced the maximum inversion relative error from 46.22 % to 9.53 %, while the mean inversion relative error was reduced from 11.09 % to 3.90 %.

Improving adversarial transferability through frequency enhanced momentum

The emergence of adversarial examples seriously affects the practical security deployment of convolutional neural networks. The existing attack algorithms perform brilliantly under white-box scenarios, but they show weak transferability when faced with unknown black-box models. Recent studies have revealed that models have different interests in different frequency components of images, and the low-frequency characteristics play a non-negligible role in the decision-making of models. In this article, we present the frequency enhanced momentum iterative attack, called FE-MI-FGSM. Specifically, we use multiple convolution kernels for Gaussian filtering of the image before each gradient update to push the processed image closer to the common decision boundaries of multiple models. Then, the average gradient of the white-box model to these processed images is obtained and used as the perturbation direction to generate adversarial examples with a high success rate of white-box attack and high transferability. The empirical results show that compared with the current mainstream gradient-based methods, our method performs better on both normally trained and adversarially trained models. Besides, our method can combine with gradient-based methods which integrate convergence algorithms or input transformations for the sake of reaching satisfactory improvement of the transferability.

HKFGCN: A novel multiple kernel fusion framework on graph convolutional network to predict microbe-drug associations

Accumulating clinical studies have consistently demonstrated that the microbes in the human body closely interact with the human host, actively participating in the regulation of drug effectiveness. Identifying the associations between microbes and drugs can facilitate the development of drug discovery, and microbes have become a new target in antimicrobial drug development. However, the discovery of microbe-drug associations relies on clinical or biological experiments, which are not only time-consuming but also financially burdensome. Thus, the utilization of computational methods to predict microbe-drug associations holds promise for reducing costs and enhancing the efficiency of biological experiments. Here, we introduce a new computational method, called HKFGCN (Heterogeneous information Kernel Fusion Graph Convolution Network), to predict the microbe-drug associations. Instead of extracting feature from a single network in previous studies, HKFGCN separately extracts topological information features from different networks, and further refines them by generating Gaussian kernel features. HKFGCN consists of three main steps. Firstly, we constructed two similarity networks and a microbe-drug association network based on numerous biological data. Second, we employed two types of encoders to extract features from these networks. Next, Gaussian kernel features were obtained from the drug and microbe features at each layer. Finally, we reconstructed the bipartite microbe-drug graph based on the learned representations. Experimental results demonstrate the excellent performance of the HKFGCN model across different datasets using the cross-validation scheme. Additionally, we conduced case studies on human immunodeficiency virus, and the results were corroborated by existing literatures. The prediction model’s code is available at https://github.com/roll-of-bubble/HKFGCN.

Incomplete Multiple Kernel Alignment Maximization for Clustering.

Multiple kernel alignment (MKA) maximization criterion has been widely applied into multiple kernel clustering (MKC) and many variants have been recently developed. Though demonstrating superior clustering performance in various applications, it is observed that none of them can effectively handle incomplete MKC, where parts or all of the pre-specified base kernel matrices are incomplete. To address this issue, we propose to integrate the imputation of incomplete kernel matrices and MKA maximization for clustering into a unified learning framework. The clustering of MKA maximization guides the imputation of incomplete kernel elements, and the completed kernel matrices are in turn combined to conduct the subsequent MKC. These two procedures are alternately performed until convergence. By this way, the imputation and MKC processes are seamlessly connected, with the aim to achieve better clustering performance. Besides theoretically analyzing the clustering generalization error bound, we empirically evaluate the clustering performance on several multiple kernel learning (MKL) benchmark datasets, and the results indicate the superiority of our algorithm over existing state-of-the-art counterparts. Our codes and data are publicly available at https://xinwangliu.github.io/.

VCSEL-based photonic spiking neural networks for ultrafast detection and tracking

Inspired by efficient biological spike-based neural networks, we demonstrate for the first time the detection and tracking of target patterns in image and video inputs at high-speed rates with networks of multiple artificial spiking optical neurons. Using photonic systems of in-parallel spiking vertical cavity surface emitting lasers (VCSELs), we demonstrate the implementation of multiple convolutional kernel operators which, in combination with optical spike signalling, enable the detection and tracking of target features in images/video feeds at an ultrafast photonic operation speed of 1 ns per pixel. Alongside a single layer optical spiking neural network (SNN) demonstration, a multi-layer network of photonic (GHz-rate) spike-firing neurons is reported where the photonic system successfully tracks a large complex feature (Handwritten Digit 3). The consecutive photonic layers perform spike-enabled image reduction and convolution operations, and interact with a software-implemented SNN, that learns the feature patterns that best identify the target to provide a high detection efficiency even in the presence of a distractor feature. This work therefore highlights the effectiveness of combining neuromorphic photonic hardware and software SNNs, for efficient learning and ultrafast operation, thanks to the use of spiking light signals, towards tackling complex AI and computer vision problems.

Incorporation of multiple random features kernel mean p‐power controller to elevate power quality in reconfigurable grid‐connected residential solar photovoltaic systems

SummaryProblems like intermittent photovoltaic (PV) power, disconnected PV arrays, etc. are brought on by the extensive integration of PV generation. In this article, a single‐stage PV‐integrated grid‐tethered system is controlled using a reconfigurative multiple random features kernel mean p‐power (MRFKMP) control algorithm. The algorithm circumvents the dependence of kernel parameter in the classic random Fourier features mapping and lower the computing complexity in the traditional multiple kernel learning adaptive filter. MRFKMP algorithm with a linear filter structure can reduce both the computational and storage cost. This is the novel application of MRFKMP algorithm to extract fundamental component of nonlinear load currents in PV‐integrated grid‐tied system. Furthermore, to increase the reliability of the system, a reconfigurable control is developed, which adaptive regulates DC link voltage from PV mode (PVM) during PV hours to active power filtering mode (AFM) during PV nonavailability/PV disconnection. On a created laboratory testbed, the applied control's practical feasibility for numerous scenarios is evaluated. Comparison with existing control algorithms reveals that the MRFKMP controller outperforms the LMF based PV system by 79% less peak‐to‐peak load current ripple and 60% less than the best performing SRF control algorithm. The grid current THD is 2.60%, which is 30% low than that presented by SRF technique.

RANGO: A Novel Deep Learning Approach to Detect Drones Disguising from Video Surveillance Systems

Video surveillance systems provide means to detect the presence of potentially malicious drones in the surroundings of critical infrastructures. In particular, these systems collect images and feed them to a deep-learning classifier able to detect the presence of a drone in the input image. However, current classifiers are not efficient in identifying drones that disguise themselves with the image background, e.g., hiding in front of a tree. Furthermore, video-based detection systems heavily rely on the image’s brightness, where darkness imposes significant challenges in detecting drones. Both these phenomena increase the possibilities for attackers to get close to critical infrastructures without being spotted and hence be able to gather sensitive information or cause physical damages, possibly leading to safety threats. In this article, we propose RANGO, a drone detection arithmetic able to detect drones in challenging images where the target is difficult to distinguish from the background. RANGO is based on a deep learning architecture that exploits a Preconditioning Operation (PREP) that highlights the target by the difference between the target gradient and the background gradient. The idea is to highlight features that will be useful for classification. After PREP, RANGO uses multiple convolution kernels to make the final decision on the presence of the drone. We test RANGO on a drone image dataset composed of multiple already-existing datasets to which we add samples of birds and planes. We then compare RANGO with multiple currently existing approaches to show its superiority. When tested on images with disguising drones, RANGO attains an increase of 6.6% mean Average Precision (mAP) compared to YOLOv5 solution. When tested on the conventional dataset, RANGO improves the mAP by approximately 2.2%, thus confirming its effectiveness also in the general scenario.

Dedicated Hardware Accelerators for Processing of Sparse Matrices and Vectors: A Survey

Performance in scientific and engineering applications such as computational physics, algebraic graph problems or Convolutional Neural Networks (CNN), is dominated by the manipulation of large sparse matrices—matrices with a large number of zero elements. Specialized software using data formats for sparse matrices has been optimized for the main kernels of interest: SpMV and SpMSpM matrix multiplications, but due to the indirect memory accesses, the performance is still limited by the memory hierarchy of conventional computers. Recent work shows that specific hardware accelerators can reduce memory traffic and improve the execution time of sparse matrix multiplication, compared to the best software implementations. The performance of these sparse hardware accelerators depends on the choice of the sparse format, COO , CSR , etc, the algorithm, inner-product , outer-product , Gustavson , and many hardware design choices. In this article, we propose a systematic survey which identifies the design choices of state-of-the-art accelerators for sparse matrix multiplication kernels. We introduce the necessary concepts and then present, compare, and classify the main sparse accelerators in the literature, using consistent notations. Finally, we propose a taxonomy for these accelerators to help future designers make the best choices depending on their objectives.

Ensemble 3D CNN and U-Net-based brain tumour classification with MKKMC segmentation

Advanced brain cancer is the deadliest type with just a few months survival rate. Existing technologies hinder the objective of forecasting cancer. This work aims to fulfil the pressing requirement for timely and precise identification of advanced-stage brain tumours, which are notorious for their markedly reduced life expectancy. It presents an innovative hybrid approach for predicting brain tumours and improves diagnostic capabilities. The Multiple Kernel K-Means Cluster Algorithm (MKKCA) is used to segment brain MRI images effectively, differentiating healthy and tumorous tissues. After segmentation, a hybrid approach with 3D-Convolutional Neural Network (CNN) and U-Net has been utilized for classification. The objective is to effectively and accurately distinguish normal and pathological brain images. To enhance the efficiency, we include the Improved Whale Optimization Algorithm (IWOA), which guarantees accurate and dependable performance via location updates. The methodology demonstrates outstanding precision with 98.5% accuracy rate, 98.56% specificity, 91% sensitivity, 87.45% precision and a recall rate of 96% with the F-Measure at 96.02%. These findings, obtained using MATLAB, demonstrate a substantial performance improvement compared to current approaches. This development not only represents a significant addition to diagnostic imaging but also a crucial role in the prediction and treatment of brain cancers.

MSEConv: A Unified Warping Framework for Video Frame Interpolation

Within the context of video frame interpolation, complex motion modeling is the task of capturing, in a video sequence, where the moving objects are located in the interpolated frame, and how to maintain the temporal consistency of motion. Existing video frame interpolation methods typically assign either a fixed size of the motion kernel or a refined optical flow to model complex motions. However, they have the limitation of data redundancy and inaccuracy representation of motion. This paper introduces a unified warping framework, named multi-scale expandable deformable convolution (MSEConv), for simultaneously performing complex motion modeling and frame interpolation. In the proposed framework, a deep fully convolutional neural network with global attention is proposed to estimate multiple small-scale kernel weights with different expansion degrees and adaptive weight allocation for each pixel synthesis. Moreover, most of the kernel-based interpolation methods can be treated as the special case of the proposed MSEConv, thus, MSEConv can be easily transferred to other kernel-based frame interpolation methods for performance improvement. To further improve the robustness of motion occlusions, an operation of mask occlusion is introduced. As a consequence, our proposed MSEConv shows strong performance on par or even better than the state-of-the-art kernel-based frame interpolation works on public datasets. Our source code and visual comparable results are available at https://github.com/Pumpkin123709/MSEConv.

Neural Generalization of Multiple Kernel Learning

Multiple Kernel Learning (MKL) is a conventional way to learn the kernel function in kernel-based methods. MKL algorithms enhance the performance of kernel methods. However, these methods have a lower complexity compared to deep models and are inferior to them regarding recognition accuracy. Deep learning models can learn complex functions by applying nonlinear transformations to data through several layers. In this paper, we show that a typical MKL algorithm can be interpreted as a one-layer neural network with linear activation functions. By this interpretation, we propose a Neural Generalization of Multiple Kernel Learning (NGMKL), which extends the conventional MKL framework to a multi-layer neural network with nonlinear activation functions. Our experiments show that the proposed method, which has a higher complexity than traditional MKL methods, leads to higher recognition accuracy on several benchmarks.

Local kernels based graph learning for multiple kernel clustering

Multiple kernel clustering (MKC) has been extensively studied in recent years. The focus of MKC is how to explore the information of base kernels. Although existing methods have promising leaning abilities, they ignore the intrinsic local structure contained in base kernels, which may negatively affect their performances. To address the above problem, a novel method, termed as consensus graph learning based on local kernels (CGLLK), is introduced. CGLLK is based on the partitions extracted by kernel k-means. Specifically, we first design a simple yet effective scheme to construct the local kernels of base kernels and then a consensus graph is applied to capture the complementary information contained in the extracted partitions of local kernels. CGLLK also considers the prior knowledge existing in base kernels. Since the partitions of local kernels and the learning stage of the consensus graph contain useful information for each other, the two processes are optimized jointly. Extensive experiments on some popular datasets are carried out to verify the effectiveness of the proposed method. Experimental results illustrate that CGLLK is much more competitive than the state-of-the-art algorithms.

Study of PARP inhibitors for breast cancer based on enhanced multiple kernel function SVR with PSO.

PARP1 is one of six enzymes required for the highly error-prone DNA repair pathway microhomology-mediated end joining (MMEJ) and needs to be inhibited when over-expressed. In order to study the PARP1 inhibitory effect of fused tetracyclic or pentacyclic dihydrodiazepinoindolone derivatives (FTPDDs) by quantitative structure-activity relationship technique, six models were established by four kinds of methods, heuristic method, gene expression programming, random forester, and support vector regression with single, double, and triple kernel function respectively. The single, double, and triple kernel functions were RBF kernel function, the integration of RBF and polynomial kernel functions, and the integration of RBF, polynomial, and linear kernel functions respectively. The problem of multi-parameter optimization introduced in the support vector regression model was solved by the particle swarm optimization algorithm. Among the models, the model established by support vector regression with triple kernel function, in which the optimal R 2 and RMSE of training set and test set were 0.9353, 0.9348 and 0.0157, 0.0288, and R2 cv of training set and test set were 0.9090 and 0.8971, shows the strongest prediction ability and robustness. The method of support vector regression with triple kernel function is a great promotion in the field of quantitative structure-activity relationship, which will contribute a lot to designing and screening new drug molecules. The information contained in the model can provide important factors that guide drug design. Based on these factors, six new FTPDDs have been designed. Using molecular docking experiments to determine the properties of new derivatives, the new drug was ultimately successfully designed.

Lq-norm multiple kernel fusion regression for self-cleansing sediment transport

Experimental and modeling studies have been conducted to develop an approach for self-cleansing rigid boundary open channel design such as drainage and sewer systems. Self-cleansing experiments in the literature are mostly performed on circular channel cross-section, while a few studies considered self-cleansing sediment transport in small rectangular channels. Experiments in this study were carried out in a rectangular channel with a length of 12.5 m, a width of 0.6 m, a depth of 0.7 m and having an automatic control system for regulating channel slope, discharge and sediment rate. Behind utilizing collected experimental data in this study, existing data in the literature for rectangular channels are used to develop self-cleansing models applicable for channel design. Through the modeling procedure, this study recommends Lq-norm multiple kernel fusion regression (LMKFR) techniques for self-cleansing sediment transport. The LMKFR is a regression technique based on the regularized kernel regression method which benefits from the combination of multiple information sources to improve the performance using the Lq-norm multiple kernel learning framework. The results obtained by LMKFR are compared to support vector regression benchmark and existing conventional regression self-cleansing sediment transport models in the literature for rectangular channels. The superiority of LMKFR is illustrated in an accurate modeling as compared with its alternatives in terms of various statistical error measurement criteria. The encouraging results of LMKFR can be linked to utilization of several kernels which are fused effectively using an Lq-norm prior that captures the intrinsic sparsity of the problem at hand. Promising performance of LMKFR technique in this study suggests it as an effective technique to be examined in similar environmental, hydrological and hydraulic problems.

Multi-Kernel Graph Attention Deep Autoencoder for MiRNA-Disease Association Prediction.

Accumulating evidence indicates that microRNAs (miRNAs) can control and coordinate various biological processes. Consequently, abnormal expressions of miRNAs have been linked to various complex diseases. Recognizable proof of miRNA-disease associations (MDAs) will contribute to the diagnosis and treatment of human diseases. Nevertheless, traditional experimental verification of MDAs is laborious and limited to small-scale. Therefore, it is necessary to develop reliable and effective computational methods to predict novel MDAs. In this work, a multi-kernel graph attention deep autoencoder (MGADAE) method is proposed to predict potential MDAs. In detail, MGADAE first employs the multiple kernel learning (MKL) algorithm to construct an integrated miRNA similarity and disease similarity, providing more biological information for further feature learning. Second, MGADAE combines the known MDAs, disease similarity, and miRNA similarity into a heterogeneous network, then learns the representations of miRNAs and diseases through graph convolution operation. After that, an attention mechanism is introduced into MGADAE to integrate the representations from multiple graph convolutional network (GCN) layers. Lastly, the integrated representations of miRNAs and diseases are input into the bilinear decoder to obtain the final predicted association scores. Corresponding experiments prove that the proposed method outperforms existing advanced approaches in MDA prediction. Furthermore, case studies related to two human cancers provide further confirmation of the reliability of MGADAE in practice.

Local Sample-Weighted Multiple Kernel Clustering With Consensus Discriminative Graph.

Multiple kernel clustering (MKC) is committed to achieving optimal information fusion from a set of base kernels. Constructing precise and local kernel matrices is proven to be of vital significance in applications since the unreliable distant-distance similarity estimation would degrade clustering performance. Although existing localized MKC algorithms exhibit improved performance compared with globally designed competitors, most of them widely adopt the KNN mechanism to localize kernel matrix by accounting for τ -nearest neighbors. However, such a coarse manner follows an unreasonable strategy that the ranking importance of different neighbors is equal, which is impractical in applications. To alleviate such problems, this article proposes a novel local sample-weighted MKC (LSWMKC) model. We first construct a consensus discriminative affinity graph in kernel space, revealing the latent local structures. Furthermore, an optimal neighborhood kernel for the learned affinity graph is output with naturally sparse property and clear block diagonal structure. Moreover, LSWMKC implicitly optimizes adaptive weights on different neighbors with corresponding samples. Experimental results demonstrate that our LSWMKC possesses better local manifold representation and outperforms existing kernel or graph-based clustering algorithms. The source code of LSWMKC can be publicly accessed from https://github.com/liliangnudt/LSWMKC.

Multi-Objective Prediction of the Sound Insulation Performance of a Vehicle Body System Using Multiple Kernel Learning–Support Vector Regression

The sound insulation performance of an electric vehicle’s body system serves as a critical metric for evaluating the noise, vibration, and harshness (NVH) quality of the vehicle. The accurate and efficient prediction of sound insulation performance is foundational for undertaking noise reduction design and optimization. Current engineering practices predominantly rely on Computer-Aided Engineering (CAE) methodologies to address this challenge. However, inherent shortcomings such as low modeling efficiency and difficulty in ensuring prediction accuracy often characterize these approaches. In an effort to overcome these limitations, we propose a decomposition framework for predicting the sound insulation performance of the electric vehicle body system. This framework is established based on a comprehensive analysis of the noise transmission paths within the system. Subsequently, the support vector regression (SVR) method is introduced to construct a machine learning model specifically designed for predicting the sound insulation performance of the body system. This approach aims to mitigate the inherent weaknesses associated with the conventional CAE processes using a ‘data-driven’ paradigm. Furthermore, the Multiple Kernel Learning (MKL) method is used to enhance the processing efficacy of the SVR model. The proposed method is validated using practical application and testing on a specific electric vehicle. The results demonstrate commendable performance in terms of prediction accuracy and robustness. This research contributes to advancing the field by presenting a more effective and reliable approach to predicting the sound insulation performance of electric vehicle body systems, offering valuable insights for noise reduction strategies and optimization efforts in the automotive industry.

Automated Buffer Sizing of Dataflow Applications in a High-level Synthesis Workflow

High-Level Synthesis (HLS) tools are mature enough to provide efficient code generation for computation kernels on FPGA hardware. For more complex applications, multiple kernels may be connected by a dataflow graph. Although some tools, such as Xilinx Vitis HLS, support dataflow directives, they lack efficient analysis methods to compute the buffer sizes between kernels in a dataflow graph. This article proposes an original method to safely approximate such buffer sizes. The first contribution computes an initial overestimation of buffer sizes without knowing the memory access patterns of kernels. The second contribution iteratively refines those buffer sizes, thanks to cosimulation. Moreover, the article introduces an open source framework using these methods to facilitate dataflow programming on FPGA using HLS. The proposed methods and framework have been tested on seven dataflow applications and outperform Vitis HLS cosimulation in five benchmarks, either in terms of BRAM and LUT usage, or in terms of exploration time. In the two other benchmarks, our best method gets results similar to Vitis HLS. Last but not least, our method admits directed cycles in the application graphs.

WMKL: multi-omics data integration enables novel cancer subtype identification via weight-boosted multi-kernel learning.

Cancer is a heterogeneous disease driven by complex molecular alterations. Cancer subtypes determined from multi-omics data can provide novel insight into personalised precision treatment. It is recognised that incorporating prior weight knowledge into multi-omics data integration can improve disease subtyping. We develop a weighted method, termed weight-boosted Multi-Kernel Learning (wMKL) which incorporates heterogeneous data types as well as flexible weight functions, to boost subtype identification. Given a series of weight functions, we propose an omnibus combination strategy to integrate different weight-related P-values to improve subtyping precision. wMKL models each data type with multiple kernel choices, thus alleviating the sensitivity and robustness issue due to selecting kernel parameters. Furthermore, wMKL integrates different data types by learning weights of different kernels derived from each data type, recognising the heterogeneous contribution of different data types to the final subtyping performance. The proposed wMKL outperforms existing weighted and non-weighted methods. The utility and advantage of wMKL are illustrated through extensive simulations and applications to two TCGA datasets. Novel subtypes are identified followed by extensive downstream bioinformatics analysis to understand the molecular mechanisms differentiating different subtypes. The proposed wMKL method provides a novel strategy for disease subtyping. The wMKL is freely available at https://github.com/biostatcao/wMKL .