Multiple Kernel Research Articles

Research on image classification based on residual group multi-scale enhanced attention network

To address the challenges associated with insufficient feature extraction and gradient degradation encountered when dealing with deepening network structures in image classification tasks, this paper presents a ResGMEANet(Residual Group Multi-scale Enhanced Attention Network). The model introduces a multi-scale attention enhancement module. This design draws inspiration from the original model's capability to independently capture feature correlations in channels and spaces. By implementing shuffle operations and feature transformations within the group, our method expands the receptive field through the utilization of multiple convolution kernels. Additionally, we incorporate an improved tensor synthesis attention, building upon the traditional convolution attention, to derive attention feature maps after feature enhancement. Evaluation on the CIFAR-10 and CIFAR-100 datasets shows that ResGMEANet outperforms both the original backbone model and several existing mainstream methods in classification accuracy. This work aims to provide a new perspective for the future by combining residual neural networks with different attention mechanisms.

Gaussian process selections in semiparametric multi‐kernel machine regression for multi‐pathway analysis

AbstractAnalyzing correlated high‐dimensional data is a challenging problem in genomics, proteomics, and other related areas. For example, it is important to identify significant genetic pathway effects associated with biomarkers in which a gene pathway is a set of genes that functionally works together to regulate a certain biological process. A pathway‐based analysis can detect a subtle change in expression level that cannot be found using a gene‐based analysis. Here, we refer to pathway as a set and gene as an element in a set. However, it is challenging to select automatically which pathways are highly associated to the outcome when there are multiple pathways. In this paper, we propose a semiparametric multikernel regression model to study the effects of fixed covariates (e.g., clinical variables) and sets of elements (e.g., pathways of genes) to address a problem of detecting signal sets associated to biomarkers. We model the unknown high‐dimension functions of multi‐sets via multiple Gaussian kernel machines to consider the possibility that elements within the same set interact with each other. Hence, our variable set selection can be considered a Gaussian process set selection. We develop our Gaussian process set selection under the Bayesian variance component‐selection framework. We incorporate prior knowledge for structural sets by imposing an Ising prior on the model. Our approach can be easily applied in high‐dimensional spaces where the sample size is smaller than the number of variables. An efficient variational Bayes algorithm is developed. We demonstrate the advantages of our approach through simulation studies and through a type II diabetes genetic‐pathway analysis.

Evaluating Explainable Machine Learning Models for Clinicians

Gaining clinicians’ trust will unleash the full potential of artificial intelligence (AI) in medicine, and explaining AI decisions is seen as the way to build trustworthy systems. However, explainable artificial intelligence (XAI) methods in medicine often lack a proper evaluation. In this paper, we present our evaluation methodology for XAI methods using forward simulatability. We define the Forward Simulatability Score (FSS) and analyze its limitations in the context of clinical predictors. Then, we applied FSS to our XAI approach defined over an ML-RO, a machine learning clinical predictor based on random optimization over a multiple kernel support vector machine (SVM) algorithm. To Compare FSS values before and after the explanation phase, we test our evaluation methodology for XAI methods on three clinical datasets, namely breast cancer, VTE, and migraine. The ML-RO system is a good model on which to test our XAI evaluation strategy based on the FSS. Indeed, ML-RO outperforms two other base models—a decision tree (DT) and a plain SVM—in the three datasets and gives the possibility of defining different XAI models: TOPK, MIGF, and F4G. The FSS evaluation score suggests that the explanation method F4G for the ML-RO is the most effective in two datasets out of the three tested, and it shows the limits of the learned model for one dataset. Our study aims to introduce a standard practice for evaluating XAI methods in medicine. By establishing a rigorous evaluation framework, we seek to provide healthcare professionals with reliable tools for assessing the performance of XAI methods to enhance the adoption of AI systems in clinical practice.

Inverse design of convolutional neural networks via nanophotonic kernels

We proposed an ultra-compact neural network based on inversely designed photonic kernels, which has the advantages of a small footprint, high computing speed, and low power consumption. The core layers of the neural network, composed of automatically optimized photonic multiplication kernels and addition kernels, can perform arbitrary photonic computations at the speed of the light wave, offering more advantages than their electronic counterparts. Furthermore, the convolution, pooling, and fully-connected layers can be realized by the appropriate permutation of the proposed optical kernels. We constructed an all-optical convolutional neural network with a high accuracy of 97.7% for recognizing handwritten digits. Our results could significantly promote the inverse design of photonic neural networks.

Comparative analysis of integrative classification methods for multi-omics data.

Recent advances in sequencing, mass spectrometry, and cytometry technologies have enabled researchers to collect multiple 'omics data types from a single sample. These large datasets have led to a growing consensus that a holistic approach is needed to identify new candidate biomarkers and unveil mechanisms underlying disease etiology, a key to precision medicine. While many reviews and benchmarks have been conducted on unsupervised approaches, their supervised counterparts have received less attention in the literature and no gold standard has emerged yet. In this work, we present a thorough comparison of a selection of six methods, representative of the main families of intermediate integrative approaches (matrix factorization, multiple kernel methods, ensemble learning, and graph-based methods). As non-integrative control, random forest was performed on concatenated and separated data types. Methods were evaluated for classification performance on both simulated and real-world datasets, the latter being carefully selected to cover different medical applications (infectious diseases, oncology, and vaccines) and data modalities. A total of 15 simulation scenarios were designed from the real-world datasets to explore a large and realistic parameter space (e.g. sample size, dimensionality, class imbalance, effect size). On real data, the method comparison showed that integrative approaches performed better or equally well than their non-integrative counterpart. By contrast, DIABLO and the four random forest alternatives outperform the others across the majority of simulation scenarios. The strengths and limitations of these methods are discussed in detail as well as guidelines for future applications.

Multimodal fusion of brain signals for robust prediction of psychosis transition

The prospective study of youths at clinical high risk (CHR) for psychosis, including neuroimaging, can identify neural signatures predictive of psychosis outcomes using algorithms that integrate complex information. Here, to identify risk and psychosis conversion, we implemented multiple kernel learning (MKL), a multimodal machine learning approach allowing patterns from each modality to inform each other. Baseline multimodal scans (n = 74, 11 converters) included structural, resting-state functional imaging, and diffusion-weighted data. Multimodal MKL outperformed unimodal models (AUC = 0.73 vs. 0.66 in predicting conversion). Moreover, patterns learned by MKL were robust to training set variations, suggesting it can identify cross-modality redundancies and synergies to stabilize the predictive pattern. We identified many predictors consistent with the literature, including frontal cortices, cingulate, thalamus, and striatum. This highlights the advantage of methods that leverage the complex pathophysiology of psychosis.

Unstructured Road Extraction in UAV Images based on Lightweight Model

There is no unified planning standard for unstructured roads, and the morphological structures of these roads are complex and varied. It is important to maintain a balance between accuracy and speed for unstructured road extraction models. Unstructured road extraction algorithms based on deep learning have problems such as high model complexity, high computational cost, and the inability to adapt to current edge computing devices. Therefore, it is best to use lightweight network models. Considering the need for lightweight models and the characteristics of unstructured roads with different pattern shapes, such as blocks and strips, a TMB (Triple Multi-Block) feature extraction module is proposed, and the overall structure of the TMBNet network is described. The TMB module was compared with SS-nbt, Non-bottleneck-1D, and other modules via experiments. The feasibility and effectiveness of the TMB module design were proven through experiments and visualizations. The comparison experiment, using multiple convolution kernel categories, proved that the TMB module can improve the segmentation accuracy of the network. The comparison with different semantic segmentation networks demonstrates that the TMBNet network has advantages in terms of unstructured road extraction.

A lightweight dynamic dual-damped wavelet-based convolutional neural network for interpretable bearing fault diagnosis

Wavelet-based convolutional neural networks (CNNS) have attracted widespread attention because they can improve the interpretability of intelligent fault diagnosis methods. However, the fault feature representation capability of typical wavelet-based convolution kernel frameworks must be strengthened to improve the diagnostic accuracy of complex faults. In the meantime, the large number of network parameters leads to high computational costs. To address these issues, a lightweight wavelet-based dynamic CNN, which comprises a dual-damping wavelet-based dynamic CNN (DWDC) block and a discrete wavelet transformation (DWT) enhancement (DWTE) block, is put forward. In the DWDC block, a wavelet convolution layer is initially designed, where a dual-damping wavelet is used as the kernel function to improve the match of the convolution kernel with fault impulses. Subsequently, a dynamic convolution layer with multiple parallel small-size convolutional kernels is designed to screen the fault features instead of a multilayer network structure, thereby greatly reducing the number of network parameters. Finally, the DWTE block is constructed by combining the DWT and residual dense block, and it can mine more fault information from the previously extracted features. The experiments on the variable speed bearing dataset, locomotive bearing dataset with constant speed and the Case Western Reserve University dataset prove that the proposed approach outperforms five classical CNN models and six advanced wavelet-based CNN models. In addition, it can effectively solve the issue of data imbalance because of its powerful feature extraction capability.

Join multiple Riemannian manifold representation and multi‐kernel non‐redundancy for image clustering

AbstractImage clustering has received significant attention due to the growing importance of image recognition. Researchers have explored Riemannian manifold clustering, which is capable of capturing the non‐linear shapes found in real‐world datasets. However, the complexity of image data poses substantial challenges for modelling and feature extraction. Traditional methods such as covariance matrices and linear subspace have shown promise in image modelling, and they are still in their early stages and suffer from certain limitations. However, these include the uncertainty of representing data using only one Riemannian manifold, limited feature extraction capacity of single kernel functions, and resulting incomplete data representation and redundancy. To overcome these limitations, the authors propose a novel approach called join multiple Riemannian manifold representation and multi‐kernel non‐redundancy for image clustering (MRMNR‐MKC). It combines covariance matrices with linear subspace to represent data and applies multiple kernel functions to map the non‐linear structural data into a reproducing kernel Hilbert space, enabling linear model analysis for image clustering. Additionally, the authors use matrix‐induced regularisation to improve the clustering kernel selection process by reducing redundancy and assigning lower weights to identical kernels. Finally, the authors also conducted numerous experiments to evaluate the performance of our approach, confirming its superiority to state‐of‐the‐art methods on three benchmark datasets.

Innovative integration of multi-scale residual networks and MK-MMD for enhanced feature representation in fault diagnosis

This article proposes a method based on multi-scale expansion of residual neural networks (ResNets) to address challenges in the operation of rotating components, such as bearings and gears, under complex conditions where they are often affected by environmental noise. This interference leads to weaker fault characteristics, making feature selection difficult and increasing the presence of extraneous information features. To tackle this issue, the proposed method first employs a multi-scale feature ResNet to extract features from vibration signals of rotating machinery. The method decomposes the signal into multiple sub-signals of different scales, extracting local features at each scale. It then uses residual connections to combine these local features to obtain a global feature representation. Furthermore, the article introduces a construction of the maximum mean discrepancy (MMD) and minimization of entropy boundaries to adapt to the differences between two domains. The method utilizes multiple kernel functions to calculate distances between data at different scales and combines these distances to obtain a comprehensive measure. By employing the MMD and minimization of entropy boundary approach, the method can more accurately determine whether signals at different scales belong to the same category, thereby improving diagnostic accuracy and robustness. Experimental results demonstrate the effectiveness of the proposed method in unsupervised cross-domain fault diagnosis tasks. Future work will focus on further optimizing the architecture of ResNets, enhancing feature extraction capabilities, and exploring advanced data augmentation methods to further improve the model’s generalization performance.

Characterizing the voxel-based approaches in radioembolization dosimetry with reDoseMC.

Yttrium-90 ( ) represents the primary radioisotope used in radioembolization procedures, while holmium-166 ( ) is hypothesized to serve as a viable substitute for due to its comparable therapeutic potential and improved quantitative imaging. Voxel-based dosimetry for these radioisotopes relies on activity images obtained through PET or SPECT and dosimetry methods, including the voxel S-value (VSV) and the local deposition method (LDM). However, the evaluation of the accuracy of absorbed dose calculations has been limited by the use of non-ideal reference standards and investigations restricted to the liver. The objective of this study was to expand upon these dosimetry characterizations by investigating the impact of image resolutions, voxel sizes, target volumes, and tissue materials on the accuracy of and dosimetry techniques. A specialized radiopharmaceutical dosimetry software called reDoseMC was developed using the Geant4 Monte Carlo toolkit and validated by benchmarking the generated kernels with published data. The decay spectra of both and were also compared. Multiple VSV kernels were generated for the liver, lungs, soft tissue, and bone for isotropic voxel sizes of 1 mm, 2 mm, and 4 mm. Three theoretical phantom setups were created with 20 or 40mm activity and mass density inserts for the same three voxel sizes. To replicate the limited spatial resolutions present in PET and SPECT images, image resolutions were modeled using a 3D Gaussian kernel with a Full Width at Half Maximum (FWHM) ranging from 0 to 16mm and with no added noise. The VSV and LDM dosimetry methods were evaluated by characterizing their respective kernels and analyzing their absorbed dose estimates calculated on theoretical phantoms. The ground truth for these estimations was calculated usingreDoseMC. The decay spectra obtained through reDoseMC showed less than a 1% difference when compared to previously published experimental data for energies below 1.9MeV in the case of and less than 1% for energies below 1.5MeV for . Additionally, the validation kernels for VSV exhibited results similar to those found in published Monte Carlo codes, with source dose depositions having less than a 3% error margin. Resolution thresholds ( ), defined as resolutions that resulted in similar dose estimates between the LDM and VSV methods, were observed for . They were 1.5mm for bone, 2.5 mm for soft tissue and liver, and 8.5 mm for lungs. For , the accuracy of absorbed dose deposition was found to be dependent on the contributions of absorbed dose from photons. Volume errors due to variations in voxel size impacted the final dose estimates. Larger target volumes yielded more accurate mean doses than smaller volumes. For both radioisotopes, the radial dose profiles for the VSV and LDM approximated but never matched the referencestandard. reDoseMC was developed and validated for radiopharmaceutical dosimetry. The accuracy of voxel-based dosimetry was found to vary widely with changes in image resolutions, voxel sizes, chosen target volumes, and tissue material; hence, the standardization of dosimetry protocols was found to be of great importance for comparable dosimetryanalysis.

基于轻量级深度学习网络RepYOLO模型的麦穗实时检测

Real-time detection has become an essential component in intelligent agriculture and industry. In this paper, a real-time wheat spike detection method based on the lightweight deep learning network RepYOLO is proposed. Addressing the small and densely packed phenotype characteristics of wheat spikes, the channel attention mechanism module CBAM from the traditional YOLOv4 algorithm is introduced and multiple convolutional kernels are merged using a structural reparameterization method. Additionally, the ATSS algorithm is incorporated to enhance the accuracy of object detection. These approaches significantly reduce the model size, improve the inference speed, and lower the memory access cost. To validate the effectiveness of the model, it is trained and tested on a large dataset of diverse wheat spike images representing various phenotypes. The experimental results demonstrate that the RepYOLO algorithm achieves an average accuracy of 98.42% with a detection speed of 8.2 FPS. On the Jetson Nano platform, the inference speed reaches 34.20 ms. Consequently, the proposed model effectively reduces the memory access cost of deep learning networks without compromising accuracy and successfully improves the utilization of CPU/MCU limited performance.

A stacked deep multi-kernel learning framework for blast induced flyrock prediction

Blasting operations are widely and frequently used for rock excavation in Civil and Mining constructions. Flyrock is one of the most important issues induced by blasting operations in open pit mines, and therefore needs to be well predicted in order to identify the safety zone to prevent the potential injuries. For this purpose, 234 sets of blasting data were collected from Sungun Copper Mine site, and a stacked deep multi-kernel learning (SD-MKL) framework was proposed to estimate the blast induced flyrock with confidence accuracy. The proposed model uses the stacking-based representation learning framework (S-RL) to achieve deep learning on small-scale training sets. A multi-kernel learning model (MKL) is used as the base module of S-RL framework, which uses a multi-feature fusion strategy to generate multiple kernels with different kernel length in order to reduce the effort in tuning hyperparameters. In addition, this study further enhanced the predictive capability of SD-MKL by introducing the boosting method into the S-RL framework and hence proposed a boosted SD-MKL model. For comparison purpose, several existing machine learning models were implemented, i.e., kernel ridge regression (KRR), support vector machine (SVM), random forest (RF), gradient boosting decision tree (GBDT), ensemble deep random vector functional link (edRVFL), SD-KRR and SD-SVM. Our experimental results showed that the proposed boosted SD-MKL achieved the best overall performance, with the lowest RMSE of 0.21/1.73, MAE of 0.08/0.78, and the highest VAF of 99.98/99.24.

A Survey of Multiple Kernel Graph Clustering Algorithms

In recent years, algorithms based on multi-kernel learning and graph clustering have garnered significant attention in the field of data mining. This article reviews four advanced multi-kernel graph clustering algorithms: nearest neighbor linear kernel weighted multi-kernel graph clustering, multi-kernel graph clustering based on simultaneous global and local structure preservation, multi-kernel graph clustering based on direct consensus relationship graph learning, and custom non-negative matrix factorization multi-kernel graph tensor clustering. These algorithms have their own characteristics and, by combining multi-kernel learning, graph clustering, and other advanced technologies, they improve the accuracy and efficiency of clustering. However, they also face challenges such as high computational complexity and parameter settings, necessitating further research and optimization. These algorithms have wide application prospects in fields such as image recognition, text classification, and bioinformatics, providing new solutions for handling complex data structures.

A novel graph-based multiple kernel learning framework for hyperspectral image classification

ABSTRACT Multiple kernel learning (MKL) is an efficient way to improve hyperspectral image classification with few training samples by integrating spectral and spatial features. Nonetheless, presenting a MKL algorithm to capture significant information and eliminate redundancy is indispensable. In this paper, a novel graph-based MKL method (GMKL) is introduced to enhance the performance of the MKL algorithm and achieve high discriminative features, resulting in maximum separability. To do so, an informative low-rank graph is determined via a new algorithm called block diagonal representation that takes into account both local and global data structures. Then, in lieu of a time-consuming search for finding the optimal combination of the basic kernels, unlike the prior works, an optimal projective direction is acquired using graph theory. Various basic kernels generated from superpixels and attribute profiles are projected to produce a discriminative combination of kernels. Eventually, the optimum kernel is given to the support vector machine classifier. Experiments have been conducted on two popular hyperspectral datasets (Indian Pines and Pavia University). For the datasets, the proposed algorithm gained 90.43% and 99.28% overall accuracy, respectively, using only 1% training samples, which is much better than the state-of-the-art MKL algorithm and deep learning-based approaches with lower computational time.

SHREG: Mitigating register redundancy in GPUs

Graphics Processing Units (GPUs) have become dominant accelerators for Machine Learning (ML) and High-Performance Computing (HPC) applications due to their massive parallelism capabilities, through the utilization of general matrix-to-matrix multiplication (GEMM) kernels. However, GEMM kernels often suffer from duplicated memory requests, mainly caused by matrix tiling used for handling large matrices. While GPUs have adopted programmable shared memory to mitigate this issue by preserving frequently reused data in shared memory, GEMM still introduces duplication in register files. Our observations show that the matrix tiling issues memory requests to the same shared memory address for neighboring threads, and this results in a substantial increase in the number of duplicated data in the register files. Such duplication degrades GPU performance by limiting warp-level parallelism due to the register shortage and redundant memory requests to shared memory. We find that the data duplication can be categorized into two types that occur with fixed patterns during the matrix tiling. Based on these observations, we introduce SHREG, an architecture design that enables different threads to share registers for overlapped data from shared memory, effectively reducing duplicated data within the register files. By leveraging the duplication patterns, SHREG utilizes register sharing and improves performance with minimal hardware overhead. Our evaluation shows that SHREG improves performance by 31.4% on various ML applications over the baseline GPU.

Optimized Ensemble Support Vector Regression Models for Predicting Stock Prices with Multiple Kernels

Optimized Ensemble Support Vector Regression Models for Predicting Stock Prices with Multiple Kernels

Learning with privileged knowledge of multiple kernels via joint prediction for CT Kernel conversion.

Most existing models for CT kernel conversion take images reconstructed with a single predetermined source kernel as input and convert them to images that are reconstructed with a target kernel. However, these models can achieve even better performance if they leverage complementary information obtained from images reconstructed with multiple different kernels. In many clinical practice scenarios, only images with one kernel can beacquired. We propose a privileged knowledge learning framework that learns privileged knowledge of other source kernels available only in the training data (called privileged information) to guide the conversion from a specific single source kernel to the target kernel, via a joint prediction (JP)task. We construct an ensemble of kernel-specific (KS) tasks where a KS network (KSNet) takes images reconstructed with a specific source kernel as input and converts them to images reconstructed with the target kernel. Then, a JP task is designed to provide extra regularization, which helps each KSNet learn more informative feature representations for kernel conversion, such as detail and structure representations. Meanwhile, we use a cross-shaped window-based attention mechanism in the JP task to highlight the most relevant features to strengthen privileged knowledge learning, thereby alleviating the problems of redundant noise unrelated to images reconstructed with target kernel and inconsistent features that arise from images reconstructed with different kernels. All KSNets can be trained collaboratively by using a JP task to improve the performance of each individual KSNet. We extensively evaluate our method on a clinical dataset with scanners from three manufacturers, that is, Siemens, GE and Philips. The experimental results demonstrate that our privileged knowledge learning framework is effective in improving CT kernel conversion. Through both quantitative and qualitative research, our privileged knowledge learning framework improves the kernel conversion results, thereby contributing to the improvement of diagnostic accuracy and the advancement of comparative research in quantitativemeasurements.

Adaptive weighted ensemble clustering via kernel learning and local information preservation

Ensemble clustering refers to learning a robust and accurate consensus result from a collection of base clustering results. Despite extensive research on this topic, it remains challenging due to the absence of raw features and real labels of samples. Current ensemble clustering methods based on co-association (CA) matrix have received much attention from researchers. However, the information contained in the CA matrix is difficult to fully exploit within its inherent representation space. To address this issue and further unveil the underlying patterns and nonlinear relationships within the CA matrix, this paper proposes a novel Adaptive Weighted method based on Kernel learning and local information preservation for Ensemble Clustering, termed AWKEC. Unlike other ensemble clustering studies, AWKEC introduces kernel ideas to ensemble clustering to learn consistent sample relationships across multiple kernel spaces. Moreover, considering the properties of the CA matrix, highly reliable local information is preserved and embedded into the learned similarity coefficient matrices via Hadamard product. The experimental results, conducted on 13 datasets and compared to 10 representative comparative methods, validate the effectiveness of the proposed AWKEC.

ConvTKG: A query-aware convolutional neural network-based embedding model for temporal knowledge graph completion

Temporal knowledge graphs (TKGs) are still far from completeness although they have benefited many artificial intelligence applications successfully. To alleviate the incompleteness of TKGs, temporal knowledge graph completion (TKGC) aims to perform link prediction and reason new time-sensitive facts from existing facts. Among the various TKGC methods, temporal knowledge graph embedding (TKGE) methods represent entities, relations, and timestamps in low-dimensional spaces, which are known for their high reasoning performance and robust scalability. Some previous researches on TKGE aim to endow the embedding quadruple with the translational characteristic by modeling the global relationship in the embedding quadruple. However, the weights of same dimensional entries of the embedding quadruple are fixed and unique, which limits the ability to capture diverse global relationships. Besides, most TKGE models view timestamps as a kind of general feature and overlook their interrelations and correlations with the queries. Moreover, the distinct characteristics of an entity as the subject entity and the object entity of facts have not been fully leveraged. In this paper, we propose a query-aware embedding model based on the convolutional neural network (CNN) named ConvTKG to perform TKGE, thus tackling the TKGC task. We design a new temporal information encoder based on the Gated Recurrent Unit (GRU) and attention mechanism to learn the query-aware representation of temporal information. More importantly, we utilize a CNN-based decoder where multiple one-dimensional convolution kernels are operated on the matrix composed of entity embedding, relation embedding, and temporal representation to capture global relationships among them. In addition, we assign two independent vectors for each entity and take advantage of the inverse relation to allow them to be learned dependently. Experiments show that ConvTKG achieves better link prediction performance than previous state-of-the-art baseline methods for TKGC on three benchmark datasets: ICEWS14, ICEWS05-15, and GDELT.