Convolutional Kernel Networks Research Articles

Inherently interpretable position-aware convolutional motif kernel networks for biological sequencing data

Artificial neural networks show promising performance in detecting correlations within data that are associated with specific outcomes. However, the black-box nature of such models can hinder the knowledge advancement in research fields by obscuring the decision process and preventing scientist to fully conceptualize predicted outcomes. Furthermore, domain experts like healthcare providers need explainable predictions to assess whether a predicted outcome can be trusted in high stakes scenarios and to help them integrating a model into their own routine. Therefore, interpretable models play a crucial role for the incorporation of machine learning into high stakes scenarios like healthcare. In this paper we introduce Convolutional Motif Kernel Networks, a neural network architecture that involves learning a feature representation within a subspace of the reproducing kernel Hilbert space of the position-aware motif kernel function. The resulting model enables to directly interpret and evaluate prediction outcomes by providing a biologically and medically meaningful explanation without the need for additional post-hoc analysis. We show that our model is able to robustly learn on small datasets and reaches state-of-the-art performance on relevant healthcare prediction tasks. Our proposed method can be utilized on DNA and protein sequences. Furthermore, we show that the proposed method learns biologically meaningful concepts directly from data using an end-to-end learning scheme.

COmic: convolutional kernel networks for interpretable end-to-end learning on (multi-)omics data.

The size of available omics datasets is steadily increasing with technological advancement in recent years. While this increase in sample size can be used to improve the performance of relevant prediction tasks in healthcare, models that are optimized for large datasets usually operate as black boxes. In high-stakes scenarios, like healthcare, using a black-box model poses safety and security issues. Without an explanation about molecular factors and phenotypes that affected the prediction, healthcare providers are left with no choice but to blindly trust the models. We propose a new type of artificial neural network, named Convolutional Omics Kernel Network (COmic). By combining convolutional kernel networks with pathway-induced kernels, our method enables robust and interpretable end-to-end learning on omics datasets ranging in size from a few hundred to several hundreds of thousands of samples. Furthermore, COmic can be easily adapted to utilize multiomics data. We evaluated the performance capabilities of COmic on six different breast cancer cohorts. Additionally, we trained COmic models on multiomics data using the METABRIC cohort. Our models performed either better or similar to competitors on both tasks. We show how the use of pathway-induced Laplacian kernels opens the black-box nature of neural networks and results in intrinsically interpretable models that eliminate the need for post hoc explanation models. Datasets, labels, and pathway-induced graph Laplacians used for the single-omics tasks can be downloaded at https://ibm.ent.box.com/s/ac2ilhyn7xjj27r0xiwtom4crccuobst/folder/48027287036. While datasets and graph Laplacians for the METABRIC cohort can be downloaded from the above mentioned repository, the labels have to be downloaded from cBioPortal at https://www.cbioportal.org/study/clinicalData?id=brca\_metabric. COmic source code as well as all scripts necessary to reproduce the experiments and analysis are publicly available at https://github.com/jditz/comics.

Randomized non-linear PCA networks

PCANet is an unsupervised Convolutional Neural Network (CNN), which uses Principal Component Analysis (PCA) to learn convolutional filters. One drawback of PCANet is that linear PCA cannot capture nonlinear structures within data. To address this problem, a straightforward approach is utilizing kernel methods by equipping the PCA method in PCANet with a kernel function. However, this practice leads to a network having cubic complexity with respect to the number of training image patches. In this paper, we propose a network called Randomized Nonlinear PCANet (RNPCANet), which uses an explicit kernel PCA to learn the convolutional filters. Although RNPCANet utilizes kernel methods for nonlinear processing of data, using kernel approximation techniques to define an explicit feature space in each stage, we theoretically show that the complexity of this model is not much higher than that of PCANet. We also show that our method links PCANets to Convolutional Kernel Networks (CKNs) as the proposed model maps the patches to a kernel feature space similar to CKNs. We evaluate our model on image recognition tasks including Coil-20, Coil-100, ETH-80, Caltech-101, MNIST, and C-Cube datasets. The experimental results show that the proposed method has superiority over PCANet and CKNs in terms of recognition accuracy.

Biological sequence modeling with convolutional kernel networks.

The growing number of annotated biological sequences available makes it possible to learn genotype-phenotype relationships from data with increasingly high accuracy. When large quantities of labeled samples are available for training a model, convolutional neural networks can be used to predict the phenotype of unannotated sequences with good accuracy. Unfortunately, their performance with medium- or small-scale datasets is mitigated, which requires inventing new data-efficient approaches. We introduce a hybrid approach between convolutional neural networks and kernel methods to model biological sequences. Our method enjoys the ability of convolutional neural networks to learn data representations that are adapted to a specific task, while the kernel point of view yields algorithms that perform significantly better when the amount of training data is small. We illustrate these advantages for transcription factor binding prediction and protein homology detection, and we demonstrate that our model is also simple to interpret, which is crucial for discovering predictive motifs in sequences. Source code is freely available at https://gitlab.inria.fr/dchen/CKN-seq. Supplementary data are available at Bioinformatics online.

Convolutional kernel networks based on a convex combination of cosine kernels

Convolutional Kernel Networks (CKNs) are efficient multilayer kernel machines, which are constructed by approximating a convolution kernel with a mapping based on Gaussian functions. In this paper, we introduce a new approximation of the same convolution kernel based on a convex combination of cosine kernels. CKNs are structurally similar to Convolutional Neural Networks (CNNs), but the convolution operation in CKNs is based on the Euclidean distance, which is not common in convolutional networks. We show that the CKN model obtained by the proposed approximation leads to the ordinary convolution operation, which is based on the inner product. From this point of view, the proposed model is a step forward towards bridging the gap between kernel methods and deep learning. In this paper, we use two methods for learning filters of the proposed CKN: Random Fourier Features, which is a randomized data-independent method for approximating shift-invariant kernels, and a novel method based on the minimization of the sum of squared errors of approximating shift-invariant kernels. Although the RFF method is much faster than ordinary CKN, it requires a high number of random features in order to obtain an acceptable accuracy. To overcome this problem, we proposed the second method, in which the filters are learned in a data-dependent fashion. We evaluate the proposed model on visual recognition datasets MNIST, CIFAR-10, C-Cube, and FERET. Our experiments show that the proposed model surpasses ordinary CKNs in terms of accuracy. Specifically, on CIFAR-10, the accuracy of the proposed method is 1.7% higher than ordinary CKN.

Learning graph affinities for spectral graph-based salient object detection

In this paper, we propose a novel method for learning graph affinities for salient object detection. First, we assume that a graph representation of an image is given with a predetermined connectivity rule and representative features for each of its nodes. Then, we learn to predict affinities related to this graph, that ensures a decent salient object detection performance, when used with a spectral graph based foreground detection method. To accomplish this task, we modify convolutional kernel networks (CKNs) for graph affinity calculation, which were originally proposed to predict similarities between images. Subsequently, we employ a spectral graph based salient object detection method – Extended Quantum Cuts (EQCut) – using these graph affinities. We show that the salient object detection error of such a system is differentiable with respect to the parameters of the CKN. Therefore, the proposed system can be trained end-to-end by applying error backpropagation and CKN parameters can be learned for salient object detection task. The comparative evaluations over a large set of benchmark datasets indicate that the proposed method has an insignificant computational burden on, but significantly outperforms the baseline EQCut – which uses color affinities – and achieves a comparable performance level with the state-of-the-art in some performance measures.

Convolutional Patch Representations for Image Retrieval: An Unsupervised Approach

Convolutional neural networks (CNNs) are able to model local stationary structures in natural images in a multi-scale fashion, when learning all model parameters with supervision. While excellent performance was achieved for image classification when large amounts of labeled visual data are available, their success for unsupervised tasks such as image retrieval has been moderate so far.Our paper focuses on this latter setting and explores several methods for learning patch descriptors without supervision with application to matching and instance-level retrieval. To that effect, we propose a new family of patch representations, based on the recently introduced convolutional kernel networks. We show that our descriptor, named Patch-CKN, performs better than SIFT as well as other convolutional networks learned by artificially introducing supervision and is significantly faster to train. To demonstrate its effectiveness, we perform an extensive evaluation on standard benchmarks for patch and image retrieval where we obtain state-of-the-art results. We also introduce a new dataset called RomePatches, which allows to simultaneously study descriptor performance for patch and image retrieval.