Learnable Kernel Research Articles

RegFormer: A Local–Nonlocal Regularization-Based Model for Sparse-View CT Reconstruction

Sparse-view computed tomography (CT) is one of the primal means to reduce radiation risk. However, the reconstruction of sparse-view CT with classic analytical method is usually contaminated by severe artifacts. By designing the regularization terms carefully, iterative reconstruction (IR) algorithms can produce promising results. Aided by the powerful deep learning techniques, learned regularization terms with convolution neural network (CNN) have attracted much attention and can further improve performance. In this paper, to further enhance the performance of existing learnable regularization-based networks, we propose a learnable local-nonlocal regularization-based model called RegFormer for sparse-view CT reconstruction. Specifically, we unroll the iterative scheme into a neural network and replace the gradient of handcrafted regularization terms with learnable kernels. The convolution layers are used to learn gradient of local regularization, resulting in excellent denoising performance. In addition, the transformer-based encoders and decoders incorporate the learned nonlocal prior into the model, preserving the structures and details. To enhance the ability to extract deep features, we propose an iteration transmission (IT) module that further improve the efficiency of each iteration. The experimental results show that our proposed RegFormer outperforms several state-of-the-art methods in artifact reduction and detail preservation.

Super-efficient Echocardiography Video Segmentation via Proxy- and Kernel-Based Semi-supervised Learning

Automatic segmentation of left ventricular endocardium in echocardiography videos is critical for assessing various cardiac functions and improving the diagnosis of cardiac diseases. It is yet a challenging task due to heavy speckle noise, significant shape variability of cardiac structure, and limited labeled data. Particularly, the real-time demand in clinical practice makes this task even harder. In this paper, we propose a novel proxy- and kernel-based semi-supervised segmentation network (PKEcho-Net) to comprehensively address these challenges. We first propose a multi-scale region proxy (MRP) mechanism to model the region-wise contexts, in which a learnable region proxy with an arbitrary shape is developed in each layer of the encoder, allowing the network to identify homogeneous semantics and hence alleviate the influence of speckle noise on segmentation. To sufficiently and efficiently exploit temporal consistency, different from traditional methods which only utilize the temporal contexts of two neighboring frames via feature warping or self-attention mechanism, we formulate the semi-supervised segmentation with a group of learnable kernels, which can naturally and uniformly encode the appearances of left ventricular endocardium, as well as extracting the inter-frame contexts across the whole video to resist the fast shape variability of cardiac structures. Extensive experiments have been conducted on two famous public echocardiography video datasets, EchoNet-Dynamic and CAMUS. Our model achieves the best performance-efficiency trade-off when compared with other state-of-the-art approaches, attaining comparative accuracy with a much faster speed. The code is available at https://github.com/JingyinLin/PKEcho-Net.

Hyper-convolutions via implicit kernels for medical image analysis

The convolutional neural network (CNN) is one of the most commonly used architectures for computer vision tasks. The key building block of a CNN is the convolutional kernel that aggregates information from the pixel neighborhood and shares weights across all pixels. A standard CNN's capacity, and thus its performance, is directly related to the number of learnable kernel weights, which is determined by the number of channels and the kernel size (support). In this paper, we present the hyper-convolution, a novel building block that implicitly encodes the convolutional kernel using spatial coordinates. Unlike a regular convolutional kernel, whose weights are independently learned, hyper-convolution kernel weights are correlated through an encoder that maps spatial coordinates to their corresponding values. Hyper-convolutions decouple kernel size from the total number of learnable parameters, enabling a more flexible architecture design. We demonstrate in our experiments that replacing regular convolutions with hyper-convolutions can improve performance with less parameters, and increase robustness against noise. We provide our code here: https://github.com/tym002/Hyper-Convolution.

Adversarial Blur-Deblur Network for Robust UAV Tracking

Unmanned aerial vehicle (UAV) tracking has been widely applied in real-world applications such as surveillance and monitoring. However, the inherent high maneuverability and agility of UAV often lead to motion blur, which can impair the visual appearance of the target object and easily degrade the existing trackers. To overcome this challenge, this work proposes a tracking-oriented adversarial blur-deblur network (ABDNet), composed of a novel deblurrer to recover the visual appearance of the tracked object, and a brand-new blur generator to produce realistic blurry images for adversarial training. More specifically, the deblurrer progressively refines the features through pixel-wise, spatial-wise, and channel-wise stages to achieve excellent deblurring performance. The blur generator adaptively fuses an image sequence with a learnable kernel to create realistic blurry images. During training, ABDNet is plugged into the state-of-the-art real-time trackers and trained with blurring-deblurring loss as well as tracking loss. During inference, the blur generator is removed, while the deblurrer and the tracker can work together for UAV tracking. Extensive experiments in both public datasets and real-world testing have validated the effectiveness of ABDNet.

Design and Analysis of Convolutional Neural Layers: A Signal Processing Perspective

Convolutional layers (CLs) are ubiquitous in contemporary deep neural network (DNN) models, commonly used for automatic feature extraction. A CL performs cross-correlation between the input to the layer and a set of learnable kernels to produce the layer output. Typically, kernel weights are randomly initialized and automatically learned during model training using the backpropagation and gradient descent algorithms to minimize a specific loss function. Modern DNN models comprise deep hierarchical stacks of CLs and pooling layers. Despite their prevalence, CLs are perceived as a magical tool for feature extraction without solid interpretations of their underlying working principles. In this work, we advance a method for designing and analyzing CLs by providing novel signal processing interpretations of the CL by exploiting the correlation and equivalent convolution functions of the layer. The proposed interpretations enable the employment of CLs to develop finite impulse response (FIR) filters, matched filters (MFs), short-time Fourier transform (STFT), discrete-time Fourier transform (DTFT), and continuous wavelet transform (CWT) algorithms. The main idea is to pre-assign the CL kernel weights to implement a specific convolution- or correlation-based DSP algorithm. Such an approach enables building self-contained DNN models in which CLs are utilized for various preprocessing and feature extractions tasks, enhancing the model portability, and cutting down the preprocessing computational cost. The proposed DSP interpretations provide an effective means to analyze and explain the operation of automatically trained CLs in the time and frequency domains by reversing the design procedures. The presented interpretations are mathematically established and experimentally validated with a comprehensive machinery fault diagnosis application example illustrating the potential of the proposed methodology.

A robust deformed convolutional neural network (CNN) for image denoising

AbstractDue to strong learning ability, convolutional neural networks (CNNs) have been developed in image denoising. However, convolutional operations may change original distributions of noise in corrupted images, which may increase training difficulty in image denoising. Using relations of surrounding pixels can effectively resolve this problem. Inspired by that, we propose a robust deformed denoising CNN (RDDCNN) in this paper. The proposed RDDCNN contains three blocks: a deformable block (DB), an enhanced block (EB) and a residual block (RB). The DB can extract more representative noise features via a deformable learnable kernel and stacked convolutional architecture, according to relations of surrounding pixels. The EB can facilitate contextual interaction through a dilated convolution and a novel combination of convolutional layers, batch normalisation (BN) and ReLU, which can enhance the learning ability of the proposed RDDCNN. To address long‐term dependency problem, the RB is used to enhance the memory ability of shallow layer on deep layers and construct a clean image. Besides, we implement a blind denoising model. Experimental results demonstrate that our denoising model outperforms popular denoising methods in terms of qualitative and quantitative analysis. Codes can be obtained at https://github.com/hellloxiaotian/RDDCNN.

Trainable segmentation for transmission electron microscope images of inorganic nanoparticles.

We present a trainable segmentation method implemented within the python package ParticleSpy. The method takes user labelled pixels, which are used to train a classifier and segment images of inorganic nanoparticles from transmission electron microscope images. This implementation is based on the trainable Waikato Environment for Knowledge Analysis (WEKA) segmentation, but is written in python, allowing a large degree of flexibility and meaning it can be easily expanded using other python packages. We find that trainable segmentation offers better accuracy than global or local thresholding methods and requires as few as 100 user-labelled pixels to produce an accurate segmentation. Trainable segmentation presents a balance of accuracy and training time between global/local thresholding and neural networks, when used on transmission electron microscope images of nanoparticles. We also quantitatively investigate the effectiveness of the components of trainable segmentation, its filter kernels and classifiers, in order to demonstrate the use cases for the different filter kernels in ParticleSpy and the most accurate classifiers for different data types. A set of filter kernels is identified that are effective in distinguishing particles from background but that retain dissimilar features. In terms of classifiers, we find that different classifiers perform optimally for different image contrast; specifically, a random forest classifier performs best for high-contrast ADF images, but that QDA and Gaussian Naïve Bayes classifiers perform better for low-contrast TEM images.

JPEG steganalysis based on denoising network and attention module

The core objective of image steganalysis is to explore the presence of weak image steganographic signals. Extracting effective steganographic signal features will play an essential role in digital image steganalysis. However, existing networks rely more on spatial rich model kernels or random learnable kernels to obtain noise residuals during the stage of steganographic signal features extraction. In this paper, we proposed a JPEG steganalysis network which based on denoising network and attention module, mainly including a noise extract block, a noise analysis block, and a judgment block. Specifically, a professional denoising convolutional neural network is first introduced in noise extract block to obtain better steganalysis features. The noise analysis block is integrated with the attention module to finely extract the steganographic signals hidden in the complex texture regions, which is quite effective in improving the signal-to-noise ratio of the stego signal. The judgment block is primarily a classifier to distinguish between cover images and stego images. Comprehensive experiments show a significant improvement in performance over the state-of-the-art steganalysis scheme. Moreover, the proposed network has better generalization capability than the compared steganalysis network for the case of cover-source and quality factor mismatch, which is critical for future steganalysis systems.

Spatial correlation filter and its application in hyperspectral ground objects recognition

ABSTRACT Research on spatial information is a hot topic in hyperspectral remote-sensing image (HSI) exploitation. This paper defines a spatial correlation filter to study the spatial variation characteristics and intensity of regionalized variables. Unlike existing spatial information expression methods based on theoretical model, the novel filter takes multi-scale and multi-directional spatial structure features into account and visualizes them, which is model-free and more comprehensive. Further, SCFB model constructed by the filter banks is proposed for pixel-wise classification of HSI and performs well when replacing the classifier component with multiple machine-learning algorithms. Accurate classification on two classic hyperspectral datasets with approximately 3% training samples and on one large dataset with 0.544% training samples indicate that the model is promising in small samples learning. In particular, supervised linear discriminant analysis (LDA) used in the feature subspace optimization part of the model significantly outperforms the other two spectral strategies. Most notably, the model is not only superior to the traditional machine-learning methods considering spectral information only, but also has advantages in accuracy, the number of learnable kernel parameters and time-consuming compared with equal-layer convolution neural network.

The framework of learnable kernel function and its application to dictionary learning of SPD data

The kernel method of machine learning is to transform data from data space to reproducing kernel Hilbert space (RKHS) and then perform machine learning in RKHS, while kernel learning is to select the best RKHS for specific applications and given learning samples. Since RKHS can be generated from kernel functions, kernel learning is to learn kernel functions. At present, the dilemma of kernel learning is that there are few kinds of kernel functions available for learning. The first contribution of this paper is to propose a new framework of kernel functions, in which the given learning samples can be embedded. Moreover, the framework contains a learnable part which can be optimized for specific applications. Symmetric positive definite (SPD) matrix data are more and more common in machine learning. However, SPD data space does not constitute a linear space and dictionary learning involves a lot of linear operations. Therefore, dictionary learning cannot be performed directly on SPD data space. The second contribution of this paper is to apply the proposed framework of kernel functions to dictionary learning of SPD data, in which SPD data are first transformed to the RKHS produced by the proposed framework, and then, both dictionary and the learnable part of the framework are learned simultaneously in RKHS. The experimental results on 4 landmark datasets show that the proposed algorithm performs better than 6 other algorithms published recently in top academic journals.

EEG Mental Recognition Based on RKHS Learning and Source Dictionary Regularized RKHS Subspace Learning

This article mainly studies Electroencephalogram (EEG) mental recognition. Because the human brain is very complex and the EEG signal is greatly affected by the environment, EEG mental recognition can be attributed to domain adaptative problems. Our main work is as follows: (1) At present, most domain adaptation learning only learns the linear subspace of Reproducing Kernel Hilbert Space (RKHS), and RKHS itself does not. Given the complexity and nonlinearity of EEG mental recognition, we propose an EEG mental recognition algorithm based on two learning. The two learning is RKHS learning and RKHS subspace learning. The source dictionary regularized RKHS subspace learning we proposed applies to EEG mental recognition and is better than pure RKHS subspace learning, but not enough. To get satisfactory results, we learn RKHS before RKHS subspace. (2) According to Moore–Aronszajn theorem, RKHS can be uniquely generated by a kernel function. The existing RKHS is rarely learnable. It is difficult to find a kernel function that can be learned and optimized. In RKHS learning, this paper uses a learnable kernel function that we published, and the kernel function is easy to optimize. (3) In RKHS subspace learning, most of the existing methods adopt the Maximum Mean Discrepancy (MMD) criterion, but it cannot make the spatial distribution of the same category of source and target domain data overlap as much as possible, and the label of the target domain data is unknown. To solve this problem, this paper proposes a framework of RKHS subspace learning based on source domain dictionary regularization. The experimental results on the brain-computer interface international competition data set (BCI competition IV 2a) show that the effect of the algorithm proposed in this paper is better than that of the other five advanced domain adaptation learning algorithms.

SPD Data Dictionary Learning Based on Kernel Learning and Riemannian Metric

The use of regional covariance descriptors to generate feature data represented by Symmetric Positive Definite (SPD) matrices from images or videos has become increasingly common in machine learning. However, SPD data itself does not constitute a vector space, and dictionary learning involves a large number of linear operations, so dictionary learning cannot be performed directly on SPD data. For this reason, a more common method is to map the SPD data to the Reproducing Kernel Hilbert Space (RKHS). The so-called kernel learning is to find the most suitable RKHS for specific tasks. RKHS can be uniquely generated by a kernel function. Therefore, RKHS learning can also be considered as kernel learning. In this article, there are two main contributions. The first contribution is to propose a framework which based on Kernel Learning and Riemannian Metric (KLRM). Usually the learnable kernel function framework is to learn some parameters in the kernel function. The second contribution is dictionary learning by applying KLRM to SPD data. The SPD data is transformed into the RKHS generated by KLRM, and RKHS after training provides the most suitable working space for dictionary learning. Under the proposed framework, we design a positive definite kernel function, which is defined by the Log-Euclidean metric. This function can be transformed into a corresponding Riemannian kernel. The experimental results provided in this paper is compared with other state-of-the-art algorithms for SPD data dictionary learning and show that the proposed algorithm achieves better results.

Deep embedding kernel

In this paper, we propose a novel supervised learning method that is called Deep Embedding Kernel (DEK). DEK combines the advantages of deep learning and kernel methods in a unified framework. More specifically, DEK is a learnable kernel represented by a newly designed deep architecture. Compared with predefined kernels, this kernel can be explicitly trained to map data to an optimized high-level feature space where data may have favorable features toward the application. Compared with typical deep learning using SoftMax or logistic regression as the top layer, DEK is expected to be more generalizable to new data. Experimental results show that DEK has superior performance than typical machine learning methods in identity detection and classification, and transfer learning, on different types of data including images, sequences, and regularly structured data.

Urban noise recognition with convolutional neural network

Urban noise recognition play a vital role in city management and safety operation, especially in the recent smart city engineering. Exiting studies on urban noise recognition are mostly based on conventional acoustic features, such as Mel-Frequency Cepstral Coefficients (MFCC) and Linear Prediction Cepstral Coefficients (LPCC), and the shallow structure based classifiers, such as support vector machine (SVM). However, the urban acoustic environment is complicated and changeable. Conventional acoustic representation and recognition methods may be insufficient in characterizing urban noises, and generally suffer from a degraded performance. In this paper, we study the recent deep neural network based urban noise recognition. The log-Mel-spectrogram, namely, the FBank feature is first derived for acoustic representation. Then, the FBank spectrum constructed with a set of FBank feature vectors from multiple acoustic signal frames is fed to a convolutional neural network (CNN) for urban noise recognition. Comprehensive studies on the dimension of FBank spectrums and the parameters in CNN, including the size of learnable kernels, the dropout rate, and the activation function, etc., are presented in the paper. An acoustic database collected in real environment covering 11 most common urban noises with more than 56,000 samples is constructed for model verification and performance evaluation. In addition, the traditional LPCC and MFCC acoustic feature combining with two popular machine learning algorithms, extreme learning machine (ELM) and support vector machine (SVM), and the FBank image feature combining with extreme learning machine (ELM), hierarchical extreme learning machine (H-ELM) and multilayer extreme learning machine (ML-ELM), have also been presented for discussions. Experimental results show that the proposed method generally outperforms conventional shallow structure based classifiers.