Convolutional Neural Network Filter Research Articles

Stance Detection of Microblog Text Based on Two-Channel CNN-GRU Fusion Network

In recent years, stance detection has become an important topic in the field of natural language processing. In earlier work, researchers have used feature engineering for stance detection but they need to define and extract appropriate features according to the particular application. This leads to poor generalization and a complex modeling process. Other researchers have applied deep learning methods. However, the popular convolutional neural network (CNN) method has the problem of information loss and a single-size CNN filter cannot accurately extract features that have different lengths from text, and so cannot deal with the diverse nature of features. In order to address these problems, we propose a two-channel CNN-GRU fusion network. First, a convolution layer with two filters with different window sizes is used to extract local features within the topic content and text content. Then, a gated recurrent unit (GRU) network is used to extract their timing characteristics. After that, the intermediate features are spliced and input to a classifier to complete the stance detection. Our method is validated using data from NLPCC 2016. The experimental results show that ACC and average F1 score of this method are 13.1% and 15.6% better than SVM method, 6.2% and 11.6% better than CNN method, 5.6% and 3.3% better than GRU method, and 1.1% and 2.2% better compared with hybrid model proposed by Nanyu, respectively, which is used as a baseline with no increase in run-time, and achieves the same accuracy with less run-time than another baseline of a semantic attention-based model proposed by Zhou. In addition, our method allows better classification than the single channel model. Finally, we find that the operation time of a multi-channel CNN-GRU increases gradually with increasing number of channels, but the classification accuracy does not improve, so a two-channel CNN-GRU is the most appropriate choice.

An Always-On 3.8 <inline-formula> <tex-math notation="LaTeX">$\mu$ </tex-math> </inline-formula>J/86% CIFAR-10 Mixed-Signal Binary CNN Processor With All Memory on Chip in 28-nm CMOS

The trend of pushing inference from cloud to edge due to concerns of latency, bandwidth, and privacy has created demand for energy-efficient neural network hardware. This paper presents a mixed-signal binary convolutional neural network (CNN) processor for always-on inference applications that achieves 3.8 $\mu \text{J}$ /classification at 86% accuracy on the CIFAR-10 image classification data set. The goal of this paper is to establish the minimum-energy point for the representative CIFAR-10 inference task, using the available design tradeoffs. The BinaryNet algorithm for training neural networks with weights and activations constrained to +1 and −1 drastically simplifies multiplications to XNOR and allows integrating all memory on-chip. A weight-stationary, data-parallel architecture with input reuse amortizes memory access across many computations, leaving wide vector summation as the remaining energy bottleneck. This design features an energy-efficient switched-capacitor (SC) neuron that addresses this challenge, employing a 1024-bit thermometer-coded capacitive digital-to-analog converter (CDAC) section for summing pointwise products of CNN filter weights and activations and a 9-bit binary-weighted section for adding the filter bias. The design occupies 6 mm2 in 28-nm CMOS, contains 328 kB of on-chip SRAM, operates at 237 frames/s (FPS), and consumes 0.9 mW from 0.6 V/0.8 V supplies. The corresponding energy per classification (3.8 $\mu \text{J}$ ) amounts to a 40 $\times $ improvement over the previous low-energy benchmark on CIFAR-10, achieved in part by sacrificing some programmability. The SC neuron array is 12.9 $\times $ more energy efficient than a synthesized digital implementation, which amounts to a 4 $\times $ advantage in system-level energy per classification.

A deep CNN based transfer learning method for false positive reduction

A low false positive (FP) rate is of great importance for the use of a Computer Aided Detection (CAD) system to detect pulmonary nodules in thoracic Computed Tomography (CT). However, due to the variations of nodules in appear and size, it is still a very challenging task to obtain a low FP rate. In this paper, we propose a deep Convolutional Neural Network (CNN) based transfer learning method for FP reduction in pulmonary nodule detection on CT slices. We utilized one of the state-of-the-art CNN models, VGG-16 [4], as a feature extractor to obtain nodule features, and used a support vector machine (SVM) for nodule classification. Firstly we transferred all the layers from a pre-trained VGG-16 model in ImageNet to our target networks. Then, we tuned the last fully connected layers to adjust the computer-vision-task-trained CNN model to pulmonary nodule classification task. The initial CNN filter weights were then optimized using the training data, i.e., the pulmonary nodule patch images and corresponding labels through back-propagation so that they better reflected the modalities in the pulmonary nodule image dataset. Finally, features learned in the fine-tuned CNN were used to train a SVM classifier. The output of the trained SVM was used for final classification. Experimental results show that the overall sensitivity of the proposed method was 87.2% with 0.39 FPs per scan, which is higher than 85.4% with 4 FPs per scan obtained by other state of art method.

Convolutional adaptive denoising autoencoders for hierarchical feature extraction

Convolutional neural networks (CNNs) are typical structures for deep learning and are widely used in image recognition and classification. However, the random initialization strategy tends to become stuck at local plateaus or even diverge, which results in rather unstable and ineffective solutions in real applications. To address this limitation, we propose a hybrid deep learning CNN-AdapDAE model, which applies the features learned by the AdapDAE algorithm to initialize CNN filters and then train the improved CNN for classification tasks. In this model, AdapDAE is proposed as a CNN pre-training procedure, which adaptively obtains the noise level based on the principle of annealing, by starting with a high level of noise and lowering it as the training progresses. Thus, the features learned by AdapDAE include a combination of features at different levels of granularity. Extensive experimental results on STL-10, CIFAR-10, andMNIST datasets demonstrate that the proposed algorithm performs favorably compared to CNN (random filters), CNNAE (pre-training filters by autoencoder), and a few other unsupervised feature learning methods.

Ambience Inhaling: Speech Noise Inhaler in Mobile Robots using Deep Learning

Audio based, machine learning human-computer interface with speech recognition systems performs sensibly well with the human voice under clean ambience, but become frail in applied technological implementation involving real-life interface. In mobile robotic systems, the speech machines are normally retrained with new changing acoustic ambience conditions are to be met. To inhale, classify, and track the real-world ambience noise with the new changing acoustic condition, we introduce an Ambience Inhaling (AI) framework in this article. This framework of an AI is to seek out complete noise information from speech data, in contrast with noise-nature discovery. Our proposed framework uses a deep convolutional neural network (CNN) based learning for classification with speech spectrogram patch segments, including a hybrid Harold Hotelling's T-square algorithm with Bayesian statistics for segmentation analysis. We use a symposium presentation-ambience as a test platform. In the symposium presentation-ambience, noise modeling is done with n-gram language having the parameter of n = 2. The impulsive or short-term noise which is superimposed with long-term noise caused degradation in classification. This degradation caused the classification errors. The provision of decision was made. The Gaussian mixture model and hidden Markova model are used with noise-only and noisy speech respectively. Time and frequency pooling are used with spectrogram also. The classification scores of 62.26%, 65.89%, and 69.12% are achieved with 5, 10 and 15 CNN filters respectively. As a significance, an AI is efficient and innovative.

Discriminatively Learned Hierarchical Rank Pooling Networks

Rank pooling is a temporal encoding method that summarizes the dynamics of a video sequence to a single vector which has shown good results in human action recognition in prior work. In this work, we present novel temporal encoding methods for action and activity classification by extending the unsupervised rank pooling temporal encoding method in two ways. First, we present discriminative rank pooling in which the shared weights of our video representation and the parameters of the action classifiers are estimated jointly for a given training dataset of labelled vector sequences using a bilevel optimization formulation of the learning problem. When the frame level features vectors are obtained from a convolutional neural network (CNN), we rank pool the network activations and jointly estimate all parameters of the model, including CNN filters and fully-connected weights, in an end-to-end manner which we coined as end-to-end trainable rank pooled CNN. Importantly, this model can make use of any existing convolutional neural network architecture (e.g., AlexNet or VGG) without modification or introduction of additional parameters. Then, we extend rank pooling to a high capacity video representation, called hierarchical rank pooling. Hierarchical rank pooling consists of a network of rank pooling functions, which encode temporal semantics over arbitrary long video clips based on rich frame level features. By stacking non-linear feature functions and temporal sub-sequence encoders one on top of the other, we build a high capacity encoding network of the dynamic behaviour of the video. The resulting video representation is a fixed-length feature vector describing the entire video clip that can be used as input to standard machine learning classifiers. We demonstrate our approach on the task of action and activity recognition. We present a detailed analysis of our approach against competing methods and explore variants such as hierarchy depth and choice of non-linear feature function. Obtained results are comparable to state-of-the-art methods on three important activity recognition benchmarks with classification performance of 76.7% mAP on Hollywood2, 69.4% on HMDB51, and 93.6% on UCF101.

A-optimal convolutional neural network

In this paper, we propose a novel data representation-classification model learning algorithm. The model is a convolutional neural network (CNN), and we learn its parameters to achieve A-optimality. The input multi-instance data are represented by a CNN model, and then classified by a linear classification model. The A-optimality of a classification model is measured by the trace of the covariance matrix of the model parameter vector. To achieve the A-optimality of the CNN model, we minimize the classification errors and a regularization term to present the classification model parameter as a function of the CNN filter parameters, and minimize its trace of the covariance matrix. We show that the minimization problem can be solved easily by transferring it to another coupled minimization problem. In the experiments over benchmark data sets of molecular, image, and seismic waveform, we show the advantages of the proposed A-optimal CNN model.

Building Correlations Between Filters in Convolutional Neural Networks.

In this paper, a new optimization approach is designed for convolutional neural network (CNN) which introduces explicit logical relations between filters in the convolutional layer. In a conventional CNN, the filters' weights in convolutional layers are separately trained by their own residual errors, and the relations of these filters are not explored for learning. Different from the traditional learning mechanism, the proposed correlative filters (CFs) are initiated and trained jointly in accordance with predefined correlations, which are efficient to work cooperatively and finally make a more generalized optical system. The improvement in CNN performance with the proposed CF is verified on five benchmark image classification datasets, including CIFAR-10, CIFAR-100, MNIST, STL-10, and street view house number. The comparative experimental results demonstrate that the proposed approach outperforms a number of state-of-the-art CNN approaches.

Convolutional deep learning for 3D object retrieval

In recent years, with the development of 3D technologies, 3D model retrieval has become a hot topic. The key point of 3D model retrieval is to extract robust feature for 3D model representation. In order to improve the effectiveness of method on 3D model retrieval, this paper proposes a feature extraction model based on convolutional neural networks (CNN). First, we extract a set of 2D images from 3D model to represent each 3D object. SIFT detector is utilized to detect interesting points from each 2D image and extract interesting patches to represent local information of each 3D model. X-means is leveraged to generate the CNN filters. Second, a single CNN layer learns low-level features which are then given as inputs to multiple recursive neural networks (RNN) in order to compose higher order features. RNNs can generate the final feature for 2D image representation. Finally, nearest neighbor is used to compute the similarity between different 3D models in order to handle the retrieval problem. Extensive comparison experiments were on the popular ETH and MV-RED 3D model datasets. The results demonstrate the superiority of the proposed method.