Convolutional Feature Extraction Research Articles

Multiscale 3-D-CNN based on spatial–spectral joint feature extraction for hyperspectral remote sensing images classification

In order to make full use of the spatial–spectral information of hyperspectral images (HSIs), a multiscale three-dimensional (3-D) convolutional neural network (CNN) model is proposed to extract the features of HSIs and classify them. First, the principal component analysis method is used to reduce the dimension of the original HSI, reduce the spectrum size on the basis of retaining the majority of spectrum information, and improve the training speed of the model. Second, at present, most of the HSI classification methods are based on spectral features and ignore the pixels’ neighborhood information on each band, thus a 3-D-CNN model is proposed for convolution feature extraction using a spatial–spectral joint method, which makes full use of the spatial–spectral information of HSI and greatly improves the classification effect of the model. Finally, traditional CNN models mostly use single-size convolution kernels. Several convolution kernels of different sizes are added to the convolution layer to extract the spatial features of different sizes, which effectively improves the classification ability of the model. The experimental results show that the multiscale 3-D-CNN model designed, when used on three datasets: Indian Pines, Pavia University, and Salinas, reached 97.54%, 99.78%, and 99.24% overall classification accuracy, respectively. Compared with the traditional single-scale two-dimensional-CNN model, the overall classification accuracy is improved by about 1% to 2%, which can better complete the classification task of HSIs.

A Deep Siamese Network with Hybrid Convolutional Feature Extraction Module for Change Detection Based on Multi-sensor Remote Sensing Images

Information extraction from multi-sensor remote sensing images has increasingly attracted attention with the development of remote sensing sensors. In this study, a supervised change detection method, based on the deep Siamese convolutional network with hybrid convolutional feature extraction module (OB-DSCNH), has been proposed using multi-sensor images. The proposed architecture, which is based on dilated convolution, can extract the deep change features effectively, and the character of “network in network” increases the depth and width of the network while keeping the computational budget constant. The change decision model is utilized to detect changes through the difference of extracted features. Finally, a change detection map is obtained via an uncertainty analysis, which combines the multi-resolution segmentation, with the output from the Siamese network. To validate the effectiveness of the proposed approach, we conducted experiments on multispectral images collected by the ZY-3 and GF-2 satellites. Experimental results demonstrate that our proposed method achieves comparable and better performance than mainstream methods in multi-sensor images change detection.

Lung Nodule Malignancy Prediction From Longitudinal CT Scans With Siamese Convolutional Attention Networks.

Goal: We propose a convolutional attention-based network that allows for use of pre-trained 2-D convolutional feature extractors and is extendable to multi-time-point classification in a Siamese structure. Methods: Our proposed framework is evaluated for single- and multi-time-point classification to explore the value that temporal information, such as nodule growth, adds to malignancy prediction. Results: Our results show that the proposed method outperforms a comparable 3-D network with less than half the parameters on single-time-point classification and further achieves performance gains on multi-time-point classification. Conclusions: Attention-based, Siamese 2-D pre-trained CNNs lead to fast training times and are effective for malignancy prediction from single-time-point or multiple-time-point imaging data.

Notice of Violation of IEEE Publication Principles: Ground-Based Cloud Image Recognition System Based on Multi-CNN and Feature Screening and Fusion

The recognition of ground-based cloud images has rich application prospects in many aspects such as weather prediction, astronomical site selection, and meteorological observation. Affected by factors such as rotation and illumination, the traditional feature extraction method is difficult to accurately describe the features of cloud images, resulting in low accuracy of ground-based cloud image recognition, and cannot meet the requirements of practical applications. With the popularity of convolutional neural networks in image processing, ground-based cloud image recognition algorithms based on convolutional neural network have become a research focus. However, the features of the ground-based cloud image are relatively shallow, and the cloud texture and other features are seriously lost in the convolution process, and it is difficult to achieve a good recognition effect. This paper proposes a ground-based cloud image recognition system based on multi-scale convolutional neural network (Multi-CNN) and multilayer perceptron neural networks (MLP). The multi-level and multi-scale convolution feature extraction is performed through convolution layers of Multi-CNN, and the local features with strong resolving power are selected through the feature screening algorithm based on DP clustering. Finally, the local features are encoded and fused for cloud image classification based on MLP. Filed test results show that our method was superior to other tested network models in terms of the recognition accuracy of 94.8% under 9 classification. In addition, ablation experiments show that the multi-scale feature extraction, screening and local feature coding in this paper have a significant effect on improving the algorithm’s ability to distinguish different cloud images.

Deep Group-Shuffling Dual Random Walks With Label Smoothing for Person Reidentification

Person reidentification (ReID) is a challenging task of finding a target pedestrian in a gallery set collected from multiple nonoverlapping camera views. Recently, state-of-the-art ReID performance has been achieved via an end-to-end trainable deep neural network framework, which integrates convolution feature extraction, similarity learning and reranking into a joint optimization framework. In such a framework, the similarity is learned via an embedding network, the reranking is conducted with a random walk, and the whole framework is optimized with a cross-entropy-based verification loss. Unfortunately, the embedding net is difficult to train well because their two-dimensional outputs mutually interfere each other when using the conventional random walk. In addition, the supervision information has not been fully exploited during the training phase due to the binary nature of the verification loss. In this paper, we propose a novel approach, called group-shuffling dual random walks with label smoothing (GSDRWLS), in which random walks are performed separately on two channels—one for positive verification and one for negative verification—and the binary verification labels are properly modified with an adaptive label smoothing technique before feeding into the verification loss in order to train the overall network effectively and to avoid the overfitting problem. Extensive experiments conducted on three large benchmark datasets, including CUHK03, Market-1501 and DukeMTMC, confirm the superior performance of our proposal.

Deep convolutional neural network-based feature extraction for steganalysis of content-adaptive JPEG steganography

Deep learning-based steganalysis techniques have significantly progressed in recent years. Compared with typical steganalysis methods using rich model features and classifiers, deep learning-based steganalysis methods usually train a deep convolutional neural network (CNN) as the steganography detector. One advantage of a deep CNN is the strong capability for discriminative feature learning. The input feature map of the fully connected layer is the steganalysis feature learned using the deep CNN. We can, therefore, extract the steganalysis feature based on the trained deep CNN. Compared with steganalysis features constructed by hand, the extraction of steganalysis features using a deep CNN can take advantage of the strong feature-learning capability of such a network. Therefore, two types of typical steganalysis frameworks are first compared. Then, a deep CNN for steganalysis feature learning is constructed, and the setting of the image preprocessing layer is discussed. Next, the detection performances of the different learned steganalysis features are compared. Finally, the detailed extraction process of the proposed learned steganalysis feature is described. The experiment results show that the proposed steganalysis method, combining learned and handcrafted steganalysis features, can significantly improve the detection performance for content-adaptive JPEG steganography.

CLR-based deep convolutional spiking neural network with validation based stopping for time series classification

Huge amount of time series data over several domains such as engineering, biomedical and finance, demands the development of efficient methods for the problem of time series classification. The classification of univariate and multivariate time series together using a single architecture is a very difficult task. In this work, a bio-inspired convolutional spiking neural network (CSNN) is proposed for both univariate and multivariate time series. For this, first we develop a simple transformation to convert raw time series sequences into matrices. The CSNN is a three staged framework which include convolutional feature extraction, spike encoding using soft leaky integrate and fire (Soft-LIf) and classification. As spikes generated are differentiable, thus the learning algorithm for CSNN uses error-backpropagation with cyclical learning rates (CLR) and RMSprop optimizer. Additionally, validation based stopping rules are employed to overcome the overfitting which also provides a set of parameters associated with low validation set loss. Thereafter, to demonstrate the accuracy and robustness of proposed CSNN model, we have used University of California (UCR) univariate as well as University of East Anglia (UEA) multivariate datasets to perform the experiments. Moreover, we conduct comparative empirical performance evaluation with benchmark methods and also with recent deep networks proposed for time series classification. Our results reveal that proposed CSNN advances the baseline methods by achieving higher performance accuracy for both univariate and multivariate datasets. It is shown that the CLR with RMSprop optimizer is able to achieve faster convergence, however CLR and adaptive rates are considered competitive to each other. In addition, we also address the optimal model selection and study the effects of different factors on the performance of proposed CSNN.

Extreme learning machine with autoencoding receptive fields for image classification

Based on the theory of local receptive field based extreme learning machine (ELM-LRF) and ELM auto encoder (ELM-AE), a new network structure is proposed to take advantage of global attributes of image and output feature of each layer in the structure. This proposed network structure is called extreme learning machine with autoencoding receptive fields (ELM-ARF), which has two parts including convolution feature extraction and feature coding. In the convolution feature extraction part, local features are extracted using orthogonalized local receptive fields. The ELM-AE theory and local receptive fields are used to encode the global receptive fields, which is used to extract global features. The pooled global features and local features are combined and input into the next layer. In the feature coding part, the shallow layer feature can be input to any deep layer through the ELM-ARF connection structure. A series of encodings are performed on the combined features in each layer to achieve a nonlinear mapping relationship from input information to target categories. In order to verify the validity of the structure, ELM-ARF is tested on four classic databases: USPS, MNIST, NORB and CIFAR10. The experimental results show that ELM-ARF effectively improves image classification accuracy by encoding the combined features that contain global attributes.

Construction of a Neural Network and its Application on Target Classification

With the arrival of big data age, the deep convolutional neural networks (DCNNs) with more hidden layers have a more complex network structure and more powerful ability than traditional machine learning methods for feature learning and feature expression. This paper first proposes a model of the DCNN to discuss the basic structure of model, convolutional feature extraction and learning algorithm of convolutional neural network; then, mainly introduces several aspects, that is, the construction of the typical network structure, the training methods, and the parameter settings of network model to be improved and optimized. Moreover, the network model is applied to the classification and recognition of Antarctic hydrological features and compared with some existing classification methods. The novelty of this paper mainly includes two aspects, i.e., the one is that the design and construction of the structure of deep neural network based on deep learning method are performed, namely, connection, weight, calculator, learning training of network, and other design. The other is classifying hydrological characteristics of Pritz Bay in Antarctica’s images by the DCNN. The results show that the correct recognition rate of the model method constructed by this paper is the highest. Finally, some problems in the current research are briefly summarized and discussed, and the new direction in the future development is forecasted.

One-Shot Scale and Angle Estimation for Fast Visual Object Tracking

Visual object tracking is the process of tracking an arbitrary object in a video, where the bounding box of the object is given in the first frame. Siamese network-based visual object tracking approaches have recently received considerable attention for their high speed and superior performance. However, for scale and angle estimation, Siamese trackers require multiple search regions, which increases the computation time, thereby decreasing the real-time tracking performance. This paper proposes a one-shot Siamese network, named Siam-OS, for fast and efficient visual object tracking. Siam-OS uses only a single search region and estimates the scale and angle of the target bounding box. This significantly reduces the number of computations required for the deep convolutional feature extraction, and thus increases the tracking speed. The experimental results with Visual Object Tracking (VOT) benchmarks show the effectiveness of the proposed Siam-OS in terms of the accuracy, robustness, expected average overlap, and speed.

Radar Target Recognition Based on Feature Pyramid Fusion Lightweight CNN

In order to improve the accuracy and robustness of radar target recognition under low SNR conditions, a novel radar high range resolution profile (HRRP) target recognition method based on feature pyramid fusion lightweight CNN is proposed in this paper. The proposed method combines the multi-scale space theory with a deep convolutional neural network. Because of the local connection characteristic of convolutional kernel, feature extracted by CNN mainly focus on the local information of the target. To make full use of both the local and global information in the target HRRP, multi-scale representation of the HRRP with different Gaussian kernels is introduced to construct the multi-channel input of the model. The generalization performance is improved by reducing the parameters of the proposed model with a depthwise separable convolution feature extraction block. Simultaneously, feature pyramid fusion is adapted to take full advantage of the features extracted by each block, which effectively improves the stability of the model and the training efficiency. The experimental results show that the multi-scale representation of the HRRP contributes to robust feature extraction. Meanwhile, the proposed feature pyramid fusion lightweight CNN can effectively prevent over-fitting and improve the stability of the model.

Improved faster R-CNN identification method for containers

In a complex port environment, the fast and effective automatic visual recognition of containers is an important part of the intelligent operation and management of ports. Due to the large amount of container image data of complex scale and shape, the traditional target detection and recognition algorithm is limited by the illumination, weather and scenes of the port, it has created challenges and difficulties in port container recognition and identification. This paper proposes a deep learning method for container target recognition detection based on the Faster R-CNN framework, the deep separable network structure is introduced into the VGG network, and the DS-VGG network is designed to improve the accuracy while reducing the network parameters to improve the recognition speed, by introducing the adversarial spatial transformer network (ASTN) to the Faster R-CNN network training to enhance the diversity of data features and improve recognition performance. In order to enhance the convolution feature extraction of container targets, a strategy training network that enhances sample target foreground features, multi-scale training learning and data amplification are used. Finally, the performance test and comparison test of the improved model proposed in this paper are carried out. The test results show that the target recognition speed is 50 frames/s on the container test set, the average accuracy rate is 97.7% and the recall rate is 94.45%. Compared with Faster R-CNN, the recognition performance is significantly improved in complex scenes such as fog, rain and night.

Ensemble Convoluted Feature Extraction for Affective Auditory P300 Brain-Computer Interfaces.

Since most of P300 brain-computer interface (BCI) methods have assumed visual events, they are not always suitable for the BCI with auditory events, and feature extraction methods appropriate for auditory P300 BCI are required. This study proposed ensemble convoluted feature extraction for affective auditory P300 BCI, which took advantage of auditory responses elicited by different affective sounds. The proposed method was compared to feature extraction method that uses the canonical correlation analysis in addition to the traditional method. Those methods were evaluated on the dataset recorded from the improved affective auditory P300 BCI system. The mean online classification accuracy was 84.1% when using the traditional feature. The offline analysis showed that the proposed ensemble convolution feature extraction achieved significantly higher accuracy (86.75%) than the traditional method. The propose feature extraction may effective for the multi-channel time-series BCI that is featured by different stimuli.

Land-use scene classification based on a CNN using a constrained extreme learning machine

ABSTRACTClassifying land-use scenes from high-resolution remote-sensing imagery with high quality and accuracy is of paramount interest for science and land management applications. In this article, we proposed a new model for land-use scene classification by integrating the recent success of convolutional neural network (CNN) and constrained extreme learning machine (CELM). In the model, the fully connected layers of a pretrained CNN have been removed. Then, CNN works as a deep and robust convolutional feature extractor. After normalization, deep convolutional features are fed to the CELM classifier. To analyse the performance, the proposed method has been evaluated on two challenging high-resolution data sets: (1) the aerial image data set consisting of 30 different aerial scene categories with sub-metre resolution and (2) a Sydney data set that is a large high spatial resolution satellite image. Experimental results show that the CNN-CELM model improves the generalization ability and reduces the training time compared to state-of-the-art methods.

Transfer-Learning-Based Online Mura Defect Classification

Flat panel displays, such as the thin film transistor liquid crystal display, the organic light-emitting diode, and the polymer light-emitting diode, have been widely applied in many fields in recent decades. To ensure the quality of these displays, defect inspection is crucial. Mura defects, which are phenomena of uneven screen displays, are the most challenging visual defects to detect. This paper presents an online sequential classifier and transfer learning (OSC-TL) method for the online training and classification of Mura defects. OSC-TL is a new method that combines a deep convolutional feature extractor and a sequential extreme learning machine classifier. It makes online sequential training in a production line possible. To demonstrate the performance of the OSC-TL method, several experiments are performed to compare the results of this method with those of other popular classification algorithms. The experimental results show that the computational resources and time consumed by OSC-TL are well below those of other common methods because of the feature transfer and the online sequential classification strategies. Consequently, the OSC-TL method has been implemented in our automated optical inspection equipment to perform online Mura defect classification. It is able to learn and recognize a Mura defect image within 1.5 milliseconds.

Weakly-Supervised Deep Embedding for Product Review Sentiment Analysis

Product reviews are valuable for upcoming buyers in helping them make decisions. To this end, different opinion mining techniques have been proposed, where judging a review sentence's orientation (e.g., positive or negative) is one of their key challenges. Recently, deep learning has emerged as an effective means for solving sentiment classification problems. A neural network intrinsically learns a useful representation automatically without human efforts. However, the success of deep learning highly relies on the availability of large-scale training data. We propose a novel deep learning framework for product review sentiment classification which employs prevalently available ratings as weak supervision signals. The framework consists of two steps: (1) learning a high level representation (an embedding space) which captures the general sentiment distribution of sentences through rating information; and (2) adding a classification layer on top of the embedding layer and use labeled sentences for supervised fine-tuning. We explore two kinds of low level network structure for modeling review sentences, namely, convolutional feature extractors and long short-term memory. To evaluate the proposed framework, we construct a dataset containing 1.1M weakly labeled review sentences and 11,754 labeled review sentences from Amazon. Experimental results show the efficacy of the proposed framework and its superiority over baselines.

An Investigation of Deep-Learning Frameworks for Speaker Verification Antispoofing

In this study, we explore the use of deep-learning approaches for spoofing detection in speaker verification. Most spoofing detection systems that have achieved recent success employ hand-craft features with specific spoofing prior knowledge, which may limit the feasibility to unseen spoofing attacks. We aim to investigate the genuine-spoofing discriminative ability from the back-end stage, utilizing recent advancements in deep-learning research. In this paper, alternative network architectures are exploited to target spoofed speech. Based on this analysis, a novel spoofing detection system, which simultaneously employs convolutional neural networks (CNNs) and recurrent neural networks (RNNs) is proposed. In this framework, CNN is treated as a convolutional feature extractor applied on the speech input. On top of the CNN processed output, recurrent networks are employed to capture long-term dependencies across the time domain. Novel features including Teager energy operator critical band autocorrelation envelope, perceptual minimum variance distortionless response, and a more general spectrogram are also investigated as inputs to our proposed deep-learning frameworks. Experiments using the ASVspoof 2015 Corpus show that the integrated CNN–RNN framework achieves state-of-the-art single-system performance. The addition of score-level fusion further improves system robustness. A detailed analysis shows that our proposed approach can potentially compensate for the issue due to short duration test utterances, which is also an issue in the evaluation corpus.