Learning High-level Features Research Articles

Deep CNN With Multi-Scale Rotation Invariance Features for Ship Classification

With the rapid development of target tracking technology, how to efficiently take advantage of useful information from optical images for ship classification becomes a challenging problem. In this paper, a novel deep learning framework fused with low-level features is proposed. Deep convolutional neural network (CNN) has been popularly used to capture structural information and semantic context because of the ability of learning high-level features; however, lacking of capability to deal with global rotation in large-scale image and losing some important information in bottom layers of the CNN limit its performance in extracting multi-scales rotation invariance features. Comparatively, some classic algorithms, such as Gabor filter or multiple scales completed local binary patterns, can effectively capture low-level texture information. In the proposed framework, low-level features are combined with high-level features obtained by deep CNN. The fused features are further fed into a typical support vector machine classifier. The proposed strategy achieves average accuracy of 98.33% on the BCCT200-RESIZE data and 88.00% on the challenging VAIS data, which demonstrates its superior classification performance when compared with some state-of-the-art methods.

Learning Efficient Binary Codes From High-Level Feature Representations for Multilabel Image Retrieval

Due to the efficiency and effectiveness of hashing technologies, they have become increasingly popular in large-scale image semantic retrieval. However, existing hash methods suppose that the data distributions satisfy the manifold assumption that semantic similar samples tend to lie on a low-dimensional manifold, which will be weakened due to the large intraclass variation. Moreover, these methods learn hash functions by relaxing the discrete constraints on binary codes to real value, which will introduce large quantization loss. To tackle the above problems, this paper proposes a novel unsupervised hashing algorithm to learn efficient binary codes from high-level feature representations. More specifically, we explore nonnegative matrix factorization for learning high-level visual features. Ultimately, binary codes are generated by performing binary quantization in the high-level feature representations space, which will map images with similar (visually or semantically) high-level feature representations to similar binary codes. To solve the corresponding optimization problem involving nonnegative and discrete variables, we develop an efficient optimization algorithm to reduce quantization loss with guaranteed convergence in theory. Extensive experiments show that our proposed method outperforms the state-of-the-art hashing methods on several multilabel real-world image datasets.

Robust Semisupervised Classification for PolSAR Image With Noisy Labels

The robustness of the supervised polarimetric synthetic aperture radar (PolSAR) image classification is severely affected by two main aspects, namely, the quantity and quality of the labeled training pixels. Specifically, limited manually labeled pixels with respect to the large scale of PolSAR image have limited the performance of the automatic classification methods, while manually labeled training pixels shall be unfaithful with the speckle and impure cell for their low qualities. In order to address the above two fundamental problems, we propose a robust semisupervised probability graphic-based classification framework. First, a semisupervised learning scheme is implemented to simultaneously exploit both labeled and unlabeled pixels for information compensation. Moreover, structural relationship among neighboring pixels inducing from the prior information is further benefit to reduce the influence of limited labeled pixels. Second, a robust classification loss function is added in the process of training classifier to enhance the robustness to the noisy labeled pixels. Third, unfaithful limited labeled data can be settled with a hybrid generative/discriminative classification framework, where labeled and unlabeled pixels are simultaneously exploited for learning high-level feature for the low-quality pixels. The effectiveness of the proposed framework on the specific aspect is validated in experiments on real PolSAR data sets, which reveal the superiority in both visual performance and classification accuracy compared with the state-of-the-art methods. Totally speaking, our model has improved the classification accuracy by at least 20% on data set Flevoland, 10% on Oberpfaffenhofen, and 5% on Weihe River than the compared ones.

Prediction of human protein subcellular localization using deep learning

Protein subcellular localization (PSL), as one of the most critical characteristics of human cells, plays an important role for understanding specific functions and biological processes in cells. Accurate prediction of protein subcellular localization is a fundamental and challenging problem, for which machine learning algorithms have been widely used. Traditionally, the performance of PSL prediction highly depends on handcrafted feature descriptors to represent proteins. In recent years, deep learning has emerged as a hot research topic in the field of machine learning, achieving outstanding success in learning high-level latent features within data samples. In this paper, to accurately predict protein subcellular locations, we propose a deep learning based predictor called DeepPSL by using Stacked Auto-Encoder (SAE) networks. In this predictor, we automatically learn high-level and abstract feature representations of proteins by exploring non-linear relations among diverse subcellular locations, addressing the problem of the need of handcrafted feature representations. Experimental results evaluated with three-fold cross validation show that the proposed DeepPSL outperforms traditional machine learning based methods. It is expected that DeepPSL, as the first predictor in the field of PSL prediction, has great potential to be a powerful computational method complementary to existing tools.

A Siamese inception architecture network for person re-identification

Person re-identification is an extremely challenging problem as person’s appearance often undergoes dramatic changes due to the large variations of viewpoints, illuminations, poses, image resolutions, and cluttered backgrounds. How to extract discriminative features is one of the most critical ways to address these challenges. In this paper, we mainly focus on learning high-level features and combine the low-level, mid-level, and high-level features together to re-identify a person across different cameras. Firstly, we design a Siamese inception architecture network to automatically learn effective semantic features for person re-identification in different camera views. Furthermore, we combine multi-level features in null space with the null Foley–Sammon transform metric learning approach. In this null space, images of the same person are projected to a single point, which minimizes the intra-class scatter to the extreme and maximizes the relative inter-class separation simultaneously. Finally, comprehensive evaluations demonstrate that our approach achieves better performance on four person re-identification benchmark datasets, including Market-1501, CUHK03, PRID2011, and VIPeR.

Spectral–spatial classification of hyperspectral images using trilateral filter and stacked sparse autoencoder

A spectral–spatial classification method using a trilateral filter (TF) and stacked sparse autoencoder (SSA) for improving the classification accuracy of hyperspectral image (HSI) is proposed. The operation is carried out in two main stages: edge-preserved smoothing and high-level feature learning. First, a reference image obtained from dual tree complex wavelet transform is adopted in a TF for smoothing the HSI. As expected, the filter not only can effectively attenuate the mixed noise (e.g., Gaussian noise and impulse noise) where the bilateral filter shows poor performance but also can produce useful spectral–spatial features from HSI by considering geometric closeness and photometric similarity between pixels simultaneously. Second, an artificial fish swarm algorithm (AFSA) is first introduced into a SSA, and the proposed deep learning architecture is used to adaptively exploit more abstract and differentiable high-level feature representations from the smoothed HSI, based on the factor that AFSA provides better trade-off among concurrency, search efficiency, and convergence rate compared with gradient descent and back-propagation algorithms in a traditional SSA. Finally, a random forest classifier is utilized to perform supervised fine-tuning and classification. Experimental results on two real HSI data sets demonstrate that the proposed method generates competitive performance compared with those of conventional methods.

Feature Selection Guided Auto-Encoder

Recently the auto-encoder and its variants have demonstrated their promising results in extracting effective features. Specifically, its basic idea of encouraging the output to be as similar as input, ensures the learned representation could faithfully reconstruct the input data. However, one problem arises that not all hidden units are useful to compress the discriminative information while lots of units mainly contribute to represent the task-irrelevant patterns. In this paper, we propose a novel algorithm, Feature Selection Guided Auto-Encoder, which is a unified generative model that integrates feature selection and auto-encoder together. To this end, our proposed algorithm can distinguish the task-relevant units from the task-irrelevant ones to obtain most effective features for future classification tasks. Our model not only performs feature selection on learned high-level features, but also dynamically endows the auto-encoder to produce more discriminative units. Experiments on several benchmarks demonstrate our method's superiority over state-of-the-art approaches.

Collaborative Random Faces-Guided Encoders for Pose-Invariant Face Representation Learning.

Learning discriminant face representation for pose-invariant face recognition has been identified as a critical issue in visual learning systems. The challenge lies in the drastic changes of facial appearances between the test face and the registered face. To that end, we propose a high-level feature learning framework called "collaborative random faces (RFs)-guided encoders" toward this problem. The contributions of this paper are three fold. First, we propose a novel supervised autoencoder that is able to capture the high-level identity feature despite of pose variations. Second, we enrich the identity features by replacing the target values of conventional autoencoders with random signals (RFs in this paper), which are unique for each subject under different poses. Third, we further improve the performance of the framework by incorporating deep convolutional neural network facial descriptors and linking discriminative identity features from different RFs for the augmented identity features. Finally, we conduct face identification experiments on Multi-PIE database, and face verification experiments on labeled faces in the wild and YouTube Face databases, where face recognition rate and verification accuracy with Receiver Operating Characteristic curves are rendered. In addition, discussions of model parameters and connections with the existing methods are provided. These experiments demonstrate that our learning system works fairly well on handling pose variations.

Tongue Images Classification Based on Constrained High Dispersal Network.

Computer aided tongue diagnosis has a great potential to play important roles in traditional Chinese medicine (TCM). However, the majority of the existing tongue image analyses and classification methods are based on the low-level features, which may not provide a holistic view of the tongue. Inspired by deep convolutional neural network (CNN), we propose a novel feature extraction framework called constrained high dispersal neural networks (CHDNet) to extract unbiased features and reduce human labor for tongue diagnosis in TCM. Previous CNN models have mostly focused on learning convolutional filters and adapting weights between them, but these models have two major issues: redundancy and insufficient capability in handling unbalanced sample distribution. We introduce high dispersal and local response normalization operation to address the issue of redundancy. We also add multiscale feature analysis to avoid the problem of sensitivity to deformation. Our proposed CHDNet learns high-level features and provides more classification information during training time, which may result in higher accuracy when predicting testing samples. We tested the proposed method on a set of 267 gastritis patients and a control group of 48 healthy volunteers. Test results show that CHDNet is a promising method in tongue image classification for the TCM study.

CNN for saliency detection with low-level feature integration

Feature matters. In this paper, a novel deep neural network framework integrated with low-level features for salient object detection is proposed for complex images. We utilise the advantage of convolutional neural networks to automatically learn the high-level features that capture the structured information and semantic context in the image. In order to better adapt a CNN model into the saliency task, we redesign the network architecture based on typical saliency datasets, which is relatively small-scale compared to ImageNet. Several low-level features are extracted, which can effectively capture contrast and spatial information in the salient regions, and incorporated to compensate with the learned high-level features at the output of the very last fully connected layer. The concatenated feature vector is further fed into a hinge-loss SVM detector in a joint discriminative learning manner and the final saliency score of each region within the bounding box is obtained by the linear combination of the detector's weights. Experiments on three challenging benchmarks demonstrate our algorithm to be effective and superior than most low-level oriented state-of-the-arts in terms of precision-recall curves, F-measure and mean absolute errors. Moreover, a series of ablation studies are conducted to verify our algorithm's simplicity and efficiency from different aspects.

Cerebral cortex classification by conditional random fields applied to intraoperative thermal imaging

Abstract Intraoperative thermal neuroimaging is a novel intraoperative imaging technique for the characterization of perfusion disorders, neural activity and other pathological changes of the brain. It bases on the correlation of (sub-)cortical metabolism and perfusion with the emitted heat of the cortical surface. In order to minimize required computational resources and prevent unwanted artefacts in subsequent data analysis workflows foreground detection is a important preprocessing technique to differentiate pixels representing the cerebral cortex from background objects. We propose an efficient classification framework that integrates characteristic dynamic thermal behaviour into this classification task to include additional discriminative features. The first stage of our framework consists of learning this representation of characteristic thermal time-frequency behaviour. This representation models latent interconnections in the time-frequency domain that cover specific, yet a priori unknown, thermal properties of the cortex. In a second stage these features are then used to classify each pixel’s state with conditional random fields. We quantitatively evaluate several approaches to learning high-level features and their impact to the overall prediction accuracy. The introduction of high-level features leads to a significant accuracy improvement compared to a baseline classifier.

Semantic Annotation of High-Resolution Satellite Images via Weakly Supervised Learning

In this paper, we focus on tackling the problem of automatic semantic annotation of high resolution (HR) optical satellite images, which aims to assign one or several predefined semantic concepts to an image according to its content. The main challenges arise from the difficulty of characterizing complex and ambiguous contents of the satellite images and the high human labor cost caused by preparing a large amount of training examples with high-quality pixel-level labels in fully supervised annotation methods. To address these challenges, we propose a unified annotation framework by combining discriminative high-level feature learning and weakly supervised feature transferring. Specifically, an efficient stacked discriminative sparse autoencoder (SDSAE) is first proposed to learn high-level features on an auxiliary satellite image data set for the land-use classification task. Inspired by the motivation that the encoder of the prelearned SDSAE can be regarded as a generic high-level feature extractor for HR optical satellite images, we then transfer the learned high-level features to semantic annotation. To compensate the difference between the auxiliary data set and the annotation data set, the transferred high-level features are further fine-tuned in a weakly supervised scheme by using the tile-level annotated training data. Finally, the fine-tuning process is formulated as an ultimate optimization problem, which can be solved efficiently with our proposed alternate iterative optimization method. Comprehensive experiments on a publicly available land-use classification data set and an annotation data set demonstrate the superiority of our SDSAE-based high-level feature learning method and the effectiveness of our weakly supervised semantic annotation framework compared with state-of-the-art fully supervised annotation methods.

Discriminant deep belief network for high-resolution SAR image classification

Classification plays an important role in many fields of synthetic aperture radar (SAR) image understanding and interpretation. Many scholars have devoted to design features to characterize the content of SAR images. However, it is still a challenge to design discriminative and robust features for SAR image classification. Recently, the deep learning has attracted much attention and has been successfully applied in many fields of computer vision. In this paper, a novel feature learning approach that is called discriminant deep belief network (DisDBN) is proposed to learning high-level features for SAR image classification, in which the discriminant features are learned by combining ensemble learning with a deep belief network in an unsupervised manner. Firstly, some subsets of SAR image patches are selected and marked with pseudo-labels to train weak classifiers. Secondly, the specific SAR image patch is characterized by a set of projection vectors that are obtained by projecting the SAR image patch onto each weak decision space spanned by each weak classifier. Finally, the discriminant features are generated by feeding the projection vectors to a DBN for SAR image classification. Experimental results demonstrate that better classification performance can be achieved by the proposed approach than the other state-of-the-art approaches.

Convolutional neural random fields for action recognition

A deep discriminative structured model, convolutional neural random fields (CNRFs), is proposed for action recognition problem. In the proposed model, a spatio-temporal convolutional neural network (CNN) is developed for invariant feature learning from raw input frames, and the CNN is combined with conditional random fields (CRFs) for capturing the interdependencies between outputs. The parameters from both CRF and CNN are learned in a joint fashion which enables structured prediction and feature learning as well. We also explore different combinations of observation and transition feature functions based on the learned high level features from convolution part. The approach enjoys the advantages of both CNN and CRF, it has the invariant feature learning ability possessed by the former and structured prediction ability of the latter. The experimental results on both segmented and unsegmented human action recognition datasets show that CNRF boosts the performance over the comparison methods by a large margin.

Modality-specific and hierarchical feature learning for RGB-D hand-held object recognition

Hand-held object recognition is an important research topic in image understanding and plays an essential role in human-machine interaction. With the easily available RGB-D devices, the depth information greatly promotes the performance of object segmentation and provides additional channel information. While how to extract a representative and discriminating feature from object region and efficiently take advantage of the depth information plays an important role in improving hand-held object recognition accuracy and eventual human-machine interaction experience. In this paper, we focus on a special but important area called RGB-D hand-held object recognition and propose a hierarchical feature learning framework for this task. First, our framework learns modality-specific features from RGB and depth images using CNN architectures with different network depth and learning strategies. Secondly a high-level feature learning network is implemented for a comprehensive feature representation. Different with previous works on feature learning and representation, the hierarchical learning method can sufficiently dig out the characteristics of different modal information and efficiently fuse them in a unified framework. The experimental results on HOD dataset illustrate the effectiveness of our proposed method.

Continuous action segmentation and recognition using hybrid convolutional neural network‐hidden Markov model model

Continuous action recognition in video is more complicated compared with traditional isolated action recognition. Besides the high variability of postures and appearances of each action, the complex temporal dynamics of continuous action makes this problem challenging. In this study, the authors propose a hierarchical framework combining convolutional neural network (CNN) and hidden Markov model (HMM), which recognises and segments continuous actions simultaneously. The authors utilise the CNN's powerful capacity of learning high level features directly from raw data, and use it to extract effective and robust action features. The HMM is used to model the statistical dependences over adjacent sub‐actions and infer the action sequences. In order to combine the advantages of these two models, the hybrid architecture of CNN‐HMM is built. The Gaussian mixture model is replaced by CNN to model the emission distribution of HMM. The CNN‐HMM model is trained using embedded Viterbi algorithm, and the data used to train CNN are labelled by forced alignment. The authors test their method on two public action dataset Weizmann and KTH. Experimental results show that the authors’ method achieves improved recognition and segmentation accuracy compared with several other methods. The superior property of features learnt by CNN is also illustrated.

Learning High-level Features for Satellite Image Classification With Limited Labeled Samples

This paper presents a novel method addressing the classification task of satellite images when limited labeled data is available together with a large amount of unlabeled data. Instead of using semi-supervised classifiers, we solve the problem by learning a high-level features, called semisupervised ensemble projection (SSEP). More precisely, we propose to represent an image by projecting it onto an ensemble of weak training (WT) sets sampled from a Gaussian approximation of multiple feature spaces. Given a set of images with limited labeled ones, we first extract preliminary features, e.g., color and textures, to form a low-level image description. We then propose a new semisupervised sampling algorithm to build an ensemble of informative WT sets by exploiting these feature spaces with a Gaussian normal affinity, which ensures both the reliability and diversity of the ensemble. Discriminative functions are subsequently learned from the resulting WT sets, and each image is represented by concatenating its projected values onto such WT sets for final classification. Moreover, we consider that the potential redundant information existed in SSEP and use sparse coding to reduce it. Experiments on high-resolution remote sensing data demonstrate the efficiency of the proposed method.

Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images.

Automated nuclear detection is a critical step for a number of computer assisted pathology related image analysis algorithms such as for automated grading of breast cancer tissue specimens. The Nottingham Histologic Score system is highly correlated with the shape and appearance of breast cancer nuclei in histopathological images. However, automated nucleus detection is complicated by 1) the large number of nuclei and the size of high resolution digitized pathology images, and 2) the variability in size, shape, appearance, and texture of the individual nuclei. Recently there has been interest in the application of "Deep Learning" strategies for classification and analysis of big image data. Histopathology, given its size and complexity, represents an excellent use case for application of deep learning strategies. In this paper, a Stacked Sparse Autoencoder (SSAE), an instance of a deep learning strategy, is presented for efficient nuclei detection on high-resolution histopathological images of breast cancer. The SSAE learns high-level features from just pixel intensities alone in order to identify distinguishing features of nuclei. A sliding window operation is applied to each image in order to represent image patches via high-level features obtained via the auto-encoder, which are then subsequently fed to a classifier which categorizes each image patch as nuclear or non-nuclear. Across a cohort of 500 histopathological images (2200 × 2200) and approximately 3500 manually segmented individual nuclei serving as the groundtruth, SSAE was shown to have an improved F-measure 84.49% and an average area under Precision-Recall curve (AveP) 78.83%. The SSAE approach also out-performed nine other state of the art nuclear detection strategies.

Object Detection in Optical Remote Sensing Images Based on Weakly Supervised Learning and High-Level Feature Learning

The abundant spatial and contextual information provided by the advanced remote sensing technology has facilitated subsequent automatic interpretation of the optical remote sensing images (RSIs). In this paper, a novel and effective geospatial object detection framework is proposed by combining the weakly supervised learning (WSL) and high-level feature learning. First, deep Boltzmann machine is adopted to infer the spatial and structural information encoded in the low-level and middle-level features to effectively describe objects in optical RSIs. Then, a novel WSL approach is presented to object detection where the training sets require only binary labels indicating whether an image contains the target object or not. Based on the learnt high-level features, it jointly integrates saliency, intraclass compactness, and interclass separability in a Bayesian framework to initialize a set of training examples from weakly labeled images and start iterative learning of the object detector. A novel evaluation criterion is also developed to detect model drift and cease the iterative learning. Comprehensive experiments on three optical RSI data sets have demonstrated the efficacy of the proposed approach in benchmarking with several state-of-the-art supervised-learning-based object detection approaches.

Efficient and robust deep learning with Correntropy-induced loss function

Deep learning systems aim at using hierarchical models to learning high-level features from low-level features. The progress in deep learning is great in recent years. The robustness of the learning systems with deep architectures is however rarely studied and needs further investigation. In particular, the mean square error (MSE), a commonly used optimization cost function in deep learning, is rather sensitive to outliers (or impulsive noises). Robust methods are needed to improve the learning performance and immunize the harmful influences caused by outliers which are pervasive in real-world data. In this paper, we propose an efficient and robust deep learning model based on stacked auto-encoders and Correntropy-induced loss function (CLF), called CLF-based stacked auto-encoders (CSAE). CLF as a nonlinear measure of similarity is robust to outliers and can approximate different norms (from $$l_0$$l0 to $$l_2$$l2) of data. Essentially, CLF is an MSE in reproducing kernel Hilbert space. Different from conventional stacked auto-encoders, which use, in general, the MSE as the reconstruction loss and KL divergence as the sparsity penalty term, the reconstruction loss and sparsity penalty term in CSAE are both built with CLF. The fine-tuning procedure in CSAE is also based on CLF, which can further enhance the learning performance. The excellent and robust performance of the proposed model is confirmed by simulation experiments on MNIST benchmark dataset.