Explicit Feature Extraction Research Articles

A brain-inspired computational model for spatio-temporal information processing

Spatio-temporal information processing is fundamental in both brain functions and AI applications. Current strategies for spatio-temporal pattern recognition usually involve explicit feature extraction followed by feature aggregation, which requires a large amount of labeled data. In the present study, motivated by the subcortical visual pathway and early stages of the auditory pathway for motion and sound processing, we propose a novel brain-inspired computational model for generic spatio-temporal pattern recognition. The model consists of two modules, a reservoir module and a decision-making module. The former projects complex spatio-temporal patterns into spatially separated neural representations via its recurrent dynamics, the latter reads out neural representations via integrating information over time, and the two modules are linked together using known examples. Using synthetic data, we demonstrate that the model can extract the frequency and order information of temporal inputs. We apply the model to reproduce the looming pattern discrimination behavior as observed in experiments successfully. Furthermore, we apply the model to the gait recognition task, and demonstrate that our model accomplishes the recognition in an event-based manner and outperforms deep learning counterparts when training data is limited.

Combining CNN and LSTM for activity of daily living recognition with a 3D matrix skeleton representation

In socially assistive robotics, human activity recognition plays a central role when the adaptation of the robot behavior to the human one is required. In this paper, we present an activity recognition approach for activities of daily living based on deep learning and skeleton data. In the literature, ad hoc features extraction/selection algorithms with supervised classification methods have been deployed, reaching an excellent classification performance. Here, we propose a deep learning approach, combining CNN and LSTM, that exploits both the learning of spatial dependencies correlating the limbs in a skeleton 3D grid representation and the learning of temporal dependencies from instances with a periodic pattern that works on raw data and so without requiring an explicit feature extraction process. These models are proposed for real-time activity recognition, and they are tested on the CAD-60 dataset. Results show that the proposed model behaves better than an LSTM model thanks to the automatic features extraction of the limbs’ correlation. “New Person” results show that the CNN-LSTM model achieves 95.4% of precision and 94.4% of recall, while the “Have Seen” results are 96.1% of precision and 94.7% of recall.

An enhanced opinion retrieval approach via implicit feature identification

Recently, there has been an enormous increase in the number of reviews of popular products. Therefore, opinion analysis has become a tedious task for customers when making decisions. As a result, opinion retrieval systems have emerged as an effective tool to analyze and represent customers’ feelings toward offered services. Conventional opinion retrieval systems retrieve and rank products according to both relevance and the overall polarity scores of the opinions. However, customer reviews are usually more detailed, including multiple features with different polarities. Consequently, feature-based opinion retrieval is necessary to extract and analyze each feature separately. Customers’ opinions are usually written with a short and unclear structure and contain many implicit linguistic features that cannot be identified by retrieval systems. As a result, the recall results are negatively affected. Few studies have focused on implicit features, as most examined explicit features. Also, implicit features extraction is a challenging task in some languages like Arabic due to difficulties with morphology. This paper proposes an enhanced retrieval approach based on feature-based opinion mining to enhance retrieval performance. In addition to explicit feature extraction, a metaheuristic optimization method with several similarity measures is utilized to identify implicit features and measure its effect on the retrieval results. The experimental results on Arabic and English datasets revealed the effectiveness of the proposed approach, whereby more features were extracted compared to the explicit feature results. Furthermore, the ranking results were improved by identifying both implicit and explicit features compared to the results obtained by the conditional random field method and association rule mining.

Fault Detection and Classification in MMC-HVDC Systems Using Learning Methods.

In this paper, we explore learning methods to improve the performance of the open-circuit fault diagnosis of modular multilevel converters (MMCs). Two deep learning methods, namely, convolutional neural networks (CNN) and auto encoder based deep neural networks (AE-based DNN), as well as stand-alone SoftMax classifier are explored for the detection and classification of faults of MMC-based high voltage direct current converter (MMC-HVDC). Only AC-side three-phase current and the upper and lower bridges’ currents of the MMCs are used directly in our proposed approaches without any explicit feature extraction or feature subset selection. The two-terminal MMC-HVDC system is implemented in Power Systems Computer-Aided Design/Electromagnetic Transients including DC (PSCAD/EMTDC) to verify and compare our methods. The simulation results indicate CNN, AE-based DNN, and SoftMax classifier can detect and classify faults with high detection accuracy and classification accuracy. Compared with CNN and AE-based DNN, the SoftMax classifier performed better in detection and classification accuracy as well as testing speed. The detection accuracy of AE-based DNN is a little better than CNN, while CNN needs less training time than the AE-based DNN and SoftMax classifier.

FiFTy: Large-Scale File Fragment Type Identification Using Convolutional Neural Networks

We present FiFTy, a modern file-type identification tool for memory forensics and data carving. In contrast to previous approaches based on hand-crafted features, we design a compact neural network architecture, which uses a trainable embedding space. Our approach dispenses with the explicit feature extraction which has been a bottleneck in legacy systems. We evaluate the proposed method on a novel dataset with 75 filetypes - the most diverse and balanced dataset reported to date. FiFTy consistently outperforms all baselines in terms of speed, accuracy and individual misclassification rates. We achieved an average accuracy of 77.5% with processing speed of ≈38 sec/GB, which is better and more than an order of magnitude faster than the previous state-of-the-art tool - Sceadan (69% at 9 min/GB). Our tool and the corresponding dataset is open-source.

A Machine Learning Approach to Road Surface Anomaly Assessment Using Smartphone Sensors

Road surface quality is essential for improving driving experience and reducing traffic accidents. Traditional road condition monitoring systems are limited in their temporal (speed) and spatial (coverage) responses needed for maintaining overall road quality. Several alternative systems have been proposed that utilize sensors mounted on vehicles. In particular, with the ubiquitous use of smartphones for navigation, smartphone-based road condition assessment has emerged as a promising new approach. In this paper, we propose to analyze different multiclass supervised machine learning techniques to effectively classify road surface conditions using accelerometer, gyroscope and GPS data collected from smartphones. Our work focuses on classification of three main class labels- smooth road, potholes, and deep transverse cracks. We hypothesize that using features from all three axes of the sensors provides more accurate results as compared to using features from only one axis. We also investigate the performance of deep neural networks to classify road conditions with and without explicit manual feature extraction. Our results indicate that models trained with features from all axes of the smartphone sensors outperform models that use only one axis. We also observe that the use of neural networks provides a significantly improved data classification. The machine learning approach discussed here can be implemented on a larger scale to monitor roads for defects that present a safety risk to commuters as well as to provide maintenance information to relevant authorities.

PSDRNN: An Efficient and Effective HAR Scheme Based on Feature Extraction and Deep Learning

Recently, in this article, due to pervasive usability, smartphone-based human activity recognition (HAR) has witnessed significant development in smart health. Meanwhile, deep recurrent neural network (DRNN) shows a strong ability to automatically extract features from the time-series data, and therefore, DRNN-based HAR schemes have achieved more effective recognition (i.e., recognition accuracy) than those adopting the traditional machine learning. However, the efficiency in training and recognition (in terms of running time) has not fully taken into account, especially for resource-constraint smartphones. To solve the above issue, we propose the PSDRNN and tri-PSDRNN schemes that employ the explicit feature extraction before DRNN. Specifically, considering that the power spectral density (PSD) feature can capture the frequency characteristics and meanwhile retain the successive time characteristics of data gathered from smartphone accelerometer, PSD feature vectors are, respectively, extracted from linear accelerations and triaxle accelerations and explicitly used as the input to the following DRNN classification model. Thorough experiments based on a real dataset demonstrate that the PSDRNN can achieve the comparable effectiveness as the xyz -DRNN (the most accurate DRNN-based HAR scheme only using acceleration data), and the average recognition and training time were reduced by 56% and 80%, respectively. Moreover, tri-PSDRNN advantages over the xyz -DRNN in terms of recognition accuracy, and the running time is still lower than the xyz -DRNN. Besides, our proposed PSDRNN scheme achieved superiority in the recognition of complex transition activities.

Two-Dimensional DOA Estimation via Deep Ensemble Learning

To achieve fast and accurate two-dimensional (2D) direction of arrival (DOA) estimation, a novel deep ensemble learning method is presented in this paper. First, a convolutional neural network (CNN) is employed to learn a mapping between the spatial covariance matrix of the received signals from the antenna elements and the directions of arrival. To avoid any explicit feature extraction step, the real and imaginary parts of the spatial covariance matrix are fed to the CNN. The output layer of the CNN uses three neurons, two of them are the sine and cosine values of the azimuth angle that are used to uniquely determine the azimuth angle, and the third neuron is a normalized value for representing the elevation angle. Second, to improve the prediction performance, since that a single CNN with limited training data has difficulties learning the highly complex and nonlinear mapping from the received signal to the angle of arrival, an ensemble learning method is proposed. Five different CNN networks are trained independently with different training conditions. The prediction results of each individual CNN are calculated as an average to obtain the final estimated results of the azimuth and elevation angles. Simulation results show that the processing time of the proposed deep ensemble learning method is dramatically reduced. In terms of the accuracy, it outperforms the neural network-based 2D DOA estimation and achieves performance comparable to the MUSIC algorithm.

Sensing OFDM Signal: A Deep Learning Approach

Spectrum sensing plays a critical role in dynamic spectrum sharing, a promising technology to address the radio spectrum shortage. In particular, sensing of orthogonal frequency division multiplexing (OFDM) signals, a widely accepted multi-carrier transmission paradigm, has received paramount interest. Despite various efforts, noise uncertainty, timing delay and carrier frequency offset (CFO) still remain as challenging problems, significantly degrading the sensing performance. In this work, we develop two novel OFDM sensing frameworks utilizing the properties of deep learning networks. Specifically, we first propose a stacked autoencoder based spectrum sensing method (SAE-SS), in which a stacked autoencoder network is designed to extract the hidden features of OFDM signals for classifying the user’s activities. Compared to the conventional OFDM sensing methods, SAE-SS is significantly superior in the robustness to noise uncertainty, timing delay, and CFO. Moreover, SAE-SS requires neither any prior information of signals (e.g., signal structure, pilot tones, cyclic prefix) nor explicit feature extraction algorithms which however are essential for the conventional OFDM sensing methods. To further improve the sensing accuracy of SAE-SS, especially under low SNR conditions, we propose a stacked autoencoder based spectrum sensing method using time-frequency domain signals (SAE-TF). SAE-TF achieves higher sensing accuracy than SAE-SS using the features extracted from both time and frequency domains, at the cost of higher computational complexity. Through extensive simulation results, both SAE-SS and SAE-TF are shown to achieve notably higher sensing accuracy than that of state of the art approaches.

Robust fingerprint classification with Bayesian convolutional networks

Fingerprint classification is vital for reducing the search time and computational complexity of the fingerprint identification system. The robustness of classifier relies on the strength of extracted features and the ability to deal with low-quality fingerprints. The proficiency to learn accurate features from raw fingerprint images rather than explicit feature extraction makes deep convolutional neural networks (DCNNs) attractive for fingerprint classification. The DCNNs use softmax for quantifying model confidence of a class for an input fingerprint image to make a prediction. However, the softmax probabilities are not a true representation of model confidence and often misleading in feature space that may not be represented with the available training examples. The primary goal of this study is to improve the efficacy of the fingerprint classification by dealing with false positives by employing Bayesian model uncertainty. The efficacy of the proposed method is shown through experimentations on NIST special database 4 (NIST-4) and fingerprint verification competition 2002 database 1-A (FVC DB1-A) 2002 and 2004 datasets. Results show that 0.8-1.0% of accuracy is improved with model uncertainty over the conventional DCNN.

A Face Emotion Recognition Method Using Convolutional Neural Network and Image Edge Computing

To avoid the complex process of explicit feature extraction in traditional facial expression recognition, a face expression recognition method based on a convolutional neural network (CNN) and an image edge detection is proposed. Firstly, the facial expression image is normalized, and the edge of each layer of the image is extracted in the convolution process. The extracted edge information is superimposed on each feature image to preserve the edge structure information of the texture image. Then, the dimensionality reduction of the extracted implicit features is processed by the maximum pooling method. Finally, the expression of the test sample image is classified and recognized by using a Softmax classifier. To verify the robustness of this method for facial expression recognition under a complex background, a simulation experiment is designed by scientifically mixing the Fer-2013 facial expression database with the LFW data set. The experimental results show that the proposed algorithm can achieve an average recognition rate of 88.56% with fewer iterations, and the training speed on the training set is about 1.5 times faster than that on the contrast algorithm.

Human activity recognition from inertial sensor time-series using batch normalized deep LSTM recurrent networks.

In recent years machine learning methods for human activity recognition have been found very effective. These classify discriminative features generated from raw input sequences acquired from body-worn inertial sensors. However, it involves an explicit feature extraction stage from the raw data, and although human movements are encoded in a sequence of successive samples in time most state-of-the-art machine learning methods do not exploit the temporal correlations between input data samples. In this paper we present a Long-Short Term Memory (LSTM) deep recurrent neural network for the classification of six daily life activities from accelerometer and gyroscope data. Results show that our LSTM can processes featureless raw input signals, and achieves 92 % average accuracy in a multi-class-scenario. Further, we show that this accuracy can be achieved with almost four times fewer training epochs by using a batch normalization approach.

SpliceVec: Distributed feature representations for splice junction prediction

Identification of intron boundaries, called splice junctions, is an important part of delineating gene structure and functions. This also provides valuable insights into the role of alternative splicing in increasing functional diversity of genes. Identification of splice junctions through RNA-seq is by mapping short reads to the reference genome which is prone to errors due to random sequence matches. This encourages identification of splicing junctions through computational methods based on machine learning. Existing models are dependent on feature extraction and selection for capturing splicing signals lying in the vicinity of splice junctions. But such manually extracted features are not exhaustive. We introduce distributed feature representation, SpliceVec, to avoid explicit and biased feature extraction generally adopted for such tasks. SpliceVec is based on two widely used distributed representation models in natural language processing. Learned feature representation in form of SpliceVec is fed to multilayer perceptron for splice junction classification task. An intrinsic evaluation of SpliceVec indicates that it is able to group true and false sites distinctly. Our study on optimal context to be considered for feature extraction indicates inclusion of entire intronic sequence to be better than flanking upstream and downstream region around splice junctions. Further, SpliceVec is invariant to canonical and non-canonical splice junction detection. The proposed model is consistent in its performance even with reduced dataset and class-imbalanced dataset. SpliceVec is computationally efficient and can be trained with user-defined data as well.

Skipping the real world: Classification of PolSAR images without explicit feature extraction

The typical processing chain for pixel-wise classification from PolSAR images starts with an optional preprocessing step (e.g. speckle reduction), continues with extracting features projecting the complex-valued data into the real domain (e.g. by polarimetric decompositions) which are then used as input for a machine-learning based classifier, and ends in an optional postprocessing (e.g. label smoothing). The extracted features are usually hand-crafted as well as preselected and represent (a somewhat arbitrary) projection from the complex to the real domain in order to fit the requirements of standard machine-learning approaches such as Support Vector Machines or Artificial Neural Networks. This paper proposes to adapt the internal node tests of Random Forests to work directly on the complex-valued PolSAR data, which makes any explicit feature extraction obsolete. This approach leads to a classification framework with a significantly decreased computation time and memory footprint since no image features have to be computed and stored beforehand. The experimental results on one fully-polarimetric and one dual-polarimetric dataset show that, despite the simpler approach, accuracy can be maintained (decreased by only less than 2% for the fully-polarimetric dataset) or even improved (increased by roughly 9% for the dual-polarimetric dataset).

On the use of convolutional neural networks for robust classification of multiple fingerprint captures

Fingerprint classification is one of the most common approaches to accelerate the identification in large databases of fingerprints. Fingerprints are grouped into disjoint classes, so that an input fingerprint is compared only with those belonging to the predicted class, reducing the penetration rate of the search. The classification procedure usually starts by the extraction of features from the fingerprint image, frequently based on visual characteristics. In this work, we propose an approach to fingerprint classification using convolutional neural networks, which avoid the necessity of an explicit feature extraction process by incorporating the image processing within the training of the classifier. Furthermore, such an approach is able to predict a class even for low-quality fingerprints that are rejected by commonly used algorithms, such as FingerCode. The study gives special importance to the robustness of the classification for different impressions of the same fingerprint, aiming to minimize the penetration in the database. In our experiments, convolutional neural networks yielded better accuracy and penetration rate than state-of-the-art classifiers based on explicit feature extraction. The tested networks also improved on the runtime, as a result of the joint optimization of both feature extraction and classification.

Urdu Nastaliq recognition using convolutional–recursive deep learning

Recent developments in recognition of cursive scripts rely on implicit feature extraction methods that provide better results as compared to traditional hand-crafted feature extraction approaches. We present a hybrid approach based on explicit feature extraction by combining convolutional and recursive neural networks for feature learning and classification of cursive Urdu Nastaliq script. The first layer extracts low-level translational invariant features using Convolutional Neural Networks (CNN) which are then forwarded to Multi-dimensional Long Short-Term Memory Neural Networks (MDLSTM) for contextual feature extraction and learning. Experiments are carried out on the publicly available Urdu Printed Text-line Image (UPTI) dataset using the proposed hierarchical combination of CNN and MDLSTM. A recognition rate of up to 98.12% for 44-classes is achieved outperforming the state-of-the-art results on the UPTI dataset.

Facial Expression Recognition with Faster R-CNN

In order to avoid the complex explicit feature extraction process and the problem of low-level data operation involved in traditional facial expression recognition, we proposed a method of Faster R-CNN (Faster Regions with Convolutional Neural Network Features) for facial expression recognition in this paper. Firstly, the facial expression image is normalized and the implicit features are extracted by using the trainable convolution kernel. Then, the maximum pooling is used to reduce the dimensions of the extracted implicit features. After that, RPNs (Region Proposal Networks) is used to generate high-quality region proposals, which are used by Faster R-CNN for detection. Finally, the Softmax classifier and regression layer is used to classify the facial expressions and predict boundary box of the test sample, respectively. The dataset is provided by Chinese Linguistic Data Consortium (CLDC), which is composed of multimodal emotional audio and video data. Experimental results show the performance and the generalization ability of the Faster R-CNN for facial expression recognition. The value of the mAP is around 0.82.

Cytopathological image analysis using deep-learning networks in microfluidic microscopy

Cytopathologic testing is one of the most critical steps in the diagnosis of diseases, including cancer. However, the task is laborious and demands skill. Associated high cost and low throughput drew considerable interest in automating the testing process. Several neural network architectures were designed to provide human expertise to machines. In this paper, we explore and propose the feasibility of using deep-learning networks for cytopathologic analysis by performing the classification of three important unlabeled, unstained leukemia cell lines (K562, MOLT, and HL60). The cell images used in the classification are captured using a low-cost, high-throughput cell imaging technique: microfluidics-based imaging flow cytometry. We demonstrate that without any conventional fine segmentation followed by explicit feature extraction, the proposed deep-learning algorithms effectively classify the coarsely localized cell lines. We show that the designed deep belief network as well as the deeply pretrained convolutional neural network outperform the conventionally used decision systems and are important in the medical domain, where the availability of labeled data is limited for training. We hope that our work enables the development of a clinically significant high-throughput microfluidic microscopy-based tool for disease screening/triaging, especially in resource-limited settings.

TASK-DEPENDENT BAND-SELECTION OF HYPERSPECTRAL IMAGES BY PROJECTION-BASED RANDOM FORESTS

Abstract. The automatic classification of land cover types from hyperspectral images is a challenging problem due to (among others) the large amount of spectral bands and their high spatial and spectral correlation. The extraction of meaningful features, that enables a subsequent classifier to distinguish between different land cover classes, is often limited to a subset of all available data dimensions which is found by band selection techniques or other methods of dimensionality reduction. This work applies Projection-Based Random Forests to hyperspectral images, which not only overcome the need of an explicit feature extraction, but also provide mechanisms to automatically select spectral bands that contain original (i.e. non-redundant) as well as highly meaningful information for the given classification task. The proposed method is applied to four challenging hyperspectral datasets and it is shown that the effective number of spectral bands can be considerably limited without loosing too much of classification performance, e.g. a loss of 1 % accuracy if roughly 13 % of all available bands are used.

TASK-DEPENDENT BAND-SELECTION OF HYPERSPECTRAL IMAGES BY PROJECTION-BASED RANDOM FORESTS

The automatic classification of land cover types from hyperspectral images is a challenging problem due to (among others) the large amount of spectral bands and their high spatial and spectral correlation. The extraction of meaningful features, that enables a subsequent classifier to distinguish between different land cover classes, is often limited to a subset of all available data dimensions which is found by band selection techniques or other methods of dimensionality reduction. This work applies Projection-Based Random Forests to hyperspectral images, which not only overcome the need of an explicit feature extraction, but also provide mechanisms to automatically select spectral bands that contain original (i.e. non-redundant) as well as highly meaningful information for the given classification task. The proposed method is applied to four challenging hyperspectral datasets and it is shown that the effective number of spectral bands can be considerably limited without loosing too much of classification performance, e.g. a loss of 1 % accuracy if roughly 13 % of all available bands are used.