CNN Feature Extractor Research Articles

The Research of Convolutional Neural Network Based on Integrated Classification in Question Classification

As a new generation of search engine, automatic question answering system (QAS) is becoming more and more important and has become one of the hotspots of computer application research and natural language processing (NLP). However, as an indispensable part of the QAS, the role of question classification is an understood thing in the system. In view of this, to further make the performance of question classification much better, both the feature extraction and the classification model were explored. On the study of existing CNN research, an improved CNN model based on Bagging integrated classification (“W2V + B-CNN” for short) is proposed and applied to question classification. Firstly, we combine the characteristics of short texts, use the Word2Vec tool to map the features of the words to a certain dimension, and organize the question sentences into the form of a two-dimensional matrix similar to the image. Then, the trained word vectors are used as the input of the CNN for feature extraction. Finally, the Bagging integrated classification algorithm is used to replace the Softmax classification of the traditional CNN for classification. In other words, the good of W2V + B-CNN model is that it can make use of the advantages of CNN and Bagging integrated classification at the same time. Overall, the new model can not only use the powerful feature extraction capabilities of CNN to extract the potential features of natural language questions but also use the good data classification capabilities of the integrated classification algorithm for feature classification at the same time, which can help improve the accuracy of the W2V + B-CNN in the application of question classification. The comparative experiment results prove that the effect of the W2V + B-CNN is significantly better than that of the CNN and other classification algorithms in question classification.

Classification of Giemsa staining chromosome using input-aware deep convolutional neural network with integrated uncertainty estimates

A fully automated chromosome analysis system can substitute cytogenetic experts for the task of chromosome karyotype analysis, which in turn can substantially increase the efficiency of disease diagnosis. However, the construction of such a system is most crucially restricted by the accuracy of chromosome classification, during karyotype analysis. To facilitate the construction of an automatic chromosome analysis system, an input-aware and probabilistic prediction convolutional neural network (IAPP-CNN) is presented in this paper for high accuracy of chromosome classification. The approach follows three stages and consists of one input-aware module, one feature extractor module and one probabilistic prediction module. In the first stage, the input-aware module develops raw images automatically into the global-scale image, the object-scale image and the part-scale image, by introducing an attention mechanism. In the second stage, the three scale images are input into the feature extraction module through three branches, then the respective feature operators are obtained via their independent CNN feature extractors. In the third stage, the probabilistic prediction module uses three dynamic probabilistic parameters to estimate the prediction of each CNN branch separately, and then combined the three CNN votes for the final decision. The feature expression ability of the key feature was improved and the network was enabled to focus on the recognizable regions in the image. Evaluation results from a large dataset of healthy patients showed that the proposed IAPP-CNN achieved the highest accuracy of 99.2% for the chromosome classification task, surpassing the performance of a competitive baseline created by state-of-the-art methods.

Vehicle re-identification method based on global and local feature fusion

Vehicle re-identification is still an issue worth discussing due to change of view, light and angle changes. At present, the key of vehicle reidentification research is to solve the problems that the difference between the same class is big and the difference between different classes is small. For the study, a vehicle re-identification method ground on view classification is proposed.The vehicle images are divided into four views, namely, front, rear, top and side, and local features are extracted by the segmented view information. Using the CNN feature extractor, the global feature representation with car ID attribute is learned. We have done experiments with the VERI-776 data sets. The mean average accuracy of 78.13% is obtained, which proves the effectiveness of the method.

Classification of Remote Sensing Image Scenes Using Double Feature Extraction Hybrid Deep Learning Approach

Over the last decade, remote sensing technology has advanced dramatically, resulting in significant improvements on image quality, data volume, and application usage. These images have essential applications since they can help with quick and easy interpretation. Many standard detection algorithms fail to accurately categorize a scene from a remote sensing image recorded from the earth. A method that uses bilinear convolution neural networks to produce a lessweighted set of models those results in better visual recognition in remote sensing images using fine-grained techniques. This proposed hybrid method is utilized to extract scene feature information in two times from remote sensing images for improved recognition. In layman's terms, these features are defined as raw, and only have a single defined frame, so they will allow basic recognition from remote sensing images. This research work has proposed a double feature extraction hybrid deep learning approach to classify remotely sensed image scenes based on feature abstraction techniques. Also, the proposed algorithm is applied to feature values in order to convert them to feature vectors that have pure black and white values after many product operations. The next stage is pooling and normalization, which occurs after the CNN feature extraction process has changed. This research work has developed a novel hybrid framework method that has a better level of accuracy and recognition rate than any prior model.

Vision-based method of automatically detecting construction video highlights by integrating machine tracking and CNN feature extraction

Automatic analysis of construction video footage is beneficial for project management tasks such as productivity analysis and safety control. However, construction videos are usually long in duration and only contain limited useful information to engineers, while the storage of video data from construction projects is challenging. To obtain and store useful video footage systematically and concisely, this research proposes a vision-based method to automatically generate video highlights from construction videos. The proposed approach is validated through two case studies: a gate scenario and an earthmoving scenario. In experiments, the proposed method has achieved 89.2% on precision and 93.3% on recall, which outperforms the feature-based method by 12.7% and 17.2% on precision and recall, respectively. Meanwhile, the proposed method reduces the required digital storage space by 94.6%. The proposed approach offers potential benefits to construction management in terms of significantly reducing video storage space and efficiently indexing construction video footage.

Analysis of Sanda action based on CCNN model of IBS distance measurement

IBS is a similarity measurement method of symbol sequences based on order statistics, and CCNN is a method based on CNN feature extraction and IBS distance measurement, which can construct crossover probability graph through image crossover comparison. IBS distance and Noribs distance are used to define the similarity between crossover probability graphs. For Sanda movement, it is difficult to carry out big data analysis with a small number of samples and the movement style of athletes will change greatly. In this paper, a CCNN model based on IBS distance measurement is proposed to analyze the action of Sanda. Firstly, the motion pictures obtained were de-noised and then simplified into a sequence diagram of human skeleton nodes by Delaunay triangulation. Then, 80% of the samples were input into the CCNN network for training and 20% for algorithm verification. The experiment found that in the case of a small number of samples, the accuracy of the simplified image analysis was 10% higher than that of the unsimplified image. In addition, compared with the accuracy of K-distance classification, Bayesian network classification and decision tree classification, it is found that the model can improve the accuracy by 2%-11.75% due to the small number of samples, and has higher accuracy and applicability.

Computer-vision classification of corn seed varieties using deep convolutional neural network

Automated classification of seed varieties is of paramount importance for seed producers to maintain the purity of a variety and crop yield. Traditional approaches based on computer vision and simple feature extraction could not guarantee high accuracy classification. This paper presents a new approach using a deep convolutional neural network (CNN) as a generic feature extractor. The extracted features were classified with artificial neural network (ANN), cubic support vector machine (SVM), quadratic SVM, weighted k-nearest-neighbor (kNN), boosted tree, bagged tree, and linear discriminant analysis (LDA). Models trained with CNN-extracted features demonstrated better classification accuracy of corn seed varieties than models based on only simple features. The CNN-ANN classifier showed the best performance, classifying 2250 test instances in 26.8 s with classification accuracy 98.1%, precision 98.2%, recall 98.1%, and F1-score 98.1%. This study demonstrates that the CNN-ANN classifier is an efficient tool for the intelligent classification of different corn seed varieties.

Device-free single-user activity recognition using diversified deep ensemble learning

WiFi-based human activity recognition (HAR) aims to recognize human activities in an off-the-shelf manner that only relies on the commercial Wi-Fi devices already installed in environments. The recent trend in HAR research is to train classifiers on top of statistical or deep neural features extracted from channel state information (CSI) data. Unfortunately, existing methods only take into account the temporal-correlation within each CSI subcarrier, while ignoring the spatial-correlation between different subcarriers. This issue has not been fully exploited yet, resulting a limited performance. To address this issue, we propose WiAReS, a WiFi-based device-free activity recognition system that takes both temporal-correlation and spatial-correlation into account. WiAReS embarks on diversified deep ensemble methods 2̌for single-user activity recognition where one user performs a single activity at a given time. More specifically, it adopts convolutional neural network (CNN) to automatically extract features from CSI measurements with the preservation of the locality of both spatial patterns and temporal patterns. To further improve recognition accuracy upon CNN-extracted features, we propose a novel ensemble architecture that fuses a multiple layer perception (MLP), a random forest (RF) and a support vector machine (SVM). Our system obtains the CSI data in PHY layer of off-the-shelf WiFi devices by installing Atheros-CSI-Tool on AR9590 based WiFi network interface cards (NICs). Comprehensive experiments have been conducted in three real environments with environmental variation to evaluate the performance of the proposed WiAReS. The experimental results demonstrate that the proposed WiARes system significantly outperforms existing methods.

A Novel Multi-Stage Training Approach for Human Activity Recognition From Multimodal Wearable Sensor Data Using Deep Neural Network

Deep neural network is an effective choice to automatically recognize human actions utilizing data from various wearable sensors. These networks automate the process of feature extraction relying completely on data. However, various noises in time series data with complex inter-modal relationships among sensors make this process more complicated. In this article, we have proposed a novel multi-stage training approach that increases diversity in this feature extraction process to make accurate recognition of actions by combining varieties of features extracted from diverse perspectives. Initially, instead of using single type of transformation, numerous transformations are employed on time series data to obtain variegated representations of the features encoded in raw data. An efficient deep CNN architecture is proposed that can be individually trained to extract features from different transformed spaces. Later, these CNN feature extractors are merged into an optimal architecture finely tuned for optimizing diversified extracted features through a combined training stage or multiple sequential training stages. This approach offers the opportunity to explore the encoded features in raw sensor data utilizing multifarious observation windows with immense scope for efficient selection of features for final convergence. Extensive experimentations have been carried out in three publicly available datasets that provide outstanding performance consistently with average five-fold cross-validation accuracy of 99.29% on UCI HAR database, 99.02% on USC HAR database, and 97.21% on SKODA database outperforming other state-of-the-art approaches.

An Automatic Detector for Fungal Spores in Microscopic Images Based on Deep Learning

HighlightsA fungal spore dataset with microscopic images was established.A fungal spore detector, FSNet, based on deep learning was proposed to automatically recognize and count fungal spores in microscopic images.A framework based on a CNN for feature extraction was designed to enrich feature maps.A loss was formulated to decrease the number of missed detections due to clustered spores.Abstract. The most common cause of spoiled grain in storage is the growth of fungi on the grain. In China, the level of fungal infection is indicated by the quantity and the categories of fungal spores in grain samples. In most Chinese grain depots, the recognition and counting of fungal spores in grain samples are conducted with microscopes, which are mainly operated manually by professionals. This method is time-consuming, laborious and error-prone. In this article, a fungal spore detector, named FSNet, was proposed to automatically recognize and count fungal spores in microscopic images. To enrich the features in feature maps, a multi-layer fusion structure based on convolutional neural network for feature extraction was designed using deep, intermediate and shallow layers. Moreover, the detection performance of FSNet was further improved by a designed loss for clustered spores, an anchor optimization and region sampling strategy. Our method achieved 91.6% mean average precision [intersection-over-union (IoU) > 0.6]. The experimental results indicated that our method was capable of recognizing and counting fungal spores in microscopic images, and it is very effective for the early detection of fungal infection. Keywords: Fungal spore detection, Deep learning, Microscopic images.

A Convolutional Neural Network Combined With Prototype Learning Framework for Brain Functional Network Classification of Autism Spectrum Disorder.

The application of deep learning methods in brain disease diagnosis is becoming a new research hotspot. This study constructed brain functional networks based on the functional magnetic resonance imaging (fMRI) data, and proposed a novel convolutional neural network combined with a prototype learning (CNNPL) framework to classify brain functional networks for the diagnosis of autism spectrum disorder (ASD). At the bottom of CNNPL, traditional CNN was employed as the basic feature extractor, while at the top of CNNPL multiple prototypes were automatically learnt on the features to represent different categories. A generalized prototype loss based on distance cross-entropy was proposed to jointly learn the parameters of the CNN feature extractor and the prototypes. The classification was implemented with prototype matching. A transfer learning strategy was introduced to our CNNPL for weight initialization in the subsequent fine-tuning phase to promote model training. We conducted systematic experiments on the aggregate multi-sites ASD dataset. Experimental results revealed that our model outperforms the current state-of-the-art methods in ASD classification and can reliably learn inter-site biomarkers, indicating the robustness of our model on large-scale dataset with inter-site variability. Furthermore, our model demonstrated robust learning capability for high-level organization of brain functionality. Our study also identified important brain regions as biomarkers associated with ASD classification. Together, our proposed model provides a promising solution for learning and classifying brain functional networks, and thus contributes to the biomarker extraction and imaging diagnosis of ASD.

Domain Adaptive Ensemble Learning.

The problem of generalizing deep neural networks from multiple source domains to a target one is studied under two settings: When unlabeled target data is available, it is a multi-source unsupervised domain adaptation (UDA) problem, otherwise a domain generalization (DG) problem. We propose a unified framework termed domain adaptive ensemble learning (DAEL) to address both problems. A DAEL model is composed of a CNN feature extractor shared across domains and multiple classifier heads each trained to specialize in a particular source domain. Each such classifier is an expert to its own domain but a non-expert to others. DAEL aims to learn these experts collaboratively so that when forming an ensemble, they can leverage complementary information from each other to be more effective for an unseen target domain. To this end, each source domain is used in turn as a pseudo-target-domain with its own expert providing supervisory signal to the ensemble of non-experts learned from the other sources. To deal with unlabeled target data under the UDA setting where real expert does not exist, DAEL uses pseudo labels to supervise the ensemble learning. Extensive experiments on three multi-source UDA datasets and two DG datasets show that DAEL improves the state of the art on both problems, often by significant margins.

Speech Enhancement With Phase Sensitive Mask Estimation Using a Novel Hybrid Neural Network

A natural choice to model strong temporal dynamics of speech is the recurrent neural network (RNN) since it can exploit the sequential information from consecutive acoustic frames and generalizes the model well to unseen speakers. Besides, the convolutional neural network (CNN) can automatically extract sophisticated speech features that can maximize the performance of a model. In this paper, we propose a hybrid neural network model integrating a new low-complexity fully-convolutional CNN and a long short-term memory (LSTM) network, a variation of RNN, to estimate a phase-sensitive mask for speech enhancement. The model is designed to take full advantages of the temporal dependencies and spectral correlations present in the input speech signal while keeping the model complexity low. Also, an attention technique is embedded to recalibrate the useful CNN-extracted features adaptively. Furthermore, a grouping strategy is employed to reduce the LSTM complexity while keeping the performance almost unchanged. Through extensive comparative experiments, we show that the proposed model significantly outperforms some known neural network-based speech enhancement methods in the presence of highly non-stationary noises, while it exhibits a relatively small number of model parameters compared to some commonly employed DNN-based methods.

Fault diagnosis for building chillers based on data self-production and deep convolutional neural network

Fault diagnosis is of great importance in the field of building chiller application, which can reduce energy waste and maintenance cost, ensure stable operation and better service. To improve the diagnosis performance, this study presents a novel fault diagnosis method based on data self-production and deep convolutional neural network (SP-CNN), in which a simple, effective data augmentation technology is proposed to well utilize the excellent feature extraction and pattern recognition capability of deep CNN to diagnose typical chiller faults. Instead of transforming the fault data into a picture for the model to process into digital type, the SP-CNN model directly transforms the data into a digitized image to avoid possible errors caused by multi-transformation. The proposed data self-production technique can effectively augment the diagnosis information and help improve the performance. The results show that the proposed method is effective and the diagnosis accuracy of the SP-CNN with SP scale of 16 reaches 97.03%, higher than that of CNN without data self-production by about 1.63%. Due to the possible occurrence of over-fitting or under-fitting, it does not necessarily mean that the larger the SP scale, the better the performance. The proposed SP-CNN model also shows a better performance than back propagation (BP) network and another CNN method in terms of the overall diagnosis accuracy and the individual accuracy for each fault. It is also found that unlike the application in computer vision, for a fault diagnosis problem, the feature sequence has a great influence on the model performance. The accuracy is further improved to 98.02% by re-arrangement of features.

Predicting defibrillation success in out-of-hospital cardiac arrested patients: Moving beyond feature design

ObjectiveOptimizing timing of defibrillation by evaluating the likelihood of a successful outcome could significantly enhance resuscitation. Previous studies employed conventional machine learning approaches and hand-crafted features to address this issue, but none have achieved superior performance to be widely accepted. This study proposes a novel approach in which predictive features are automatically learned. MethodsA raw 4s VF episode immediately prior to first defibrillation shock was feed to a 3-stage CNN feature extractor. Each stage was composed of 4 components: convolution, rectified linear unit activation, dropout and max-pooling. At the end of feature extractor, the feature map was flattened and connected to a fully connected multi-layer perceptron for classification. For model evaluation, a 10 fold cross-validation was employed. To balance classes, SMOTE oversampling method has been applied to minority class. ResultsThe obtained results show that the proposed model is highly accurate in predicting defibrillation outcome (Acc = 93.6 %). Since recommendations on classifiers suggest at least 50 % specificity and 95 % sensitivity as safe and useful predictors for defibrillation decision, the reported sensitivity of 98.8 % and specificity of 88.2 %, with the analysis speed of 3 ms/input signal, indicate that the proposed model possesses a good prospective to be implemented in automated external defibrillators. ConclusionsThe learned features demonstrate superiority over hand-crafted ones when performed on the same dataset. This approach benefits from being fully automatic by fusing feature extraction, selection and classification into a single learning model. It provides a superior strategy that can be used as a tool to guide treatment of OHCA patients in bringing optimal decision of precedence treatment. Furthermore, for encouraging replicability, the dataset has been made publicly available to the research community.

Multi-label ocular disease classification with a dense correlation deep neural network

Early diagnosis and timely treatment of ocular diseases are vital to prevent irreversible vision loss. Color fundus photography is an effective and economic tool for fundus screening. Since few symptoms are visible in the early disease stages, automatic and robust diagnosing algorithms according to color fundus photographs are in urgent need. Existing studies concentrate on image-level diagnoses treating the eyes independently without utilizing the useful correlation information between the left and right eyes. Besides, they commonly target only one or several ocular disease categories at a time. Considering the importance of both patient-level diagnosis correlating bilateral eyes and multi-label disease classification, we propose a patient-level multi-label ocular disease classification model based on convolutional neural networks. Specifically, a dense correlation network (DCNet) is designed to tackle the problem. DCNet consists of three major modules, a backbone CNN for feature extraction, a spatial correlation module for feature correlation, and a classifier for classification score generation. The backbone CNN extracts two sets of features from the left and right color fundus photographs, respectively. Subsequently, the spatial correlation module captures the pixel-wise correlations between the two feature sets. Then, the processed features are fused to get a patient-level representation. The final disease classification is conducted with the patient-level representation. Adopting a multi-label soft margin loss, the effectiveness of the proposed model is evaluated on a publicly available dataset, and the classification performance is improved with a large margin compared with multiple baseline methods.

Automated shopping system using computer vision

The shopping experiment made by amazon go in USA is one of the most interesting applications of computer vision recently. They allow you to shop and automatically charge your virtual card for whatever goods you purchased using cameras and wireless systems, so no checkouts or waiting lines are required. However, amazon didn’t reveal yet the details of how their system components are implemented. In this paper, we introduce a complete system for computer vision based automated shopping. The proposed system contains barcode scanning of objects, data registration, image capturing for offline training stage, motion (change) detection, CNN and SVM for object classification and charging/discharging customers. Our system can be integrated with the wireless data transmission to do the whole shopping process. First, the proposed method extracts the objects’ barcodes to register their details, and take sample images of objects for classifier training. We employ a pre-trained CNN (i.e. ResNet50) for feature extraction and a multi-class SVM for training. After training our classifier, we have a real-time operation stage (i.e. test stage). We assume that a camera is embedded above products on each shelf to capture videos of the products. We employ a change detector to understand any added or removed items. If the item is removed from or added to the shelve, the moving object is input to CNN feature extractor, and then SVM classifier for identification and pricing. Results show that the proposed system is fast and effective.

SCANet

Continuous authentication monitors the security of a system throughout the login session on mobile devices. In this article, we present SCANet, a two-stream convolutional neural network--based continuous authentication system that leverages the accelerometer and gyroscope on smartphones to monitor users’ behavioral patterns. We are among the first to use two streams of data—frequency domain data and temporal difference domain data—from the two sensors as the inputs of the convolutional neural network (CNN). SCANet utilizes the two-stream CNN to learn and extract representative features and then performs the principal component analysis to select the top 25 features with high discriminability. With the CNN-extracted features, SCANet exploits the one-class support vector machine to train the classifier in the enrollment phase. Based on the trained CNN and classifier, SCANet identifies the current user as a legitimate user or an impostor in the continuous authentication phase. We evaluate the effectiveness of the two-stream CNN and the performance of SCANet on our dataset and BrainRun dataset, and the experimental results demonstrate that CNN achieves 90.04% accuracy, and SCANet reaches an average of 5.14% equal error rate on two datasets and takes approximately 3 s for user authentication.

Echo-ID: Smart User Identification Leveraging Inaudible Sound Signals

In this article, we present a novel user identification mechanism for smart spaces called Echo-ID (referred to as E-ID). Our solution relies on inaudible sound signals for capturing the user's behavioral tapping/typing characteristics while s/he types the PIN on a PIN-PAD, and uses them to identify the corresponding user from a set of ${N}$ enrolled inhabitants. E-ID proposes an all-inclusive pipeline that generates and transmits appropriate sound signals, and extracts a user-specific imprint from the recorded signals (E-Sign). For accurate identification of the corresponding user given an E-Sign sample, E-ID makes use of deep-learning (i.e., CNN for feature extraction) and SVM classifier (for making the identification decision). We implemented a proof of the concept of E-ID by leveraging the commodity speaker and microphone. Our evaluations revealed that E-ID can identify the users with an average accuracy of 93% to 78% from an enrolled group of 2-5 subjects, respectively.

Video Summarization Using Feature Vector Clustering

With ever-growing utilization of online and offline videos and increasing video content, Video Summarization serves as the best aid for video browsing. It involves domain explicit semantic comprehension of a video and understanding of user expectations. Generally, video summarization systems include extracting video features, analyzing the visual variations and selecting video frames. Over the years, various methodologies have been developed for the same. Different supervised and unsupervised algorithms have been established and these models have been trained on various factors or various rewards. The challenges these methods face stand as a motivation for the approach this paper discusses. Like in many cases, summary frames may be repeated if some scene or concept appears more than once. This paper presents a novel approach based on clustering of video frames based on their feature vectors. The clustering takes into consideration the semantic factor of video frames. Each concept cluster gives a representative frame which then forms the summary set, here concept cluster refers to the independent entity present in a video which can be easily distinguished by another concept or entity. This entity can be a scene of a mountain or different persons. It also aims to increase system performance by removing the redundancy. The system is developed using a CNN for feature extraction and a clustering algorithm that takes into consideration the similarity factor between these vectors. The model is evaluated on the measures Precision and Recall and tested on the VSUMM dataset. The results outperform some of the established methodologies and serve the summarization purpose.