Convolutional Neural Network Feature Extractor Research Articles

Hybrid Eye-Tracking on a Smartphone with CNN Feature Extraction and an Infrared 3D Model.

This paper describes a low-cost, robust, and accurate remote eye-tracking system that uses an industrial prototype smartphone with integrated infrared illumination and camera. Numerous studies have demonstrated the beneficial use of eye-tracking in domains such as neurological and neuropsychiatric testing, advertising evaluation, pilot training, and automotive safety. Remote eye-tracking on a smartphone could enable the significant growth in the deployment of applications in these domains. Our system uses a 3D gaze-estimation model that enables accurate point-of-gaze (PoG) estimation with free head and device motion. To accurately determine the input eye features (pupil center and corneal reflections), the system uses Convolutional Neural Networks (CNNs) together with a novel center-of-mass output layer. The use of CNNs improves the system’s robustness to the significant variability in the appearance of eye-images found in handheld eye trackers. The system was tested with 8 subjects with the device free to move in their hands and produced a gaze bias of 0.72°. Our hybrid approach that uses artificial illumination, a 3D gaze-estimation model, and a CNN feature extractor achieved an accuracy that is significantly (400%) better than current eye-tracking systems on smartphones that use natural illumination and machine-learning techniques to estimate the PoG.

Dynamic prediction for attitude and position in shield tunneling: A deep learning method

Abstract Quality management in shield tunneling projects is a challenging problem. To improve the quality of segment erection, the shield driver must constantly adjust the shield's attitude and position to reduce a snakelike motion for fitting the design tunnel axis based on the driver's experience, which is untimely and less reliable during excavation. Considering the disadvantage of this control method, this paper presents a predictive framework for the attitude and position in shield tunneling by applying a hybrid deep learning model. This framework contains a wavelet transform noise filter, convolutional neural network feature extractor, and long short-term memory predictor for determining the attitude and position of the shield machine in the future. The prediction framework is tested with the collected data of Mixshield operated in the river-crossing tunnel project of Yangtze Sanyang Road, Wuhan, China. Six variables characterizing the shield attitude and position are selected to validate the feasibility and performance of our method. Results reveal that the proposed model outperforms the other three similar models in predictive accuracy and provides decision support for adjusting the attitude and position in shield tunneling.

An approach for medical event detection in Chinese clinical notes of electronic health records

BackgroundMedical event detection in narrative clinical notes of electronic health records (EHRs) is a task designed for reading text and extracting information. Most of the previous work of medical event detection treats the task as extracting concepts at word granularity, which omits the overall structural information of the clinical notes. In this work, we treat each clinical note as a sequence of short sentences and propose an end-to-end deep neural network framework.MethodsWe redefined the task as a sequence labelling task at short sentence granularity, and proposed a novel tag system correspondingly. The dataset were derived from a third-level grade-A hospital, consisting of 2000 annotated clinical notes according to our proposed tag system. The proposed end-to-end deep neural network framework consists of a feature extractor and a sequence labeller, and we explored different implementations respectively. We additionally proposed a smoothed Viterbi decoder as sequence labeller without additional parameter training, which can be a good alternative to conditional random field (CRF) when computing resources are limited.ResultsOur sequence labelling models were compared to four baselines which treat the task as text classification of short sentences. Experimental results showed that our approach significantly outperforms the baselines. The best result was obtained by using the convolutional neural networks (CNNs) feature extractor and the sequential CRF sequence labeller, achieving an accuracy of 92.6%. Our proposed smoothed Viterbi decoder achieved a comparable accuracy of 90.07% with reduced training parameters, and brought more balanced performance across all categories, which means better generalization ability.ConclusionsEvaluated on our annotated dataset, the comparison results demonstrated the effectiveness of our approach for medical event detection in Chinese clinical notes of EHRs. The best feature extractor is the CNNs feature extractor, and the best sequence labeller is the sequential CRF decoder. And it was empirically verified that our proposed smoothed Viterbi decoder could bring better generalization ability while achieving comparable performance to the sequential CRF decoder.

Automatic Audio Chord Recognition With MIDI-Trained Deep Feature and BLSTM-CRF Sequence Decoding Model

With the advances of machine learning technologies, data-driven feature extraction and sequence modeling approaches are being widely explored for automatic chord recognition tasks. Currently, there is a bottleneck in the amount of enough annotated data for training robust acoustic models, as hand-annotating time-synchronized chord labels requires professional musical skills and considerable labor. To cope with this limitation, in this paper, we propose a convolutional neural network (CNN) based deep feature extractor, which is trained on a large set of time, synchronized musical instrument digital interface audio data pairs and can robustly estimate pitch class activations of real-world music audio recordings. The CNN feature extractor plus a bidirectional long short-term memory conditional random field decoding model forms the proposed hybrid system for automatic chord recognition. Experiments show that the proposed model is compatible for both regular major/minor triad chord classification and larger vocabulary chord recognition, and outperforms other state-of-the-art chord recognition systems.

Theoretical learning guarantees applied to acoustic modeling

In low-resource scenarios, for example, small datasets or a lack in computational resources available, state-of-the-art deep learning methods for speech recognition have been known to fail. It is possible to achieve more robust models if care is taken to ensure the learning guarantees provided by the statistical learning theory. This work presents a shallow and hybrid approach using a convolutional neural network feature extractor fed into a hierarchical tree of support vector machines for classification. Here, we show that gross errors present even in state-of-the-art systems can be avoided and that an accurate acoustic model can be built in a hierarchical fashion. Furthermore, we present proof that our algorithm does adhere to the learning guarantees provided by the statistical learning theory. The acoustic model produced in this work outperforms traditional hidden Markov models, and the hierarchical support vector machine tree outperforms a multi-class multilayer perceptron classifier using the same features. More importantly, we isolate the performance of the acoustic model and provide results on both the frame and phoneme level, considering the true robustness of the model. We show that even with a small amount of data, accurate and robust recognition rates can be obtained.

An Analysis of CNN Feature Extractor Based on KL Divergence

Convolutional neural networks (CNNs) have brought in exciting progress in many computer vision tasks. But the feature extraction process executed by CNN still keeps a black box to us, and we have not fully understood its working mechanism. In this paper, we propose a method to evaluate CNN features and further to analyze the CNN feature extractor, which is inspired by Bayes Classification Theory and KL divergence (KLD). Experiments have shown that CNN can promote feature discrimativeness by gradually increasing the intra-class KLD, and meanwhile promote feature robustness by gradually decreasing the inner-class KLD during training. Experiments also reveal that, with the deepening of network, CNN can gradually improve separability information density in feature space and encode much more separability information into the final feature vectors.