CNN-based Classifier Research Articles

0792 Mandibular Movement Monitoring with Artificial Intelligence Analysis for the Diagnosis of Sleep Bruxism

Abstract Introduction Sleep bruxism (BXM) is the result of rhythmic muscular masticatory activity (RMMA) and can be captured by masseters surface electromyography (sEMG). Despite the multiple adverse negative consequences of BXM, a simple reliable home diagnostic device is currently unavailable, with in laboratory audio-video polysomnography (type I PSG) remaining the gold standard diagnostic tool. Mandibular movements (MM) recordings during sleep can readily identify RMMA, are simple to set up and can be easily repeated from night to night. Here, we aimed to identify stereotypical MM in patients with BXM, and to develop RMMA automatic detection and BXM diagnosis using an artificial intelligence-based approach. Methods MM were recorded by a dedicated sensor (Sunrise, Namur, Belgium) in 12 patients with BXM during type I PSG. The Sunrise system consists of a coin-sized hardware that is comfortably placed on the subject’s chin. Its embedded inertial measurement unit communicates via Bluetooth with a smartphone and automatically transfers MM signals to a cloud-based infrastructure at the end of the night. Data processing and analysis are then performed in Python programming language. A time series cluster analysis was applied to sequences of masseters sEMG and MM signals during BXM episodes (n=300) and during spontaneous micro-arousals (n=300). Then, a convolutional neuronal network (CNN) was developed to identify BXM and distinguish it from spontaneous micro-arousals while exclusively relying on MM signal. Results Based on the cluster analysis, BXM periods were characterized by a specific pattern of MM signals (higher frequency and amplitude), which was closely associated with the sEMG signals but clearly differed from the MM signal patterns during micro-arousals. CNN-based classifier distinguished the BXM events from other RMMAs during micro-arousals and respiratory efforts with an overall accuracy of 91%. Conclusion Sleep bruxism can be automatically identified, quantified, and characterized with mandibular movements analysis supported by artificial intelligence technology. Support This work was supported by the French National Research Agency (ANR-12-TECS-0010), in the framework of the “Investissements d’avenir” program (ANR-15-IDEX-02). https://life.univ-grenoble-alpes.fr.

GC-Net: Gridding and Clustering for Traffic Object Detection With Roadside LiDAR

The emerging intelligent transportation systems puts higher demands on the collection and analysis of the traffic data. LiDAR can provide high-precision point clouds of traffic objects, making it a promising choice for the surveillance device. This article focuses on the traffic object detection with roadside LiDAR: estimating both positions and categories of them. To overcome the challenges posed by point clouds, we propose GC-net, which is based on a three-stage pipeline, including gridding, clustering, and classification. First, we design a one-to-one mapping on raw point cloud as data preprocessing, which transforms the data structure from the graph to the grid. Then, we propose an efficient clustering algorithm: Grid- Density-Based Spatial Clustering of Applications with Noise to search the traffic objects. It exploits index information in the grid data to simplify the computational complexity. Last, we train a CNN-based classifier to categorize the found objects by extracting the local features, which performs well even the global shapes are defective. It only employs object-wise supervision, which reduces the difficulty of creating datasets. Based on the point clouds collected in real urban traffic scenarios, comparative experiences show that the proposed GC-net achieves a superior performance both in detection accuracy and computational speed, which are significant indicators for the real-time traffic surveillance systems.

Deep learning based classification of solid lipid-poor contrast enhancing renal masses using contrast enhanced CT.

Establish a workflow that utilizes convolutional neural nets (CNN) to classify solid, lipid-poor, contrast enhancing renal masses using multiphase contrast enhanced CT (CECT) images and to assess the performance of the resulting network. In this institutional review board approved study of 143 patients with predominantly solid, lipid-poor, contrast enhancing renal lesions (46 benign and 97 malignant), patients with a pre-operative multiphase CECT of the abdomen and pelvis obtained between June 2009 and June 2015 were retrospectively queried. Benign renal masses included oncocytoma and lipid-poor angiomyolipoma and the malignant group included clear cell, papillary, and chromophobe carcinomas.Region of interests of whole tumor volumes were manually segmented, and CT phase images with the largest cross-section of the segmented tumor in the axial plane were used for assessment. Post-surgical pathological evaluation was used to establish diagnosis.The segmented images of renal masses were used as input to a CNN. The data were augmented and split into training (83.9%) and validation sets (16.1%) to determine the hyperparameters of the CNN. Thereafter. the performance of the resulting CNN was quantified using eightfold cross-validation. The CNN-based classifier demonstrated an overall accuracy of 78% (95% confidence interval: 76-80%), sensitivity of 70% (95% confidence interval: 66-74%), specificity of 81% (79-83%) and an area under the curve of 0.82. A CNN-based classifier to diagnose solid enhancing malignant renal masses based on multiphase CECT images was developed. It was established that a CNN-based classifier could be trained to accurately distinguish malignant renal lesions.

CNN-Based Automatic Prioritization of Bug Reports

Software systems often receive a large number of bug reports. Triagers read through such reports and assign different priorities to different reports so that important and urgent bugs could be fixed on time. However, manual prioritization is tedious and time-consuming. To this end, in this article, we propose a convolutional neural network (CNN) based automatic approach to predict the multiclass priority for bug reports. First, we apply natural language processing (NLP) techniques to preprocess textual information of bug reports and covert the textual information into vectors based on the syntactic and semantic relationship of words within each bug report. Second, we perform the software engineering domain specific emotion analysis on bug reports and compute the emotion value for each of them using a software engineering domain repository. Finally, we train a CNN-based classifier that generates a suggested priority based on its input, i.e., vectored textual information and emotion values. To the best of our knowledge, it is the first CNN-based approach to bug report prioritization. We evaluate the proposed approach on open-source projects. Results of our cross-project evaluation suggest that the proposed approach significantly outperforms the state-of-the-art approaches and improves the average F1-score by more than 24%.

Classification of Eddy Sea Surface Temperature Signatures Under Cloud Coverage

Mesoscale oceanic eddies have a visible signature on sea surface temperature (SST) satellite images, portraying diverse patterns of coherent vortices, temperature gradients, and swirling filaments. However, learning the regularities of such signatures defines a challenging pattern recognition task, due to their complex structure but also to the cloud coverage which can corrupt a large fraction of the image. We introduce a novel deep learning approach to classify sea temperature eddy signatures, even if they are corrupted by strong cloud coverage. A large dataset of SST image patches is automatically retained and used to train a CNN-based classifier. Classification is performed with very high accuracy on coherent eddy signatures and is robust to a high level of cloud coverage, surpassing human expert efficiency on this task. This methodology can serve to validate and correct detections on satellite altimetry, the standard method used until now to track mesoscale eddies.

Optimization of Convolutional Neural Networks for Imbalanced Set Classification

Abstract Various research works demonstrate that the distribution of training data has a significant impact on the performance of Convolutional Neural Networks (CNNs). The CNN classifiers using balanced datasets achieve very accurate performance when compared to shallow classifiers. However, considering the CNNs originally are not designed to handle imbalanced datasets, such cases have a severely negative impact on their overall performance. In this research we focus on examining the effect of imbalanced data sets on performance outcomes of various CNN-based classifier models. The work also discusses implementation of some newly suggested optimizers as well as traditional ones for Deep Learning (DL), along with different approaches of learning rate adaptation including a method based on cyclic increase and decrease during training. We compare the impact of using two different methods of ensembling in order to further increase the classifier performance. Additionally, we experimentally evaluate the proposed approaches on two imbalanced image datasets.

Joint learning of Siamese network with top-down modulation and hard example mining for visual tracking

The development of deep convolutional neural networks (CNNs) has facilitated a type of Siamese network-based tracking methods. Such kind of trackers generally include two basic modules, i.e., the appearance model for target representation and the discriminative classifier to determine the location. In the existing methods, much attention has been paid on feature fusion for more robust appearance representation. However, they ignore the intrinsic relationship between features from multiple layers. Furthermore, the problem of data imbalance between positive and negative samples in offline training limits the discriminative ability of the CNN-based classifier. We investigate the joint learning of representation and discriminative classifier under the Siamese network for robust tracking. We integrate the top-down modulation for feature fusion by considering the intrinsic relationship between features from different layers. To solve the data imbalance problem, we propose an advanced hinge loss objective function to mine the hard examples in offline training, which helps to improve the robustness of the similarity measure. Experimental results on the public tracking benchmark datasets show that the proposed method obtains favorable tracking accuracy against the state-of-the-art trackers with a real-time tracking speed.

CNN-Based Ternary Classification for Image Steganalysis

This study proposes a convolutional neural network (CNN)-based steganalytic method that allows ternary classification to simultaneously identify WOW and UNIWARD, which are representative adaptive image steganographic algorithms. WOW and UNIWARD have very similar message embedding methods in terms of measuring and minimizing the degree of distortion of images caused by message embedding. This similarity between WOW and UNIWARD makes it difficult to distinguish between both algorithms even in a CNN-based classifier. Our experiments particularly show that WOW and UNIWARD cannot be distinguished by simply combining binary CNN-based classifiers learned to separately identify both algorithms. Therefore, to identify and classify WOW and UNIWARD, WOW and UNIWARD must be learned at the same time using a single CNN-based classifier designed for ternary classification. This study proposes a method for ternary classification that learns and classifies cover, WOW stego, and UNIWARD stego images using a single CNN-based classifier. A CNN structure and a preprocessing filter are also proposed to effectively classify/identify WOW and UNIWARD. Experiments using BOSSBase 1.01 database images confirmed that the proposed method could make a ternary classification with an accuracy of approximately 72%.

Semantic Segmentation of 3D LiDAR Data in Dynamic Scene Using Semi-Supervised Learning

This work studies the semantic segmentation of 3D LiDAR data in dynamic scenes for autonomous driving applications. A system of semantic segmentation using 3D LiDAR data, including range image segmentation, sample generation, inter-frame data association, track-level annotation, and semi-supervised learning, is developed. To reduce the considerable requirement of fine annotations, a CNN-based classifier is trained by considering both supervised samples with manually labeled object classes and pairwise constraints, where a data sample is composed of a segment as the foreground and neighborhood points as the background. A special loss function is designed to account for both annotations and constraints, where the constraint data are encouraged to be assigned to the same semantic class. A dataset containing 1838 frames of LiDAR data, 39 934 pairwise constraints and 57 927 human annotations is developed. The performance of the method is examined extensively. The qualitative and quantitative experiments show that the combination of a few annotations and large amount of constraint data significantly enhances the effectiveness and scene adaptability, resulting in greater than 10% improvement.

On the use of distributed semantics of tweet metadata for user age prediction

Social media data represent an important resource for behavioral analysis of the aging population. This paper addresses the problem of age prediction from Twitter dataset, where the prediction issue is viewed as a classification task. For this purpose, an innovative model based on Convolutional Neural Network is devised. To this end, we rely on language-related features and social media specific metadata. More specifically, we introduce two features that have not been previously considered in the literature: the content of URLs and hashtags appearing in tweets. We also employ distributed representations of words and phrases present in tweets, hashtags and URLs, pre-trained on appropriate corpora in order to exploit their semantic information in age prediction. We show that our CNN-based classifier, when compared with baseline models, yields an improvement of up to 12.3% for Dutch dataset, 9.8% for English1 dataset, and 6.6% for English2 dataset in the micro-averaged F1 score.

Adaptive Deep Convolutional Neural Networks for Scene-Specific Object Detection

A deep convolutional neural network (CNN) becomes a widely used tool for object detection. Many previous works have achieved excellent performance on object detection benchmarks. However, these works present generic detectors whose performance will drop rapidly when they are applied to a surveillance scene. In this paper, we propose an efficient method to construct a scene-specific regression model based on a generic CNN-based classifier. Our regression model is an adaptive deep CNN (ADCNN), which can predict object locations in the surveillance scene. First, we transfer the generic CNN-based classifier to the surveillance scene by selecting useful kernels. Second, we learn the context information of the surveillance scene in our regression model for accurate location prediction. Our main contributions are: 1) a transfer learning method that selects useful kernels in the generic CNN-based classifier; 2) a special architecture that can effectively learn the local and global context information in the surveillance scene; and 3) a new objective function to effectively train parameters in ADCNN. Compared with some state-of-the-art models, ADCNN achieves the best performance on three surveillance data sets for pedestrian detection and one surveillance data set for vehicle detection.

Study of the Application of Deep Convolutional Neural Networks (CNNs) in Processing Sensor Data and Biomedical Images.

The fast progress in research and development of multifunctional, distributed sensor networks has brought challenges in processing data from a large number of sensors. Using deep learning methods such as convolutional neural networks (CNN), it is possible to build smarter systems to forecasting future situations as well as precisely classify large amounts of data from sensors. Multi-sensor data from atmospheric pollutants measurements that involves five criteria, with the underlying analytic model unknown, need to be categorized, so do the Diabetic Retinopathy (DR) fundus images dataset. In this work, we created automatic classifiers based on a deep convolutional neural network (CNN) with two models, a simpler feedforward model with dual modules and an Inception Resnet v2 model, and various structural tweaks for classifying the data from the two tasks. For segregating multi-sensor data, we trained a deep CNN-based classifier on an image dataset extracted from the data by a novel image generating method. We created two deepened and one reductive feedforward network for DR phase classification. The validation accuracies and visualization results show that increasing deep CNN structure depth or kernels number in convolutional layers will not indefinitely improve the classification quality and that a more sophisticated model does not necessarily achieve higher performance when training datasets are quantitatively limited, while increasing training image resolution can induce higher classification accuracies for trained CNNs. The methodology aims at providing support for devising classification networks powering intelligent sensors.

Detection, localisation and tracking of pallets using machine learning techniques and 2D range data

The problem of autonomous transportation in industrial scenarios is receiving a renewed interest due to the way it can revolutionise internal logistics, especially in unstructured environments. This paper presents a novel architecture allowing a robot to detect, localise, and track (possibly multiple) pallets using machine learning techniques based on an on-board 2D laser rangefinder only. The architecture is composed of two main components: the first stage is a pallet detector employing a Faster Region-based Convolutional Neural Network (Faster R-CNN) detector cascaded with a CNN-based classifier; the second stage is a Kalman filter for localising and tracking detected pallets, which we also use to defer commitment to a pallet detected in the first stage until sufficient confidence has been acquired via a sequential data acquisition process. For fine-tuning the CNNs, the architecture has been systematically evaluated using a real-world dataset containing 340 labeled 2D scans, which have been made freely available in an online repository. Detection performance has been assessed on the basis of the average accuracy over k-fold cross-validation, and it scored 99.58% in our tests. Concerning pallet localisation and tracking, experiments have been performed in a scenario where the robot is approaching the pallet to fork. Although data have been originally acquired by considering only one pallet as per specification of the use case we consider, artificial data have been generated as well to mimic the presence of multiple pallets in the robot workspace. Our experimental results confirm that the system is capable of identifying, localising and tracking pallets with a high success rate while being robust to false positives.

Matrix Completion for Graph-Based Deep Semi-Supervised Learning

Convolutional Neural Networks (CNNs) have provided promising achievements for image classification problems. However, training a CNN model relies on a large number of labeled data. Considering the vast amount of unlabeled data available on the web, it is important to make use of these data in conjunction with a small set of labeled data to train a deep learning model. In this paper, we introduce a new iterative Graph-based Semi-Supervised Learning (GSSL) method to train a CNN-based classifier using a large amount of unlabeled data and a small amount of labeled data. In this method, we first construct a similarity graph in which the nodes represent the CNN features corresponding to data points (labeled and unlabeled) while the edges tend to connect the data points with the same class label. In this graph, the missing label of unsupervised nodes is predicted by using a matrix completion method based on rank minimization criterion. In the next step, we use the constructed graph to calculate triplet regularization loss which is added to the supervised loss obtained by initially labeled data to update the CNN network parameters.

Multi-Stage Hough Space Calculation for Lane Markings Detection via IMU and Vision Fusion.

It is challenging to achieve robust lane detection based on a single frame, particularly when complicated driving scenarios are present. A novel approach based on multiple frames is proposed in this paper by taking advantage of the fusion of vision and Inertial Measurement Units (IMU). Hough space is employed as a storage medium where lane markings can be stored and visited conveniently. The detection of lane markings is achieved by the following steps. Firstly, primary line segments are extracted from a basic Hough space, which is calculated by Hough Transform. Secondly, a CNN-based classifier is introduced to measure the confidence probability of each line segment, and transforms the basic Hough space into a probabilistic Hough space. In the third step, pose information provided by the IMU is applied to align previous probabilistic Hough spaces to the current one and a filtered probabilistic Hough space is acquired by smoothing the primary probabilistic Hough space across frames. Finally, valid line segments with probability higher than 0.7 are extracted from the filtered probabilistic Hough space. The proposed approach is applied experimentally, and the results demonstrate a satisfying performance compared to various existing methods.

Improving Pest Bird Detection in a Vineyard Environment using Super-Resolution and Deep Learning

Pest bird detection, classification, and recognition in vineyard environment are challenging because of their varying shapes, small size, movement, and outdoor environment. Motion is often used to detect flying birds in outdoor environment from video sequences. However, motion detection is sensitive to noise as well as background movement of leaves and give rise to false detection. The high-quality image resolution is desired for performance improvement in pattern recognition and analysis. This work presents the integration of super-resolution technology to enhance quality of small moving objects which were later on classified as birds or false positives using deep learning. Implementation of the super-resolution enhanced the image resolution which offers high pixel density and more details about the scene. With the implementation of super-resolution, the CNN-based classifier received enhanced feature information to perform more informed decision in classifying birds. The classification accuracy shows a significant rise from 70% to more than 90% after resolution enhancement. Results also show that the model trained with combined varying spatial resolution for the same set of images performs almost equally over any spatial resolution.

Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks

ImportanceConvolutional neural networks (CNNs) achieve expert-level accuracy in the diagnosis of pigmented melanocytic lesions. However, the most common types of skin cancer are nonpigmented and nonmelanocytic, and are more difficult to diagnose.ObjectiveTo compare the accuracy of a CNN-based classifier with that of physicians with different levels of experience.Design, Setting, and ParticipantsA CNN-based classification model was trained on 7895 dermoscopic and 5829 close-up images of lesions excised at a primary skin cancer clinic between January 1, 2008, and July 13, 2017, for a combined evaluation of both imaging methods. The combined CNN (cCNN) was tested on a set of 2072 unknown cases and compared with results from 95 human raters who were medical personnel, including 62 board-certified dermatologists, with different experience in dermoscopy.Main Outcomes and MeasuresThe proportions of correct specific diagnoses and the accuracy to differentiate between benign and malignant lesions measured as an area under the receiver operating characteristic curve served as main outcome measures.ResultsAmong 95 human raters (51.6% female; mean age, 43.4 years; 95% CI, 41.0-45.7 years), the participants were divided into 3 groups (according to years of experience with dermoscopy): beginner raters (<3 years), intermediate raters (3-10 years), or expert raters (>10 years). The area under the receiver operating characteristic curve of the trained cCNN was higher than human ratings (0.742; 95% CI, 0.729-0.755 vs 0.695; 95% CI, 0.676-0.713; P < .001). The specificity was fixed at the mean level of human raters (51.3%), and therefore the sensitivity of the cCNN (80.5%; 95% CI, 79.0%-82.1%) was higher than that of human raters (77.6%; 95% CI, 74.7%-80.5%). The cCNN achieved a higher percentage of correct specific diagnoses compared with human raters (37.6%; 95% CI, 36.6%-38.4% vs 33.5%; 95% CI, 31.5%-35.6%; P = .001) but not compared with experts (37.3%; 95% CI, 35.7%-38.8% vs 40.0%; 95% CI, 37.0%-43.0%; P = .18).Conclusions and RelevanceNeural networks are able to classify dermoscopic and close-up images of nonpigmented lesions as accurately as human experts in an experimental setting.

Towards Real-Time Multi-Sensor Golf Swing Classification Using Deep CNNs

In recent years, smart sports equipment and body sensor systems have become popular in professional and amateur sports. One of a few remaining problems in real-time applications is the discovery of knowledge from the embedded sensors data. In sports training, such knowledge helps accelerated motor learning. The authors start with exploring the possibilities of the classification of golf swing performance with the 1-D convolutional neural network (CNN) in real-time. They thoroughly investigate multiple golf swing data classifiers based on CNNs fed with multi-sensor signals. The authors test the possibilities of real-time performance of CNN methods on the multi-length sequences. In addition, they thoroughly evaluate the performance of their well-trained CNN-based classifier on the aforementioned test set in terms of common indicators. Experiments and corresponding results show that the authors' models can satisfy the real-time requirement of the accuracy of the classification and outperform support vector machine (SVM).

Multi-sensor Golf Swing Classification Using Deep CNN

In recent years smart sport equipments have achieved great success in professional and amateur sports, as well as body sensory systems; now discovering interesting knowledge in the surge of data from those embedded sensors used in sports is necessary and the focus of our research. In this paper, we investigate golf swing data classification method based on deep convolutional neural network (deep CNN) fed with multi-sensor golf swing signals. Our smart golf club integrates two orthogonally affixed strain gage sensors, 3-axis accelerometer and 3-axis gyroscope, and collects real-world golf swing data from professional and amateur golf players. Furthermore we explore the performance of our well-trained CNN-based classifier and evaluate it on the real-world test set in terms of common indicators including accuracy, precision-recall, and F1-score. Experiments and corresponding results show that our CNN-based model can satisfy the requirement of accuracy of golf swing classification, and outperforms support vector machine (SVM) method.

Classification Algorithms for Human and Dog Movement Based on Micro-Doppler Signals

We propose classification algorithms for human and dog movement. The proposed algorithms use micro-Doppler signals obtained from humans and dogs moving in four different directions. A two-stage classifier based on a support vector machine (SVM) is proposed, which uses a radial-based function (RBF) kernel and 16SUPth/SUP-order linear predictive code (LPC) coefficients as feature vectors. With the proposed algorithms, we obtain the best classification results when a first-level SVM classifies the type of movement, and then, a second-level SVM classifies the moving object. We obtain the correct classification probability 95.54% of the time, on average. Next, to deal with the difficult classification problem of human and dog running, we propose a twolayer convolutional neural network (CNN). The proposed CNN is composed of six (6x6) convolution filters at the first and second layers, with (5x5) max pooling for the first layer and (2x2) max pooling for the second layer. The proposed CNN-based classifier adopts an auto regressive spectrogram as the feature image obtained from the 16SUPth/SUP-order LPC vectors for a specific time duration. The proposed CNN exhibits 100% classification accuracy and outperforms the SVM-based classifier. These results show that the proposed classifiers can be used for human and dog classification systems and also for classification problems using data obtained from an ultrawideband (UWB) sensor.