Proposed Convolutional Neural Network Architecture Research Articles

Autonomous Traffic Sign (ATSR) Detection and Recognition using Deep CNN

Automatic detection and recognition of traffic signs is very important and could potentially be used for driver assistance to reduce accidents and eventually in driverless automobiles. In this paper, Deep Convolutional Neural Network (CNN) is used to develop an Autonomous Traffic and Road Sign (ATRS) detection and recognition system. The proposed system works in real time detecting and recognizing traffic sign images. The contribution of this paper is also a newly developed database of 24 different traffic signs collected from random road sides in Saudi Arabia. The images were taken from different angles and including other parameters and conditions. A total of 2718 images were collected to form the database which we named Saudi Arabian Traffic and Road Signs (SA-TRS-2018). The CNN architecture was used with varying parameters in order to achieve the best recognition rates. Experimental results show that the proposed CNN architecture achieved an accuracy of 100%, thus higher than those achieved in similar previous studies.

Joint Alternate Small Convolution and Feature Reuse for Hyperspectral Image Classification

A hyperspectral image (HSI) contains fine and rich spectral information and spatial information of ground objects, which has great potential in applications. It is also widely used in precision agriculture, marine monitoring, military reconnaissance and many other fields. In recent years, a convolutional neural network (CNN) has been successfully used in HSI classification and has provided it with outstanding capacity for improving classification effects. To get rid of the bondage of strong correlation among bands for HSI classification, an effective CNN architecture is proposed for HSI classification in this work. The proposed CNN architecture has several distinct advantages. First, each 1D spectral vector that corresponds to a pixel in an HSI is transformed into a 2D spectral feature matrix, thereby emphasizing the difference among samples. In addition, this architecture can not only weaken the influence of strong correlation among bands on classification, but can also fully utilize the spectral information of hyperspectral data. Furthermore, a 1 × 1 convolutional layer is adopted to better deal with HSI information. All the convolutional layers in the proposed CNN architecture are composed of small convolutional kernels. Moreover, cascaded composite layers of the architecture consist of 1 × 1 and 3 × 3 convolutional layers. The inputs and outputs of each composite layer are stitched as the inputs of the next composite layer, thereby accomplishing feature reuse. This special module with joint alternate small convolution and feature reuse can extract high-level features from hyperspectral data meticulously and comprehensively solve the overfitting problem to an extent, in order to obtain a considerable classification effect. Finally, global average pooling is used to replace the traditional fully connected layer to reduce the model parameters and extract high-dimensional features from the hyperspectral data at the end of the architecture. Experimental results on three benchmark HSI datasets show the high classification accuracy and effectiveness of the proposed method.

Pulmonary Artery-Vein Classification in CT Images Using Deep Learning.

Recent studies show that pulmonary vascular diseases may specifically affect arteries or veins through different physiologic mechanisms. To detect changes in the two vascular trees, physicians manually analyze the chest computed tomography (CT) image of the patients in search of abnormalities. This process is time consuming, difficult to standardize, and thus not feasible for large clinical studies or useful in real-world clinical decision making. Therefore, automatic separation of arteries and veins in CT images is becoming of great interest, as it may help physicians to accurately diagnose pathological conditions. In this paper, we present a novel, fully automatic approach to classify vessels from chest CT images into arteries and veins. The algorithm follows three main steps: first, a scale-space particles segmentation to isolate vessels; then a 3-D convolutional neural network (CNN) to obtain a first classification of vessels; finally, graph-cuts' optimization to refine the results. To justify the usage of the proposed CNN architecture, we compared different 2-D and 3-D CNNs that may use local information from bronchus- and vessel-enhanced images provided to the network with different strategies. We also compared the proposed CNN approach with a random forests (RFs) classifier. The methodology was trained and evaluated on the superior and inferior lobes of the right lung of 18 clinical cases with noncontrast chest CT scans, in comparison with manual classification. The proposed algorithm achieves an overall accuracy of 94%, which is higher than the accuracy obtained using other CNN architectures and RF. Our method was also validated with contrast-enhanced CT scans of patients with chronic thromboembolic pulmonary hypertension to demonstrate that our model generalizes well to contrast-enhanced modalities. The proposed method outperforms state-of-the-art methods, paving the way for future use of 3-D CNN for artery/vein classification in CT images.

Automatic Recognition of Flock Behavior of Chickens with Convolutional Neural Network and Kinect Sensor

In this paper, we propose an automatic convolutional neural network (CNN)-based method to recognize the chicken behavior within a poultry farm using a Kinect sensor. It resolves the hardships in flock behavior image classification by leveraging a data-driven mechanism and exploiting non-manually extracted multi-scale image features which combine both the local and global characteristics of the image. To our best knowledge, this is probably the first attempt of deep learning strategy in the field of domestic animal behavior recognition. To testify the performance of our proposed method, we conducted experiments between state-of-the-art methods and our method. Experimental results witness that our proposed approach outperforms the state-of-the-art methods both in effectiveness and efficiency. Our proposed CNN architecture for recognizing flock behavior of chickens produces an extremely impressive accuracy of 99.17%.

A deep learning method based on convolutional neural network for automatic modulation classification of wireless signals

Automatic modulation classification (AMC) plays an important role in many fields to identify the modulation type of wireless signals. In this paper, we introduce deep learning to signal recognition. Based on architecture analysis of the convolutional neural network (CNN), we used real signal data generated by instruments as dataset, and proposed an improved CNN architecture to achieve compatible recognition accuracy of modulation classification. According to various conditions of signal noise ratio (SNR), we test the proposed CNN architecture with the real sampled signals. Experiments results show that the high-layer network is not necessary for modulation recognition with high SNR signals. The proposed CNN architecture has higher average classification accuracy than RESNET and is more compatible for modulation classification of signals with lower SNR.

A robust deep convolutional neural network for the classification of abnormal cardiac rhythm using single lead electrocardiograms of variable length

Objective: Atrial fibrillation (AF) is a major cause of hospitalization and death in the United States. Moreover, as the average age of individuals increases around the world, early detection and diagnosis of AF become even more pressing. In this paper, we introduce a novel deep learning architecture for the detection of normal sinus rhythm, AF, other abnormal rhythms, and noise. Approach: We have demonstrated through a systematic approach many hyperparameters, input sets, and optimization methods that yielded influence in both training time and performance accuracy. We have focused on these properties to identify an optimal 13-layer convolutional neural network (CNN) model which was trained on 8528 short single-lead ECG recordings and evaluated on a test dataset of 3658 recordings. Main results: The proposed CNN architecture achieved a state-of-the-art performance in identifying normal, AF and other rhythms with an average F1-score of 0.83. Significance: We have presented a robust deep learning-based architecture that can identify abnormal cardiac rhythms using short single-lead ECG recordings. The proposed architecture is computationally fast and can also be used in real-time cardiac arrhythmia detection applications.

Wrist sensor-based tremor severity quantification in Parkinson's disease using convolutional neural network

Tremor is a commonly observed symptom in patients of Parkinson's disease (PD), and accurate measurement of tremor severity is essential in prescribing appropriate treatment to relieve its symptoms. We propose a tremor assessment system based on the use of a convolutional neural network (CNN) to differentiate the severity of symptoms as measured in data collected from a wearable device. Tremor signals were recorded from 92 PD patients using a custom-developed device (SNUMAP) equipped with an accelerometer and gyroscope mounted on a wrist module. Neurologists assessed the tremor symptoms on the Unified Parkinson's Disease Rating Scale (UPDRS) from simultaneously recorded video footages. The measured data were transformed into the frequency domain and used to construct a two-dimensional image for training the network, and the CNN model was trained by convolving tremor signal images with kernels. The proposed CNN architecture was compared to previously studied machine learning algorithms and found to outperform them (accuracy = 0.85, linear weighted kappa = 0.85). More precise monitoring of PD tremor symptoms in daily life could be possible using our proposed method.

White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks.

White matter hyperintensities (WMH) are a feature of sporadic small vessel disease also frequently observed in magnetic resonance images (MRI) of healthy elderly subjects. The accurate assessment of WMH burden is of crucial importance for epidemiological studies to determine association between WMHs, cognitive and clinical data; their causes, and the effects of new treatments in randomized trials. The manual delineation of WMHs is a very tedious, costly and time consuming process, that needs to be carried out by an expert annotator (e.g. a trained image analyst or radiologist). The problem of WMH delineation is further complicated by the fact that other pathological features (i.e. stroke lesions) often also appear as hyperintense regions. Recently, several automated methods aiming to tackle the challenges of WMH segmentation have been proposed. Most of these methods have been specifically developed to segment WMH in MRI but cannot differentiate between WMHs and strokes. Other methods, capable of distinguishing between different pathologies in brain MRI, are not designed with simultaneous WMH and stroke segmentation in mind. Therefore, a task specific, reliable, fully automated method that can segment and differentiate between these two pathological manifestations on MRI has not yet been fully identified. In this work we propose to use a convolutional neural network (CNN) that is able to segment hyperintensities and differentiate between WMHs and stroke lesions. Specifically, we aim to distinguish between WMH pathologies from those caused by stroke lesions due to either cortical, large or small subcortical infarcts. The proposed fully convolutional CNN architecture, called uResNet, that comprised an analysis path, that gradually learns low and high level features, followed by a synthesis path, that gradually combines and up-samples the low and high level features into a class likelihood semantic segmentation. Quantitatively, the proposed CNN architecture is shown to outperform other well established and state-of-the-art algorithms in terms of overlap with manual expert annotations. Clinically, the extracted WMH volumes were found to correlate better with the Fazekas visual rating score than competing methods or the expert-annotated volumes. Additionally, a comparison of the associations found between clinical risk-factors and the WMH volumes generated by the proposed method, was found to be in line with the associations found with the expert-annotated volumes.

Convolutional Neural Network for Histopathological Analysis of Osteosarcoma

Pathologists often deal with high complexity and sometimes disagreement over osteosarcoma tumor classification due to cellular heterogeneity in the dataset. Segmentation and classification of histology tissue in H&E stained tumor image datasets is a challenging task because of intra-class variations, inter-class similarity, crowded context, and noisy data. In recent years, deep learning approaches have led to encouraging results in breast cancer and prostate cancer analysis. In this article, we propose convolutional neural network (CNN) as a tool to improve efficiency and accuracy of osteosarcoma tumor classification into tumor classes (viable tumor, necrosis) versus nontumor. The proposed CNN architecture contains eight learned layers: three sets of stacked two convolutional layers interspersed with max pooling layers for feature extraction and two fully connected layers with data augmentation strategies to boost performance. The use of a neural network results in higher accuracy of average 92% for the classification. We compare the proposed architecture with three existing and proven CNN architectures for image classification: AlexNet, LeNet, and VGGNet. We also provide a pipeline to calculate percentage necrosis in a given whole slide image. We conclude that the use of neural networks can assure both high accuracy and efficiency in osteosarcoma classification.

Exudate detection for diabetic retinopathy with convolutional neural networks.

Exudate detection is an essential task for computer-aid diagnosis of diabetic retinopathy (DR), so as to monitor the progress of DR. In this paper, deep convolutional neural network (CNN) is adopted to achieve pixel-wise exudate identification. The CNN model is first trained with expert labeled exudates image patches and then saved as off-line classifier. In order to achieve pixel-level accuracy meanwhile reduce computational time, potential exudate candidate points are first extracted with morphological ultimate opening algorithm. Then the local region (64 × 64) surrounding the candidate points are forwarded to the trained CNN model for classification/identification. A pixel-wise accuracy of 91.92%, sensitivity of 88.85% and specificity of 96% is achieved with the proposed CNN architecture on the test database.

Two-phase deep convolutional neural network for reducing class skewness in histopathological images based breast cancer detection

Different types of breast cancer are affecting lives of women across the world. Common types include Ductal carcinoma in situ (DCIS), Invasive ductal carcinoma (IDC), Tubular carcinoma, Medullary carcinoma, and Invasive lobular carcinoma (ILC). While detecting cancer, one important factor is mitotic count – showing how rapidly the cells are dividing. But the class imbalance problem, due to the small number of mitotic nuclei in comparison to the overwhelming number of non-mitotic nuclei, affects the performance of classification models.This work presents a two-phase model to mitigate the class biasness issue while classifying mitotic and non-mitotic nuclei in breast cancer histopathology images through a deep convolutional neural network (CNN). First, nuclei are segmented out using blue ratio and global binary thresholding. In Phase-1 a CNN is then trained on the segmented out 80×80 pixel patches based on a standard dataset. Hard non-mitotic examples are identified and augmented; mitotic examples are oversampled by rotation and flipping; whereas non-mitotic examples are undersampled by blue ratio histogram based k-means clustering. Based on this information from Phase-1, the dataset is modified for Phase-2 in order to reduce the effects of class imbalance.The proposed CNN architecture and data balancing technique yielded an F-measure of 0.79, and outperformed all the methods relying on specific handcrafted features, as well as those using a combination of handcrafted and CNN-generated features.

Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification

The ability of deep convolutional neural networks (CNNs) to learn discriminative spectro-temporal patterns makes them well suited to environmental sound classification. However, the relative scarcity of labeled data has impeded the exploitation of this family of high-capacity models. This study has two primary contributions: first, we propose a deep CNN architecture for environmental sound classification. Second, we propose the use of audio data augmentation for overcoming the problem of data scarcity and explore the influence of different augmentations on the performance of the proposed CNN architecture. Combined with data augmentation, the proposed model produces state-of-the-art results for environmental sound classification. We show that the improved performance stems from the combination of a deep, high-capacity model and an augmented training set: this combination outperforms both the proposed CNN without augmentation and a “shallow” dictionary learning model with augmentation. Finally, we examine the influence of each augmentation on the model's classification accuracy for each class, and observe that the accuracy for each class is influenced differently by each augmentation, suggesting that the performance of the model could be improved further by applying class-conditional data augmentation.

Deep Learning Enabled Fault Diagnosis Using Time-Frequency Image Analysis of Rolling Element Bearings

Traditional feature extraction and selection is a labor-intensive process requiring expert knowledge of the relevant features pertinent to the system. This knowledge is sometimes a luxury and could introduce added uncertainty and bias to the results. To address this problem a deep learning enabled featureless methodology is proposed to automatically learn the features of the data. Time-frequency representations of the raw data are used to generate image representations of the raw signal, which are then fed into a deep convolutional neural network (CNN) architecture for classification and fault diagnosis. This methodology was applied to two public data sets of rolling element bearing vibration signals. Three time-frequency analysis methods (short-time Fourier transform, wavelet transform, and Hilbert-Huang transform) were explored for their representation effectiveness. The proposed CNN architecture achieves better results with less learnable parameters than similar architectures used for fault detection, including cases with experimental noise.

Deep learning for decentralized parking lot occupancy detection

A smart camera is a vision system capable of extracting application-specific information from the captured images. The paper proposes a decentralized and efficient solution for visual parking lot occupancy detection based on a deep Convolutional Neural Network (CNN) specifically designed for smart cameras. This solution is compared with state-of-the-art approaches using two visual datasets: PKLot, already existing in literature, and CNRPark-EXT. The former is an existing dataset, that allowed us to exhaustively compare with previous works. The latter dataset has been created in the context of this research, accumulating data across various seasons of the year, to test our approach in particularly challenging situations, exhibiting occlusions, and diverse and difficult viewpoints. This dataset is public available to the scientific community and is another contribution of our research. Our experiments show that our solution outperforms and generalizes the best performing approaches on both datasets. The performance of our proposed CNN architecture on the parking lot occupancy detection task, is comparable to the well-known AlexNet, which is three orders of magnitude larger.

Convolutional neural networks for hyperspectral image classification

As a powerful visual model, convolutional neural networks (CNNs) have demonstrated remarkable performance in various visual recognition problems, and attracted considerable attention in recent years. However, due to the highly correlated bands and insufficient training samples of hyperspectral image data, it still remains a challenging problem to effectively apply the CNN models on hyperspectral images. In this paper, an efficient CNN architecture has been proposed to boost its discriminative capability for hyperspectral image classification, in which the original data is used as the input and the final CNN outputs are the predicted class-related results. The proposed CNN infrastructure has several distinct advantages. Firstly, different from traditional classification methods those need hand-crafted features, the CNN model used here is designed to deal with the problem of hyperspectral image analysis in an end-to-end way. Secondly, the parameters of the CNN model are optimized from a small training set, while the over-fitting problem of the neural network has been alleviated to some extent. Finally, in order to better deal with the hyperspectral image information, 1 × 1 convolutional layers have been adopted, and an average pooling layer and larger dropout rates have also been employed in the whole CNN procedure. The experiments on three benchmark data sets have demonstrated that the proposed CNN architecture considerably outperforms other state-of-the-art methods.

Detecting predatory conversations in social media by deep Convolutional Neural Networks

Automatic identification of predatory conversations in chat logs helps the law enforcement agencies act proactively through early detection of predatory acts in cyberspace. In this paper, we describe the novel application of a deep learning method to the automatic identification of predatory chat conversations in large volumes of chat logs. We present a classifier based on Convolutional Neural Network (CNN) to address this problem domain. The proposed CNN architecture outperforms other classification techniques that are common in this domain including Support Vector Machine (SVM) and regular Neural Network (NN) in terms of classification performance, which is measured by F1-score. In addition, our experiments show that using existing pre-trained word vectors are not suitable for this specific domain. Furthermore, since the learning algorithm runs in a massively parallel environment (i.e., general-purpose GPU), the approach can benefit a large number of computation units (neurons) compared to when CPU is used. To the best of our knowledge, this is the first time that CNNs are adapted and applied to this application domain.

Facial expression recognition in the wild based on multimodal texture features

Facial expression recognition in the wild is a very challenging task. We describe our work in static and continuous facial expression recognition in the wild. We evaluate the recognition results of gray deep features and color deep features, and explore the fusion of multimodal texture features. For the continuous facial expression recognition, we design two temporal–spatial dense scale-invariant feature transform (SIFT) features and combine multimodal features to recognize expression from image sequences. For the static facial expression recognition based on video frames, we extract dense SIFT and some deep convolutional neural network (CNN) features, including our proposed CNN architecture. We train linear support vector machine and partial least squares classifiers for those kinds of features on the static facial expression in the wild (SFEW) and acted facial expression in the wild (AFEW) dataset, and we propose a fusion network to combine all the extracted features at decision level. The final achievement we gained is 56.32% on the SFEW testing set and 50.67% on the AFEW validation set, which are much better than the baseline recognition rates of 35.96% and 36.08%.

Gender classification: a convolutional neural network approach

An approach using a convolutional neural network (CNN) is proposed for real-time gender classification based on facial images. The proposed CNN architecture exhibits a much reduced design complexity when compared with other CNN solutions applied in pattern recognition. The number of processing layers in the CNN is reduced to only four by fusing the convolutional and subsampling layers. Unlike in conventional CNNs, we replace the convolution operation with cross-correlation, hence reducing the computational load. The network is trained using a second-order backpropagation learning algorithm with annealed global learning rates. Performance evaluation of the proposed CNN solution is conducted on two publicly available face databases of SUMS and AT&T. We achieve classification accuracies of 98.75% and 99.38% on the SUMS and AT&T databases, respectively. The neural network is able to process and classify a 32 × 32 pixel face image in less than 0.27 ms, which corresponds to a very high throughput of over 3700 images per second. Training converges within less than 20 epochs. These results correspond to a superior classification performance, verifying that the proposed CNN is an effective real-time solution for gender recognition.

Convolutional Neural Networks with Fused Layers Applied to Face Recognition

In this paper, we propose an effective convolutional neural network (CNN) model to the problem of face recognition. The proposed CNN architecture applies fused convolution/subsampling layers that result in a simpler model with fewer network parameters; that is, a smaller number of neurons, trainable parameters, and connections. In addition, it does not require any complex or costly image preprocessing steps that are typical in existing face recognizer systems. In this work, we enhance the stochastic diagonal Levenberg–Marquardt algorithm, a second-order back-propagation algorithm to obtain faster network convergence and better generalization ability. Experimental work completed on the ORL database shows that a recognition accuracy of 100% is achieved, with the network converging within 15 epochs. The average processing time of the proposed CNN face recognition solution, executed on a 2.5 GHz Intel i5 quad-core processor, is 3 s per epoch, with a recognition speed of less than 0.003 s. These results show that the proposed CNN model is a computationally efficient architecture that exhibits faster processing and learning times, and also produces higher recognition accuracy, outperforming other existing work on face recognizers based on neural networks.