Common Convolutional Neural Network Research Articles

GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition

The dominant approaches for named entity recognitionm (NER) mostly adopt complex recurrent neural networks (RNN), e.g., long-short-term-memory (LSTM). However, RNNs are limited by their recurrent nature in terms of computational efficiency. In contrast, convolutional neural networks (CNN) can fully exploit the GPU parallelism with their feedforward architectures. However, little attention has been paid to performing NER with CNNs, mainly owing to their difficulties in capturing the long-term context information in a sequence. In this paper, we propose a simple but effective CNN-based network for NER, i.e., gated relation network (GRN), which is more capable than common CNNs in capturing long-term context. Specifically, in GRN we firstly employ CNNs to explore the local context features of each word. Then we model the relations between words and use them as gates to fuse local context features into global ones for predicting labels. Without using recurrent layers that process a sentence in a sequential manner, our GRN allows computations to be performed in parallel across the entire sentence. Experiments on two benchmark NER datasets (i.e., CoNLL2003 and Ontonotes 5.0) show that, our proposed GRN can achieve state-of-the-art performance with or without external knowledge. It also enjoys lower time costs to train and test.

Scale-space approximated convolutional neural networks for retinal vessel segmentation

Background and objective: Retinal fundus images are widely used to diagnose retinal diseases and can potentially be used for early diagnosis and prevention of chronic vascular diseases and diabetes. While various automatic retinal vessel segmentation methods using deep learning have been proposed, they are mostly based on common CNN structures developed for other tasks such as classification. MethodsWe present a novel and simple multi-scale convolutional neural network (CNN) structure for retinal vessel segmentation. We first provide a theoretical analysis of existing multi-scale structures based on signal processing. In previous structures, multi-scale representations are achieved through downsampling by subsampling and decimation. By incorporating scale-space theory, we propose a simple yet effective multi-scale structure for CNNs using upsampling, which we term scale-space approximated CNN (SSANet). Based on further analysis of the effects of the SSA structure within a CNN, we also incorporate residual blocks, resulting in a multi-scale CNN that outperforms current state-of-the-art methods. ResultsQuantitative evaluations are presented as the area-under-curve (AUC) of the receiver operating characteristic (ROC) curve and the precision-recall curve, as well as accuracy, for four publicly available datasets, namely DRIVE, STARE, CHASE_DB1, and HRF. For the CHASE_DB1 set, the SSANet achieves state-of-the-art AUC value of 0.9916 for the ROC curve. An ablative analysis is presented to analyze the contribution of different components of the SSANet to the performance improvement. ConclusionsThe proposed retinal SSANet achieves state-of-the-art or comparable accuracy across publicly available datasets, especially improving segmentation for thin vessels, vessel junctions, and central vessel reflexes.

Deep learning in medical image analysis: A third eye for doctors

Aim and scopeArtificial intelligence (AI) in medicine is a fast-growing field. The rise of deep learning algorithms, such as convolutional neural networks (CNNs), offers fascinating perspectives for the automation of medical image analysis. In this systematic review article, we screened the current literature and investigated the following question: “Can deep learning algorithms for image recognition improve visual diagnosis in medicine?” Materials and methodsWe provide a systematic review of the articles using CNNs for medical image analysis, published in the medical literature before May 2019. Articles were screened based on the following items: type of image analysis approach (detection or classification), algorithm architecture, dataset used, training phase, test, comparison method (with specialists or other), results (accuracy, sensibility and specificity) and conclusion. ResultsWe identified 352 articles in the PubMed database and excluded 327 items for which performance was not assessed (review articles) or for which tasks other than detection or classification, such as segmentation, were assessed. The 25 included papers were published from 2013 to 2019 and were related to a vast array of medical specialties. Authors were mostly from North America and Asia. Large amounts of qualitative medical images were necessary to train the CNNs, often resulting from international collaboration. The most common CNNs such as AlexNet and GoogleNet, designed for the analysis of natural images, proved their applicability to medical images. ConclusionCNNs are not replacement solutions for medical doctors, but will contribute to optimize routine tasks and thus have a potential positive impact on our practice. Specialties with a strong visual component such as radiology and pathology will be deeply transformed. Medical practitioners, including surgeons, have a key role to play in the development and implementation of such devices.

Impact of pre-analytical variables on deep learning accuracy in histopathology.

Machine learning (ML) binary classification in diagnostic histopathology is an area of intense investigation. Several assumptions, including training image quality/format and the number of training images required, appear to be similar in many studies irrespective of the paucity of supporting evidence. We empirically compared training image file type, training set size, and two common convolutional neural networks (CNNs) using transfer learning (ResNet50 and SqueezeNet). Thirty haematoxylin and eosin (H&E)-stained slides with carcinoma or normal tissue from three tissue types (breast, colon, and prostate) were photographed, generating 3000 partially overlapping images (1000 per tissue type). These lossless Portable Networks Graphics (PNGs) images were converted to lossy Joint Photographic Experts Group (JPG) images. Tissue type-specific binary classification ML models were developed by the use of all PNG or JPG images, and repeated with a subset of 500, 200, 100, 50, 30 and 10 images. Eleven models were generated for each tissue type, at each quantity of training images, for each file type, and for each CNN, resulting in 924 models. Internal accuracies and generalisation accuracies were compared. There was no meaningful significant difference in accuracies between PNG and JPG models. Models trained with more images did not invariably perform better. ResNet50 typically outperformed SqueezeNet. Models were generalisable within a tissue type but not across tissue types. Lossy JPG images were not inferior to lossless PNG images in our models. Large numbers of unique H&E-stained slides were not required for training optimal ML models. This reinforces the need for an evidence-based approach to best practices for histopathological ML.

Deep Metadata Fusion for Traffic Light to Lane Assignment

We present a deep metadata fusion approach that connects image data and heterogeneous metadata inside a Convolutional Neural Network (CNN). This approach enables us to assign all relevant traffic lights to their associated lanes. To achieve this, a common CNN topology is trained by down-sampled and transformed input images to predict an indication vector. The indication vector contains the column positions of all the relevant traffic lights that are associated with lanes. In parallel, we fuse prepared and adaptively weighted Metadata Feature Maps (MFM) with the convolutional feature map input of a selected convolutional layer. The results are compared to rule-based, only-metadata, and only-vision approaches. In addition, human performance of the traffic light to ego-vehicle lane assignment has been measured by a subjective test. The proposed approach outperforms all other approaches. It achieves about 93.0% average precision for a real-world dataset. In a more complex dataset, 87.1% average precision is achieved. In particular, the new approach reaches significantly higher results with 93.7% to 91.0% average accuracy for a real-world dataset in contrast to lower human performance.

Smoke Detection on Video Sequences Using 3D Convolutional Neural Networks

Research on video smoke detection has become a hot topic in fire disaster prevention and control as it can realize early detection. Conventional methods use handcrafted features rely on prior knowledge to recognize whether a frame contains smoke. Such methods are often proposed for fixed fire scene and sensitive to the environment resulting in false alarms. In this paper, we use convolutional neural networks (CNN), which are state-of-the-art for image recognition tasks to identify smoke in video. We develop a joint detection framework based on faster RCNN and 3D CNN. An improved faster RCNN with non-maximum annexation is used to realize the smoke target location based on static spatial information. Then, 3D CNN realizes smoke recognition by combining dynamic spatial–temporal information. Compared with common CNN methods using image for smoke detection, 3D CNN improved the recognition accuracy significantly. Different network structures and data processing methods of 3D CNN have been compared, including Slow Fusion and optical flow. Tested on a dataset that comprises smoke video from multiple sources, the proposed frameworks are shown to perform very well in smoke location and recognition. Finally, the framework of two-stream 3D CNN performs the best, with a detection rate of 95.23% and a low false alarm rate of 0.39% for smoke video sequences.

A common convolutional neural network model to classify plain, rolled and latent fingerprints

Fingerprint classification helps in reducing the number of comparisons during the matching stage in automatic fingerprint identification system. In this study, a convolutional neural network model is proposed for classification of plain, rolled and latent fingerprints. We first propose a new convolutional neural network model initialised with random weights and train the model on fingerprint images. Then we fine-tune two pre-trained convolutional neural network models on fingerprint images. Finally, we compare these three models: two pre-trained models and a custom convolutional neural network model initialised with random weights. We show that pre-trained models can achieve the state-of-the-art results on other similar tasks with no or little fine-tuning. We also show that training data size and depth of the network have a serious impact on the overall performance of deep networks. Dropout is used to enhance the performance of deep networks where the labelled training data is not of sufficient size. All the three models trained on NIST DB4 fingerprint and IIIT-D latent fingerprint databases report good accuracy. By only fine-tuning the pre-trained convolutional neural network model, we get the accuracy of 99%, easily out-performing the state-of-the-art.

A Multiple-Feature Reuse Network to Extract Buildings from Remote Sensing Imagery

Automatic building extraction from remote sensing imagery is important in many applications. The success of convolutional neural networks (CNNs) has also led to advances in using CNNs to extract man-made objects from high-resolution imagery. However, the large appearance and size variations of buildings make it difficult to extract both crowded small buildings and large buildings. High-resolution imagery must be segmented into patches for CNN models due to GPU memory limitations, and buildings are typically only partially contained in a single patch with little context information. To overcome the problems involved when using different levels of image features with common CNN models, this paper proposes a novel CNN architecture called a multiple-feature reuse network (MFRN) in which each layer is connected to all the subsequent layers of the same size, enabling the direct use of the hierarchical features in each layer. In addition, the model includes a smart decoder that enables precise localization with less GPU load. We tested our model on a large real-world remote sensing dataset and obtained an overall accuracy of 94.5% and an 85% F1 score, which outperformed the compared CNN models, including a 56-layer fully convolutional DenseNet with 93.8% overall accuracy and an F1 score of 83.5%. The experimental results indicate that the MFRN approach to connecting convolutional layers improves the performance of common CNN models for extracting buildings of different sizes and can achieve high accuracy with a consumer-level GPU.

Automated prediction of dosimetric eligibility of patients with prostate cancer undergoing intensity-modulated radiation therapy using a convolutional neural network.

The quality of radiotherapy has greatly improved due to the high precision achieved by intensity-modulated radiation therapy (IMRT). Studies have been conducted to increase the quality of planning and reduce the costs associated with planning through automated planning method; however, few studies have used the deep learning method for optimization of planning. The purpose of this study was to propose an automated method based on a convolutional neural network (CNN) for predicting the dosimetric eligibility of patients with prostate cancer undergoing IMRT. Sixty patients with prostate cancer who underwent IMRT were included in the study. Treatment strategy involved division of the patients into two groups, namely, meeting all dose constraints and not meeting all dose constraints, by experienced medical physicists. We used AlexNet (i.e., one of common CNN architectures) for CNN-based methods to predict the two groups. An AlexNet CNN pre-trained on ImageNet was fine-tuned. Two dataset formats were used as input data: planning computed tomography (CT) images and structure labels. Five-fold cross-validation was used, and performance metrics included sensitivity, specificity, and prediction accuracy. Class activation mapping was used to visualize the internal representation learned by the CNN. Prediction accuracies of the model with the planning CT image dataset and that with the structure label dataset were 56.7 ± 9.7% and 70.0 ± 11.3%, respectively. Moreover, the model with structure labels focused on areas associated with dose constraints. These results revealed the potential applicability of deep learning to the treatment planning of patients with prostate cancer undergoing IMRT.

Multi-feature data fusion based on common space model and recurrent convolutional neural networks for EEG tristimania recognition used in upper limb rehabilitation exercises

Multi-feature data fusion based on common space model and recurrent convolutional neural networks for EEG tristimania recognition used in upper limb rehabilitation exercises

Auto-Balanced Filter Pruning for Efficient Convolutional Neural Networks

In recent years considerable research efforts have been devoted to compression techniques of convolutional neural networks (CNNs). Many works so far have focused on CNN connection pruning methods which produce sparse parameter tensors in convolutional or fully-connected layers. It has been demonstrated in several studies that even simple methods can effectively eliminate connections of a CNN. However, since these methods make parameter tensors just sparser but no smaller, the compression may not transfer directly to acceleration without support from specially designed hardware. In this paper, we propose an iterative approach named Auto-balanced Filter Pruning, where we pre-train the network in an innovative auto-balanced way to transfer the representational capacity of its convolutional layers to a fraction of the filters, prune the redundant ones, then re-train it to restore the accuracy. In this way, a smaller version of the original network is learned and the floating-point operations (FLOPs) are reduced. By applying this method on several common CNNs, we show that a large portion of the filters can be discarded without obvious accuracy drop, leading to significant reduction of computational burdens. Concretely, we reduce the inference cost of LeNet-5 on MNIST, VGG-16 and ResNet-56 on CIFAR-10 by 95.1%, 79.7% and 60.9%, respectively.

Relative location prediction in CT scan images using convolutional neural networks

Background and objectiveRelative location prediction in computed tomography (CT) scan images is a challenging problem. Many traditional machine learning methods have been applied in attempts to alleviate this problem. However, the accuracy and speed of these methods cannot meet the requirement of medical scenario. In this paper, we propose a regression model based on one-dimensional convolutional neural networks (CNN) to determine the relative location of a CT scan image both quickly and precisely. MethodsIn contrast to other common CNN models that use a two-dimensional image as an input, the input of this CNN model is a feature vector extracted by a shape context algorithm with spatial correlation. Normalization via z-score is first applied as a pre-processing step. Then, in order to prevent overfitting and improve model's performance, 20% of the elements of the feature vectors are randomly set to zero. This CNN model consists primarily of three one-dimensional convolutional layers, three dropout layers and two fully-connected layers with appropriate loss functions. ResultsA public dataset is employed to validate the performance of the proposed model using a 5-fold cross validation. Experimental results demonstrate an excellent performance of the proposed model when compared with contemporary techniques, achieving a median absolute error of 1.04 cm and mean absolute error of 1.69 cm. The time taken for each relative location prediction is approximately 2 ms. ConclusionResults indicate that the proposed CNN method can contribute to a quick and accurate relative location prediction in CT scan images, which can improve efficiency of the medical picture archiving and communication system in the future.

Multi-Pixel Simultaneous Classification of PolSAR Image Using Convolutional Neural Networks.

Convolutional neural networks (CNN) have achieved great success in the optical image processing field. Because of the excellent performance of CNN, more and more methods based on CNN are applied to polarimetric synthetic aperture radar (PolSAR) image classification. Most CNN-based PolSAR image classification methods can only classify one pixel each time. Because all the pixels of a PolSAR image are classified independently, the inherent interrelation of different land covers is ignored. We use a fixed-feature-size CNN (FFS-CNN) to classify all pixels in a patch simultaneously. The proposed method has several advantages. First, FFS-CNN can classify all the pixels in a small patch simultaneously. When classifying a whole PolSAR image, it is faster than common CNNs. Second, FFS-CNN is trained to learn the interrelation of different land covers in a patch, so it can use the interrelation of land covers to improve the classification results. The experiments of FFS-CNN are evaluated on a Chinese Gaofen-3 PolSAR image and other two real PolSAR images. Experiment results show that FFS-CNN is comparable with the state-of-the-art PolSAR image classification methods.

Recurrent Neural Networks to Correct Satellite Image Classification Maps

While initially devised for image categorization, convolutional neural networks (CNNs) are being increasingly used for the pixelwise semantic labeling of images. However, the proper nature of the most common CNN architectures makes them good at recognizing but poor at localizing objects precisely. This problem is magnified in the context of aerial and satellite image labeling, where a spatially fine object outlining is of paramount importance. Different iterative enhancement algorithms have been presented in the literature to progressively improve the coarse CNN outputs, seeking to sharpen object boundaries around real image edges. However, one must carefully design, choose and tune such algorithms. Instead, our goal is to directly learn the iterative process itself. For this, we formulate a generic iterative enhancement process inspired from partial differential equations, and observe that it can be expressed as a recurrent neural network (RNN). Consequently, we train such a network from manually labeled data for our enhancement task. In a series of experiments we show that our RNN effectively learns an iterative process that significantly improves the quality of satellite image classification maps.

Application of Convolutional Neural Networks to Four-Class Motor Imagery Classification Problem

In this paper the use of a novel feature extraction method oriented to convolutional neural networks (CNN) is discussed in order to solve four-class motor imagery classification problem. Analysis of viable CNN architectures and their influence on the obtained accuracy for the given task is argued. Furthermore, selection of optimal feature map image dimension, filter sizes and other CNN parameters used for network training is investigated. Methods for generating 2D feature maps from 1D feature vectors are presented for commonly used feature types. Initial results show that CNN can achieve high classification accuracy of 68% for the four-class motor imagery problem with less complex feature extraction techniques. It is shown that optimal accuracy highly depends on feature map dimensions, filter sizes, epoch count and other tunable factors, therefore various fine-tuning techniques must be employed. Experiments show that simple FFT energy map generation techniques are enough to reach the state of the art classification accuracy for common CNN feature map sizes. This work also confirms that CNNs are able to learn a descriptive set of information needed for optimal electroencephalogram (EEG) signal classification.DOI: http://dx.doi.org/10.5755/j01.itc.46.2.17528