Deep Convolutional Neural Network Structure Research Articles

Research on Inception Module Incorporated Siamese Convolutional Neural Networks to Realize Face Recognition

Face recognition is an active research subject of biometrics due to its significant research and application prospects. The performance of face recognition can be affected by a series of uncontrollable factors, such as illumination, expression, posture and occlusion, which restricts its real-world applications. Therefore, improving the robustness of face recognition to environmental changes became an urgent problem. In this paper, a simplified deep convolutional neural network structure having high robustness under unlimited conditions is designed for face recognition. This structure can improve training speed and face recognition accuracy, and be suitable for small-scale data sets. Inception Module Incorporated Siamese Convolutional Neural Networks (IMISCNN) is developed based on effective reduction of external interference and better features extraction by adopting the Siamese network structure. A cyclical learning rate strategy is also introduced in IMISCNN for better model convergence. Compared to classical face recognition algorithms, such as PCA, PCA and SVM, CNN, PCANet, and the original SNN et al. The accuracy of IMISCNN in CASIA-webface and Extended Yale B standard face database is 99.36% and 99.21%, respectively. Its feasibility and effectiveness have been verified in our experiments.

Efficient Outdoor Video Semantic Segmentation Using Feedback-Based Fully Convolution Neural Network

In this article, we focus on efficient semantic segmentation problem from sequential two-dimensional images, in which all pixels are classified into certain classes for scene understanding. Such problem is challenging because it involves constraints of both spatial and temporal consistencies, which have large difficulties in explicitly determining such structural constraints. Traditionally, such a problem is tackled using structured prediction method, such as conditional random field (CRF). However, pure CRF method suffers from very high complexity in computing high-order potentials and slow performance during inference step, which is unsuitable for efficient video segmentation in real scenario. In this article, a novel feedback-based deep fully convolutional neural network (CNN) is proposed to inherently incorporate spatial context through appending output feedback mechanism. The proposed method has the following contributions: 1) spatial context in images are easily captured through iterative feedback refinement, without the expensive postprocess step such as CRF refinement; 2) easily integrated with generic deep CNN structure; and 3) the inference time is greatly reduced for efficient image segmentation. Compared to current state-of-the-art methods, our proposed method was shown to provide up to 14% better accuracy on semantic segmentation task in challenging Camvid and Cityscapes datasets, while taking up to relatively 980% shorter inference time. The proposed method also shows its effectiveness for real-time road detection task of autonomous driving.

A Fast and Robust Deep Convolutional Neural Networks for Complex Human Activity Recognition Using Smartphone.

As a significant role in healthcare and sports applications, human activity recognition (HAR) techniques are capable of monitoring humans’ daily behavior. It has spurred the demand for intelligent sensors and has been giving rise to the explosive growth of wearable and mobile devices. They provide the most availability of human activity data (big data). Powerful algorithms are required to analyze these heterogeneous and high-dimension streaming data efficiently. This paper proposes a novel fast and robust deep convolutional neural network structure (FR-DCNN) for human activity recognition (HAR) using a smartphone. It enhances the effectiveness and extends the information of the collected raw data from the inertial measurement unit (IMU) sensors by integrating a series of signal processing algorithms and a signal selection module. It enables a fast computational method for building the DCNN classifier by adding a data compression module. Experimental results on the sampled 12 complex activities dataset show that the proposed FR-DCNN model is the best method for fast computation and high accuracy recognition. The FR-DCNN model only needs 0.0029 s to predict activity in an online way with 95.27% accuracy. Meanwhile, it only takes 88 s (average) to establish the DCNN classifier on the compressed dataset with less precision loss 94.18%.

Study of the Application of Deep Convolutional Neural Networks (CNNs) in Processing Sensor Data and Biomedical Images.

The fast progress in research and development of multifunctional, distributed sensor networks has brought challenges in processing data from a large number of sensors. Using deep learning methods such as convolutional neural networks (CNN), it is possible to build smarter systems to forecasting future situations as well as precisely classify large amounts of data from sensors. Multi-sensor data from atmospheric pollutants measurements that involves five criteria, with the underlying analytic model unknown, need to be categorized, so do the Diabetic Retinopathy (DR) fundus images dataset. In this work, we created automatic classifiers based on a deep convolutional neural network (CNN) with two models, a simpler feedforward model with dual modules and an Inception Resnet v2 model, and various structural tweaks for classifying the data from the two tasks. For segregating multi-sensor data, we trained a deep CNN-based classifier on an image dataset extracted from the data by a novel image generating method. We created two deepened and one reductive feedforward network for DR phase classification. The validation accuracies and visualization results show that increasing deep CNN structure depth or kernels number in convolutional layers will not indefinitely improve the classification quality and that a more sophisticated model does not necessarily achieve higher performance when training datasets are quantitatively limited, while increasing training image resolution can induce higher classification accuracies for trained CNNs. The methodology aims at providing support for devising classification networks powering intelligent sensors.

End-to-End Automatic Image Annotation Based on Deep CNN and Multi-Label Data Augmentation

Automatic image annotation is a key step in image retrieval and image understanding. In this paper, we present an end-to-end automatic image annotation method based on a deep convolutional neural network (CNN) and multi-label data augmentation. Different from traditional annotation models that usually perform feature extraction and annotation as two independent tasks, we propose an end-to-end automatic image annotation model based on deep CNN (E2E-DCNN). E2E-DCNN transforms the image annotation problem into a multi-label learning problem. It uses a deep CNN structure to carry out the adaptive feature learning before constructing the end-to-end annotation structure using multiple cross-entropy loss functions for training. It is difficult to train a deep CNN model using small-scale datasets or scale up multi-label datasets using traditional data augmentation methods; hence, we propose a multi-label data augmentation method based on Wasserstein generative adversarial networks (ML-WGAN). The ML-WGAN generator can approximate the data distribution of a single multi-label image. The images generated by ML-WGAN can assist in the reduction of the over-fitting problem of training a deep CNN model and enhance the generalization ability of the trained CNN model. We optimize the network structure by using deformable convolution and spatial pyramid pooling. We experiment the proposed E2E-DCNN model with data augmentation by the proposed ML-WGAN on several public datasets. The experimental results demonstrate that the proposed model outperforms the state-of-the-art automatic image annotation models.

Normalization in Training U-Net for 2-D Biomedical Semantic Segmentation

2D biomedical semantic segmentation is important for robotic vision in surgery. Segmentation methods based on Deep Convolutional Neural Network (DCNN) can out-perform conventional methods in terms of both accuracy and levels of automation. One common issue in training a DCNN for biomedical semantic segmentation is the internal covariate shift where the training of convolutional kernels is encumbered by the distribution change of input features, hence both the training speed and performance are decreased. Batch Normalization (BN) is the first proposed method for addressing internal covariate shift and is widely used. Instance Normalization (IN) and Layer Normalization (LN) have also been proposed. Group Normalization (GN) is proposed more recently and has not yet been applied to 2D biomedical semantic segmentation, however, no specific validations on GN were given. Most DCNNs for biomedical semantic segmentation adopt BN as the normalization method by default, without reviewing its performance. In this paper, four normalization methods - BN, IN, LN and GN are compared in details, specifically for 2D biomedical semantic segmentation. U-Net is adopted as the basic DCNN structure. Three datasets regarding the Right Ventricle (RV), aorta, and Left Ventricle (LV) are used for the validation. The results show that detailed subdivision of the feature map, i.e. GN with a large group number or IN, achieves higher accuracy. This accuracy improvement mainly comes from better model generalization. Codes are uploaded and maintained at Xiao-Yun Zhou's Github.

Transferring Ensemble Representations Using Deep Convolutional Neural Networks for Small-Scale Image Classification

The deep convolutional neural networks (DCNN) require large number of training data to avoid overfitting, which makes it unsuitable for processing small-scale image datasets. The transfer learning using DCNN (TCNN) reuses pre-trained layers to generate a mid-level image representation so that the optimization of more than millions CNN parameters can be avoided. By this way, overfitting problem in small-scale data can be alleviated. However, although now many public DCNNs have been trained and can be reused, the existing TCNNs are formed by only a single pre-trained DCNN structure and cannot make full use of multiple structures of pre-trained DCNNs. At the same time, the existing ensemble CNNs have not enough good representation ability. To address this problem, we combine the conventional ideas of ensemble CNNs and propose three ensemble TCNNs (TECNN). They are the voting method based on the combination of all TCNNs, the PickOver method by finding the optimal combination, and weighted method by finding weighted combination. Different from the existing ensemble CNNs, the proposed methods do not need to retrain the component CNNs and generate ensemble transferring representations by transferring the pre-trained mid-level parameters. The mathematical models of those three methods are also provided. Their versions of using fine-tuning are also compared in the experiments. In addition, we replace the Softmax classifier with ensemble linear classifiers in the full-connection layer. They outperform the current state of the art algorithms on Caltech ImageNet and some internet image data. All this research has released as an open source library called Transferring Image Ensemble Representations using Deep Convolutional Neural Networks (TECNN). The source codes and relevant datasets in different versions are available from: http://www.cquptshuyinxia.com/TECNN.html .

Ensemble Learning for Facial Age Estimation Within Non-Ideal Facial Imagery

Human facial age estimation has been widely used in many computer vision applications, including security surveillance, forensics, biometrics, human–computer interaction (HCI), and so on. We propose a facial age estimation method oriented to non-ideal facial imagery. The method consists of image preprocessing, feature extraction, and age predication. First, we preprocess non-ideal input images in RGB stream, luminance modified (LM) stream, and YIQ stream. Then, we leverage the deep convolutional neural networks (DCNNs) to extract the feature of images preprocessed in each stream. To reduce the training data volume and training complexity, we adopt the transfer learning to build the DCNN structure. With the extracted feature, the weak classifier equipped at every stream is designed to obtain a weak classification prediction of the age range. Moreover, in order to generate estimation, we use the ensemble learning to fuse the three weak classifiers. We design an integrated strategy algorithm based on the combination of voting method and weighted average method. The simulation results show that our proposed algorithm can improve the an exact match (AEM) and an error of one age category (AEO) by 4.75% and 6.75% compared with the best AEM and AEO of the three weak classifiers. Furthermore, in comparison with the unweighted average method, our proposed algorithm can improve the AEM and AEO by 8.68% and 12.79%, respectively.

Retinal Vessels Segmentation Based on Dilated Multi-Scale Convolutional Neural Network

Accurate segmentation of retinal vessels is a basic step in diabetic retinopathy (DR) detection. Most methods based on deep convolutional neural network (DCNN) have small receptive fields, and hence they are unable to capture global context information of larger regions, with difficult to identify pathological. The final segmented retina vessels contain more noise with low classification accuracy. Therefore, in this paper, we propose a DCNN structure named as D-Net. In the encoding phase, we reduced the loss of feature information by reducing the downsampling factor, which reduced the difficulty of tiny thin vessels segmentation. We use the combined dilated convolution to effectively enlarge the receptive field of the network and alleviate the “grid problem” that exists in the standard dilated convolution. In the proposed multi-scale information fusion module (MSIF), parallel convolution layers with different dilation rates are used, so that the model can obtain more dense feature information and better capture retinal vessel information of different sizes. In the decoding module, the skip layer connection is used to propagate context information to higher resolution layers, so as to prevent low-level information from passing the entire network structure. Finally, our method was verified on DRIVE, STARE, and CHASE dataset. The experimental results show that our network structure outperforms some state-of-art method, such as N 4 -fields, U-Net, and DRIU in terms of accuracy, sensitivity, specificity, and ${AUC_{ROC}}$ . Particularly, D-Net outperforms U-Net by 1.04 %, 1.23 %, and 2.79% in DRIVE, STARE, and CHASE dataset, respectively.

Deep Semantic Segmentation of Kidney and Space-Occupying Lesion Area Based on SCNN and ResNet Models Combined with SIFT-Flow Algorithm.

Renal segmentation is one of the most fundamental and challenging task in computer aided diagnosis systems. In order to overcome the shortcomings of automatic kidney segmentation based on deep network for abdominal CT images, a two-stage semantic segmentation of kidney and space-occupying lesion area based on SCNN and ResNet models combined with SIFT-flow transformation is proposed in paper, which is divided into two stages: image retrieval and semantic segmentation. To facilitate the image retrieval, a metric learning-based approach is firstly adopted to construct a deep convolutional neural network structure using SCNN and ResNet network to extract image features and minimize the impact of interference factors on features, so as to obtain the ability to represent the abdominal CT scan image with the same angle under different imaging conditions. And then, SIFT Flow transformation is introduced, which adopts MRF to fuse label information, priori spatial information and smoothing information to establish the dense matching relationship of pixels so that the semantics can be transferred from the known image to the target image so as to obtain the semantic segmentation result of kidney and space-occupying lesion area. In order to validate effectiveness and efficiency of our proposed method, we conduct experiments on self-establish CT dataset, focus on kidney organ and most of which have tumors inside of the kidney, and abnormal deformed shape of kidney. The experimental results qualitatively and quantitatively show that the accuracy of kidney segmentation is greatly improved, and the key information of the proportioned tumor occupying a small area of the image are exhibited a good segmentation results. In addition, our algorithm has also achieved ideal results in the clinical verification, which is suitable for intelligent medicine equipment applications.

Automatic recognition and classification of multi-channel microseismic waveform based on DCNN and SVM

The recognition and classification of the multi-channel microseismic waveform are important for mine hazard prediction. It is widely used to design the corresponding waveform feature for recognition and classification of the microseismic waveform by hand. The process of designing features manually is arduous and the results of recognition and classification are not ideal. In this paper, we propose a method combining Deep Convolutional Neural Networks (DCNN) with Support Vector Machine (SVM) for identifying the microseismic waveform automatically. We constructed a DCNN structure to train the optimal weight model named the DCNN-Model. The DCNN-Model is used as a tool for extracting features from multi-channel waveforms. After combining the extracted features, we used SVM to classify multi-channel waveforms. We compared the outputs of other classifiers, such as Random Forest and k-Nearest Neighbor (KNN). To extend the dataset of DCNN training and extract the essential characteristics of waveform images more accurately, we pre-process the raw data by means of filtering and de-nosing. The experiment shows that the recognition and classification method is of practical value, and the accuracy rate can reach as high as 98.18%.

Comparison of convolutional neural network models for food image classification

ABSTRACTAccording to some estimates of World Health Organization, in 2014, more than 1.9 billion adults were overweight. About 13% of the world’s adult population were obese. 39% of adults were overweight. The worldwide prevalence of obesity more than doubled between 1980 and 2014. Nowadays, mobile applications recording food intake of people become popular. If an improved food classification system is introduced, users take the photo of their meals and system classifies photos into the categories. Hence, we proposed a deep convolutional neural network structure trained from scratch and compared its performance with pre-trained structures Alexnet and Caffenet in INISTA 2017. This study is the extended version of it. Three different deep convolutional neural networks were trained from scratch by using different learning methods: stochastic gradient descent, Nesterov’s accelerated gradient and Adaptive Moment Estimation, and compared with Alexnet and Caffenet fine-tuned with the same learning algorithms. Train, validation and test datasets were generated from Food11 and Food101 datasets. All tests were implemented through NVIDIA Digit interface on GeForce GTX1070. According to the test results, although pre-trained models provided better results than proposed structures, their performances were comparable. Moreover, learning optimization methods accelerated and improved the performances of all the compared models.

Extensive exploration of comprehensive vehicle attributes using D‐CNN with weighted multi‐attribute strategy

As a classical machine learning method, multi-task learning (MTL) has been widely applied in computer vision technology. Due to deep convolutional neural network (D-CNN) having strong ability of feature representation, the combination of MTL and D-CNN has attracted much attention from researchers recently. However, this kind of combination has rarely been explored in the field of vehicle analysis. The authors propose a D-CNN enhanced with weighted multi-attribute strategy for extensive exploration of comprehensive vehicle attributes over surveillance images. Specifically, regarding to recognising vehicle model and make/manufacturer, several related attributes as auxiliary tasks are incorporated in the training process of D-CNN structure. The proposed strategy focuses more on the main task compared with traditional MTL methods, which has assigned different weights for the main task and auxiliary tasks rather than treating all involved tasks equally. To the extent of their knowledge, this is the first report relating to the combination of D-CNN and weighted MTL for exploration of comprehensive vehicle attributes. The following experiments will show that the proposed approach outperforms the state-of-the-art method for the vehicle recognition and improves the accuracy rate by about 10% for the analysis of other vehicle attributes on the recently public CompCars dataset.