Global Average Pooling Layer Research Articles

A method of inpainting moles and acne on the high‐resolution face photos

With the rapid development of mobile phones, more and more high-resolution photos are taken. The demand for high-resolution image inpainting is becoming increasingly urgent. In order to repair high-resolution face images automatically and quickly, this paper proposes an improved generative adversarial networks method. Firstly, we made a high-resolution dataset for training and testing, and abandoned the traditional 256 ×256 size data. Secondly, since the existing methods can only repair the mask with fixed size and shape on the image, when the global average pooling layer is used in the network, the improved network can repair the moles and acne with arbitrary sizes and shapes on the human face photos. Finally, in order to achieve optimal performance of the network, a mixed loss function is used in training. The experimental results prove that our method has not only achieved good results in qualitative results, but also achieved excellent results in quantitative results.

Combination of temporal‐channels correlation information and bilinear feature for action recognition

In this study, the authors focus on improving the spatio–temporal representation ability of three‐dimensional (3D) convolutional neural networks (CNNs) in the video domain. They observe two unfavourable issues: (i) the convolutional filters only dedicate to learning local representation along input channels. Also they treat channel‐wise features equally, without emphasising the important features; (ii) traditional global average pooling layer only captures first‐order statistics, ignoring finer detail features useful for classification. To mitigate these problems, they proposed two modules to boost 3D CNNs’ performance, which are temporal‐channel correlation (TCC) and bilinear pooling module. The TCC module can capture the information of inter‐channel correlations over the temporal domain. Moreover, the TCC module generates channel‐wise dependencies, which can adaptively re‐weight the channel‐wise features. Therefore, the network can focus on learning important features. With regards to the bilinear pooling module, it can capture more complex second‐order statistics in deep features and generate a second‐order classification vector. We can get more accurate classification results by combining the first‐order and second‐order classification vector. Extensive experiments show that adding our proposed modules to I3D network could consistently improve the performance and outperform the state‐of‐the‐art methods. The code and models are available at https://github.com/caijh33/I3D_TCC_Bilinear.

Improvement of the AlexNet Networks for Large-Scale Recognition Applications

In this paper, the deep convolutional neural networks (DCNNs) are studied to perform the complex feature extraction on the image in the convolution layer and to improve the final test accuracy of the network. By improving the DCNNs algorithm and framework, it can enhance the accurate extraction of the image features. We replace the fully connection layer of the original network with the global average pooling layer. In the absence of the large number of calculations of network parameters, the final effect is not changed; thereby, it increases the speed of the network. The simulation result is given to show the effectiveness of the DCNNs algorithm by comparing the training accuracy and test accuracy of the five improvement algorithms.

Depth-Wise Separable Convolution Neural Network with Residual Connection for Hyperspectral Image Classification

The neural network-based hyperspectral images (HSI) classification model has a deep structure, which leads to the increase of training parameters, long training time, and excessive computational cost. The deepened network models are likely to cause the problem of gradient disappearance, which limits further improvement for its classification accuracy. To this end, a residual unit with fewer training parameters were constructed by combining the residual connection with the depth-wise separable convolution. With the increased depth of the network, the number of output channels of each residual unit increases linearly with a small amplitude. The deepened network can continuously extract the spectral and spatial features while building a cone network structure by stacking the residual units. At the end of executing the model, a 1 × 1 convolution layer combined with a global average pooling layer can be used to replace the traditional fully connected layer to complete the classification with reduced parameters needed in the network. Experiments were conducted on three benchmark HSI datasets: Indian Pines, Pavia University, and Kennedy Space Center. The overall classification accuracy was 98.85%, 99.58%, and 99.96% respectively. Compared with other classification methods, the proposed network model guarantees a higher classification accuracy while spending less time on training and testing sample sites.

Utilization of smartphone and tablet camera photographs to predict healing of diabetes-related foot ulcers

The objective of this study was to build a machine learning model that can predict healing of diabetes-related foot ulcers, using both clinical attributes extracted from electronic health records (EHR) and image features extracted from photographs. The clinical information and photographs were collected at an academic podiatry wound clinic over a three-year period. Both hand-crafted color and texture features and deep learning-based features from the global average pooling layer of ResNet-50 were extracted from the wound photographs. Random Forest (RF) and Support Vector Machine (SVM) models were then trained for prediction. For prediction of eventual wound healing, the models built with hand-crafted imaging features alone outperformed models built with clinical or deep-learning features alone. Models trained with all features performed comparatively against models trained with hand-crafted imaging features. Utilization of smartphone and tablet photographs taken outside of research settings hold promise for predicting prognosis of diabetes-related foot ulcers.

Blind Image Quality Measurement by Exploiting High-Order Statistics With Deep Dictionary Encoding Network

Blind image quality measurement (BIQM) has achieved great progress due to the deployment of deep neural networks (DNNs) for training end-to-end models. Most of the existing DNN-based BIQM methods simply aggregate the local deep feature maps with a global max or average pooling layer to generate holistic feature vectors for quality prediction. However, such pooling strategy fails to capture the high-order statistics of local feature descriptors. Inspired by the success of dictionary encoding-based BIQM methods, this article proposes a deep dictionary encoding network (Deep-DEN) that can well capture the high-order statistics of local deep features in an end-to-end manner. In Deep-DEN, dictionary encoding is encapsulated into a single learnable layer attached to the end of a backbone network and followed by a fully connected layer for quality prediction. As a result, high-order statistics of the extracted local deep features in the backbone network and quality prediction functions are simultaneously optimized in a fully supervised manner. The performance of Deep-DEN has been extensively evaluated on several benchmarks and the superiority has been well validated by comparisons with other state-of-the-art BIQM methods.

A deep neural network compression algorithm based on knowledge transfer for edge devices

The computation and storage capacity of the edge device are limited, which seriously restrict the application of deep neural network in the device. Toward to the intelligent application of the edge device, we introduce the deep neural network compression algorithm based on knowledge transfer, a three-stage pipeline: lightweight, multi-level knowledge transfer and pruning that reduce the network depth, parameter and operation complexity of the deep learning neural networks. We lighten the neural networks by using a global average pooling layer instead of a fully connected layer and replacing a standard convolution with separable convolutions. Next, the multi-level knowledge transfer minimizes the difference between the output of the ”student network” and the ”teacher network” in the middle and logits layer, increasing the supervised information when training the ”student network”. Lastly, we prune the network by cutting off the unimportant convolution kernels with a global iterative pruning strategy. The experiment results show that the proposed method improve the efficiency up to 30% than the knowledge distillation method in reducing the loss of classification performance. Benchmarked on GPU (Graphics Processing Unit) server, Raspberry Pi 3 and Cambricon-1A, the parameters of the compressed network after using our knowledge transfer and pruning method have achieved more than 49.5 times compression and the time efficiency of a single feedforward operation has been improved more than 3.2 times.

Improved VGG model-based efficient traffic sign recognition for safe driving in 5G scenarios

The rapid development and application of AI in intelligent transportation systems has widely impacted daily life. The application of an intelligent visual aid for traffic sign information recognition can provide assistance and even control vehicles to ensure safe driving. The field of autonomous driving is booming, and great progress has been made. Many traffic sign recognition algorithms based on convolutional neural networks (CNNs) have been proposed because of the fast execution and high recognition rate of CNNs. However, this work addresses a challenging question in the autonomous driving field: how can traffic signs be recognized in real time and accurately? The proposed method designs an improved VGG convolutional neural network and has significantly superior performance compared with existing schemes. First, some redundant convolutional layers are removed efficiently from the VGG-16 network, and the number of parameters is greatly reduced to further optimize the overall architecture and accelerate calculation. Furthermore, the BN (batch normalization) layer and GAP (global average pooling) layer are added to the network to improve the accuracy without increasing the number of parameters. The proposed method needs only 1.15 M when using the improved VGG-16 network. Finally, extensive experiments on the German Traffic Sign Recognition Benchmark (GTSRB) Dataset are performed to evaluate our proposed scheme. Compared with traditional methods, our scheme significantly improves recognition accuracy while maintaining good real-time performance.

A resource conscious human action recognition framework using 26-layered deep convolutional neural network

Vision-based human action recognition (HAR) is a hot topic of research from the decade due to a few popular applications such as visual surveillance and robotics. For correct action recognition, various local and global points are requires known as features. These features modified during the variation in human movement. But due to a bit change in several human actions, the features of these actions are mixed that degrade the recognition performance. In this article, we design a new 26-layered Convolutional Neural Network (CNN) architecture for accurate complex action recognition. The features are extracted from the global average pooling layer and fully connected (FC) layer, and fused by a proposed high entropy-based approach. Further, we propose a feature selection method name Poisson distribution along with Univariate Measures (PDaUM). Few of fused CNN features are irrelevant, and few of them are redundant that makes the incorrect prediction among complex human actions. Therefore, the proposed PDaUM based approach selects only the strongest features that later passed to the Extreme Learning Machine (ELM) and Softmax for final recognition. Four datasets are using for experimental analysis - HMDB51 (51 classes), UCF Sports (10 classes), KTH (6 classes), and Weizmann (10 classes). On these datasets, the ELM classifier gives an improved performance as compared to a Softmax classifier. The achieved accuracy on each dataset is 81.4%, 99.2%, 98.3%, and 98.7%, respectively. Comparison with existing techniques, it is shown that the proposed architecture gives better performance in terms of accuracy and testing time.

Deep Learning-Based Solar-Cell Manufacturing Defect Detection With Complementary Attention Network

The automatic defects detection for solar cell electroluminescence (EL) images is a challenging task, due to the similarity of defect features and complex background features. To address this problem, in this article a novel complementary attention network (CAN) is designed by connecting the novel channel-wise attention subnetwork with spatial attention subnetwork sequentially, which adaptively suppresses the background noise features and highlights the defect features simultaneously by employing the complementary advantage of the channel features and spatial position features. In CAN, the novel channel-wise attention subnetwork applies convolution operation to integrate the concatenated and discriminative output features extracted by global average pooling layer and global max pooling layer, which can make fully use of these informative features. Furthermore, a region proposal attention network (RPAN) is proposed by embedding CAN into region proposal network in faster R-CNN (convolution neutral network) to extract more refined defective region proposals, which is used to construct a novel end-to-end faster RPAN-CNN framework for detecting defects in raw EL image. Finally, some experimental results on a large-scale EL dataset including 3629 images, 2129 of which are defective, show that the proposed method performs much better than other methods in terms of defects classification and detection results in raw solar cell EL images.

Detection of defects in voltage-dependent resistors using stacked-block-based convolutional neural networks

Voltage-dependent resistors (VDRs) are important circuit-protection devices. Their performance is affected by packaging quality. To identify VDR packaging defects more accurately and efficiently, we have proposed a convolutional neural network (CNN)-based VDR appearance quality inspection method that includes four stages: image acquisition, data augmentation, neural architecture design, and CNN training and testing. In designing the neural architecture, we have proposed two VDR-oriented network blocks, which consist of a compressed subnet and a multiscale subnet. Then, a stacking-block-based neural architecture design (BlockNAD) strategy is employed to determine the number of blocks. The last block is connected to a classification layer composed of a global average pooling (GAP) layer and a full connection (FC) layer. Further, using a VDR dataset containing 8058 images, we compared the identification performances of the candidate networks with different structures on 12 categories of VDR defects by adopting a variety of indicators, such as the mean average precision (mAP) and average test time per sample. The experimental results of the proposed method demonstrate competitive results compared to the state-of-the-art methods in identifying VDR defects, with a mAP value of approximately 99.9% and an average test time per sample of approximately 3 ms.

Visually Interpretable Representation Learning for Depression Recognition from Facial Images

Recent evidence in mental health assessment have demonstrated that facial appearance could be highly indicative of depressive disorder. While previous methods based on the facial analysis promise to advance clinical diagnosis of depressive disorder in a more efficient and objective manner, challenges in visual representation of complex depression pattern prevent widespread practice of automated depression diagnosis. In this paper, we present a deep regression network termed DepressNet to learn a depression representation with visual explanation. Specifically, a deep convolutional neural network equipped with a global average pooling layer is first trained with facial depression data, which allows for identifying salient regions of input image in terms of its severity score based on the generated depression activation map (DAM). We then propose a multi-region DepressNet, with which multiple local deep regression models for different face regions are jointly leaned and their responses are fused to improve the overall recognition performance. We evaluate our method on two benchmark datasets, and the results show that our method significantly boosts state-of-the-art performance of the visual-based depression recognition. Most importantly, the DAM induced by our learned deep model may help reveal the visual depression pattern on faces and understand the insights of automated depression diagnosis.

Two-stream feature aggregation deep neural network for scene classification of remote sensing images

Scene classification of high-spatial resolution (HSR) images has a wide range of potential applications in various fields, and it has become a research hotspot in remote sensing community. Recently, deep transfer learning-based methods have attracted tremendous attention due to powerful ability of feature extraction. In this paper, a novel architecture termed two-stream feature aggregation deep neural network (TFADNN) is developed for HSR scene classification. The TFADNN method contains two parallel parts, including the stream of discriminative features and the stream of general features. In the first stream, the fully connected layers of pre-trained CNNs are replaced by a global average pooling layer to remove the limitation on the size of input images. As for the second stream, the multi-scale nonlinear encoding based bag-of-visual-words (MNBoVW) model is proposed to process convolutional features, and the global representations can be obtained. Then, weighted fusion is adopted to integrate two-stream features. As a result, the TFADNN method can learn the discriminative features from HSR images with arbitrary sizes, and the experimental results on two challenging datasets indicate that the TFADNN method achieves satisfactory classification performance compared with some state-of-the-art methods.

Classification of Heart Sounds Using Convolutional Neural Network

Heart sounds play an important role in the diagnosis of cardiac conditions. Due to the low signal-to-noise ratio (SNR), it is problematic and time-consuming for experts to discriminate different kinds of heart sounds. Thus, objective classification of heart sounds is essential. In this study, we combined a conventional feature engineering method with deep learning algorithms to automatically classify normal and abnormal heart sounds. First, 497 features were extracted from eight domains. Then, we fed these features into the designed convolutional neural network (CNN), in which the fully connected layers that are usually used before the classification layer were replaced with a global average pooling layer to obtain global information about the feature maps and avoid overfitting. Considering the class imbalance, the class weights were set in the loss function during the training process to improve the classification algorithm’s performance. Stratified five-fold cross-validation was used to evaluate the performance of the proposed method. The mean accuracy, sensitivity, specificity and Matthews correlation coefficient observed on the PhysioNet/CinC Challenge 2016 dataset were 86.8%, 87%, 86.6% and 72.1% respectively. The proposed algorithm’s performance achieves an appropriate trade-off between sensitivity and specificity.

The Effect of Different Deep Network Architectures upon CNN-Based Gaze Tracking

In this paper, we explore the effect of using different convolutional layers, batch normalization and the global average pooling layer upon a convolutional neural network (CNN) based gaze tracking system. A novel method is proposed to label the participant’s face images as gaze points retrieved from eye tracker while watching videos for building a training dataset that is closer to human visual behavior. The participants can swing their head freely; therefore, the most real and natural images can be obtained without too many restrictions. The labeled data are classified according to the coordinate of gaze and area of interest on the screen. Therefore, varied network architectures are applied to estimate and compare the effects including the number of convolutional layers, batch normalization (BN) and the global average pooling (GAP) layer instead of the fully connected layer. Three schemes, including the single eye image, double eyes image and facial image, with data augmentation are used to feed into neural network to train and evaluate the efficiency. The input image of the eye or face for an eye tracking system is mostly a small-sized image with relatively few features. The results show that BN and GAP are helpful in overcoming the problem to train models and in reducing the amount of network parameters. It is shown that the accuracy is significantly improved when using GAP and BN at the mean time. Overall, the face scheme has a highest accuracy of 0.883 when BN and GAP are used at the mean time. Additionally, comparing to the fully connected layer set to 512 cases, the number of parameters is reduced by less than 50% and the accuracy is improved by about 2%. A detection accuracy comparison of our model with the existing George and Routray methods shows that our proposed method achieves better prediction accuracy of more than 6%.

Understanding the learning mechanism of convolutional neural networks in spectral analysis

Deep learning approaches, especially convolutional neural network (CNN) models, have achieved excellent performances in vibrational spectral analysis. The critical drawback of the CNN approach is the lack of interpretation, and it is regarded as a black box. Interpreting the learning mechanism of chemometric models is critical for intuitive understanding and further application. In this study, an interpretable CNN model with a global average pooling layer is presented for Raman and mid-infrared spectral data analysis. A class activation mapping (CAM)-based approach is leveraged to visualize the active variables in the whole spectrum. The visualization of active variables shows a discriminative pattern in which the most contributed variables peaked around theoretical chemical characteristic bands. The visualization of the feature maps by three convolutional layers demonstrates the data transformation pipeline and how the CNN model hierarchically extracts informative spectral features. The first layer acts as a Savitzky-Golay filter and learns spectral shape characteristics, while the second layer learns enhanced patterns from typical spectral peaks on a few correlated variables. The third layer shows stable activations on critical spectral peaks. A partial least squares - linear discriminant analysis (PLS-LDA) model is presented for comparison on classification accuracy and model interpretation. The CNN model yields mean classification accuracies of 99.01 and 100% for E. coli and meat datasets on the test set, while the PLS-LDA models obtain accuracies of 98.83 and 100%. Both the CNN and PLS-LDA models demonstrate stable patterns on active variables while CNN models are more stable than PLS-LDA models on classification performances for various dataset partitions with Monte-Carlo cross-validation.

An automatic sorting system for unwashed eggs using deep learning

Egg quality and safety are significant concerns of consumers and modern food industries. This study proposes a novel and precise assessment of egg sorting using a deep convolutional neural network (CNN), which is a state-of-the-art computer vision method to perform classification tasks. To classify unwashed egg images, VGG16 architecture was modified by a global average pooling layer, dense layers, a batch normalization layer, and a dropout layer. The modified model was trained based on intact, bloody, and broken (breakage, crack, or hole on the eggshell) eggs, which were combined with being dirty. Performance evaluation of the CNN model through 5-fold cross-validation showed that it outperforms traditional machine vision-based models. The accuracy, precision, sensitivity, specificity, and area under the curve were 96.55, 95.59, 94.92, 97.39, and 96.16%, respectively. The CNN model achieved an average overall accuracy of 94.84 by 5-fold cross-validation.

Real-time defect detection network for polarizer based on deep learning

Quality analysis of the polarizer of a production line can be performed using image processing technology. The existing method of detecting defective images based on deep learning can ensure accurate classification; however, its detection speed is low, the model requires a large amount of memory, and it is difficult to meet the real-time requirements of online detection systems when hardware resources are limited. Therefore, in this study a lightweight polarizer defect detection network, called DDN, was developed based on deep learning. First, a parallel module was designed to build the network. This module has two main advantages. First, it mixes different convolution template sizes, and can fuse the features of different scales and extract more defect features than the traditional convolution layer. Second, depthwise separable convolution is used to replace full convolution in this module, which significantly reduces the number of parameters and the multiply-accumulate operations. Finally, a global average pooling (GAP) layer is used instead of a fully connected layer. The GAP layer has no parameters to optimize, which substantially reduces the number of network parameters. Experimental results show that the proposed method is better than existing methods in terms of classification speed, precision, and memory consumption for polarizer detection, and can satisfy real-time requirements.

Lightweight efficient network for defect classification of polarizers

SummaryIn industrial production, detecting polarizer defects online and in real time is necessary. Existing methods of detecting polarizer defects based on deep learning can ensure the accuracy of classification; however, there are several issues associated with these methods. These include the models having low detection speed, consuming large amounts of memory, and being difficult to be transplanted into online detection systems. To solve the aforementioned problems, a lightweight efficient network (LWEN) structure based on deep learning was designed, which improves the standard convolution layer and the fully connected (FC) layer to minimize the training model size and increase the speed of classification without reducing the accuracy of classification. First, a new building block, the shunt module, was designed to build the LWEN. Subsequently, a global average pooling layer was used to reduce the spatial resolution to 1 before the FC layer. These key technologies were designed to reduce the number of network parameters and minimize the model size of the network. Experimental results show that the proposed LWEN outperforms the state‐of‐the‐art approaches in terms of classification accuracy, speed, and model size.

A Data-Driven-Based Fault Diagnosis Approach for Electrical Power DC-DC Inverter by Using Modified Convolutional Neural Network With Global Average Pooling and 2-D Feature Image

A novel convolutional neural network namely the modified CNN-GAP model is proposed for fast fault diagnosis of the DC-DC inverter. This method improves the model structure of the traditional CNN by using a global average pooling layer to replace the fully connected layer of 2~3 layers. The improved CNN-GAP method mainly contains an input layer, a feature extraction layer, a global average pooling (GAP) layer, and a Softmax output layer. Firstly, the raw 1-D time-series data directly input into the input layer of the established CNN-GAP diagnosis model. The 2-D feature maps are reconstructed in the input layer. Secondly, the representative features are automatically extracted from the 2-D feature maps by using multiple convolutional layers and pooling layers. Thirdly, the dimension transformation and size compression of the output image of the feature extraction layer is completed by the GAP layer. Finally, the fault diagnosis result of the DC-DC inverter is automatically output in the Softmax output layer. The proposed method is used for diagnosing the open-circuit fault of the IGBT in the isolated DC-DC inverter. The proposed method is more accurate and effective than other mainstream intelligent diagnosis methods including the SVM, KNN, DNN, and traditional CNN. The experiment results show that the diagnostic accuracy is up to 99.95%, and the testing time can reduce by more than 15%. The improved CNN-GAP method could greatly reduce the model parameter quantity of the traditional CNN more than 80%, which is more suitable for rapid fault diagnosis in electronic devices.