Deployment Of Convolutional Neural Networks Research Articles

Diabetic Retinopathy Stage Detection Using CNN and Inception V3

The project explores the deployment of Convolutional Neural Networks (CNN) and the Inception V3 model for the automated detection and classification of diabetic retinopathy stages using fundus images. Recognizing diabetic retinopathy as a leading cause of blindness among the working-age population globally, this research aims to streamline the diagnostic process, traditionally reliant on the manual examination by ophthalmologists. Through the utilization of the DRIVE and STARE datasets, the project benchmarks the performance of CNN and Inception V3 models in accurately categorizing the severity of diabetic retinopathy into five distinct stages. The comparison between these models is grounded on parameters such as accuracy, loss, and predicted value, with findings indicating Inception V3's superiority in both performance metrics and diagnostic precision. This advancement could significantly contribute to early and more accessible detection of diabetic retinopathy, thereby mitigating progression towards blindness. Furthermore, the project underscores the potential of deep learning algorithms in enhancing diagnostic methodologies for retinal diseases, paving the way for future explorations in the field of medical imaging and artificial intelligence.

Improving adversarial transferability through frequency enhanced momentum

The emergence of adversarial examples seriously affects the practical security deployment of convolutional neural networks. The existing attack algorithms perform brilliantly under white-box scenarios, but they show weak transferability when faced with unknown black-box models. Recent studies have revealed that models have different interests in different frequency components of images, and the low-frequency characteristics play a non-negligible role in the decision-making of models. In this article, we present the frequency enhanced momentum iterative attack, called FE-MI-FGSM. Specifically, we use multiple convolution kernels for Gaussian filtering of the image before each gradient update to push the processed image closer to the common decision boundaries of multiple models. Then, the average gradient of the white-box model to these processed images is obtained and used as the perturbation direction to generate adversarial examples with a high success rate of white-box attack and high transferability. The empirical results show that compared with the current mainstream gradient-based methods, our method performs better on both normally trained and adversarially trained models. Besides, our method can combine with gradient-based methods which integrate convergence algorithms or input transformations for the sake of reaching satisfactory improvement of the transferability.

ESSENCE: Exploiting Structured Stochastic Gradient Pruning for Endurance-Aware ReRAM-Based In-Memory Training Systems

Processing-in-memory (PIM) enables energyefficient deployment of convolutional neural networks (CNNs) from edge to cloud. Resistive random-access memory (ReRAM) is one of the most commonly used technologies for PIM architectures. One of the primary limitations of ReRAM-based PIM in neural network training arises from the limited write endurance due to the frequent weight updates. To make ReRAM-based architectures viable for CNN training, the write endurance issue needs to be addressed. This work aims to reduce the number of weight reprogrammings without compromising the final model accuracy. We propose the ESSENCE framework with an endurance-aware structured stochastic gradient pruning method, which dynamically adjusts the probability of gradient update based on the current update counts. Experimental results with multiple CNNs and datasets demonstrate that the proposed method can extend ReRAM’s life time for training. For instance, with the ResNet20 network and CIFAR-10 dataset, ESSENCE can save the mean update counts of up to 10.29× compared to the SGD method and effectively reduce the maximum update counts compared with No Endurance method. Furthermore, aggressive tuning method based on ESSENCE can boost the mean update count savings by up to 14.41×.

Efficient Dynamic Reconfigurable CNN Accelerator for Edge Intelligence Computing on FPGA

This paper proposes an efficient dynamic reconfigurable CNN accelerator (EDRCA) for FPGAs to tackle the issues of limited hardware resources and low energy efficiency in the deployment of convolutional neural networks on embedded edge computing devices. First, a configuration layer sequence optimization method is proposed to minimize the configuration time overhead and improve performance. Second, accelerator templates for dynamic regions are designed to create a unified high-speed interface and enhance operational performance. The dynamic reconfigurable technology is applied on the Xilinx KV260 FPGA platform to design the EDRCA accelerator, resolving the hardware resource constraints in traditional accelerator design. The YOLOV2-TINY object detection network is used to test the EDRCA accelerator on the Xilinx KV260 platform using floating point data. Results at 250 MHz show a computing performance of 75.1929 GOPS, peak power consumption of 5.25 W, and power efficiency of 13.6219 GOPS/W, indicating the potential of the EDRCA accelerator for edge intelligence computing.

VNGEP: Filter pruning based on von Neumann graph entropy

To facilitate the deployment of convolutional neural networks on resource-limited devices, neural network pruning, especially filter pruning, has been shown to be a promising approach. Precise channel importance evaluation requires the extraction of global feature information. In this paper, we propose a novel filter pruning method, VNGEP, which combines intra- and inter-channel information to evaluate channel importance comprehensively. We describe the feature map correlation information from a graph perspective. The feature map tensor is converted into an undirected weighted graph, where the edge weights are the cosine distance between feature maps. To characterize the individual information of the feature maps, we present the feature map energy matrix and embed it into the above undirected weighted graph. Ultimately, the filter importance is measured by the von Neumann graph entropy, thus identifying filters that are less informative and more replaceable by other filters. Moreover, we systematically investigate the computational feasibility and data sensitivity of the proposed scheme. Pruning experimental results for several models on different datasets show the superiority of our approach. For example, with ResNet110, our method achieves a 48.3% and 52.1% reduction in model size and FLOPs, respectively, along with a 1.07% improvement in accuracy over the baseline model on CIFAR10. With ResNet50, we achieve an accuracy improvement of 0.25% over the baseline model on ImageNet, while the model size and FLOPs are reduced by 40.8% and 44.8%, respectively.

Automatic blight disease detection in potato (Solanum tuberosum L.) and tomato (Solanum lycopersicum, L. 1753) plants using deep learning

Early and late blight are two diseases which pose a huge risk to both potato (Solanum tuberosum L.) and tomato (Solanum lycopersicum, L. 1753) crops and make farmers run at a loss. The early and automatic detection of these diseases would save time as well as enable farmers to act quickly on crops which have been affected. Machine learning and deep learning technology provide many solutions for the detection of the blight diseases in affected crops, and are common in the literature. However, explanation methods for such solutions are not common, but are necessary, considering some machine learning models are seen as black boxes. This study proposes a ResNet-9 model which detects the blight disease state of potato and tomato leaf images, which farmers can leverage. With the data obtained from the popular “Plant Village Dataset”, there were 3,990 initial training data samples. After augmenting the training set and a rigorous hyperparameter optimization procedure, the model was trained with these hyperparameter values, and examined on the test set, which contained 1,331 images. A test accuracy of 99.25%, 99.67% overall precision, 99.33% overall recall and 99.33% overall F1-score values were achieved. To fully understand the model, explanations for the proposed model were provided through saliency maps, which showed the reasoning behind the predictions of the model. It was observed that the ResNet-9 model considered the shape of the leaf, diseased areas present and general green areas of the leaf for its predictions and this makes us understand the model predictions better and see that the model behaves as expected. Our results could contribute to the testing and deployment of Convolutional Neural Network (CNN) models for classification of proximal sensing images of potato (Solanum tuberosum L.) and tomato (Solanum lycopersicum, L. 1753) plant leaves. Further studies would benefit from this modeling framework and would have the chance to test several other variables to determine the leaf infections in an earlier stage for crop protection.

Memory optimization at Edge for Distributed Convolution Neural Network

AbstractInternet of Things (IoT) edge intelligence has emerged by optimizing the deep learning (DL) models deployed on resource‐constraint devices for quick decision‐making. In addition, edge intelligence reduces network overload and latency by bringing intelligent analytics closer to the source. On the other hand, DL models need a lot of computing resources. As a result, they have high computational workloads and memory footprint, making it impractical to deploy and execute on IoT edge devices with limited capabilities. In addition, existing layer‐based partitioning methods generate many intermediate results, resulting in a huge memory footprint. In this article, we propose a framework to provide a comprehensive solution that enables the deployment of convolutional neural networks (CNNs) onto distributed IoT devices for faster inference and reduced memory footprint. This framework considers a pretrained YOLOv2 model, and a weight pruning technique is applied to the pre‐trained model to reduce the number of non‐contributing parameters. We use the fused layer partitioning method to vertically partition the fused layers of the CNN and then distribute the partition among the edge devices to process the input. In our experiment, we have considered multiple Raspberry Pi as edge devices. Raspberry Pi with a neural computing stick is a gateway device to combine the results from various edge devices and get the final output. Our proposed model achieved inference latency of 5 to 7 seconds for to fused layer partitioning for five devices with a 9% improvement in memory footprint.

Quantized convolutional neural networks through the lens of partial differential equations

Quantization of convolutional neural networks (CNNs) is a common approach to ease the computational burden involved in the deployment of CNNs, especially on low-resource edge devices. However, fixed-point arithmetic is not natural to the type of computations involved in neural networks. In this work, we explore ways to improve quantized CNNs using PDE-based perspective and analysis. First, we harness the total variation (TV) approach to apply edge-aware smoothing to the feature maps throughout the network. This aims to reduce outliers in the distribution of values and promote piecewise constant maps, which are more suitable for quantization. Secondly, we consider symmetric and stable variants of common CNNs for image classification and graph convolutional networks for graph node classification. We demonstrate through several experiments that the property of forward stability preserves the action of a network under different quantization rates. As a result, stable quantized networks behave similarly to their non-quantized counterparts even though they rely on fewer parameters. We also find that at times, stability even aids in improving accuracy. These properties are of particular interest for sensitive, resource-constrained, low-power or real-time applications like autonomous driving.

Neural network pruning based on channel attention mechanism

Network pruning facilitates the deployment of convolutional neural networks in resource-limited environments by reducing redundant parameters. However, most of the existing methods ignore the differences in the contributions of the output feature maps. In response to the above, we propose a novel neural network pruning method based on the channel attention mechanism. In this paper, we firstly utilise the principal component analysis algorithm to reduce the influence of noisy data on feature maps. Then, we propose an improved Leaky-Squeeze-and-Excitation block to evaluate the contribution of each output feature map using the channel attention mechanism. Finally, we effectively remove lower contribution channels without reducing the model performance as much as possible. Extensive experimental results show that our proposed method achieves significant improvements over the state-of-the-art in terms of FLOPs and parameters reduction with similar accuracies. For example, with VGG-16-baseline, our proposed method reduces parameters by 83.3% and FLOPs by 66.3%, with only a loss of 0.13% in top-5 accuracy. Furthermore, it effectively balances pruning efficiency and prediction accuracy.

Task‐aware swapping for efficient DNN inference on DRAM‐constrained edge systems

Object detection at the edge side is a common task in various environments. The deployment of convolutional neural networks in intelligent edge systems is very challenging because of the highly constrained main-memory space. This study aims at operating neural networks with a reduced memory requirement. The basic idea is that tasks of the same type would involve the same critical subnetwork. We propose identifying the critical network connections by considering the importance of channels. During runtime, the proposed method detects the task types and timely swaps the model parameters of the critical subnetworks from the external storage into dynamic random access memory (DRAM). Compared with conventional network pruning, the proposed approach further reduced the DRAM requirement by 34.6% while maintaining a high inference accuracy.

A Flexible and Efficient FPGA Accelerator for Various Large-Scale and Lightweight CNNs

To enable efficient deployment of convolutional neural networks (CNNs) on embedded platforms for different computer vision applications, several convolution variants have been introduced, such as depthwise convolution (DWCV), transposed convolution (TPCV), and dilated convolution (DLCV). To address the utilization degradation issue occurred in a general convolution engine for these emerging operators, a highly flexible and reconfigurable hardware accelerator is proposed to efficiently support various CNN-based vision tasks. Firstly, to avoid workload imbalance of TPCV, a zero transfer and skipping (ZTS) method is proposed to reorganize the computation process. To eliminate the redundant zero calculations of TPCV and DLCV, a sparsity-alike processing (SAP) method is proposed based on weight-oriented dataflow. Secondly, the DWCV or pooling layers are configured to be directly executed after standard convolutions without external memory accesses. Furthermore, a programmable execution schedule is introduced to gain better flexibility. Finally, the proposed accelerator is evaluated on Intel Arria 10 SoC FPGA. Experimental results show state-of-the-art performance on both large-scale and lightweight CNNs for image segmentation or classification. Specifically, the accelerator can achieve a processing speed up to 339.9 FPS and computational efficiency up to 0.58 GOPS/DSP, which is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3.3\times $ </tex-math></inline-formula> better than the prior art evaluated on the same network.

Searching for Network Width With Bilaterally Coupled Network.

Searching for a more compact network width recently serves as an effective way of channel pruning for the deployment of convolutional neural networks (CNNs) under hardware constraints. To fulfil the searching, a one-shot supernet is usually leveraged to efficiently evaluate the performance w.r.t. different network widths. However, current methods mainly follow a unilaterally augmented (UA) principle for the evaluation of each width, which induces the training unfairness of channels in supernet. In this article, we introduce a new supernet called Bilaterally Coupled Network (BCNet) to address this issue. In BCNet, each channel is fairly trained and responsible for the same amount of network widths, thus each network width can be evaluated more accurately. Besides, we propose to reduce the redundant search space and present the BCNetV2 as the enhanced supernet to ensure rigorous training fairness over channels. Furthermore, we leverage a stochastic complementary strategy for training the BCNet, and propose a prior initial population sampling method to boost the performance of the evolutionary search. We also propose a new open-source width search benchmark on macro structures named Channel-Bench-Macro for the better comparisons of the width search algorithms with MobileNet- and ResNet-like architectures. Extensive experiments on the benchmark datasets demonstrate that our method can achieve state-of-the-art performance.

Study of CNN Capacity Applied to Left Ventricle Segmentation in Cardiac MRI

CNN (Convolutional Neural Network) models have been successfully used for segmentation of the left ventricle (LV) in cardiac MRI (Magnetic Resonance Imaging), providing clinical measurements. In practice, two questions arise with deployment of CNNs: 1) when is it better to use a shallow model instead of a deeper one? 2) how the size of a dataset might change the network performance? We propose a framework to answer them, by experimenting with deep and shallow versions of three U-Net families, trained from scratch in six subsets varying from 100 to 10,000 images, different network sizes, learning rates and regularization values. 1620 models were evaluated using 5-fold cross-validation by loss and DICE. The results indicate that: sample size affects performance more than architecture or hyper-parameters; in small samples the performance is more sensitive to hyper-parameters than architecture; the performance difference between shallow and deeper networks is not the same across families.

A FPGA-based end-to-end acceleration framework for fast deployment of Convolutional Neural Networks

Nowadays, CNNs has delivered the state-of-the-art performance in the field of computer vision, image classification, etc. As CNNs going deeper, it becomes more difficult to implement CNNs applications based on general-purpose computing platforms. Recently, many FPGA-based CNNs accelerators have been proposed, these accelerators achieved high performance on specific CNNs models, however they are somewhat lack of reconfigurability to fit different applications. To deal with this problem, an end-to-end acceleration framework was proposed in this paper, which consists of a parameterized hardware accelerator and a fully automatic software framework. Parallel computation and pipeline optimization are deployed in the hardware design to achieve high performance. Simultaneously, runtime reconfigurability is implemented by using a global register list. By encapsulating the underlying driver, a three-layer software framework is provided for users to deploy their pre-trained models. A typical CNNs model used for handwritten digital recognition was selected to test and verify the accelerator. The experimental result shows that the accelerator can reach a recognition speed of 22.65FPS under the clock frequency of 100MHz, comparing with ARM Cortex-A9 working at 650MHz, it can achieve 25.9 times of acceleration effect, with only 1.59W power consumption.

High-Performance CNN Accelerator on FPGA Using Unified Winograd-GEMM Architecture

Deep neural networks have revolutionized a variety of applications in varying domains like autonomous vehicles, weather forecasting, cancer detection, surveillance, traffic management, and so on. The convolutional neural network (CNN) is the state-of-the-art technique for many machine learning tasks in the image and video processing domains. Deployment of CNNs on embedded systems with lower processing power and smaller power budget is a challenging task. Recent studies have shown the effectiveness of field-programmable gate array (FPGA) as a hardware accelerator for the CNNs that can deliver high performance at low power budgets. Majority of computations in CNNs involve 2-D convolution. Winograd minimal filtering-based algorithm is the most efficient technique for calculating convolution for smaller filter sizes. CNNs also consist of fully connected layers that are computed using general element-wise matrix multiplication (GEMM). In this article, we propose a unified architecture named UniWiG, where both Winograd-based convolution and GEMM can be accelerated using the same set of processing elements. This approach leads to efficient utilization of FPGA hardware resources while computing all layers in the CNN. The proposed architecture shows performance improvement in the range of $1.4\times $ to $4.02\times $ with only 13% additional FPGA resources with respect to the baseline GEMM-based architecture. We have mapped popular CNN models like AlexNet and VGG-16 onto the proposed accelerator and the measured performance compares favorably with other state-of-the-art implementations. We have also analyzed the vulnerability of the accelerator to the side-channel attacks. Preliminary investigations show that the UniWiG architecture is more robust to memory side-channel attacks than direct convolution-based techniques.

A High-Efficiency FPGA-Based Accelerator for Binarized Neural Network

Although the convolutional neural network (CNN) has exhibited outstanding performance in various applications, the deployment of CNN on embedded and mobile devices is limited by the massive computations and memory footprint. To address these challenges, Courbariaux and co-workers put forward binarized neural network (BNN) which quantizes both the weights and activations to [Formula: see text]1. From the perspective of hardware, BNN can greatly simplify the computation and reduce the storage. In this work, we first present the algorithm optimizations to further binarize the first layer and the padding bits of BNN; then we propose a fully binarized CNN accelerator. With the Shuffle–Compute structure and the memory-aware computation schedule scheme, the proposed design can boost the performance for feature maps of different sizes and make full use of the memory bandwidth. To evaluate our design, we implement the accelerator on the Zynq ZC702 board, and the experiments on the SVHN and CIFAR-10 datasets show the state-of-the-art performance efficiency and resource efficiency.

A Review on Convolutional Neural Network in Bearing Fault Diagnosis

As the degradation of bearing yield to an enormous adverse impact on machinery and the damage that comes within could jeopardize human precious life. Hence, the bearing fault diagnosis is indisputably indispensable. This paper is predominantly focused on the utilization of Convolutional Neural Network (CNN) in bearing fault diagnosis of the rolling bearing. By deployment of CNN, an accurate diagnosis can be achieved without the necessity of pre-training the data. The function of CNN in diagnosing the bearing and architecture development of CNN are discussed. Lastly, to establish new and significant contribution in this area, new challenges are pinpointed.