Unimportant Channels Research Articles

Three-Stage Global Channel Pruning for Resources-Limited Platform.

Deep neural networks (DNNs) have demonstrated remarkable performance in many fields, and deploying them on resource-limited devices has drawn more and more attention in industry and academia. Typically, there are great challenges for intelligent networked vehicles and drones to deploy object detection tasks due to the limited memory and computing power of embedded devices. To meet these challenges, hardware-friendly model compression approaches are required to reduce model parameters and computation. Three-stage global channel pruning, which involves sparsity training, channel pruning, and fine-tuning, is very popular in the field of model compression for its hardware-friendly structural pruning and ease of implementation. However, existing methods suffer from problems such as uneven sparsity, damage to the network structure, and reduced pruning ratio due to channel protection. To solve these issues, the present article makes the following significant contributions. First, we present an element-level heatmap-guided sparsity training method to achieve even sparsity, resulting in higher pruning ratio and improved performance. Second, we propose a global channel pruning method that fuses both global and local channel importance metrics to identify unimportant channels for pruning. Third, we present a channel replacement policy (CRP) to protect layers, ensuring that the pruning ratio can be guaranteed even under high pruning rate conditions. Evaluations show that our proposed method significantly outperforms the state-of-the-art (SOTA) methods in terms of pruning efficiency, making it more suitable for deployment on resource-limited devices.

Deep Neural Network-Based Flood Monitoring System Fusing RGB and LWIR Cameras for Embedded IoT Edge Devices

Floods are among the most common disasters, causing loss of life and enormous damage to private property and public infrastructure. Monitoring systems that detect and predict floods help respond quickly in the pre-disaster phase to prevent and mitigate flood risk and damages. Thus, this paper presents a deep neural network (DNN)-based real-time flood monitoring system for embedded Internet of Things (IoT) edge devices. The proposed system fuses long-wave infrared (LWIR) and RGB cameras to overcome a critical drawback of conventional RGB camera-based systems: severe performance deterioration at night. This system recognizes areas occupied by water using a DNN-based semantic segmentation network, whose input is a combination of RGB and LWIR images. Flood warning levels are predicted based on the water occupancy ratio calculated by the water segmentation result. The warning information is delivered to authorized personnel via a mobile message service. For real-time edge computing, the heavy semantic segmentation network is simplified by removing unimportant channels while maintaining performance by utilizing the network slimming technique. Experiments were conducted based on the dataset acquired from the sensor module with RGB and LWIR cameras installed in a flood-prone area. The results revealed that the proposed system successfully conducts water segmentation and correctly sends flood warning messages in both daytime and nighttime. Furthermore, all of the algorithms in this system were embedded on an embedded IoT edge device with a Qualcomm QCS610 System on Chip (SoC) and operated in real time.

Research On Pruning Methods for Mobilenet Convolutional Neural Network

This paper comprehensively reviews pruning methods for MobileNet convolutional neural networks. MobileNet is a lightweight convolutional neural network suitable for resource-constrained environments such as mobile devices.Various pruning methods can be applied to reduce the model's storage space and computational complexity, including channel pruning, kernel pruning, and weight pruning. Channel pruning removes unimportant channels to reduce redundant parameters and computations in the model, while kernel pruning reduces redundant calculations by pruning convolutional kernels. Weight pruning involves setting small-weighted elements to zero to remove unimportant weights. These pruning methods can be used individually or in combination. After pruning, fine-tuning is necessary to restore the model's performance. Factors such as pruning rate, pruning order, and pruning location need to be considered to achieve a balance between reducing model size and computational complexity while minimizing performance loss. Pruning methods based on MobileNet convolutional neural networks reduce the parameter count and computational complexity, improving model lightweightness and inference efficiency. These methods are of significant value in resource-constrained environments such as mobile devices. This review provides insights into pruning methods for MobileNet convolutional neural networks and their applications in lightweight and efficient model deployment. Further advancements such as automated pruning methods driven by reinforcement learning algorithms can enhance the pruning process to achieve optimal model compression effects. Future research should focus on adapting and optimizing these pruning methods for specific problem domains and achieving even higher compression ratios and computational speedups.

Expanding and Refining Hybrid Compressors for Efficient Object Re-Identification.

Recent object re-identification (Re-ID) methods gain high efficiency via lightweight student models trained by knowledge distillation (KD). However, the huge architectural difference between lightweight students and heavy teachers causes students to have difficulties in receiving and understanding teachers' knowledge, thus losing certain accuracy. To this end, we propose a refiner-expander-refiner (RER) structure to enlarge a student's representational capacity and prune the student's complexity. The expander is a multi-branch convolutional layer to expand the student's representational capacity to understand a teacher's knowledge comprehensively, which does not require any feature-dimensional adapter to avoid knowledge distortions. The two refiners are 1×1 convolutional layers to prune the input and output channels of the expander. In addition, in order to alleviate the competition accuracy-related and pruning-related gradients, we design a common consensus gradient resetting (CCGR) method, which discards unimportant channels according to the intersection of each sample's unimportant channel judgment. Finally, the trained RER can be simplified into a slim convolutional layer via re-parameterization to speed up inference. As a result, we propose an expanding and refining hybrid compressing (ERHC) method. Extensive experiments show that our ERHC has superior inference speed and accuracy, e.g., on the VeRi-776 dataset, given the ResNet101 as a teacher, ERHC saves 75.33% model parameters (MP) and 74.29% floating-point of operations (FLOPs) without sacrificing accuracy.

Adaptive Channel Allocation for Robust Differentiable Architecture Search.

Differentiable architecture search (DARTS) has attracted much attention due to its simplicity and significant improvement in efficiency. However, the excessive accumulation of the skip connection, when training epochs become large, makes it suffer from weak stability and low robustness, thus limiting its practical applications. Many works have attempted to restrict the accumulation of skip connections by indicators or manual design. These methods, however, are susceptible to human priors and hyperparameters. In this work, we suggest a more subtle and direct approach that no longer explicitly searches for skip connections in the search stage, based on the paradox that skip connections were proposed to guarantee the performance of very deep networks, but the networks in the search stage of DARTS are actually very shallow. Instead, by introducing channel importance ranking and channel allocation strategy, the skip connections are implicitly searched and automatically refilled unimportant channels in the evaluation stage. Our method, dubbed adaptive channel allocation (ACA) strategy, is a general-purpose approach for DARTS, which universally works in DARTS variants without introducing human priors, indicators, or hyperparameters. Extensive experiments on various datasets and DARTS variants verify that the ACA strategy is the most effective one among the existing methods in improving robustness and dealing with the collapse issue when training epochs become large.

Network pruning via resource reallocation

Channel pruning is broadly recognized as an effective approach to obtain a small compact model through eliminating unimportant channels from a large cumbersome network. Contemporary methods typically perform iterative pruning procedure from the original over-parameterized model, which is both tedious and expensive especially when the pruning is aggressive. In this paper, we propose a simple yet effective channel pruning technique, termed network Pruning via rEsource rEalLocation (PEEL), to quickly produce a desired slim model with negligible cost. Specifically, PEEL first constructs a predefined backbone and then conducts resource reallocation on it to shift parameters from less informative layers to more important layers in one round, thus amplifying the positive effect of these informative layers. To demonstrate the effectiveness of PEEL , we perform extensive experiments on ImageNet with ResNet-18, ResNet-50, MobileNetV2, MobileNetV3-small and EfficientNet-B0. Experimental results show that structures uncovered by PEEL exhibit competitive performance with state-of-the-art pruning algorithms under various pruning settings. Encouraging results are also observed when applying PEEL to compress the semantic segmentation model. Our code is available at https://github.com/cardwing/Codes-for-PEEL.

Pulmonary arteries segmentation from CT images using PA-Net with attention module and contour loss.

Pulmonary embolism is a kind of cardiovascular disease that threatens human life and health. Since pulmonary embolism exists in the pulmonary artery, improving the segmentation accuracy of pulmonary artery is the key to the diagnosis of pulmonary embolism. Traditional medical image segmentation methods have limited effectiveness in pulmonary artery segmentation. In recent years, deep learning methods have been gradually adopted to solve complex problems in the field of medical image segmentation. Due to the irregular shape of the pulmonary artery and the adjacent-complex tissues, the accuracy of the existing pulmonary artery segmentation methods based on deep learning needs to be improved. Therefore, the purpose of this paper is to develop a segmentation network, which can obtain higher segmentation accuracy and further improve the diagnosis effect. In this study, the pulmonary artery segmentation performance from the network model and loss function is improved, proposing a pulmonary artery segmentation network (PA-Net) to segment the pulmonary artery region from 2D CT images. Reverse Attention and edge attention are used to enhance the expression ability of the boundary. In addition, to better use feature information, the channel attention module is introduced in the decoder to highlight the important channel features and suppress the unimportant channels. Due to blurred boundaries, pixels near the boundaries of the pulmonary artery may be difficult to segment. Therefore, a new contour loss function based on the active contour model is proposed in this study to segment the target region by assigning dynamic weights to false positive and false negative regions and accurately predict the boundary structure. The experimental results show that the segmentation accuracy of this proposed method is significantly improved in comparison with state-of-the-art segmentation methods, and the Dice coefficient is 0.938±0.035, which is also confirmed from the 3D reconstruction results. Our proposed method can accurately segment pulmonary artery structure. This new development will provide the possibility for further rapid diagnosis of pulmonary artery diseases such as pulmonary embolism. Code is available at https://github.com/Yuanyan19/PA-Net.

Identification of plant leaf diseases by deep learning based on channel attention and channel pruning.

Plant diseases cause significant economic losses and food security in agriculture each year, with the critical path to reducing losses being accurate identification and timely diagnosis of plant diseases. Currently, deep neural networks have been extensively applied in plant disease identification, but such approaches still suffer from low identification accuracy and numerous parameters. Hence, this paper proposes a model combining channel attention and channel pruning called CACPNET, suitable for disease identification of common species. The channel attention mechanism adopts a local cross-channel strategy without dimensionality reduction, which is inserted into a ResNet-18-based model that combines global average pooling with global max pooling to effectively improve the features' extracting ability of plant leaf diseases. Based on the model's optimum feature extraction condition, unimportant channels are removed to reduce the model's parameters and complexity via the L1-norm channel weight and local compression ratio. The accuracy of CACPNET on the public dataset PlantVillage reaches 99.7% and achieves 97.7% on the local peanut leaf disease dataset. Compared with the base ResNet-18 model, the floating point operations (FLOPs) decreased by 30.35%, the parameters by 57.97%, the model size by 57.85%, and the GPU RAM requirements by 8.3%. Additionally, CACPNET outperforms current models considering inference time and throughput, reaching 22.8 ms/frame and 75.5 frames/s, respectively. The results outline that CACPNET is appealing for deployment on edge devices to improve the efficiency of precision agriculture in plant disease detection.

Differentiable Network Pruning via Polarization of Probabilistic Channelwise Soft Masks.

Channel pruning has been demonstrated as a highly effective approach to compress large convolutional neural networks. Existing differentiable channel pruning methods usually use deterministic soft masks to scale the channelwise outputs and explore an appropriate threshold on the masks to remove unimportant channels, which sometimes causes unexpected damage to the network accuracy when there are no sweet spots that clearly separate important channels from redundant ones. In this article, we introduce a new differentiable channel pruning method based on polarization of probabilistic channelwise soft masks (PPSMs). We use variational inference to approximate the posterior distributions of the masks and simultaneously exploit a polarization regularization to push the probabilistic masks towards either 0 or 1; thus, the channels with near-zero masks can be safely eliminated with little hurt on network accuracy. Our method significantly relieves the difficulty faced by the existing methods to find an appropriate threshold on the masks. The joint inference and polarization of probabilistic soft masks enable PPSM to yield better pruning results than the state of the arts. For instance, our method prunes 65.91% FLOPs of ResNet50 on the ImageNet dataset with only 0.7% model accuracy degradation.

Acceleration of Infrared Target Detection via Efficient Channel Pruning

Deployment of modern detection models is difficult for infrared target detection used in robot vision systems due to their heavy computational burden. To alleviate this situation, a simple but efficient channel pruning method is proposed for model acceleration. Specifically, a soft-gated module combined with batch normalization (SGBN) is designed as a standalone layer to substitute the standard batch normalization (BN) layer during training. The conversion between SGBN and BN is easy, and the training overhead introduced is almost negligible after replacement. By controlling the sparsity of the scaling factor in SGBN, unimportant channels with small output are blocked automatically and globally, which is simultaneous with model training. Removing these redundant channels no longer requires fine-tuning, thus significantly speeding up the pruning process. Experiments of pruning different detection models on the infrared dataset show the effectiveness of our method. For example, the parameters and FLOPs of pruned CetnerNet are reduced by 72.70% and 40.20%, respectively, without accuracy loss. The inference speed on the CPU is 12.01ms faster. Extended studies on the classification task also demonstrate its great potential when transferring to other applications.

Latency-aware automatic CNN channel pruning with GPU runtime analysis

The huge storage and computation cost of convolutional neural networks (CNN) make them challenging to meet the real-time inference requirement in many applications. Existing channel pruning methods mainly focus on removing unimportant channels in a CNN model based on rule-of-thumb designs, using reduced floating-point operations (FLOPs) and parameter numbers to measure the pruning quality. The inference latency of pruned models is often overlooked. In this paper, we propose a latency-aware automatic CNN channel pruning method (LACP), which aims to search low latency and accurate pruned network structure automatically. We evaluate the inaccuracy of measuring pruning quality by FLOPs and the number of parameters, and use the model inference latency as the direct optimization metric. To bridge model pruning and inference acceleration, we analyze the inference latency of convolutional layers on GPU. Results show that the inference latency of convolutional layers exhibits a staircase pattern along with channel number due to the GPU tail effect. Based on that observation, we greatly shrink the search space of network structures. Then we apply an evolutionary procedure to search a computationally efficient pruned network structure, which reduces the inference latency and maintains the model accuracy. Experiments and comparisons with state-of-the-art methods on three image classification datasets show that our method can achieve better inference acceleration with less accuracy loss.

Compressing convolutional neural networks via intermediate features

Deep convolutional neural network (CNN) is difficult to deploy to mobile and portable devices due to its large number of parameters and floating-point operations (FLOPs). To tackle this problem, we propose a novel channel pruning method. We use the modified squeeze-and-excitation blocks (MSEB) to measure the importance of the channels in the convolutional layers. The unimportant channels, including convolutional kernels related to them, are pruned directly, which greatly reduces the storage cost and the number of calculations. For ResNet with basic blocks, we propose an approach to consistently prune all residual blocks in the same stage to ensure that the compact network structure is dimensionally correct. After pruning we retrain the compact network from scratch to restore the accuracy. Finally, we verify our method on CIFAR-10, CIFAR-100 and ILSVRC-2012. The results indicate that the performance of the compact network is better than the original network when the pruning rate is small. Even when the pruning amplitude is large, the accuracy can also be maintained or decreased slightly. On the CIFAR-100, when reducing the parameters and FLOPs up to 82% and 62% respectively, the accuracy of VGG-19 even improve by 0.54% after retraining. The source code is available at https://github.com/JingfeiChang/UCP.

SuperPruner: Automatic Neural Network Pruning via Super Network

Most network pruning methods rely on rule-of-thumb for human experts to prune the unimportant channels. This is time-consuming and can lead to suboptimal pruning. In this paper, we propose an effective SuperPruner algorithm, which aims to find optimal pruned structure instead of pruning unimportant channels. We first train a VerifyNet, a kind of super network, which is able to roughly evaluate the performance of any given network structure. The particle swarm optimization algorithm is then used to search for optimal network structure. Lastly, the weights in the VerifyNet are used as the initial weights of the optimal pruned structure to make fine-tuning. VerifyNet is a network performance evaluation; our algorithm can quickly prune the network under any hardware constraints. Our algorithm can be applied in multiple fields such as object recognition and semantic segmentation. Extensive experiment results demonstrate the effectiveness of SuperPruner. For example, on CIFAR-10, the pruned VGG16 achieves 93.18% Top-1 accuracy and reduces 74.19% of FLOPs and 89.25% of parameters. Compared with state-of-the-art methods, our algorithm can achieve higher pruned ratio with less accuracy cost.

Multi-threshold channel pruning method based on L1 regularization

Aiming at the problem of single detection scene and poor detection speed, a multi-threshold channel pruning method based on L1 regularization is proposed. L1 regularization is applied to the scaling factor of the BN layer to push the scaling factor of the BN layer to zero, thereby identifying unimportant channels, and determining the initial threshold of channel pruning according to the scaling factor after sparse training; The most suitable pruning threshold of the convolutional layer is obtained through minimum error reconstruction. The least square method and the method of weighting the channel are used to reconstruct the error before and after pruning, and the most suitable pruning threshold is obtained by minimizing the error. Through the data set simulation experiment, the YOLOv3 network is pruned, so that the network has a smaller network structure and faster detection speed, which is suitable for deployment on mobile devices.

CCPrune: Collaborative channel pruning for learning compact convolutional networks

Deep convolutional neural networks (CNNs) is difficult to deploy on resource-constrained devices due to its huge amount of computation. Channel pruning is an effective method to reduce the amount of computation and accelerate network inference. Most of channels pruning methods use statistics from a single structure (convolutional layer or batch normalization layer) of the sparse network to evaluate the importance of channels. The limitation of these methods is that it may often mistakenly delete the important channels. In view of this, we propose a novel method, namely Collaborative Channel Pruning (CCPrune), to evaluate the importance of channels, which combines the convolution layer weights and the BN layer scaling factors. The proposed method first introduces the regularization on the convolution layer weights and the BN layer scaling factors respectively. Then combine the weight of the convolutional layer and the scaling factor of the BN layer to evaluate the importance of the channel. Finally, it can delete the unimportant channels without reduces the performance of the model. The experimental results well demonstrate the effectiveness of our method. On CIFAR-10, it can reduce the FLOPs of VGG-19 by 85.50% while only slightly reducing the accuracy of the model, and it can reduce the FLOPs of Resnet-50 by 78.31% without reducing the accuracy of the model, respectively.

Dynamical Channel Pruning by Conditional Accuracy Change for Deep Neural Networks.

Channel pruning is an effective technique that has been widely applied to deep neural network compression. However, many existing methods prune from a pretrained model, thus resulting in repetitious pruning and fine-tuning processes. In this article, we propose a dynamical channel pruning method, which prunes unimportant channels at the early stage of training. Rather than utilizing some indirect criteria (e.g., weight norm, absolute weight sum, and reconstruction error) to guide connection or channel pruning, we design criteria directly related to the final accuracy of a network to evaluate the importance of each channel. Specifically, a channelwise gate is designed to randomly enable or disable each channel so that the conditional accuracy changes (CACs) can be estimated under the condition of each channel disabled. Practically, we construct two effective and efficient criteria to dynamically estimate CAC at each iteration of training; thus, unimportant channels can be gradually pruned during the training process. Finally, extensive experiments on multiple data sets (i.e., ImageNet, CIFAR, and MNIST) with various networks (i.e., ResNet, VGG, and MLP) demonstrate that the proposed method effectively reduces the parameters and computations of baseline network while yielding the higher or competitive accuracy. Interestingly, if we Double the initial Channels and then Prune Half (DCPH) of them to baseline's counterpart, it can enjoy a remarkable performance improvement by shaping a more desirable structure.

Visual Sorting of Express Parcels Based on Multi-Task Deep Learning.

Visual sorting of express parcels in complex scenes has always been a key issue in intelligent logistics sorting systems. With existing methods, it is still difficult to achieve fast and accurate sorting of disorderly stacked parcels. In order to achieve accurate detection and efficient sorting of disorderly stacked express parcels, we propose a robot sorting method based on multi-task deep learning. Firstly, a lightweight object detection network model is proposed to improve the real-time performance of the system. A scale variable and the joint weights of the network are used to sparsify the model and automatically identify unimportant channels. Pruning strategies are used to reduce the model size and increase the speed of detection without losing accuracy. Then, an optimal sorting position and pose estimation network model based on multi-task deep learning is proposed. Using an end-to-end network structure, the optimal sorting positions and poses of express parcels are estimated in real time by combining pose and position information for joint training. It is proved that this model can further improve the sorting accuracy. Finally, the accuracy and real-time performance of this method are verified by robotic sorting experiments.

Detecting Citrus in Orchard Environment by Using Improved YOLOv4

Real-time detection of fruits in orchard environments is one of crucial techniques for many precision agriculture applications, including yield estimation and automatic harvesting. Due to the complex conditions, such as different growth periods and occlusion among leaves and fruits, detecting fruits in natural environments is a considerable challenge. A rapid citrus recognition method by improving the state-of-the-art You Only Look Once version 4 (YOLOv4) detector is proposed in this paper. Kinect V2 camera was used to collect RGB images of citrus trees. The Canopy algorithm and the K-Means++ algorithm were then used to automatically select the number and size of the prior frames from these RGB images. An improved YOLOv4 network structure was proposed to better detect smaller citrus under complex backgrounds. Finally, the trained network model was used for sparse training, pruning unimportant channels or network layers in the network, and fine-tuning the parameters of the pruned model to restore some of the recognition accuracy. The experimental results show that the improved YOLOv4 detector works well for detecting different growth periods of citrus in a natural environment, with an average increase in accuracy of 3.15% (from 92.89% to 96.04%). This result is superior to the original YOLOv4, YOLOv3, and Faster R-CNN. The average detection time of this model is 0.06 s per frame at 1920 × 1080 resolution. The proposed method is suitable for the rapid detection of the type and location of citrus in natural environments and can be applied to the application of citrus picking and yield evaluation in actual orchards.

Lightweight Attention Pyramid Network for Object Detection and Instance Segmentation

Feature pyramids of convolutional neural networks (ConvNets)—from bottom to top—are used by most recent researchers for the improvement of object detection accuracy, but they seldom aim to address the correlation of each feature channel and the fusion of low-level features and high-level features. In this paper, an Attention Pyramid Network (APN) is proposed, which mainly contains the adaptive transformation module and feature attention block. The adaptive transformation module utilizes the multiscale feature fusion, and makes full use of the accurate target location information of low-level features and the semantic information of high-level features. Then, the feature attention block strengthens the features of important channels and weakens the features of unimportant channels through learning. By implementing the APN in a basic Mask R-CNN system, our method achieves state-of-the-art results on the MS COCO dataset and 2018 WAD database without bells and whistles. In addition, the structure of the APN makes the network parameters lighter, and runs at 4 ms on average, which is ignorable when compared to the inference time of the backbone of ConvNet.

Learning Sparse Convolutional Neural Network via Quantization With Low Rank Regularization

With the refinement of tasks in artificial intelligence, bringing in exponential level increments in computation cost and storage. Therefore, the augment of computation resource for complicated neural networks severely hinders their applications on limited-power devices in recent years. As a result, there is an impending necessity to compress and accelerate the deep networks by special ways. Considering the different peculiarities of weight quantization and sparse regularization, in this paper, we propose a low rank sparse quantization (LRSQ) method to quantize network weights and regularize the corresponding structures at the same time. Our LRSQ can: 1) obtain low-bit quantized networks to reduce memory and computation cost and 2) learn a compact structure from complex convolutional networks for subsequent channel pruning which has significant reduction on FLOPs. In experimental sections, we evaluate the proposed method on several popular models such as VGG-7/16/19 and ResNet-18/34/50, and results show that this method can dramatically reduce parameters and channels of the network with slight inference accuracy loss. Furthermore, we also visualize and analyze the four-dimensional weight tensors, which shows the low rank and group-sparsity structure of it. Finally, we try pruning unimportant channels which are zero-channels in our quantized model, and finding even a little better precision than the standard full-precision network.