Efficient Convolutional Neural Network Research Articles

TransCNN: Hybrid CNN and transformer mechanism for surveillance anomaly detection

Surveillance video anomaly detection (SVAD) is a challenging task due to the variations in object scale, discrimination and unexpected events, the impact of the background, and the wide range of definitions of anomalous events in different surveillance contexts. In this work, we introduce an end-to-end hybrid convolution neural network (CNN) and vision transformer-based framework for anomaly detection. The proposed framework uses spatial and temporal information from a surveillance video to detect anomalous events and operates in two steps: in the first step, an efficient backbone CNN model is used for spatial feature extraction, while in the second step, these features are passed from the transformer-based model to learn the long-term temporal relationships between various complex surveillance events. The features from the backbone model are fed to a sequential learning model in which temporal self-attention is utilised to generate an attention map; this allows the proposed framework to learn the spatiotemporal features effectively and to detect anomalous events. Our experimental results on various benchmark VAD datasets prove the validity of the proposed framework, which outperforms other state-of-the-art approaches by achieving high AUC values of 94.6%, 98.4%, and 89.6% on the ShanghaiTech, UCSD Ped2 and CUHK avenue datasets, respectively.

Pruning feature maps for efficient convolutional neural networks

In recent years, convolutional neural networks have become increasingly important in the field of machine learning, especially for computer vision. However, deep network models are difficult to deploy on hardware-constrained devices because of huge number of parameters, storage requirements and computational cost. This paper proposes a pruning feature maps method to delete redundant feature information in the deep network. Thus, it can simplify network structure at the same time reduce the computational complexity and speed up the operation. We first define a small chi-square supervised set. The feature maps of this set and the training set are extracted. Then, two variance matrices are constructed. The differences between the variance matrices are then used to establish chi-square distances. Through continuous experiments, the optimal position threshold is set, and the feature maps corresponding to the channels below the position threshold are pruned. Experiment performances show that the method can reduce network redundancy, storage space and network complexity totally up to 70%. This is accomplished without appreciably diminishing the network’s accuracy. In the worst case, the accuracy difference for images classification by using simplified network and network before simplification was less than 0.4%. The use of a small pre-trained network also speeds up the network training. Together, these improvements constitute an important step towards the effective implementation of CNNs on constrained devices.

Breast Cancer Detection Using Image Processing and Machine Learning

Different breast cancer detection systems have been developed to help clinicians analyze screening mammograms. Breast cancer has been increasing gradually so scientists work to develop new methods to reduce the risks of this life-threatening disease. Convolutional Neural Networks (CNNs) have shown much promise In the field of medical imaging because of recent developments in deep learning. However, CNN’s based methods have been restricted due to the small size of the few public breast cancer datasets. This research has developed a new framework and introduced it to detect breast cancer. This framework utilizes Convolutional Neural Networks (CNNs) and image processing to achieve its goal because CNNs have been an important success in image recognition, reaching human performance. An efficient and fast CNN pre-trained object detector called RetinaNet has been used in this research. RetinaNet is an uncomplicated one-stage object detector. A two-stage transfer learning has been used with the selected detector to improve the performance. RetinaNet model is initially trained with a general-purpose dataset called COCO dataset. The transfer learning is then used to apply the RetinaNet model to another dataset of mammograms called the CBIS-DDSM dataset. Finally, the second transfer learning is used to test the RetinaNet model onto a small dataset of mammograms called the INbreast dataset. The results of the proposed two-stage transfer learning (RetinaNet → CBIS-DDSM → INbreast) are better than the other state-of-the-art methods on the public INbreast dataset. Furthermore, the True Positive Rate (TPR) is 0.99 ± 0.02 at 1.67 False Positives per Image (FPPI), which is better than the one-stage transfer learning with a TPR of 0.94 ± 0.02 at 1.67 FPPI.

Building an efficient convolution neural network from scratch: A case study on detecting and localizing slums

Designing a convolution neural network from scratch is one of the biggest challenges facing the creation of reproducible models. Despite feeding the model with an adequate amount of labeled data to mitigate fluctuations during training, the model still suffers from high variance in the final overall accuracy and loss among identical training runs. Many of the reasons behind this are the randomness in data shuffling and augmentation, the behavior of gradient decent function, and the non-determinism in convnet layers and floating-point computation of the GPU.The method used to address the aforementioned issues, specifically in the case of negative transfer learning, is divided into three steps: First, designing an efficient lightweight convnet architecture with respect to available resources. Second, mitigating oscillations during training. Third, after setting the random seed across the training, select the appropriate weight initialization. Our extensive experiments in the use case of binary slum localization and detection show that our method improves the reproducibility of our model from scratch with an accuracy of 98.88±1.15%, a loss of 0.03±0.05 for a confidence level of 99.73%. These results make this model a strong competitor to the pre-trained models using transfer learning.

Fire Detection with Image Processing

Abstract: Convolutional neural networks (CNNs) have yielded state-of-theart performance in image classification and other computer vision tasks. Their application in fire detection systems will substantially improve detection accuracy, which will eventually minimize fire disasters and reduce the ecological and social ramifications. However, the major concern with CNNbased fire detection systems is their implementation in real-world surveillance networks, due to their high memory and computational requirements for inference. In this paper, we propose an original, energy-friendly, and computationally efficient CNN architecture, inspired by the SqueezeNet architecture for fire detection, localization, and semantic understanding of the scene of the fire. It uses smaller convolutional kernels and contains no dense, fully connected layers, which helps keep the computational requirements to a minimum. Despite its low computational needs, the experimental results demonstrate that our proposed solution achieves accuracies that are comparable to other, more complex models, mainly due to its increased depth. Moreover, this paper shows how a tradeoff can be reached between fire detection accuracy and efficiency, by considering the specific characteristics of the problem of interest and the variety of fire data

Efficient convolutional neural networks on Raspberry Pi for image classification

With the good performance of deep learning in the field of computer vision (CV), the convolutional neural network (CNN) architectures have become main backbones of image recognition tasks. With the widespread use of mobile devices, neural network models based on platforms with low computing power are gradually being paid attention. However, due to the limitation of computing power, deep learning algorithms are usually not available on mobile devices. This paper proposes a lightweight convolutional neural network TripleNet, which can operate easily on Raspberry Pi. Adopted from the concept of block connections in ThreshNet, the newly proposed network model compresses and accelerates the network model, reduces the amount of parameters of the network, and shortens the inference time of each image while ensuring the accuracy. Our proposed TripleNet and other State-of-the-Art (SOTA) neural networks perform image classification experiments with the CIFAR-10 and SVHN datasets on Raspberry Pi. The experimental results show that, compared with GhostNet, MobileNet, ThreshNet, EfficientNet, and HarDNet, the inference time of TripleNet per image is shortened by 15%, 16%, 17%, 24%, and 30%, respectively. The detail codes of this work are available at https://github.com/RuiyangJu/TripleNet.

Deepview: Deep-Learning-Based Users Field of View Selection in 360° Videos for Industrial Environments

The industrial demands of immersive videos for virtual reality/augmented reality applications are crescendo, where the video stream provides a choice to the user viewing object of interest with the illusion of “being there.” However, in industry 4.0, streaming of such huge-sized video over the network consumes a tremendous amount of bandwidth, where the users are only interested in specific regions of the immersive videos. Furthermore, for delivering full excitement videos and minimizing the bandwidth consumption, the automatic selection of the user’s Region of Interest in a 360° video is very challenging because of subjectivity and difference in contentment. To tackle these challenges, we employ two efficient convolutional neural networks for salient object detection and memorability computation in a unified framework to find the most prominent portion of a 360° video. The proposed system is four-fold: 1) preprocessing; 2) intelligent visual interest predictor; 3) final viewport selection; and 4) virtual camera steerer. First, an input 360° video frame is split into three Field of Views (FoVs), each with a viewing angle of 120°. Next, each FoV is passed to the object detection and memorability prediction model for visual interestingness computation. Furthermore, the FoV is supplied as a viewport, containing the most salient and memorable objects. Finally, a virtual camera steerer is designed using enriched salient features from YOLO and LSTM that are forwarded to the dense optical flow to follow the salient object inside the immersive video. Performance evaluation of the proposed system over our own collected data from various Websites as well as on public data sets indicates the effectiveness for diverse categories of 360° videos and helps in the minimization of the bandwidth usage, making it suitable for industry 4.0 applications.

NAND-SPIN-based processing-in-MRAM architecture for convolutional neural network acceleration

The performance and efficiency of running large-scale datasets on traditional computing systems exhibit critical bottlenecks due to the existing "power wall" and "memory wall" problems. To resolve those problems, processing-in-memory (PIM) architectures are developed to bring computation logic in or near memory to alleviate the bandwidth limitations during data transmission. NAND-like spintronics memory (NAND-SPIN) is one kind of promising magnetoresistive random-access memory (MRAM) with low write energy and high integration density, and it can be employed to perform efficient in-memory computation operations. In this work, we propose a NAND-SPIN-based PIM architecture for efficient convolutional neural network (CNN) acceleration. A straightforward data mapping scheme is exploited to improve the parallelism while reducing data movements. Benefiting from the excellent characteristics of NAND-SPIN and in-memory processing architecture, experimental results show that the proposed approach can achieve $\sim$2.6$\times$ speedup and $\sim$1.4$\times$ improvement in energy efficiency over state-of-the-art PIM solutions.

Efficient Convolutional Neural Networks for Semiconductor Wafer Bin Map Classification.

The results obtained in the wafer test process are expressed as a wafer map and contain important information indicating whether each chip on the wafer is functioning normally. The defect patterns shown on the wafer map provide information about the process and equipment in which the defect occurred, but automating pattern classification is difficult to apply to actual manufacturing sites unless processing speed and resource efficiency are supported. The purpose of this study was to classify these defect patterns with a small amount of resources and time. To this end, we explored an efficient convolutional neural network model that can incorporate three properties: (1) state-of-the-art performances, (2) less resource usage, and (3) faster processing time. In this study, we dealt with classifying nine types of frequently found defect patterns: center, donut, edge-location, edge-ring, location, random, scratch, near-full type, and None type using open dataset WM-811K. We compared classification performance, resource usage, and processing time using EfficientNetV2, ShuffleNetV2, MobileNetV2 and MobileNetV3, which are the smallest and latest light-weight convolutional neural network models. As a result, the MobileNetV3-based wafer map pattern classifier uses 7.5 times fewer parameters than ResNet, and the training speed is 7.2 times and the inference speed is 4.9 times faster, while the accuracy is 98% and the F1 score is 89.5%, achieving the same level. Therefore, it can be proved that it can be used as a wafer map classification model without high-performance hardware in an actual manufacturing system.

Artificial Intelligence-based health diagnostic of Lithium-ion battery leveraging transient stage of constant current and constant voltage charging

State of health (SOH) estimation is essential to the health diagnostic of lithium-ion battery. The data-driven approach with charging feature extraction is promising for online SOH estimation and has been widely explored over years. However, their deployment can be barriered by the lack of complete charging data in real-world applications. Motivated by this, this paper proposes an artificial intelligence-based SOH estimator using the transient phase between constant current (CC) and constant voltage (CV) charging, which is easily obtained in real-world charging scenarios. Specifically, a convolutional neural network (CNN) model is proposed to explain the relationship between the charging data and the SOH. Following this endeavor, the transfer learning is exploited for model mitigation and SOH estimation on different battery types, relying on much reduced amount of data for efficient CNN model re-training. The validation experiments are conducted based on the aging data obtained on LiNiCoAlO2 (NCA) and LiCoO2 (LCO) cells. Results suggest that the proposed method realizes accurate SOH estimation requiring only a short segment from the CC-CV transient phase, so that can meet a broad range of real-world charging scenarios. Moreover, the efficient model transfer promises expected performance with different battery types.The short version of the paper was presented at ICAE2021, Nov 29 - Dec 5, 2021. This paper is a substantial extension of the short version of the conference paper.

Light Field Image Super-Resolution Network via Joint Spatial-Angular and Epipolar Information

This paper proposes a novel efficient convolutional neural network (CNN) for light field (LF) spatial super-resolution (SR). Due to the high dimensionality of LF data, it is important to decompose the original 4D LF data structure into a lower data subspace helping to reduce memory and computational costs. In this context, we propose a novel hybrid CNN structure working on different 2D data representations to fully handle the 4D structure. Our proposed network can simultaneously exploit information from three different types of 4D LF representations: spatial, angular, and epipolar plane image (EPI) information. Based on pixel similarities of each data subspace, we propose an inter-intra spatial feature extractor (II-SFE) and an inter-intra angular feature extractor (II-AFE) that can exploit the pixel correlations among the spatial-angular domain. Moreover, based on the correlated epipolar geometry among pixels, we also propose a multi-orientation epipolar feature extractor (MO-EFE) by decomposing an LF image into four orientation stacks of views and feeding them into different network branches to learn rich sub-pixel information in the EPI domain. The extracted features of each 2D subspace are integrated by our proposed attention fusion module (AFM), which adaptively rescales the embedding weights based on their correlations. Experimental results on both small- and wide-baseline benchmark LF datasets confirm the effectiveness of the proposed method compared to the well-known state-of-the-art methods in terms of objective and subjective performance.

Edge Solution for Real-Time Motor Fault Diagnosis Based on Efficient Convolutional Neural Network

Real-time motor fault diagnosis can detect motor faults on time and prompt the repair or replacement of faulty motors which minimizes the potential losses caused by motor faults. Deep learning (DL) methods have been intensively applied in motor fault diagnosis. Most DL algorithms need to be trained with sufficient computation resources on cloud or local servers. However, uploading the raw data and downloading the command instructions to the edge will cause inevitable time delays and security concerns. This paper develops a DL algorithm based on efficient convolutional neural networks (ECNN) that can be deployed on an edge computing node for real-time motor fault diagnosis and dynamic control. The effectiveness, efficiency, and robustness of the ECNN model have been validated by experiments, and the results indicate that the ECNN model can achieve 100 % accuracy in recognition of 10 types of motor conditions, with the inference time and memory usage less than 14 ms and 44 KiB, respectively. The comparison results demonstrate that the ECNN model yields higher accuracy than the classical shallow neural networks, and it also presents the advantages of smaller model volume, lower prediction time, and higher accuracy as compared with the DL models. The proposed method shows significant potential for practical application in real-time motor fault detection and control.

Depth-wise Decomposition for Accelerating Separable Convolutions in Efficient Convolutional Neural Networks

Very deep convolutional neural networks (CNNs) have been firmly established as the primary methods for many computer vision tasks. However, most state-of-the-art CNNs are large, which results in high inference latency. Depth-wise separable convolution has been proposed for image recognition tasks on platforms with limited computation power, such as robots and self-driving cars. Any regular deep CNN has a depth-wise separable counterpart, which is faster, but less accurate, when equally trained. In this paper, we propose a novel decomposition approach based on SVD, namely depth-wise decomposition, for converting regular convolutions into depth-wise separable convolutions post-training, while maintaining high accuracy. We show that our approach generalizes to the multi-channel and multi-layer cases, by applying Generalized Singular Value Decomposition (GSVD). We conduct thorough experiments with the ShuffleNet V2 model on a large-scale image recognition dataset: ImageNet. Our approach outperforms the baseline, channel decomposition. Moreover, our approach improves the Top-1 accuracy of ShuffleNet V2 by ∼2%.

A specific fine-grained identification model for plasma-treated rice growth using multiscale shortcut convolutional neural network.

As an agricultural innovation, low-temperature plasma technology is an environmentally friendly green technology that increases crop quality and productivity. However, there is a lack of research on the identification of plasma-treated rice growth. Although traditional convolutional neural networks (CNN) can automatically share convolution kernels and extract features, the outputs are only suitable for entry-level categorization. Indeed, shortcuts from the bottom layers to fully connected layers can be established feasibly in order to utilize spatial and local information from the bottom layers, which contain small distinctions necessary for fine-grain identification. In this work, 5000 original images which contain the basic growth information of rice (including plasma treated rice and the control rice) at the tillering stage were collected. An efficient multiscale shortcut CNN (MSCNN) model utilizing key information and cross-layer features was proposed. The results show that MSCNN outperforms the mainstream models in terms of accuracy, recall, precision and F1 score with 92.64%, 90.87%, 92.88% and 92.69%, respectively. Finally, the ablation experiment, comparing the average precision of MSCNN with and without shortcuts, revealed that the MSCNN with three shortcuts achieved the best performance with the highest precision.

A Novel Efficient Patient Monitoring FER System Using Optimal DL-Features

Automated Facial Expression Recognition (FER) serves as the backbone of patient monitoring systems, security, and surveillance systems. Real-time FER is a challenging task, due to the uncontrolled nature of the environment and poor quality of input frames. In this paper, a novel FER framework has been proposed for patient monitoring. Preprocessing is performed using contrast-limited adaptive enhancement and the dataset is balanced using augmentation. Two lightweight efficient Convolution Neural Network (CNN) models MobileNetV2 and Neural search Architecture Network Mobile (NasNetMobile) are trained, and feature vectors are extracted. The Whale Optimization Algorithm (WOA) is utilized to remove irrelevant features from these vectors. Finally, the optimized features are serially fused to pass them to the classifier. A comprehensive set of experiments were carried out for the evaluation of real-time image datasets FER-2013, MMA, and CK+ to report performance based on various metrics. Accuracy results show that the proposed model has achieved 82.5% accuracy and performed better in comparison to the state-of-the-art classification techniques in terms of accuracy. We would like to highlight that the proposed technique has achieved better accuracy by using 2.8 times lesser number of features.

Global and Local Structure Network for Image Classification

Principal Component Analysis Network (PCANet) is a feature learning algorithm which is widely used in face recognition and object classification. However, original PCANet still has some shortages. One is that PCA algorithm only extracts features by considering the global structure. The other lies in that the original PCANet only employs one particular single layer convolutional results, which loses the information of other convolutional layers. In this paper, we propose a new simple and efficient convolutional neural network called global and local structure network (GLSNet) to address the problems. The network extracts the features both from the global structure and the local structure of the original data space. Specifically, a principal component analysis (PCA) convolutional layer which learns the filters by PCA algorithm is used to remove the noises and redundant information at the first stage. Then at the second stage, another PCA convolution is added to extract features by considering the global structure. As for the local structure, we use the neighborhood preserving embedding (NPE) algorithm to learn the convolutional filters. At the output stage, the global structure feature extracted by PCA convolution and the local structure feature extracted by NPE convolution is concatenated as a united feature. Furthermore, the first layer convolutional feature is also taken into consideration to obtain shallow-level information. Finally, these features are concatenated as a united feature, and a spatial pyramid pooling layer is followed to pool above the united features. To test the effectiveness of the proposed algorithm, the experiments on some image datasets, including three types: human face dataset, object dataset, and handprinted dataset, proceeded. And it performs better than the original PCANet and some improvement algorithms of PCANet, such as PLDANet, and MMPCANet.

A lightweight CNN-based knowledge graph embedding model with channel attention for link prediction.

Knowledge graph (KG) embedding is to embed the entities and relations of a KG into a low-dimensional continuous vector space while preserving the intrinsic semantic associations between entities and relations. One of the most important applications of knowledge graph embedding (KGE) is link prediction (LP), which aims to predict the missing fact triples in the KG. A promising approach to improving the performance of KGE for the task of LP is to increase the feature interactions between entities and relations so as to express richer semantics between them. Convolutional neural networks (CNNs) have thus become one of the most popular KGE models due to their strong expression and generalization abilities. To further enhance favorable features from increased feature interactions, we propose a lightweight CNN-based KGE model called IntSE in this paper. Specifically, IntSE not only increases the feature interactions between the components of entity and relationship embeddings with more efficient CNN components but also incorporates the channel attention mechanism that can adaptively recalibrate channel-wise feature responses by modeling the interdependencies between channels to enhance the useful features while suppressing the useless ones for improving its performance for LP. The experimental results on public datasets confirm that IntSE is superior to state-of-the-art CNN-based KGE models for link prediction in KGs.

An Aggressively Pruned CNN Model With Visual Attention for Near Real-Time Wood Defects Detection on Embedded Processors

The use of Convolutional Neural Networks (CNN) for the application of wood defects detection has gained significant attention in recent years. In industrial settings, these tasks are typically performed in a strict and consistent environment, making the use of large and complex CNN models unnecessary. Despite this, recent research has continued to focus on the use of such models to achieve increasingly accurate detections. These models require costly machines for inference, making adoption less likely especially for manufacturers in the developing nations. In view of this limitation, this paper proposes a set of strategies to achieve a highly efficient CNN model for fast and accurate wood defects detection based on the YOLOv4-Tiny architecture. The model has been improved with Efficient Channel Attention (ECA) to better select multi-scale features from the backbone network and has been drastically reduced in size through an iterative pruning and recovery process. This results in an 88% reduction in model parameters while retaining accuracy comparable to most state-of-the-art (SOTA) methods. Consequently, the model can perform near real-time inference directly on a general-purpose embedded processor without external hardware accelerators. This research hopes to motivate the development of efficient defect detectors that can run on low-cost embedded devices.

Efficient Privacy-Preserving Inference Outsourcing for Convolutional Neural Networks

Inference outsourcing enables model owners to deploy their machine learning models on cloud servers to serve users. In this paradigm, the privacy of model owners and users should be considered. Existing solutions focus on Convolutional Neural Networks (CNNs) but their efficiency is much lower than GALA, which is a solution that only protects user privacy. Furthermore, these solutions adopt approximations that reduce the model accuracy and thus require model owners to retrain the models. In this paper, we present an efficient CNN inference outsourcing solution that protects the privacy of both model owners and users. Specifically, we design secure two-party computation protocols based on two non-colluding cloud servers, which calculate with additive secret shares of the model and the user’s input. Our protocols avoid the expensive permutation operations in linear calculations and approximations in non-linear calculations. We implement our solution on realistic CNNs and experimental results show that our solution is even 2–4 times faster than GALA.

YOLOv5-light: efficient convolutional neural networks for flame detection

Flame detection is critical to human society. Current flame detection methods have problems about huge model volume and large parameters. We proposed an improved YOLOv5-light that reduces the parameters while maintaining the same detection accuracy. First, we use the channel separation shuffling method to ensure the information exchange between channels and optimize the residual structure. Second, depthwise separable convolution is used as the downsampling module in the backbone network to reduce the model’s parameters further. Finally, the Ghost module is used to improve the feature extraction module in the head network to reduce computational cost caused by ordinary convolution. The final experimental results show that the proposed YOLOv5-light network achieves detection accuracy similar to that of YOLOv5, with a 65.6% decrease in parameters and a model volume of only 34.8% of the latter.