Lightweight CNN Research Articles

Intelligent fault diagnosis of rotating machinery based on a novel lightweight convolutional neural network

For most existing fault diagnosis methods, feature extraction is always based on a complex artificial design and the complete feature extraction from an original signal. With the gradual complication of modern industrial machinery and equipment, it has become more difficult for traditional feature extractors to achieve the desired results. Deep convolutional neural networks (DCNNs) have been developed as effective techniques for fault classification but require large-scale high-intensity computing and prohibitive hardware resource requirements. This paper proposes a lightweight CNN that can be easily used for the fault diagnosis of rotating machinery by adjusting the network structure and optimizing the network. First, the raw vibration acceleration signal is transformed into a two-dimensional gray image. Second, two mature and commonly used modules named LeNet and NIN are combined to form a new model with a simple structure. Then, through parameter adjustment and optimization, an improved and optimized CNN with a lightweight structure and fewer parameters is constructed. The experimental verification has shown that this method has high accuracy and stability in fault diagnosis. Finally, the application of this new network for the fault diagnosis of rolling bearings with different damage levels but similar fault types shows high diagnostic accuracy and good generalization ability. In addition, we attempt to explain how the feature filters of a CNN work by visualizing the convolutional layer of the network.

Collaborative Image Relevance Learning for Visual Re-Ranking

In content-based image retrieval, the initial retrieval result may be unsatisfactory, which can be refined with visual re-ranking techniques, such as query expansion, geometric verification, etc . In this work, we approach visual re-ranking from a novel perspective. Observing that the contextual similarity of images from a retrieval result list exhibits strong visual relevance, we propose to collaboratively learn the semantic relevance among images for visual re-ranking. In our approach, we represent the image set of a fixed-length retrieval list into a correlation matrix, and learn the relevance of all image pairs simultaneously with a lightweight CNN model. To optimize the CNN model, a weighted MSE loss is defined, which takes into account the sparsity of labels. To find the optimal length of retrieval result list for different queries, we present a query sensitive selection method. We conduct comprehensive experiments on five benchmark datasets, and demonstrate the generality, and effectiveness of the proposed visual re-ranking method.

I-FireNet: 에지 AI 환경에서 실시간 산불감지를 위해 일반화 성능을 향상한 경량형 합성곱 신경망

i-FireNet: 에지 AI 환경에서 실시간 산불감지를 위해 일반화 성능을 향상한 경량형 합성곱 신경망

Extracting road maps from high-resolution satellite imagery using refined DSE-LinkNet

Road detection and extraction have gained momentum in recent past years with crucial applications such as urban planning, autonomous driving, automated map update, providing aid to rescue missions, etc. The current methodologies generate the disconnected road segments, cause boundary loss, and also they are incapable of handling the imbalanced class distribution problems. In this paper, we propose a fully convolutional architecture, named as refined DSE-LinkNet, to extract the connected and precise road maps. We use a pre-trained encoder by combining the layers of the two very efficient and light-weight CNN models: DenseNet and SE-Net that makes the proposed model more expressive with faster convergence. We introduce a new module, Fusion block, in our architecture that enhances its precise localisation as well as classification ability by capturing multilevel as well as multiscale features. To address the imbalanced class distribution problem, a new aggregate loss function is proposed by integrating binary cross-entropy, Jaccard coefficient, and Lovasz sigmoid loss functions. The experiments are performed on a publicly available dataset, DeepGlobe Road Extraction Challenge 2018, to show its efficacy over the D-LinkNet, winner of DeepGlobe Challenge 2018, by achieving IoU of 0.69 with lesser number of parameters and better computational complexity.

MBBNet: An edge IoT computing-based traffic light detection solution for autonomous bus

Abstract Traffic light detection is a key module in the autonomous driving system to enhance the interactions between drivers and unmanned vehicles. In recent studies, deep neural networks are widely used for traffic light detection and resource/power consumption is a major concern for model deployment in vehicular edge devices. This paper proposes a novel light-weight deep CNN model that integrates the multi-backbone of state-of-the-art architectures for the self-driving traffic light detection. The MBBNet (Multi-BackBone Network) consists of three common convolutional backbones, i.e., the normal, residual and highway (DenseNet) convolutional modules. Simple ensemble of those backbones may incur high computational load. Therefore, channel compression is adopted to control the model parameters, while guaranteeing the accuracy for mobile and embedded hardware. Evaluation of a dataset collected from real road conditions demonstrate the robustness of our detection system, and it achieves higher accuracy (accuracy > 0.94 and A v e r a g e _ I O U > 74.05 % ) for self-driving buses. In terms of resource consumption, the trained model size is 1.35 MB, and can process high-resolution images (1280 × 960) at 14 FPS (frames per second) on low-power edge devices.

Indoor Video Flame Detection Based on Lightweight Convolutional Neural Network

At present, all CNN-based fire detection algorithms identify fire by means of a single frame image, all of which demonstrate low accuracy under strong interferences or complex backgrounds such as flickering light or backgrounds with high level of brightness. To increase the accuracy of fire detection, this paper presents a neural network model which combines lightweight CNN with SRU. In this algorithm, the scene content is extracted by CNN and the dynamic characteristics of the flames are extracted from sequential frames. In this paper, Resnet18+SRU (V1-type) and Mobilenets+SRU (V2-type) are proposed. Based on the characteristics of flames at a fixed position within a short period of time, a 3D convolutional layer is added between the Mobilenets and the SRU in the V2-type model, resulting in the V3-type model. Based on a cross validation set containing multiple types of interference in an indoor environment, experiments were conducted to compare the three models proposed in this paper with other models. The experiment results showed that the accuracy of the method proposed in this paper is above 96%, about 25% higher than the accuracy of CNN-based fire alarm via single-frame image, and that the V3-type models with 3D convolutional layer has the highest accuracy and best overall performance.

A lightweight face detector by integrating the convolutional neural network with the image pyramid

Recently, significant improvements on face detection have been achieved by convolutional neural networks. However, the speed of a CNN-based face detector is hampered by the large computational complexity and the huge number of parameters. It is still challenging to achieve real-time face detection as well as maintain high accuracy. In this work, we introduce a single-stage face detector with an extremely lightweight CNN to achieve fast and accurate detection. Specifically, our method has a structure to integrate the network with the image pyramid for fully utilizing the calculated features. Benefiting from weight sharing, the network size still can keep small. We also analysis the detection capability of anchors with various scales, and reserve the most effective anchors in our model. Besides, to avoid the too difficult training samples which the small network can’t learn, each ground truth face is assigned with a 0-or-1 weight during training. When tested on WIDERFACE and FDDB, it outperforms existing lightweight face detectors on accuracy with the smallest model size. The outstanding detection performance and lightweight model size signify its effectiveness and practicability.

Deep Learning based Fruit Freshness Classification and Detection with CMOS Image sensors and Edge processors

CMOS Image sensors play a vital role in the exponentially growing field of Artificial Intelligence (AI). Applications like image classification, object detection and tracking are just some of the many problems now solved with the help of AI, and specifically deep learning. In this work, we target image classification to discern between six categories of fruits — fresh/ rotten apples, fresh/ rotten oranges, fresh/ rotten bananas. Using images captured from high speed CMOS sensors along with lightweight CNN architectures, we show the results on various edge platforms. Specifically, we show results using ON Semiconductor’s global-shutter based, 12MP, 90 frame per second image sensor (XGS-12), and ON Semiconductor’s 13 MP AR1335 image sensor feeding into MobileNetV2, implemented on NVIDIA Jetson platforms. In addition to using the data captured with these sensors, we utilize an open-source fruits dataset to increase the number of training images. For image classification, we train our model on approximately 30,000 RGB images from the six categories of fruits. The model achieves an accuracy of 97% on edge platforms using ON Semiconductor’s 13 MP camera with AR1335 sensor. In addition to the image classification model, work is currently in progress to improve the accuracy of object detection using SSD and SSDLite with MobileNetV2 as the feature extractor. In this paper, we show preliminary results on the object detection model for the same six categories of fruits.

Data augmentation for bearing fault detection with a light weight CNN

Bearings are vital part of rotary machines. A failure of bearing has a negative impact on schedules, production operation and even human casualties. Therefore, in prior achieving fault detection and diagnosis (FDD) of bearing is ensuring the safety and reliable operation of rotating machinery systems. However, there are some challenges of the industrial FDD problems. First, there are unbalanced samples because industrial faults rarely occur. Conse-quently, the labeled data which can refer to failure information are limited in the industry and data augmentation methods are critical pre-processing be-fore training data driven models. Second, due to many learnable parameters in model and data of long sequence, both lead to time delay for FDD. There-fore, this paper proposes various data preprocessing methods and Light-Convolutional Neural Network (LCNN).

Automatic Skin Cancer Detection in Dermoscopy Images Based on Ensemble Lightweight Deep Learning Network

The complex detection background and lesion features make the automatic detection of dermoscopy image lesions face many challenges. The previous solutions mainly focus on using larger and more complex models to improve the accuracy of detection, there is a lack of research on significant intra-class differences and inter-class similarity of lesion features. At the same time, the larger model size also brings challenges to further algorithm application; In this paper, we proposed a lightweight skin cancer recognition model with feature discrimination based on fine-grained classification principle. The propose model includes two common feature extraction modules of lesion classification network and a feature discrimination network. Firstly, two sets of training samples (positive and negative sample pairs) are input into the feature extraction module (Lightweight CNN) of the recognition model. Then, two sets of feature vectors output from the feature extraction module are used to train the two classification networks and feature discrimination networks of the recognition model at the same time, and the model fusion strategy is applied to further improve the performance of the model, the proposed recognition method can extract more discriminative lesion features and improve the recognition performance of the model in a small amount of model parameters; In addition, based on the feature extraction module of the proposed recognition model, U-Net architecture, and migration training strategy, we build a lightweight semantic segmentation model of lesion area of dermoscopy image, which can achieve high precision lesion area segmentation end-to-end without complicated image preprocessing operation; The performance of our approach was appraised through widespread experiments comparative and feature visualization analysis, the outcome indicates that the proposed method has better performance than the start-of-the-art deep learning-based approach on the ISBI 2016 skin lesion analysis towards melanoma detection challenge dataset.

A lightweight CNN model and its application in intelligent practical teaching evaluation

In this paper, we propose a lightweight CNN model. Firstly, we standardize the existing CNN model structure based on the minimum computing unit, and second we apply a parameter control solution to solve the problem of parameter redundancy in the model. At last we build a lightweight nonaligned CNN model. The experimental results show that the model parameters can be reduced by more than 50% when the test error is almost the same. Through deep learning, the proposed model is applied to the practical teaching system to achieve the intelligent evaluation effect of the practical teaching process, while improve the quality and efficiency of teaching.

A Lightweight CNN Model for Refining Moving Vehicle Detection From Satellite Videos

With the recent development of earth observation technology, the satellites have obtained the ability to capture city-scale videos, which enable potential applications in intelligent traffic management. Because of the broad field-of-view, the moving vehicles in satellite videos are usually composed of only tens of pixels, making it difficult to differentiate true objects from noise and other distractors. In addition, the edges of tall building tops are often mistakenly detected as moving vehicles because of the effects of motion parallax. This article proposed a terse framework that can effectively suppress false targets, achieving high precision and recall. The study involves three parts: 1) An adaptive filtering method is proposed to reduce noise, thus making the detection algorithm more reliable; 2) Several background subtraction models are tested, and the best one is chosen to produce the preliminary detection results at high recall but low accuracy; 3) A lightweight convolutional neural network (LCNN) is designed and trained on a small collection of samples, and then used to eliminate false targets. The experiments and evaluations demonstrate that our method can largely improve the precision at the expense of a slight reduction of recall.

Class-specific discriminant regularization in real-time deep CNN models for binary classification problems

In this paper, we first propose lightweight deep CNN models, capable of effectively operating in real-time on-drone for high-resolution video input, addressing various binary classification problems, e.g. crowd, face, football player, and bicycle detection, in the context of media coverage of specific sport events by drones with increased decisional autonomy. Furthermore, we propose a novel class-specific discriminant regularizer in order to improve the generalization ability of the proposed real-time models, exploiting the nature of the considered two-class problems. The experimental evaluation on four datasets validates the effectiveness of the proposed regularizer in enhancing the generalization ability of the proposed models.

Helping the Visually Impaired See via Image Multi-labeling Based on SqueezeNet CNN

This work presents a deep learning method for scene description. (1) Background: This method is part of a larger system, called BlindSys, that assists the visually impaired in an indoor environment. The method detects the presence of certain objects, regardless of their position in the scene. This problem is also known as image multi-labeling. (2) Methods: Our proposed deep learning solution is based on a light-weight pre-trained CNN called SqueezeNet. We improved the SqueezeNet architecture by resetting the last convolutional layer to free weights, replacing its activation function from a rectified linear unit (ReLU) to a LeakyReLU, and adding a BatchNormalization layer thereafter. We also replaced the activation functions at the output layer from softmax to linear functions. These adjustments make up the main contributions in this work. (3) Results: The proposed solution is tested on four image multi-labeling datasets representing different indoor environments. It has achieved results better than state-of-the-art solutions both in terms of accuracy and processing time. (4) Conclusions: The proposed deep CNN is an effective solution for predicting the presence of objects in a scene and can be successfully used as a module within BlindSys.

S-DenseNet: A DenseNet Compression Model Based on Convolution Grouping Strategy Using Skyline Method

DenseNet has strong expressiveness in many computer vision tasks, but the complexity of its model makes it difficult to deploy to devices with limited computing resources. In this paper, we propose S-DenseNet, a compact model of DenseNet which makes the extracting features of DenseNet more comprehensively and reduces the parameter redundancy at the same time. Firstly, we investigate the parameter redundancy of DenseNet and show the possibility of compressing DenseNet. For example, we reduced its parameters by 10% as keeping in the same accuracy. Secondly, we propose an algorithm based on the Skyline method for converting standard convolution into group convolution automatically. It can choose the filter weights automatically and restore the standard convolution filter with a high sparsity one called S-Conv. Finally, we combine S-Conv with dense connections and formed a more effective network architecture, that is S-DenseNet. Experiments show that in Cifar10, Cifar100 and SVHN datasets, we use only 40% parameters and 20% FLOPs of DenseNet, and the accuracy is improved by 1%. Compared with the compression models of DenseNet and other CNNs, we use less complexity in the model and achieve higher or similar accuracy. On ImageNet dataset, compared with lightweight CNN models, traditional CNNs and their compression models, we achieve the higher or similar Top-1 accuracy with less complexity.

Radar Target Recognition Based on Feature Pyramid Fusion Lightweight CNN

In order to improve the accuracy and robustness of radar target recognition under low SNR conditions, a novel radar high range resolution profile (HRRP) target recognition method based on feature pyramid fusion lightweight CNN is proposed in this paper. The proposed method combines the multi-scale space theory with a deep convolutional neural network. Because of the local connection characteristic of convolutional kernel, feature extracted by CNN mainly focus on the local information of the target. To make full use of both the local and global information in the target HRRP, multi-scale representation of the HRRP with different Gaussian kernels is introduced to construct the multi-channel input of the model. The generalization performance is improved by reducing the parameters of the proposed model with a depthwise separable convolution feature extraction block. Simultaneously, feature pyramid fusion is adapted to take full advantage of the features extracted by each block, which effectively improves the stability of the model and the training efficiency. The experimental results show that the multi-scale representation of the HRRP contributes to robust feature extraction. Meanwhile, the proposed feature pyramid fusion lightweight CNN can effectively prevent over-fitting and improve the stability of the model.

Explorations on visual localization from active to passive

In this paper, we novelly consider visual localization in active and passive two ways, with simple definition that active localization assists device to estimate location of its interest while passive localization aids device to estimate its own location in environment. Expecting to indicate some insights into visual localization, we specifically performed two explorations on active localization and more importantly explored to upgrade them from active to passive localization with extra geometry information available. In order to produce unconstrained and accurate 2D location estimation of interested object, we constructed an active localization system by fusing detection, tracking and recognition. Based on recognition, we proposed a collaborative strategy making mutual enhancement between detection and tracking possible to obtain better performance on 2D location estimation. Meanwhile, to actively estimate semantic location of interested visual region, we employed latest state-of-the-art light weight CNN models specifically designed for efficiency and trained two of them with large place dataset in perspective of scene recognition. What’s more, using depth information available from RGB-D camera, we improved the active system for 2D location of interested object to a passive system for relative 3D location of device to the interested object. Firstly estimated was the 3D location of the interested object in the coordinate system of device, then relative location of device to the interested object in world coordinate system was deduced with appropriate assumption. Evaluations both subjectively on a RGB-D sequence obtained in a lab environment and practically on a robotic platform in an office environment indicated that the improved system was suitable for autonomous following robot. As well, the active system for rough semantic location estimation of interested visual region was promoted to a passive system for fine location estimation of device, with available 3D map describing the visited environment. In perspective of place recognition, we first adopted one of the efficient CNN models previously trained for semantic location estimation as a base to generate CNN features for both retrieval of candidate loops in the map and geometrical consistency checking of retrieved loops, then true loops were used to deduce fine location of device itself in environment. Comparison with state-of-the-art results reflected that the promoted system was adequate for long-term robotic autonomy. Achieving favorable performances, the presented four explorations have implied adequacy for elaborating on some insights into visual localization.

BornoNet: Bangla Handwritten Characters Recognition Using Convolutional Neural Network

Bangla handwriting recognition is becoming an important issue in several years but it becomes a challenge to get good performance due to the alignment and many of them are similar. A simple, lightweight CNN model has been proposed in this paper for classifying Bangla Handwriting Character, which contains 50 basic Bangla characters (11 vowels and 39 consonants). Experiments have been made on three datasets along with the BanglaLekha-Isolated [1] CMATERdb [2] and the ISI [3] dataset. For character recognition, the proposed BornoNet model gets 98%, 96.81%, 95.71%, and 96.40% validation accuracy respectively for CMATERdb, ISI, BanglaLekha-Isolated dataset and mixed dataset. Also proposed model was trained with one dataset and cross-validated with other two datasets. Proposed model achieved the best accuracy rate so far for BanglaLekha-Isolated, CMATERdb and ISI datasets.The proposed BornoNet model can be found on https://github.com/shahariarrabby/BornoNet

Automatic Color Correction for Multisource Remote Sensing Images with Wasserstein CNN

In this paper a non-parametric model based on Wasserstein CNN is proposed for color correction. It is suitable for large-scale remote sensing image preprocessing from multiple sources under various viewing conditions, including illumination variances, atmosphere disturbances, and sensor and aspect angles. Color correction aims to alter the color palette of an input image to a standard reference which does not suffer from the mentioned disturbances. Most of current methods highly depend on the similarity between the inputs and the references, with respect to both the contents and the conditions, such as illumination and atmosphere condition. Segmentation is usually necessary to alleviate the color leakage effect on the edges. Different from the previous studies, the proposed method matches the color distribution of the input dataset with the references in a probabilistic optimal transportation framework. Multi-scale features are extracted from the intermediate layers of the lightweight CNN model and are utilized to infer the undisturbed distribution. The Wasserstein distance is utilized to calculate the cost function to measure the discrepancy between two color distributions. The advantage of the method is that no registration or segmentation processes are needed, benefiting from the local texture processing potential of the CNN models. Experimental results demonstrate that the proposed method is effective when the input and reference images are of different sources, resolutions, and under different illumination and atmosphere conditions.