Lightweight Deep Neural Network Research Articles

Noise Reduction of Nuclear Magnetic Resonance Spectroscopy Using Lightweight Deep Neural Network

Noise Reduction of Nuclear Magnetic Resonance Spectroscopy Using Lightweight Deep Neural Network

Intelligent fault diagnosis of multi-sensor rolling bearings based on variational mode extraction and a lightweight deep neural network

Intelligent fault diagnosis of multi-sensor rolling bearings based on variational mode extraction and a lightweight deep neural network

Intelligent fault diagnosis of multi-sensor rolling bearings based on variational mode extraction and a lightweight deep neural network

Intelligent fault diagnosis of multi-sensor rolling bearings based on variational mode extraction and a lightweight deep neural network

Toward DNN of LUTs: Learning Efficient Image Restoration with Multiple Look-Up Tables.

The widespread usage of high-definition screens on edge devices stimulates a strong demand for efficient image restoration algorithms. The way of caching deep learning models in a look-up table (LUT) is recently introduced to respond to this demand. However, the size of a single LUT grows exponentially with the increase of its indexing capacity, which restricts its receptive field and thus the performance. To overcome this intrinsic limitation of the single-LUT solution, we propose a universal method to construct multiple LUTs like a neural network, termed MuLUT. Firstly, we devise novel complementary indexing patterns, as well as a general implementation for arbitrary patterns, to construct multiple LUTs in parallel. Secondly, we propose a re-indexing mechanism to enable hierarchical indexing between cascaded LUTs. Finally, we introduce channel indexing to allow cross-channel interaction, enabling LUTs to process color channels jointly. In these principled ways, the total size of MuLUT is linear to its indexing capacity, yielding a practical solution to obtain superior performance with the enlarged receptive field. We examine the advantage of MuLUT on various image restoration tasks, including super-resolution, demosaicing, denoising, and deblocking. MuLUT achieves a significant improvement over the single-LUT solution, e.g., up to 1.1dB PSNR for super-resolution and up to 2.8dB PSNR for grayscale denoising, while preserving its efficiency, which is 100× less in energy cost compared with lightweight deep neural networks. Our code and trained models are publicly available at https://github.com/ddlee-cn/MuLUT.

Non-sample fuzzy based convolutional neural network model for noise artifact in biomedical images

The use of a light-weight deep learning Convolutional Neural Network (CNN) augmented with the power of Fuzzy Non-Sample Shearlet Transformation (FNSST) has successfully solved the problem of reducing noise and artifacts in Low-Dose Computed Tomography (LDCT) pictures. Both the Normal-Dose Computed Tomography (NDCT) and the Low-Dose Computed Tomography (LDCT) images from the dataset are subjected to the FNSST decomposition procedure during the training phase, producing high-frequency sub-images that act as input for the CNN. The CNN creates a meaningful connection between the high-frequency sub-images from LDCT and their corresponding residual sub-images during the training operation. The CNN is given the capacity to distinguish between LDCT high-frequency sub-images and expected high-frequency sub-images, which frequently have varying levels of noise or artifacts, especially in a fuzzy setting. The FNSST-CNN then successfully distinguishes LDCT high-frequency sub-images from the expected high-frequency sub-images during the testing phase, thereby reducing noise and artifacts. When compared to other approaches like KSVD, BM3D, and conventional image domain CNNs, the performance of FNSST-CNN is impressive as shown by better peak signal-to-noise ratios, stronger structural similarity, and a closer likeness to NDCT pictures.

Autonomous Identification of Bridge Concrete Cracks Using Unmanned Aircraft Images and Improved Lightweight Deep Convolutional Networks

The identification of the development of structural defects is an important part of bridge structure damage diagnosis, and cracks are considered the most typical and highly dangerous structural disease. However, existing deep learning‐based methods are mostly aimed at the scene of concrete cracks, while they rarely focus on designing network architectures to improve the vision‐based model performance from the perspective of unmanned aircraft system (UAS) inspection, which leads to a lack of specificity. Because of this, this study proposes a novel lightweight deep convolutional neural network‐based crack pixel‐level segmentation network for UAS‐based inspection scenes. Firstly, the classical encoder‐decoder architecture UNET is utilized as the base model for bridge structural crack identification, and the hourglass‐shaped depthwise separable convolution is introduced to replace the traditional convolutional operation in the UNET model to reduce model parameters. Then, a kind of lightweight and efficient channel attention module is used to improve model feature fuzzy ability and segmentation accuracy. We conducted a series of experiments on bridge structural crack detection tasks by utilizing a long‐span bridge as the research item. The experimental results show that the constructed method achieves an effective balance between reasoning accuracy and efficiency with the value of 97.62% precision, 97.23% recall, 97.42% accuracy, and 93.25% IOU on the bridge concrete crack datasets, which are significantly higher than those of other state‐of‐the‐art baseline methods. It can be inferred that the application of hourglass‐shaped depth‐separable volumes can actively reduce basic model parameters. Moreover, the lightweight and efficient attention modules can achieve local cross‐channel interaction without dimensionality reduction and improve the network segmentation performance.

Tensor-Empowered Hardware-Friendly Lightweight Deep Neural Networks for Vehicular Edge Computing

Tensor-Empowered Hardware-Friendly Lightweight Deep Neural Networks for Vehicular Edge Computing

Lightweight Deep Neural Network Model With Padding-Free Downsampling

Lightweight Deep Neural Network Model With Padding-Free Downsampling

Deep reinforcement learning with light-weight vision model for sequential robotic object sorting

Training an agent to generate cooperative joint learning policies for executing object sorting in cluttered scenes is complex. This paper presents a comparative model-free deep reinforcement learning (DRL) system built from variants of end-to-end lightweight deep neural networks that jointly complement three primitive policies (pushing, grasping, and placing) that operate from visual observations to actions. The end goal of the proposed DRL systems is to perform object sorting in very dynamic and complex environmental scenarios of cluttered and random objects (irregular and regular blocks) existing in different color variations. For this, we investigated DRL methods built from 12 custom instances of Pixel-wise Q-valued Critic Networks (PQCN) from four main backbone networks (DenseNet121, DenseNet169, MobileNetV3, and SqueezeNet) individually combined with custom fully convolutional neural networks (FCN) for learning the image-based observation space through affordance mapping paradigm while considering both dual and single transfer learning schemes. Additionally, we explored three categories of gradient-based optimization methods and considered custom reward functions with varying discount factors. The results show that the PQCN constructed by integrating DenseNet121 or MobileNetV3 with FCN and trained by a dual transfer learning scheme (with a discount factor of γ=0.70) yielded the best performance in most categories of our evaluation in the training phases. Overall, the PQCN-DenseNet121, when trained with a dual transfer learning process, yielded the best generalization (sorting success rate) in most of the evaluation metrics across all object categories in the testing phase. Furthermore, our study provides a new benchmark for validating deep reinforcement learning algorithms while sorting object blocks under varying degrees of complexity. This research finds practical relevance in industrial fields such as manufacturing and packaging, building construction, plant sorting, automobile assembly lines, and waste sorting. Here is the article’s source code: https://github.com/Emmanuel-Okafor/DRL_for_Object_Sorting.git.

A lightweight deep neural network model and its applications based on channel pruning and group vector quantization

A lightweight deep neural network model and its applications based on channel pruning and group vector quantization

Lightweight Pixel Difference Networks for Efficient Visual Representation Learning.

Recently, there have been tremendous efforts in developing lightweight Deep Neural Networks (DNNs) with satisfactory accuracy, which can enable the ubiquitous deployment of DNNs in edge devices. The core challenge of developing compact and efficient DNNs lies in how to balance the competing goals of achieving high accuracy and high efficiency. In this paper we propose two novel types of convolutions, dubbed Pixel Difference Convolution (PDC) and Binary PDC (Bi-PDC) which enjoy the following benefits: capturing higher-order local differential information, being computationally efficient, and can be integrated well into existing DNNs. With PDC and Bi-PDC, we further present two lightweight deep networks named Pixel Difference Networks (PiDiNet) and Binary PiDiNet (Bi-PiDiNet) respectively to learn highly efficient yet more accurate representations for visual tasks including edge detection and object recognition. Extensive experiments on popular datasets (BSDS500, ImageNet, LFW, YTF, etc.) show that PiDiNet and Bi-PiDiNet achieve the best accuracy-efficiency trade-off. For edge detection, PiDiNet is the first network that can be trained without ImageNet, and can achieve the human-level performance on BSDS500 at 100 FPS and with [Formula: see text]1M parameters. For object recognition, among existing Binary DNNs, Bi-PiDiNet achieves the best accuracy and a nearly 2× reduction of computational cost on ResNet18.

Tiny-PPG: A lightweight deep neural network for real-time detection of motion artifacts in photoplethysmogram signals on edge devices

Photoplethysmogram (PPG) signals are easily contaminated by motion artifacts in real-world settings, despite their widespread use in Internet-of-Things (IoT) based wearable and smart health devices for cardiovascular health monitoring. This study proposed a lightweight deep neural network, called Tiny-PPG, for accurate and real-time PPG artifact segmentation on IoT edge devices. The model was trained and tested on a public dataset, PPG DaLiA, which featured complex artifacts with diverse lengths and morphologies during various daily activities of 15 subjects using a watch-type device (Empatica E4). The model structure, training method and loss function were specifically designed to balance detection accuracy and speed for real-time PPG artifact detection in resource-constrained embedded devices. To optimize the model size and capability in multi-scale feature representation, the model employed depth-wise separable convolution and atrous spatial pyramid pooling modules, respectively. Additionally, the contrastive loss was also utilized to further optimize the feature embeddings. With additional model pruning, Tiny-PPG achieved state-of-the-art detection accuracy of 87.4% while only having 19,726 model parameters (0.15 megabytes), and was successfully deployed on an STM32 embedded system for real-time PPG artifact detection. Therefore, this study provides an effective solution for resource-constraint IoT smart health devices in PPG artifact detection.

Defect detection of MicroLED with low distinction based on deep learning

MicroLED has great prospects in visible light communication, optical detection and other fields. To solve limited efficiency and reliability issues caused by current manual quality control processes in MicroLED production environments, we propose a micro-vision-based automatic scanning system (MVASS) used to capture the MicroLED defect images and performing defect prediction inference. Since MicroLED defects are weak, they are very similar to normal MicroLEDs with low distinction, and the large number of MicroLEDs in an image, making a poor detection effect for the existing method. In this paper, a real-time detection method named AMS-YOLO base on lightweight deep convolutional neural network for MicroLED defects with low-distinction is proposed, utilizing the attention residual module (ARM) to extract multi-level effective features by focusing on the key features in the defect images by self-attention, and introducing the multi-scale feature fast fusion module (M3FM) to fuses the multi-level feature effectively and quickly to better capture the global information of the defect image, and using the selective feature pyramid module (SFPM) to selectively fuse the extracted features to enhance the network's classification and positioning capabilities and ultimately achieve precise MicroLED classification and detection. Experimental results on the MicroLED dataset show that AMS-YOLO demonstrates a better performance, with the mAP, Precision, Recall and speed increasing by 3.0 %, 3.5 %, 6.0 % and 4.3 % respectively, compared to the YOLOv5s.

DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks.

Convolutional neural network (CNN) architectures are generally heavy on memory and computational requirements which make them infeasible for embedded systems with limited hardware resources. We propose dual convolutional kernels (DualConv) for constructing lightweight deep neural networks. DualConv combines 3×3 and 1×1 convolutional kernels to process the same input feature map channels simultaneously and exploits the group convolution technique to efficiently arrange convolutional filters. DualConv can be employed in any CNN model such as VGG-16 and ResNet-50 for image classification, you only look once (YOLO) and R-CNN for object detection, or fully convolutional network (FCN) for semantic segmentation. In this work, we extensively test DualConv for classification since these network architectures form the backbone for many other tasks. We also test DualConv for image detection on YOLO-V3. Experimental results show that, combined with our structural innovations, DualConv significantly reduces the computational cost and number of parameters of deep neural networks while surprisingly achieving slightly higher accuracy than the original models in some cases. We use DualConv to further reduce the number of parameters of the lightweight MobileNetV2 by 54% with only 0.68% drop in accuracy on CIFAR-100 dataset. When the number of parameters is not an issue, DualConv increases the accuracy of MobileNetV1 by 4.11% on the same dataset. Furthermore, DualConv significantly improves the YOLO-V3 object detection speed and improves its accuracy by 4.4% on PASCAL visual object classes (VOC) dataset.

Smart-DNN+: A Memory-efficient Neural Networks Compression Framework for the Model Inference

Deep Neural Networks (DNNs) have achieved remarkable success in various real-world applications. However, running a Deep Neural Network (DNN) typically requires hundreds of megabytes of memory footprints, making it challenging to deploy on resource-constrained platforms such as mobile devices and IoT. Although mainstream DNNs compression techniques such as pruning, distillation, and quantization can reduce the memory overhead of model parameters during DNN inference, they suffer from three limitations: (i) low model compression ratio for the lightweight DNN structures with little redundancy, (ii) potential degradation in model inference accuracy, and (iii) inadequate memory compression ratio is attributable to ignoring the layering property of DNN inference. To address these issues, we propose a lightweight memory-efficient DNN inference framework called Smart-DNN+, which significantly reduces the memory costs of DNN inference without degrading the model quality. Specifically, ① Smart-DNN+ applies a layerwise binary-quantizer with a remapping mechanism to greatly reduce the model size by quantizing the typical floating-point DNN weights of 32-bit to the 1-bit signs layer by layer. To maintain model quality, ② Smart-DNN+ employs a bucket-encoder to keep the compressed quantization error by encoding the multiple similar floating-point residuals into the same integer bucket IDs. When running the compressed DNN in the user’s device, ③ Smart-DNN+ utilizes a partially decompressing strategy to greatly reduce the required memory overhead by first loading the compressed DNNs in memory and then dynamically decompressing the required materials for model inference layer by layer. Experimental results on popular DNNs and datasets demonstrate that Smart-DNN+ achieves lower 0.17%–0.92% memory costs at lower runtime overheads compared with the states of the art without degrading the inference accuracy. Moreover, Smart-DNN+ potentially reduces the inference runtime up to 2.04× that of conventional DNN inference workflow.

Large-scale Video Analytics with Cloud–Edge Collaborative Continuous Learning

Deep learning–based video analytics demands high network bandwidth to ferry the large volume of data when deployed on the cloud. When incorporated at the edge side, only lightweight deep neural network (DNN) models are affordable due to computational constraint. In this article, a cloud–edge collaborative architecture is proposed combining edge-based inference with cloud-assisted continuous learning. Lightweight DNN models are maintained at the edge servers and continuously retrained with a more comprehensive model on the cloud to achieve high video analytics performance while reducing the amount of data transmitted between edge servers and the cloud. The proposed design faces the challenge of constraints of both computation resources at the edge servers and network bandwidth of the edge–cloud links. An accuracy gradient-based resource allocation algorithm is proposed to allocate the limited computation and network resources across different video streams to achieve the maximum overall performance. A prototype system is implemented and experiment results demonstrate the effectiveness of our system with up to 28.6% absolute mean average precision gain compared with alternative designs.

Robust Payload Recognition Based on Sensor-Over-Muscle-Independence Deep Learning for the Control of Exoskeletons

Force myography (FMG) can detect changes in the muscle volume which can be interpreted to recognize human intention. FMG data, however, is highly dependent on the placement of sensors over muscles, and interpretation becomes challenging if the sensors get displaced. This paper presents a robust sensor over muscle independence (SOMI) preprocessing algorithm combined with lightweight deep neural network (DNN) which shows high classification accuracy of the FMG data. SOMI organizes the irregular sensory data into regular patterns. Proposed algorithm makes the DNN insensitive to not only position and rotation shift but also to the flip of the sensors arrangement. A custom designed FMG band is used for payload recognition to experimentally validate the proposed method with five payload statuses and eight subjects. A five-fold cross validation comparative study demonstrated that the proposed method is 19.8% and 7.1% more accurate than support vector machine (SVM) and DNN without SOMI, respectively, and showed superior performance against k-nearest neighbors (KNN) and decision tree (DT). SOMI empowered a lightweight DNN to maintain the accuracy over 98% for different arbitrary wearing schemes of the FMG band over both left and right upper arms.

Sound-Based Real-Time Scrap Falling State Monitoring in the Shearing Process of Wide Heavy Plate Mill

The scrap falling state in the shearing process of a Wide Heavy Plate Mill depends on manual monitoring and judgment. The steel plate can only move forward after the scrap falls, otherwise, the equipment will be damaged. An experienced operator can use the sound they hear to estimate the falling state. To automatically recognize sound events, a scheme of sound data acquisition processing and recognition for monitoring the scrap state of the shearing process is proposed in this paper. Firstly, a pickup is installed at the monitoring point of the crop shear to collect the sound. Afterward, the sound data is processed and finally fed into the model for recognition. Since many sound segments in the field are not related to key actions, an endpoint detection technique based on a short-term energy threshold and the variable window length strategy is applied in the data processing section to obtain the sound segments to be identified. The Log-Mel spectrogram feature of the sound segment is then extracted. After a data filling to ensure a uniform data shape, the feature map is fed into a lightweight deep convolution neural network called MobileNetV2 to quickly identify sound events. For the shearing process events, the model recognition accuracy can reach 95% of the F1 score in a realistic noise environment and we can identify the scrap falling 0.3s after the action starts. Finally, the model was applied to the UrbanSound8K dataset and achieved 95% recognition accuracy, validating the performance of the method.

Development of Speech Command Control Based TinyML System for Post-Stroke Dysarthria Therapy Device

Post-stroke dysarthria (PSD) is a widespread outcome of a stroke. To help in the objective evaluation of dysarthria, the development of pathological voice recognition and technology has a lot of attention. Soft robotics therapy devices have been received as an alternative rehabilitation and hand grasp assistance for improving activity daily living (ADL). Despite the significant progress in this field, most soft robotic therapy devices use a complex, bulky, lack of pathological voice recognition model, large computational power, and stationary controller. This study aims to develop a portable wirelessly multi-controller with a simulated dysarthric vowel speech in Bahasa Indonesia and non-dysarthric micro speech recognition, using tiny machine learning (TinyMl) system for hardware efficiency. The speech interface using INMP441, compute with a lightweight Deep Convolutional Neural network (DCNN) design and embedded into ESP-32. Feature model using Short Time Fourier Transform (STFT) and fed into CNN. This method has proven useful in micro-speech recognition with low computational power in both speech scenarios with a level of accuracy above 90%. Realtime inference performance on ESP-32 using hand prosthetics, with 3-level household noise intensity respectively 24db,42db, and 62db, and has respectively resulted from 95%, 85%, and 50% Accuracy. Wireless connectivity success rate with both controllers is around 0.2 - 0.5 ms.

Radar-Based Road Surface Classification System for Personal Mobility Devices

As personal mobility devices (PMDs) have been widely adopted, the accident rate also increased. For safe driving, the speed of the PMDs should be controlled by recognizing the condition of the road surface. In recent advances in artificial intelligence, various studies have been conducted to recognize the type of material based on radar sensors and machine learning. However, there are very limited studies addressing radar-based material recognition in noisy radar signal environments like PMDs. In this paper, we proposed a road surface classification scheme based on the 60 GHz pulsed radar, which has low cost and low power usage to be applied to various PMDs. We also developed a lightweight deep neural network model to recognize the type of road surface from the radar data after removing unnecessary features using principal component analysis (PCA). The proposed mechanism is robust against ambient noise and can recognize surface types on roads of variable shapes with high accuracy and low computational complexity. The results of experiments showed that the proposed lightweight surface classification system improved accuracy with only about 40% of the parameters used in the previous study.