Lightweight Backbone Research Articles

DESENet: a bilateral network with detail-enhanced semantic encoder for real-time semantic segmentation

Abstract Real-time semantic segmentation is widely used in various domains such as autonomous driving. Most real-time semantic segmentation networks adopt an encoder-decoder structure. During encoding, feature maps undergo multiple downsampling stages to enhance computational efficiency and enlarge the receptive field. However, this process may lead to the loss of detailed information, which may not be fully recovered during upsampling. Moreover, upsampling semantic features may cause blurring due to the lack of spatial detail information. To mitigate these issues, we use spatially guided cross-resolution self-attention to improve the upsampling of the semantic features by supplementing detailed information from the spatial branch. Furthermore, we incorporate an inductive bias into the cross-resolution attention mechanism to enhance its ability to learn generalized features. Additionally, we design a semantic feature extraction block, and a spatial feature extraction branch to construct a lightweight backbone. The results on Cityscapes and CamVid show that the proposed model achieves a good balance between accuracy and efficiency. Specifically, it obtains 74.2 and 71.5 mIoU on the two test datasets with 1.31M parameters, respectively. Code is available on https://github.com/clearwater753/DECENet.

Infrared target detection algorithm based on multipath coordinate attention mechanism

Abstract The current generation of infrared target detection algorithms frequently exhibits a high degree of dependency on parameter configurations within complex operational environments. This often results in a reduction in detection accuracy, an increase in the number of model parameters, and a slowing of the detection process. To address these limitations, a new algorithm, CGhostNet-Attention-YOLO (CAY), is proposed in this paper. Firstly, we designed a lightweight backbone network, CGhostNet, with the objective of improving feature extraction efficiency, thereby enabling accurate and real-time feature extraction. Furthermore, we proposed a Multipath Coordinate Attention (MPCA) mechanism, which incorporates both channel and positional information, thereby facilitating enhanced context awareness and the comprehension of relationships between different positions. This effectively enhances the model's ability to comprehend the overall meaning and addresses the issue of missed detections in infrared targets, significantly improving detection accuracy. Moreover, we employed the Inner-SIoU loss function to accelerate model convergence, reduce loss, and enhance the robustness of the model. Finally, comparative experiments were conducted on our dataset (IFD) as well as publicly available datasets, including FLIR, Pascal VOC, and NEU-DET. The results demonstrate that the CAY algorithm achieved a mean Average Precision (mAP@0.5) of 81.3% on the IFD dataset, 86.1% on the FLIR dataset, 79.2% on the Pascal VOC dataset, and 79.9% on the NEU-DET dataset, with a 27% reduction in the number of parameters. These findings validate the feasibility of the proposed algorithm.

LSKNet: A Foundation Lightweight Backbone for Remote Sensing

LSKNet: A Foundation Lightweight Backbone for Remote Sensing

SES-ReNet: Lightweight deep learning model for human detection in hazy weather conditions

Accurate detection of people in outdoor scenes plays an essential role in improving personal safety and security. However, existing human detection algorithms face significant challenges when visibility is reduced and human appearance is degraded, particularly in hazy weather conditions. To address this problem, we present a novel lightweight model based on the RetinaNet detection architecture.The model incorporates a lightweight backbone feature extractor, a dehazing functionality based on knowledge distillation (KD), and a multi-scale attention mechanism based on the Squeeze and Excitation (SE) principle. KD is done from a larger network trained on unhazed clear images, whereas SE is done at low-level and high-level features of the network. Experimental results have shown remarkable result, outperforming state-of-the-art methods while running at 22 FPS. The combination of high accuracy and real-time capabilities makes our approach a promising solution for effective human detection in challenging weather conditions and suitable for real-time applications.

A multi-slice attention fusion and multi-view personalized fusion lightweight network for Alzheimer’s disease diagnosis

ObjectiveAlzheimer’s disease (AD) is a type of neurological illness that significantly impacts individuals’ daily lives. In the intelligent diagnosis of AD, 3D networks require larger computational resources and storage space for training the models, leading to increased model complexity and training time. On the other hand, 2D slices analysis may overlook the 3D structural information of MRI and can result in information loss.ApproachWe propose a multi-slice attention fusion and multi-view personalized fusion lightweight network for automated AD diagnosis. It incorporates a multi-branch lightweight backbone to extract features from sagittal, axial, and coronal view of MRI, respectively. In addition, we introduce a novel multi-slice attention fusion module, which utilizes a combination of global and local channel attention mechanism to ensure consistent classification across multiple slices. Additionally, a multi-view personalized fusion module is tailored to assign appropriate weights to the three views, taking into account the varying significance of each view in achieving accurate classification results. To enhance the performance of the multi-view personalized fusion module, we utilize a label consistency loss to guide the model’s learning process. This encourages the acquisition of more consistent and stable representations across all three views.Main resultsThe suggested strategy is efficient in lowering the number of parameters and FLOPs, with only 3.75M and 4.45G respectively, and accuracy improved by 10.5% to 14% in three tasks. Moreover, in the classification tasks of AD vs. CN, AD vs. MCI and MCI vs. CN, the accuracy of the proposed method is 95.63%, 86.88% and 85.00%, respectively, which is superior to the existing methods.ConclusionsThe results show that the proposed approach not only excels in resource utilization, but also significantly outperforms the four comparison methods in terms of accuracy and sensitivity, particularly in detecting early-stage AD lesions. It can precisely capture and accurately identify subtle brain lesions, providing crucial technical support for early intervention and treatment.

Gene expression prediction from histology images via hypergraph neural networks.

Spatial transcriptomics reveals the spatial distribution of genes in complex tissues, providing crucial insights into biological processes, disease mechanisms, and drug development. The prediction of gene expression based on cost-effective histology images is a promising yet challenging field of research. Existing methods for gene prediction from histology images exhibit two major limitations. First, they ignore the intricate relationship between cell morphological information and gene expression. Second, these methods do not fully utilize the different latent stages of features extracted from the images. To address these limitations, we propose a novel hypergraph neural network model, HGGEP, to predict gene expressions from histology images. HGGEP includes a gradient enhancement module to enhance the model's perception of cell morphological information. A lightweight backbone network extracts multiple latent stage features from the image, followed by attention mechanisms to refine the representation of features at each latent stage and capture their relations with nearby features. To explore higher-order associations among multiple latent stage features, we stack them and feed into the hypergraph to establish associations among features at different scales. Experimental results on multiple datasets from disease samples including cancers and tumor disease, demonstrate the superior performance of our HGGEP model than existing methods.

A lightweight detection algorithm for tooth cracks in optical images

ObjectivesCracked tooth syndrome (CTS) is one of the major causes of tooth loss, presents the problem of early microcrack symptoms that are difficult to distinguish. This paper aims to investigate the practicality and feasibility of an improved object detection algorithm for automatically detecting cracks in dental optical images. MethodsA total of 286 teeth were obtained from Sun Yat-sen University and Guangdong University of Technology, and simulated cracks were generated using thermal expansion and contraction. Over 3000 images of cracked teeth were collected, including 360 real clinical images. To make the model more lightweight and better suited for deployment on embedded devices, this paper improves the YOLOv8 model for detecting tooth cracks through model pruning and backbone replacement. Additionally, the impact of image enhancement modules and coordinate attention modules on optimizing our model was analyzed. ResultsThrough experimental validation, we conclude that that model pruning reduction maintains performance better than replacing a lightweight backbone network on a tooth crack detection task. This approach achieved a reduction in parameters and GFLOPs by 16.8 % and 24.3 %, respectively, with minimal impact on performance. These results affirm the effectiveness of the proposed method in identifying and labeling tooth fractures. In addition, this paper demonstrated that the impact of image enhancement modules and coordinate attention mechanisms on YOLOv8's performance in the task of tooth crack detection was minimal. ConclusionsAn improved object detection algorithm has been proposed to reduce model parameters. This lightweight model is easier to deploy and holds potential for assisting dentists in identifying cracks on tooth surfaces.

LSR-Det: A Lightweight Detector for Ship Detection in SAR Images Based on Oriented Bounding Box

Convolutional neural networks (CNNs) have significantly advanced in recent years in detecting arbitrary-oriented ships in synthetic aperture radar (SAR) images. However, challenges remain with multi-scale target detection and deployment on satellite-based platforms due to the extensive model parameters and high computational complexity. To address these issues, we propose a lightweight method for arbitrary-oriented ship detection in SAR images, named LSR-Det. Specifically, we introduce a lightweight backbone network based on contour guidance, which reduces the number of parameters while maintaining excellent feature extraction capability. Additionally, a lightweight adaptive feature pyramid network is designed to enhance the fusion capability of the ship features across different layers with a low computational cost by incorporating adaptive ship feature fusion modules between the feature layers. To efficiently utilize the fused features, a lightweight rotating detection head is designed, incorporating the idea of sharing the convolutional parameters, thereby improving the network’s ability to detect multi-scale ship targets. The experiments conducted on the SAR ship detection dataset (SSDD) and the rotating ship detection dataset (RSDD-SAR) demonstrate that LSR-Det achieves an average precision (AP50) of 98.5% and 97.2% with 3.21 G floating point operations (FLOPs) and 0.98 M parameters, respectively, outperforming the current popular SAR arbitrary-direction ship target detection methods.

Stepwise Attention-Guided Multiscale Fusion Network for Lightweight and High-Accurate SAR Ship Detection

Many exceptional deep learning networks have demonstrated remarkable proficiency in general object detection tasks. However, the challenge of detecting ships in synthetic aperture radar (SAR) imagery increases due to the complex and various nature of these scenes. Moreover, sophisticated large-scale models necessitate substantial computational resources and hardware expenses. To address these issues, a new framework is proposed called a stepwise attention-guided multiscale feature fusion network (SAFN). Specifically, we introduce a stepwise attention mechanism designed to selectively emphasize relevant information and filter out irrelevant details of objects in a step-by-step manner. Firstly, a novel LGA-FasterNet is proposed, which incorporates a lightweight backbone FasterNet with lightweight global attention (LGA) to realize expressive feature extraction while reducing the model’s parameters. To effectively mitigate the impact of scale and complex background variations, a deformable attention bidirectional fusion network (DA-BFNet) is proposed, which introduces a novel deformable location attention (DLA) block and a novel deformable recognition attention (DRA) block, strategically integrating through bidirectional connections to achieve enhanced features fusion. Finally, we have substantiated the robustness of the new framework through extensive testing on the publicly accessible SAR datasets, HRSID and SSDD. The experimental outcomes demonstrate the competitive performance of our approach, showing a significant enhancement in ship detection accuracy compared to some state-of-the-art methods.

A lightweight feature extraction backbone for gangue detection with improved Resnet50

ABSTRACT In response to the sizable parameters and floating point operations (FLOPs) of feature extraction backbone of gangue detection algorithm, a lightweight backbone was proposed by improving Resnet50. Initially, the partial 3 × 3 convolution was used to replace regular 3 × 3 convolution in original Resnet50 for reducing parameters and FLOPs in convolution operation. Subsequently, layers of batch normalization and rectified linear unit after 3 × 3 convolution were removed for their further reduction. Finally, bottleneck attention module was integrated at the bottleneck of Resnet50 to improve the mean average precision (mAP) in gangue detection. Based on three proposed methods, the ablation and comparative experiments were carried out. The reduction in parameters and FLOPs and increase in mAP were analyzed and discussed according to experimental results. It could be concluded that the parameters and FLOPs of the obtained lightweight backbone were no more than 56.36% and 57.42% of those of the popular backbones, while the decrease amplitude of its mAP value was no more than 0.2%.

Lightweight transmission line defect identification method based on OFN network and distillation method

AbstractEdge devices are increasingly utilized for defect detection in power line inspection, necessitating algorithms that optimize model size and accuracy. This study introduces a lightweight detection method using the Optimized Feature Network with MobileOne‐FPN‐NASHead (OFN network) OFN network and distillation technique. The OFN incorporates a lightweight backbone, neural network search, re‐parametrization, and a feature pyramid to create a compact yet effective detection network. To enhance feature learning, a heterogeneous distillation approach is applied, leveraging a modified YOLOv8 as a teacher network. This modification includes an explicit visual centre for improved global and local information extraction, crucial for dense target detection in power line inspections. Additionally, the Minimize the points distance IoU (MPDloU) loss function is used to improve localization accuracy over the the Complete‐Intersection Over Union (CIoU) loss. Experimental results show a 1.1% mean Average Precision (mAP) increase for the enhanced YOLOv8 and a 70.2% mAP for the OFN network with 18.95 GFLOPs and 343 FPS, achieving a commendable balance between model efficiency and detection performance. The research underscores the viability of the OFN for edge computing in power line defect detection, highlighting the potential of innovative algorithmic structures in this application.

Lightweight detection of small tools for safer construction

Construction sites present significant potential safety hazards to the workers, with hand tools being a major source of injuries. This paper presents a Lightweight approach for Small Tools Detection (LSTD) method using a deep neural network for real-time detection of small construction tools. LSTD utilizes a lightweight backbone with Dynamic Feature Extraction, Accurate Separated Head, and Integrated Feature Fusion, reducing parameters by 73% and computations by 28% versus YOLOv5 while achieving 87.3% mean Average Precision (mAP) on challenging construction site datasets. Additional modules enhance detection recall and robustness to appearance variation and scale changes. Extensive experiments demonstrate LSTD's superior performance in misty conditions and illumination changes. With high accuracy in a compact 2.87 M parameter network, LSTD brings ubiquitous worker safety monitoring via edge devices closer to reality. The proposed model marks a significant advancement in improving safety in high-risk construction environments.

Melon ripeness detection by an improved object detection algorithm for resource constrained environments

BackgroundRipeness is a phenotype that significantly impacts the quality of fruits, constituting a crucial factor in the cultivation and harvesting processes. Manual detection methods and experimental analysis, however, are inefficient and costly.ResultsIn this study, we propose a lightweight and efficient melon ripeness detection method, MRD-YOLO, based on an improved object detection algorithm. The method combines a lightweight backbone network, MobileNetV3, a design paradigm Slim-neck, and a Coordinate Attention mechanism. Additionally, we have created a large-scale melon dataset sourced from a greenhouse based on ripeness. This dataset contains common complexities encountered in the field environment, such as occlusions, overlapping, and varying light intensities. MRD-YOLO achieves a mean Average Precision of 97.4% on this dataset, achieving accurate and reliable melon ripeness detection. Moreover, the method demands only 4.8 G FLOPs and 2.06 M parameters, representing 58.5% and 68.4% of the baseline YOLOv8n model, respectively. It comprehensively outperforms existing methods in terms of balanced accuracy and computational efficiency. Furthermore, it maintains real-time inference capability in GPU environments and demonstrates exceptional inference speed in CPU environments. The lightweight design of MRD-YOLO is anticipated to be deployed in various resource constrained mobile and edge devices, such as picking robots. Particularly noteworthy is its performance when tested on two melon datasets obtained from the Roboflow platform, achieving a mean Average Precision of 85.9%. This underscores its excellent generalization ability on untrained data.ConclusionsThis study presents an efficient method for melon ripeness detection, and the dataset utilized in this study, alongside the detection method, will provide a valuable reference for ripeness detection across various types of fruits.

A comprehensive survey of deep learning-based lightweight object detection models for edge devices

This study concentrates on deep learning-based lightweight object detection models on edge devices. Designing such lightweight object recognition models is more difficult than ever due to the growing demand for accurate, quick, and low-latency models for various edge devices. The most recent deep learning-based lightweight object detection methods are comprehensively described in this work. Information on the lightweight backbone architectures used by these object detectors has been listed. The training and inference processes concerning to deep learning applications on edge devices is being discussed. To raise readers’ awareness of this developing domain, a variety of applications for deep learning-based lightweight object detectors and related utilities have been offered. Designing potent, lightweight object detectors based on deep learning has been suggested as a counter to such problems. On well-known datasets such as MS-COCO and PASCAL-VOC, we thoroughly examine the performance of certain conventional deep learning-based lightweight object detectors.

An automated lightweight approach for detecting dead fish in a recirculating aquaculture system

Accurate methods for detecting dead fish are of great significance for fisheries because of their potential to improve production and reduce water pollution. Small-scale fisheries typically apply manual observation, but this approach is labor-intensive and time-consuming, which limits its application on large-scale farms. Advances in computer vision tools have provided the foundation for revolutionary methods of automated identification of individual-animal behavior. However, computer vision systems often suffer from unstable accuracy, low efficiency, and limited detection capability. Thus, the aim of this study was to design a deep learning system, designated “Deadfish-YOLO”, to detect dead fish based solely on standard underwater camera images with no additional hardware. First, eight selected rearing tanks were monitored by a self-developed computer version system. From these tank images, a dataset containing 18,114 frames was built by manual labeling. Next, a lightweight backbone network was generated with YOLOv4 to ensure fast computations, and an attention mechanism was introduced into the model to suppress unimportant features. Finally, the ReLU-memristor-like activation function was adopted to improve neural-network performance. The accuracy and processing speed of Deadfish-YOLO were superior to those of other state-of-the-art single- and two-stage detection models; Deadfish-YOLO ran at 85 frames per second with a mean average precision of 0.946 and an average ratio of 0.924 between intersection and union. These results demonstrate that Deadfish-YOLO may be used to automatically monitor dead fish in real circulating aquaculture systems. In addition, the results of this study should facilitate the wider application of artificial-intelligence-based animal monitoring in aquaculture.

Pineapple Detection with YOLOv7-Tiny Network Model Improved via Pruning and a Lightweight Backbone Sub-Network

High-complexity network models are challenging to execute on agricultural robots with limited computing capabilities in a large-scale pineapple planting environment in real time. Traditional module replacement often struggles to reduce model complexity while maintaining stable network accuracy effectively. This paper investigates a pineapple detection framework with a YOLOv7-tiny model improved via pruning and a lightweight backbone sub-network (the RGDP-YOLOv7-tiny model). The ReXNet network is designed to significantly reduce the number of parameters in the YOLOv7-tiny backbone network layer during the group-level pruning process. Meanwhile, to enhance the efficacy of the lightweight network, a GSConv network has been developed and integrated into the neck network, to further diminish the number of parameters. In addition, the detection network incorporates a decoupled head network aimed at separating the tasks of classification and localization, which can enhance the model’s convergence speed. The experimental results indicate that the network before pruning optimization achieved an improvement of 3.0% and 2.2%, in terms of mean average precision and F1 score, respectively. After pruning optimization, the RGDP-YOLOv7-tiny network was compressed to just 2.27 M in parameter count, 4.5 × 109 in computational complexity, and 5.0MB in model size, which were 37.8%, 34.1%, and 40.7% of the original YOLOv7-tiny network, respectively. Concurrently, the mean average precision and F1 score reached 87.9% and 87.4%, respectively, with increases of 0.8% and 1.3%. Ultimately, the model’s generalization performance was validated through heatmap visualization experiments. Overall, the proposed pineapple object detection framework can effectively enhance detection accuracy. In a large-scale fruit cultivation environment, especially under the constraints of hardware limitations and limited computational power in the real-time detection processes of agricultural robots, it facilitates the practical application of artificial intelligence algorithms in agricultural engineering.

A Efficient and Accurate UAV Detection Method Based on YOLOv5s

Due to the limited computational resources of portable devices, target detection models for drone detection face challenges in real-time deployment. To enhance the detection efficiency of low, slow, and small unmanned aerial vehicles (UAVs), this study introduces an efficient drone detection model based on YOLOv5s (EDU-YOLO), incorporating lightweight feature extraction and balanced feature fusion modules. The model employs the ShuffleNetV2 network and coordinate attention mechanisms to construct a lightweight backbone network, significantly reducing the number of model parameters. It also utilizes a bidirectional feature pyramid network and ghost convolutions to build a balanced neck network, enriching the model’s representational capacity. Additionally, a new loss function, EloU, replaces CIoU to improve the model’s positioning accuracy and accelerate network convergence. Experimental results indicate that, compared to the YOLOv5s algorithm, our model only experiences a minimal decrease in mAP by 1.1%, while reducing GFLOPs from 16.0 to 2.2 and increasing FPS from 153 to 188. This provides a substantial foundation for networked optoelectronic detection of UAVs and similar slow-moving aerial targets, expanding the defensive perimeter and enabling earlier warnings.

Welding Seam Tracking and Inspection Robot Based on Improved YOLOv8s-Seg Model.

A weld is the main connection form of special equipment, and a weld is also the most vulnerable part of special equipment. Therefore, an effective detection of a weld is of great significance to improve the safety of special equipment. The traditional inspection method is not only time-consuming and labor-intensive, but also expensive. The welding seam tracking and inspection robot can greatly improve the inspection efficiency and save on inspection costs. Therefore, this paper proposes a welding seam tracking and inspection robot based on YOLOv8s-seg. Firstly, the MobileNetV3 lightweight backbone network is used to replace the backbone part of YOLOv8s-seg to reduce the model parameters. Secondly, we reconstruct C2f and prune the number of output channels of the new building module C2fGhost. Finally, in order to make up for the precision loss caused by the lightweight model, we add an EMA attention mechanism after each detection layer in the neck part of the model. The experimental results show that the accuracy of weld recognition reaches 97.8%, and the model size is only 4.88 MB. The improved model is embedded in Jetson nano, a robot control system for seam tracking and detection, and TensorRT is used to accelerate the reasoning of the model. The total reasoning time from image segmentation to path fitting is only 54 ms, which meets the real-time requirements of the robot for seam tracking and detection, and realizes the path planning of the robot for inspecting the seam efficiently and accurately.

Research on Student Classroom Behavior Detection Based on the Real-Time Detection Transformer Algorithm

With the rapid development of artificial intelligence and big data technology, intelligent education systems have become a key research focus in the field of modern educational technology. This study aims to enhance the intelligence level of educational systems by accurately detecting student behavior in the classroom using deep learning techniques. We propose a method for detecting student classroom behavior based on an improved RT DETR (Real-Time Detection Transformer) object detection algorithm. By combining actual classroom observation data with AI-generated data, we create a comprehensive and diverse student behavior dataset (FSCB-dataset). This dataset not only more realistically simulates the classroom environment but also effectively addresses the scarcity of datasets and reduces the cost of dataset construction. The study introduces MobileNetV3 as a lightweight backbone network, reducing the model parameters to one-tenth of the original while maintaining nearly the same accuracy. Additionally, by incorporating learnable position encoding and dynamic upsampling techniques, the model significantly improves its ability to recognize small objects and complex scenes. Test results on the FSCB-dataset show that the improved model achieves significant improvements in real-time performance and computational efficiency. The lightweight network is also easy to deploy on mobile devices, demonstrating its practicality in resource-constrained environments.

A lightweight target tracking algorithm based on online correction for meta-learning

The traditional Siamese network based object tracking algorithms suffer from high computational complexity, making them difficult to run on embedded devices. Moreover, when faced with long-term tracking tasks, their success rates significantly decline. To address these issues, we propose a lightweight long-term object tracking algorithm called Meta-Master-based Ghost Fast Tracking (MGTtracker),which based on meta-learning. This algorithm integrates the Ghost mechanism to create a lightweight backbone network called G-ResNet, which accurately extracts target features while operating quickly. We design a tiny adaptive weighted fusion feature pyramid network (TiFPN) to enhance feature information fusion and mitigate interference from similar objects. We introduce a lightweight region regression network, the Ghost Decouple Net (GDNet) for target position prediction. Finally, we propose a meta-learning-based online template correction mechanism called Meta-Master to overcome error accumulation in long-term tracking tasks and the difficulty of reacquiring targets after loss. We evaluate the algorithm on public datasets OTB100, VOT2020, VOT2018LT, and LaSOT and deploy it for performance testing on Jetson Xavier NX. Experimental results demonstrate the effectiveness and superiority of the algorithm. Compared to existing classic object tracking algorithms, our approach achieves a faster running speed of 25 FPS on NX, and real-time correction enhances the algorithm’s robustness. Although similar in accuracy and EAO metrics, our algorithm outperforms similar algorithms in speed and effectively addresses the issues of significant cumulative errors and easy target loss during tracking. Code is released at https://github.com/ygh96521/MGTtracker.git.