Low-level Feature Maps Research Articles

Detection Method for Bolt Missing Pins of Transmission Line Based on Improved SSD Network

The bolt pin structure plays a role in connecting and fixing various components in transmission lines, and the pins are prone to detachment, which may lead to the disintegration of key components and cause serious consequences. The bolt pin target size in the transmission line inspection image is small, the background is complex, and the detection effect using traditional object detection algorithms is poor. This article proposes a method for detecting missing bolts in transmission lines based on an improved SSD (Single Shot MultiBox Detector) network. Using the traditional SSD network as the foundation, a feature fusion module is first added to the network to enrich the semantic information of low-level feature maps and the detail information of high-level feature maps through the mutual fusion of high-level and low-level feature maps. Secondly, embedding a channel attention mechanism module enhances the channel features corresponding to the bolt pin structure and suppresses channel features that are irrelevant to the background; At the same time, the introduction of Focal Loss loss function reduces the unevenness of positive and negative sample classification. Finally, experiments were conducted on the collected bolt missing pin detection dataset, and the results showed that the method proposed in this paper has good performance in detecting missing pins, and the detection accuracy has been significantly improved.

Segmentation Method of Zanthoxylum bungeanum Cluster Based on Improved Mask R-CNN

The precise segmentation of Zanthoxylum bungeanum clusters is crucial for developing picking robots. An improved Mask R-CNN model was proposed in this study for the segmentation of Zanthoxylum bungeanum clusters in natural environments. Firstly, the Swin-Transformer network was introduced into the model’s backbone as the feature extraction network to enhance the model’s feature extraction capabilities. Then, the SK attention mechanism was utilized to fuse the detailed information into the mask branch from the low-level feature map of the feature pyramid network (FPN), aiming to supplement the image detail features. Finally, the distance intersection over union (DIOU) loss function was adopted to replace the original bounding box loss function of Mask R-CNN. The model was trained and tested based on a self-constructed Zanthoxylum bungeanum cluster dataset. Experiments showed that the improved Mask R-CNN model achieved 84.0% and 77.2% in detection mAP50box and segmentation mAP50mask, respectively, representing a 5.8% and 4.6% improvement over the baseline Mask R-CNN model. In comparison to conventional instance segmentation models, such as YOLACT, Mask Scoring R-CNN, and SOLOv2, the improved Mask R-CNN model also exhibited higher segmentation precision. This study can provide valuable technology support for the development of Zanthoxylum bungeanum picking robots.

GKE-TUNet: Geometry-Knowledge Embedded TransUNet Model for Retinal Vessel Segmentation Considering Anatomical Topology.

Automated retinal vessel segmentation is crucial for computer-aided clinical diagnosis and retinopathy screening. However, deep learning faces challenges in extracting complex intertwined structures and subtle small vessels from densely vascularized regions. To address these issues, we propose a novel segmentation model, called Geometry-Knowledge Embedded TransUNet (GKE-TUNet), which incorporates explicit embedding of topological features of retinal vessel anatomy. In the proposed GKE-TUNet model, a skeleton extraction network is pre-trained to extract the anatomical topology of retinal vessels from refined segmentation labels. During vessel segmentation, the dense skeleton graph is sampled as a graph of key-points and connections and is incorporated into the skip connection layer of TransUNet. The graph vertices are used as node features and correspond to positions in the low-level feature maps. The graph attention network (GAT) is used as the graph convolution backbone network to capture the shape semantics of vessels and the interaction of key locations along the topological direction. Finally, the node features obtained by graph convolution are read out as a sparse feature map based on their corresponding spatial coordinates. To address the problem of sparse feature maps, we employ convolution operators to fuse sparse feature maps with low-level dense feature maps. This fusion is weighted and connected to deep feature maps. Experimental results on the DRIVE, CHASE-DB1, and STARE datasets demonstrate the competitiveness of our proposed method compared to existing ones.

Improved DeepSORT-Based Object Tracking in Foggy Weather for AVs Using Sematic Labels and Fused Appearance Feature Network.

The presence of fog in the background can prevent small and distant objects from being detected, let alone tracked. Under safety-critical conditions, multi-object tracking models require faster tracking speed while maintaining high object-tracking accuracy. The original DeepSORT algorithm used YOLOv4 for the detection phase and a simple neural network for the deep appearance descriptor. Consequently, the feature map generated loses relevant details about the track being matched with a given detection in fog. Targets with a high degree of appearance similarity on the detection frame are more likely to be mismatched, resulting in identity switches or track failures in heavy fog. We propose an improved multi-object tracking model based on the DeepSORT algorithm to improve tracking accuracy and speed under foggy weather conditions. First, we employed our camera-radar fusion network (CR-YOLOnet) in the detection phase for faster and more accurate object detection. We proposed an appearance feature network to replace the basic convolutional neural network. We incorporated GhostNet to take the place of the traditional convolutional layers to generate more features and reduce computational complexities and costs. We adopted a segmentation module and fed the semantic labels of the corresponding input frame to add rich semantic information to the low-level appearance feature maps. Our proposed method outperformed YOLOv5 + DeepSORT with a 35.15% increase in multi-object tracking accuracy, a 32.65% increase in multi-object tracking precision, a speed increase by 37.56%, and identity switches decreased by 46.81%.

DLSW-YOLOv8n: A Novel Small Maritime Search and Rescue Object Detection Framework for UAV Images with Deformable Large Kernel Net

Unmanned aerial vehicle maritime search and rescue target detection is susceptible to external factors, which can seriously reduce detection accuracy. To address these challenges, the DLSW-YOLOv8n algorithm is proposed combining Deformable Large Kernel Net (DL-Net), SPD-Conv, and WIOU. Firstly, to refine the contextual understanding ability of the model, the DL-Net is integrated into the C2f module of the backbone network. Secondly, to enhance the small target characterization representation, a spatial-depth layer is used instead of pooling in the convolution module, and an additional detection head is integrated into the low-level feature map. The loss function is improved to enhance small target localization performance. Finally, a UAV maritime target detection dataset is employed to demonstrate the effectiveness of the proposed algorithm, whose results show that DLSW-YOLOv8n achieves a detection accuracy of 79.5%, which represents an improvement of 13.1% compared to YOLOv8n.

MAGNet: Multi-scale Awareness and Global fusion Network for RGB-D salient object detection

In recent years, excellent RGB-D salient object detection performance has been achieved. However, existing detection methods generally require a large number of model parameters in pursuit of high accuracy. To alleviate this problem, we propose a Multi-scale Awareness and Global fusion Network for RGB-D salient object detection, named MAGNet. MAGNet has 16.1M Params and 9.9G FLOPs. Specifically, we notice that convolutional neural networks (CNNs) can strongly perceive local spatial structures, whereas attention mechanisms can perform global correlation analysis of input information. Therefore, we exploit the advantages of both methods to design two kinds of cross-modal feature fusion modules. To reduce the computational complexity of the model, we design a multi-scale awareness fusion module (MAFM) to fully leverage the rich textural information and edge information in low-level feature maps. For the high-level feature maps, we incorporate an attention mechanism and a CNN to design a global fusion module (GFM), which enables the model to better capture the semantic information of different modalities by learning the correspondence between RGB and depth images. Then, we employ the proposed multi-level convolution module (MCM) to generate the predicted map through a step-by-step decoding process, which can gradually recover finer detection results. Finally, extensive experimental results on six datasets show that the proposed MAGNet not only achieves advanced detection performance but also drastically reduces the number of model parameters. Source code is available at https://github.com/mingyu6346/MAGNet.

BSANet: Boundary-aware and scale-aggregation networks for CMR image segmentation

The accurate segmentation of distinct cardiac regions from cardiac magnetic resonance (CMR) images is pivotal for enhancing the diagnosis and prognosis of heart diseases. However, the complex structure of the heart and the low contrast in CMR images pose significant challenges to achieving precise segmentation. In response to these difficulties, we propose a novel approach named BSANet, which integrates the inherent local characteristics of convolution with the global feature extraction of the Transformer through an elaborately designed multi-scale Boundary-Aware (MBA) module and a Scale-Aggregation TransFormer (SAT) module. The MBA module is specifically tailored to address the issue of blurry boundaries in CMR images. It enhances the edge extraction ability within low-level feature maps. On the other hand, the SAT module is designed to tackle the complexity and diversity of heart structures. This module retains crucial scale information to enhance segmentation performance while efficiently leveraging the computational capabilities of the Transformer via spatial reduction. Experimental results performed on three datasets show the superior performance of BSANet compared to the state-of-the-art methods. Notably, on the ACDC dataset, the proposed BSANet achieves the highest Dice score of 92.39%, while maintaining the lowest parameters. Furthermore, BSANet demonstrates optimal generalization performance on the RVSC and Sunnybrook datasets, underscoring its potential as a valuable tool for radiologists in clinical diagnosis.

Proportional feature pyramid network based on weight fusion for lane detection.

Lane detection under extreme conditions presents a highly challenging task that requires capturing each crucial pixel to predict the complex topology of lane lines and differentiate the various lane types. Existing methods predominantly rely on deep feature extraction networks with substantial parameters or the fusion of multiple prediction modules, resulting in large model sizes, embedding difficulties, and slow detection speeds. This article proposes a Proportional Feature Pyramid Network (P-FPN) through fusing the weights into the FPN for lane detection. For obtaining a more accurately detecting result, the cross refinement block is introduced in the P-FPN network. The cross refinement block takes the feature maps and anchors as inputs and gradually refines the anchors from high to low level feature maps. In our method, the high-level features are explored to predict lanes coarsely while local-detailed features are leveraged to improve localization accuracy. Extensive experiments on two widely used lane detection datasets, The Chinese Urban Scene Benchmark for Lane Detection (CULane) and the TuSimple Lane Detection Challenge (TuSimple) datasets, demonstrate that the proposed method achieves competitive results compared with several state-of-the-art approaches.

A Novel Building Extraction Network via Multi-Scale Foreground Modeling and Gated Boundary Refinement

Deep learning-based methods for building extraction from remote sensing images have been widely applied in fields such as land management and urban planning. However, extracting buildings from remote sensing images commonly faces challenges due to specific shooting angles. First, there exists a foreground–background imbalance issue, and the model excessively learns features unrelated to buildings, resulting in performance degradation and propagative interference. Second, buildings have complex boundary information, while conventional network architectures fail to capture fine boundaries. In this paper, we designed a multi-task U-shaped network (BFL-Net) to solve these problems. This network enhances the expression of the foreground and boundary features in the prediction results through foreground learning and boundary refinement, respectively. Specifically, the Foreground Mining Module (FMM) utilizes the relationship between buildings and multi-scale scene spaces to explicitly model, extract, and learn foreground features, which can enhance foreground and related contextual features. The Dense Dilated Convolutional Residual Block (DDCResBlock) and the Dual Gate Boundary Refinement Module (DGBRM) individually process the diverted regular stream and boundary stream. The former can effectively expand the receptive field, and the latter utilizes spatial and channel gates to activate boundary features in low-level feature maps, helping the network refine boundaries. The predictions of the network for the building, foreground, and boundary are respectively supervised by ground truth. The experimental results on the WHU Building Aerial Imagery and Massachusetts Buildings Datasets show that the IoU scores of BFL-Net are 91.37% and 74.50%, respectively, surpassing state-of-the-art models.

Research and Implementation of Millet Ear Detection Method Based on Lightweight YOLOv5.

As the millet ears are dense, small in size, and serious occlusion in the complex grain field scene, the target detection model suitable for this environment requires high computing power, and it is difficult to deploy the real-time detection of millet ears on mobile devices. A lightweight real-time detection method for millet ears is based on YOLOv5. First, the YOLOv5s model is improved by replacing the YOLOv5s backbone feature extraction network with the MobilenetV3 lightweight model to reduce model size. Then, using the multi-feature fusion detection structure, the micro-scale detection layer is augmented to reduce high-level feature maps and low-level feature maps. The Merge-NMS technique is used in post-processing for target information loss to reduce the influence of boundary blur on the detection effect and increase the detection accuracy of small and obstructed targets. Finally, the models reconstructed by different improved methods are trained and tested on the self-built millet ear data set. The AP value of the improved model in this study reaches 97.78%, F1-score is 94.20%, and the model size is only 7.56 MB, which is 53.28% of the standard YoloV5s model size, and has a better detection speed. Compared with other classical target detection models, it shows strong robustness and generalization ability. The lightweight model performs better in the detection of pictures and videos in the Jetson Nano. The results show that the improved lightweight YOLOv5 millet detection model in this study can overcome the influence of complex environments, and significantly improve the detection effect of millet under dense distribution and occlusion conditions. The millet detection model is deployed on the Jetson Nano, and the millet detection system is implemented based on the PyQt5 framework. The detection accuracy and detection speed of the millet detection system can meet the actual needs of intelligent agricultural machinery equipment and has a good application prospect.

SGNet: a fast and accurate semantic segmentation network based on semantic guidance

ABSTRACT In this paper, we design a fast and accurate lightweight semantic segmentation method for mobile robots. Accurate semantic segmentation usually requires obtaining high-resolution feature maps with strong semantic representation. Among the widely utilized methods, dilated convolution is computationally intensive and the feature pyramid accuracy is not as accurate. Inspired by the Flow Alignment Module (FAM) proposed by SFNet, we propose a Semantic Guided Upsampling Module (SGUM) for learning semantic offsets between neighboring level feature maps to solve the semantic misalignment problem and effectively fuse high level semantic features into low-level high-resolution feature maps. The integration of the SGUM into the semantic segmentation network with different backbones shows better performance than other methods on various datasets, showing the effectiveness of the SGUM on semantic feature fusion as well as its generalizability.

Robust Image Inpainting Forensics by Using an Attention-Based Feature Pyramid Network

Deep learning has injected a new sense of vitality into the field of image inpainting, allowing for the creation of more realistic inpainted images that are difficult to distinguish from the original ones. However, this also means that the malicious use of image inpainting technology to tamper with images could lead to more serious consequences. In this paper, we use an attention-based feature pyramid network (AFPN) to locate the inpainting traces left by deep learning. AFPN employs a feature pyramid to extract low- and high-level features of inpainted images. It further utilizes a multi-scale convolution attention (MSCA) module to optimize the high-level feature maps. The optimized high-level feature map is then fused with the low-level feature map to detect inpainted regions. Additionally, we introduce a fusion loss function to improve the training effectiveness. The experimental results show that AFPN exhibits remarkable precision in deep inpainting forensics and effectively resists JPEG compression and additive noise attacks.

Multiscale self-calibrated pulmonary nodule detection network fusing dual attention mechanism

Objective. In this study, we propose a model called DEPMSCNet (a multiscale self-calibration network) that has a high sensitivity and low false positive rate for detecting pulmonary nodules. Approach. First, at the feature extraction stage, we propose to use the REPSA-MSC module instead of the traditional convolutional neural network. The module extracts multiscale information from the feature map based on the image pyramid strategy while introducing adaptive convolutional branches to detect contextual information at each position of the multiscale, thereby expanding the receptive field and improving sensitivity. At the same time, multiple branches are adaptively weighted by channel attention, and the weights of different branches are adjusted to better generate pixel-level attention. Secondly, the proposed DSAM (dual-path spatial attention module) operates at the information fusion stage. This module fully exploits the rich spatial information of CT scans, obtains receptive field information from two branches, combines low-level feature map information with high-level semantic information, and enhances location-related information to effectively improve specificity. Thirdly, the focal loss function is used to solve the problem of positive and negative sample imbalance. Main results. The proposed model has been evaluated on the public lung nodule analysis (LUNA16) challenge dataset. The technique outperforms the most recent state-of-the-art detection algorithms in terms of sensitivity and specificity, obtaining a sensitivity of 0.988 and a competitive performance metric (CPM) of 0.963. Significance. Ablation experiments show that the two modules proposed in this paper effectively reduce false positives and improve sensitivity. This model effectively reduces the number of false positive nodules that doctors see on CT scans.

A Multi-Scale Object Detector Based on Coordinate and Global Information Aggregation for UAV Aerial Images

Unmanned aerial vehicle (UAV) image object detection has great application value in the military and civilian fields. However, the objects in the captured images from UAVs have problems of large-scale variation, complex backgrounds, and a large proportion of small objects. To resolve these problems, a multi-scale object detector based on coordinate and global information aggregation is proposed, named CGMDet. Firstly, a Coordinate and Global Information Aggregation Module (CGAM) is designed by aggregating local, coordinate, and global information, which can obtain features with richer context information. Secondly, a Feature Fusion Module (FFM) is proposed, which can better fuse features by learning the importance of different scale features and improve the representation ability of multi-scale features by reusing feature maps to help models better detect multi-scale objects. Moreover, more location information of low-level feature maps is integrated to improve the detection results of small targets. Furthermore, we modified the bounding box regression loss of the model to make the model more accurately regress the bounding box and faster convergence. Finally, we tested the CGMDet on VisDrone and UAVDT datasets. The proposed CGMDet improves mAP0.5 by 1.9% on the VisDrone dataset and 3.0% on the UAVDT dataset.

FRBNet: Feedback refinement boundary network for semantic segmentation in breast ultrasound images

Ultrasound breast tumour segmentation is a key step in computer-aided diagnosis and provides an important basis for clinical diagnosis and analysis. Accurate segmentation of breast tumours from ultrasound images is a challenging task due to the characteristics of black shadows, blurred boundaries and uneven colour intensity variations between classes. Currently, most breast tumour segmentation methods focus on extracting multi-scale information and fusing contextual information while underestimating the importance of feature information that can assist in identifying object boundaries in segmentation tasks. The loss of boundary feature information can easily lead to discontinuity or inaccuracy of the target boundary when the network generates the final prediction map. To address this problem, we propose a new feedback refinement boundary network (FRBNet) for accurate segmentation of breast tumour regions in ultrasound images, which mainly consists of a channel calibration module (CCM), boundary detection (BD) module, and feedback refinement module (FRM). Specifically, before fusing low-level feature maps with high-level feature maps, CCM first adopts the method of redistributing feature channel responses to enhance the channels carrying key target information and suppress the noisy channels in low-level feature maps. The BD module then improves the quality of the boundaries in the segmentation results by additionally learning the boundaries of breast tumours to provide accurate boundary feature information for subsequent prediction. The FRM employs a feedback mechanism that complementarily fuses the coarse prediction map and the feature map containing the target boundary feature information, thus achieving the best prediction results before generating the final prediction map. Experimental results on a public ultrasound breast dataset show that our network outperforms other medical image segmentation methods.

Long and short-range relevance context network for semantic segmentation

The semantic information can ensure better pixel classification, and the spatial information of the low-level feature map can ensure the detailed location of the pixels. However, this part of spatial information is often ignored in capturing semantic information, it is a huge loss for the spatial location of the image semantic category itself. To better alleviate this problem, we propose a Long and Short-Range Relevance Context Network. Specifically, we first construct a Long-Range Relevance Context Module to capture the global semantic context of the high-level feature and the ignored local spatial context information. At the same time, we build a Short-Range Relevance Context Module to capture the piecewise spatial context information in each stage of the low-level features in the form of jump connections. The whole network adopts a coding and decoding structure to better improve the segmentation results. Finally, we conduct a large number of experiments on three semantic segmentation datasets (PASCAL VOC2012, Cityscapes and ADE20K datasets) to verify the effectiveness of the network.

RoIA: Region of Interest Attention Network for Surface Defect Detection

Surface defect detection plays an important role in manufacturing and has aroused widespread interests. However, it is challenging as defects are highly similar to non-defects. To address this issue, this paper proposes a Region of Interest Attention (RoIA) network based on deep learning for automatically identifying surface defects. It consists of three parts: multi-level feature preservation (MFP) module, region proposal attention (RPA) module, and skip dense connection detection (SDCD) ones, where MFP is designed to differentiate defect features and texture information by feature reserved block, RPA is developed to locate the position of the defects by capturing global and local context information, and SDCD is proposed to better predict defect categories by propagating the fine-grained details from low-level feature map to high-level one. Experimental results conducted on three public datasets (e.g., NEU-DET, DAGM and Magnetic-Tile) demonstrate that the proposed method can significantly improve the detection performance than state-of-the-art ones and achieve an average defect detection accuracy of 99.49%

DARGS: Image inpainting algorithm via deep attention residuals group and semantics

To solving the problems that the existing image inpainting methods lack authenticity, do not deal with the information of missing and non-missing regions flexibly, and do not deal with the image feature information on different stages effectively, we propose an image restoration method combining Semantic Priors and Deep Attention Residual Group. The image restoration method mainly consists of Semantic Priors Network, Deep Attention Residual Group, and Full-scale Skip Connection. The Semantic Priors Network learns the complete semantic prior information of the visual elements in the missing regions and completes the missing regions with the learned semantic information. The deep attention residual set allows the generator to focus more on not only the missing regions of the image but also to learn the features of each channel adaptive. The full-scale jump joins can combine the low-level feature maps containing image boundaries with the high-level feature maps containing image textures and details to repair the missing regions. In this paper, a full comparison experiment is conducted on the CelebA-HQ and the Paris StreetView datasets, and the experimental results show that proposed method outperforms the current representative state-of-the-art image restoration methods.

Context Attention Fusion Network for crowd counting

Context information, which plays a crucial role in many computer vision tasks, can benefit deep networks to construct better cognitive competence from more comprehensive surrounding. However, most existing crowd counting methods often overlook the importance of context information extracted from both global and local views. To address or alleviate this problem, this paper proposes a novel Context Attention Fusion Network, which is abbreviated as CAFNet for crowd counting. The core idea behind CAFNet is the interaction of multiple context information, including local context, cross-level context, and cross-layer context. To explore local context, we design a local context aggregation module to extract hierarchically local semantic information and then integrate them adaptively. To utilize cross-level context, a guidance attention fusion module is designed to fuse low-level feature map as the guidance of high-level context information so that the spatial details can be effectively compensated. To make full use of cross-layer features, a multi-layer context fusion module is developed to exchange the potentiality of multi-layer information to generate a high-resolution density map. Experimental results on four challenging datasets manifest that the newly proposed CAFNet can deliver impressive results compared with other state-of-the-art crowd counting models.

Small and overlapping worker detection at construction sites

Although there has been study on worker detection using computer vision (CV) for the safety of construction sites, it is still challenging to identify employees who are obstructed or have poor vision. To solve these problems, we propose a method of small and overlapping target (worker) detection at a complex construction site named SOC-YOLO. The method is based on YOLOv5 and utilizes distance intersection over union (DIoU) non-maximum suppression (NMS), incorporating weighted triplet attention, expansion feature-level, and Soft-pool. Workers can be captured with overlap, particularly in large-scale construction sites, using the DIoU-based loss function, and NMS contributed to accuracy improvement. Next, we propose a weighted-triplet attention mechanism that can extract feature information from space more effectively and channel attention when learning object detection networks, using a simple average approach based on the same weight between the existing triplet attention. Next, we propose a model that adds additional predictive heads and residual connections to address the poor detection accuracy of workers photographed over long distances. A low-level feature map containing more information regarding small targets is used by extending the feature level. Finally, Softpool-spatial pyramid pooling fast (Softpool-SPPF) is proposed to solve the problem of inconsistent input image sizes. Softpool-SPPF performs an spatial pyramid pooling (SPP) function while preserving more functional information for accurate small target detection. Experiments were conducted using published worker detection datasets and handmade datasets, and the results showed increase from 81.26% to 84.63% average precision (AP) for small objects, from 67.52% to 73.88% mAP for minute objects, from 74.56% to77.57% for overlapping objects. The proposed method is expected to be useful for safety monitoring by applying it to the construction site worker tracking model.