Feature Pyramid Research Articles

Using U-Net models in deep learning for brain tumor detection from MRI scans

Tumor diseases in the nervous system are both dangerous and complex. Magnetic Resonance Imaging (MRI) is crucial for detecting brain disease; however, identifying the presence of tumors from these is time-consuming and requires a professional doctor. Utilizing deep learning for tumor detection in MRI images can reduce waiting times and enhance detection accuracy. We propose a method employing two U-Net models: ResNeXt-50 and EfficientNet architectures, integrated with a Feature Pyramid Network (FPN) for segmenting brain tumor. The models were trained on the BraTS 2021 dataset, consisting of 3,929 MRI scan images with 3,929 corresponding masks, divided into training, testing, and evaluation sets in a 70:15:15 ratio. The results indicate that the hybrid model, which combines EfficientNet and FPN, delivers superior performance, with an average Intersection over Union (IoU) accuracy of 0.90 on the test set compared to 0.50 for ResNeXt-50, and Dice accuracy of 0.92 compared to 0.66 for ResNeXt-50. Furthermore, we developed a web application that implements the EfficientNet with FPN model, facilitating convenient tumor detection from uploaded MRI images for doctors.

ADYOLOv5-Face: An Enhanced YOLO-Based Face Detector for Small Target Faces

Benefiting from advancements in generic object detectors, significant progress has been achieved in the field of face detection. Among these algorithms, the You Only Look Once (YOLO) series plays an important role due to its low training computation cost. However, we have observed that face detectors based on lightweight YOLO models struggle with accurately detecting small faces. This is because they preserve more semantic information for large faces while compromising the detailed information for small faces. To address this issue, this study makes two contributions to enhance detection performance, particularly for small faces: (1) modifying the neck part of the architecture by integrating a Gather-and-Distribute mechanism instead of the traditional Feature Pyramid Network to tackle the information fusion challenges inherent in YOLO-based models; and (2) incorporating an additional detection head specifically designed for detecting small faces. To evaluate the performance of the proposed face detector, we introduce a new dataset named XD-Face for the face detection task. In the experimental section, the proposed model is trained using the Wider Face dataset and evaluated on both Wider Face and XD-face datasets. Experimental results demonstrate that the proposed face detector outperforms other excellent face detectors across all datasets involving small faces and achieved improvements of 1.1%, 1.09%, and 1.35% in the AP50 metric on the WiderFace validation dataset compared to the baseline YOLOv5s-based face detector.

Colon polyp detection based on multi-scale and multi-level feature fusion and lightweight convolutional neural network

Early diagnosis and treatment of colorectal polyps are crucial for preventing colorectal cancer. This paper proposes a lightweight convolutional neural network for the automatic detection and auxiliary diagnosis of colorectal polyps. Initially, a 53-layer convolutional backbone network is used, incorporating a spatial pyramid pooling module to achieve feature extraction with different receptive field sizes. Subsequently, a feature pyramid network is employed to perform cross-scale fusion of feature maps from the backbone network. A spatial attention module is utilized to enhance the perception of polyp image boundaries and details. Further, a positional pattern attention module is used to automatically mine and integrate key features across different levels of feature maps, achieving rapid, efficient, and accurate automatic detection of colorectal polyps. The proposed model is evaluated on a clinical dataset, achieving an accuracy of 0.9982, recall of 0.9988, F1 score of 0.9984, and mean average precision (mAP) of 0.9953 at an intersection over union (IOU) threshold of 0.5, with a frame rate of 74 frames per second and a parameter count of 9.08 M. Compared to existing mainstream methods, the proposed method is lightweight, has low operating configuration requirements, high detection speed, and high accuracy, making it a feasible technical method and important tool for the early detection and diagnosis of colorectal cancer.

Optimization Study of Coal Gangue Detection in Intelligent Coal Selection Systems Based on the Improved Yolov8n Model

To address the low recognition accuracy of models for coal gangue images in intelligent coal preparation systems—especially in identifying small target coal gangue due to factors such as camera angle changes, low illumination, and motion blur—we propose an improved coal gangue separation model, Yolov8n-improvedGD(GD—Gangue Detection), based on Yolov8n. The optimization strategy includes integrating the GCBlock(Global Context Block) from GCNet(Global Context Network) into the backbone network to enhance the model’s ability to capture long-range dependencies in images and improve recognition performance. The CGFPN (Contextual Guidance Feature Pyramid Network) module is designed to optimize the feature fusion strategy and enhance the model’s feature expression capabilities. The GSConv-SlimNeck architecture is employed to optimize computational efficiency and enhance feature map fusion capabilities, thereby improving the model’s robustness. A 160 × 160 scale detection head is incorporated to enhance the sensitivity and accuracy of small coal and gangue detection, mitigate the effects of low-quality data, and improve target localization accuracy.

CSSDet: small object detection via cross-scale feature enhancement on drone-view images

ABSTRACT Object detection on drone-view images is vital for applications like intelligent transportation, abnormal behavior detection, and urban surveillance. However, the diverse perspectives and altitudes from which Unmanned Aerial Vehicle (UAV) capture scenes pose challenges due to multi-scale differences and small object sizes. To address this, we propose CSSDet, emphasizing multi-scale object detection and small object enhancement. CSSDet integrates an additional detection head for small objects and employs a bidirectional weighted feature pyramid network for effective cross-scale feature fusion, enhancing global perceptual capability. Additionally, we introduce coordinate attention and Involution modules to mitigate information loss. Experiments are conducted on two publicly available datasets, VisDrone and UAVDT. Quantitative results from the experiments indicate that the proposed CSSDet achieves optimal performance across a wide range of evaluation metrics, with particular strengths demonstrated in multi-scale object detection and small-size object detection. Qualitative results showcase its remarkable effectiveness in various aspects of object detection, including small object detection, objects at different scales and categories, multi-perspective scenarios, and complex scenes. The substantial experimental findings confirm that CSSDet attains comprehensive object detection performance on drone-view images. The source code is available at https://github.com/Gui-Cheng/CSSDet.

MFP-YOLO: a multi-scale feature perception network for CT bone metastasis detection.

Bone metastasis is one of the most common forms of metastasis in the late stages of malignancy. The early detection of bone metastases can help clinicians develop appropriate treatment plans. CT images are essential for diagnosing and assessing bone metastases in clinical practice. However, early bone metastasis lesions occupy a small part of the image and display variable sizes as the condition progresses, which adds complexity to the detection. To improve diagnostic efficiency, this paper proposes a novel algorithm-MFP-YOLO. Building on the YOLOv5 algorithm, this approach introduces a feature extraction module capable of capturing global information and designs a new content-aware feature pyramid structure to improve the network's capability in processing lesions of varying sizes. Moreover, this paper innovatively applies a transformer-structure decoder to bone metastasis detection. A dataset comprising 3921 CT images was created specifically for this task. The proposed method outperforms the baseline model with a 5.5% increase in precision and a 7.7% boost in recall. The experimental results indicate that this method can meet the needs of bone metastasis detection tasks in real scenarios and provide assistance for medical diagnosis.

Ship detection method based on attention guidance and multi-sample decision making

Single-stage target detection methods have the characteristics of fast training speed and short detection time. However, its feature pyramid network is difficult to suppress the background and noise information of SAR ship images, and the detection head has prediction errors. To address this problem, this paper proposes a detection model based on attention guidance and multi-sample decision for synthetic aperture radar ship detection. Firstly, an attention guidance network is proposed and added to the highest level of the feature pyramid to suppress background and noise interference, thereby improving the representation ability of features. Secondly, a multi-sample decision network is proposed to participate in the prediction of target position. This network alleviates the impact of prediction errors on detection results by increasing the number of samples output in the regression branch. Finally, a novel maximum likelihood loss function is designed. This loss function constructs a maximum likelihood function using the samples output from the multi-sample decision network, which is used to standardize the training of the decision network and further improve the accuracy of target positioning. Taking the RetinaNet network model as the baseline method, compared with the baseline method and the current advanced target detection methods, this method shows the highest detection accuracy on the ship detection dataset SSDD, with AP reaching 52.8%. Compared with the baseline method, the proposed method improves the AP evaluation index 3.4% ∼ 5.7%, and the training parameter Params only increases by 2.03 M, and the frame rate FPS only decreases 0.5Iter/s.

Visible light infrared matching algorithm based on improved SuperPoint and linear converter

In order to solve the problems of high difficulty and mismatch rate in heterogeneous image matching of visible light and infrared images, a deep learning matching algorithm based on improved SuperPoint and linear transformer is proposed. The algorithm first introduces the idea of feature pyramid to construct a feature description branch based on the SuperPoint network structure, and trains it based on the hinge loss function, so as to better learn the multi-scale deep features of visible light and infrared images and increase the similarity of image point pair descriptors with the same name; in the feature matching module, the linear transformer is used to improve the SuperGlue matching algorithm and aggregate features to improve matching performance. The proposed algorithm is experimentally verified on multiple data sets. The results show that compared with the existing algorithms, the algorithm achieves better matching results and improves the matching accuracy.

Sugarcane-YOLO: An Improved YOLOv8 Model for Accurate Identification of Sugarcane Seed Sprouts

Sugarcane is a crop that propagates through seed sprouts on nodes. Accurate identification of sugarcane seed sprouts is crucial for sugarcane planting and the development of intelligent sprout-cutting equipment. This paper proposes a sugarcane seed sprout recognition method based on the YOLOv8s model by adding the simple attention mechanism (SimAM) module to the neck network of the YOLOv8s model and adding the spatial-depth convolution (SPD-Conv) to the tail convolution part. Meanwhile, the E-IoU loss function is chosen to increase the model’s regression speed. Additionally, a small-object detection layer, P2, is incorporated into the feature pyramid network (FPN), and the large-object detection layer, P5, is eliminated to further improve the model’s recognition accuracy and speed. Then, the improvement of each part is tested and analyzed, and the effectiveness of the improved modules is verified. Finally, the Sugarcane-YOLO model is obtained. On the sugarcane seed and sprout dataset, the Sugarcane-YOLO model performed better and was more balanced in accuracy and detection speed than other mainstream models, and it was the most suitable model for seed and sprout recognition by automatic sugarcane-cutting equipment. Experimental results showed that the Sugarcane-YOLO achieved a mAP50 value of 99.05%, a mAP72 value of 81.3%, a mAP50-95 value of 71.61%, a precision of 97.42%, and a recall rate of 98.63%.

Aero-engine defect detection by integrating attention and multi-scale features

The structural integrity of aircraft engines is related to flight safety. At present, aircraft engine defect detection based on borescope technology is mainly manual operation. In order to improve the detection accuracy and efficiency, an intelligent aircraft engine defect detection algorithm that integrates attention and multi-scale features is proposed to assist borescope work. First, to address the class imbalance problem of defect samples in the original borescope image, a multi- sample fusion data enhancement method based on geometric transformation and Poisson image editing is used to enrich small sample images and construct a defect dataset. Then, the coordinated attention module (CA) is integrated into the baseline network YOLOv5 to emphasize the extraction of defect features and enhance the network's distinction between defect targets and complex backgrounds. A weighted bidirectional feature pyramid structure (BiFPN) is constructed in the neck network to complete higher-level feature fusion and improve the expression ability of multi-scale targets. Finally, the bounding box regression loss function is defined as the EIOU loss to achieve fast and accurate positioning and identification of defect targets. Experimental results show that the average accuracy of defect detection of the proposed algorithm reaches 89.7%, which is improved compared with the baseline network 6.3%, and the size of the trained model is only 14.0 M. Therefore, the proposed method can effectively detect the main defects of aircraft engines.

CBS-YOLOv5: fault detection algorithm of electrolyzer plate with low-resolution infrared images based on improved YOLOv5

Abstract In the process of copper electrorefining, accurate detection of electrode plate faults is extremely challenging due to the low resolution of captured infrared images, significant noise interference, and dense electrode plate arrangements. To address these challenges, this paper proposes an improved YOLOv5-based electrode plate fault detection algorithm called CBS-YOLOv5. This algorithm introduces several innovations over the original YOLOv5, including: the incorporation of coordinate attention to enhance the ability of the feature extraction network to separate target information from noise; the construction of a small object detection module to improve the detection of dense small objects by increasing the resolution of the feature map; the replacement of the traditional path aggregation network with a Bi-directional Feature Pyramid Network (BiFPN) for more flexible multi-scale feature fusion; and the integration of the swin transformer to optimize the cross-stage partial bottleneck structure, significantly enhancing the model’s ability to detect densely packed small objects. Experimental results show that the proposed CBS-YOLOv5 model achieves an accuracy of 88.1%, which is an improvement of 5.7% over the base model. Furthermore, this algorithm demonstrates exceptional detection capabilities for dense small objects in low-resolution infrared images while maintaining real-time detection speed, making it suitable for various complex industrial scenarios, including fault detection in non-ferrous metal electrolysis processes. CBS-YOLOv5 not only improves detection accuracy and robustness but also has broad application prospects, offering a new solution for intelligent manufacturing and industrial inspection.

A classification and recognition model for multiple fruit tree leaf diseases

Abstract Fruit tree leaf diseases affect both fruit survival rate and orchard revenue. Rapid and accurate identification of tree leaf diseases not only prevents the spread of diseases but also ensures healthy tree growth and improves fruit quality. Existing models mostly focus on individual types of tree leaf diseases, with limited applications in the identification of multiple tree leaf diseases. In response to the insensitivity of current classification models to disease region features and the issue of increased error rates due to the presence of similar diseases, this study proposes a residual network model based on the ECA channel attention mechanism and meta-Acon adaptive activation function.The model employs ResNet34 as the backbone network and incorporates the ECA module after each residual block to focus on relevant information. Additionally, a new activation function called meta-Acon is introduced to enhance the model's generalization ability through its dynamic learning capability. Finally, the model's recognition performance is improved by fusing bottom-level features with features from other layers using the Feature Pyramid Network (FPN).Experimental results on a dataset augmented with Mosaic processing show that the FPEM-ResNet34 model achieves a classification accuracy of 98.46%. Compared to other common models such as VGG-16, Inception-V1,AlexNet, and ResNet50, the proposed method mentioned in this paper demonstrates more effective improvement in the accuracy of tree leaf classification, making it highly valuable for practical applications..

Quantitative assessment of cracks in concrete structures using active-learning-integrated transformer and unmanned robotic platform

Quantitative assessment of cracks in concrete bridges is crucial for structural health monitoring and digital twin. However, the training of crack segmentation models relies heavily on annotation resources, and their segmentation capabilities are often unsatisfactory in terms of the accuracy of boundary location of thin cracks encountered in practice. In this paper, an active-learning-integrated crack segmentation transformer (ACS-Former) framework is proposed to maximize segmentation performance with limited annotation resources. The two-branch ACS-Former includes (1) a feature pyramid transformer (FPT) for multi-scale crack segmentation and (2) boundary difficulty-aware active learning (BDAL) to select informative images for labeling and incorporation into FPT training. Additionally, an adhesive climbing robot is proposed for image collection of hard-to-access components of large bridges. The on-site operational feasibility and practicability of the proposed ACS-Former and climbing robot are demonstrated by field experiments performed on in-service bridges, including the quantification of cracks narrower than 0.2 mm, as required by engineering codes.

Temporal-enhanced Radar and Camera Fusion for Object Detection

Recently, object detection methods based on multimodal fusion have gained widespread adoption in autonomous driving, proving to be valuable for detecting objects in dynamic environments. Among them, millimetre wave (mmWave) radar is commonly utilized as an effective complement to cameras, as it is almost unaffected by harsh weather conditions. However, current approaches that fuse mmWave radar and camera often overlook the correlation between the two modalities, failing to fully exploit their complementary features. To address this, we propose a temporal-enhanced radar and camera fusion network to explore the correlation between these two modalities and learn a comprehensive representation for object detection. In our model, a temporal fusion model is introduced to fuse mmWave radar features from different moments, thus mitigating the problem of mmWave radar point-object mismatch due to object movement. Moreover, a new correlation-based fusion strategy using the dedicated mask cross attention is proposed to fuse mmWave radar and vision features more effectively. Finally, we design a gate feature pyramid network that selects shallow texture information based on deep semantic information to obtain more representative features. The experimental results on the nuScenes benchmark demonstrate the effectiveness of our proposed method.

Nutritional composition analysis in food images: an innovative Swin Transformer approach.

Accurate recognition of nutritional components in food is crucial for dietary management and health monitoring. Current methods often rely on traditional chemical analysis techniques, which are time-consuming, require destructive sampling, and are not suitable for large-scale or real-time applications. Therefore, there is a pressing need for efficient, non-destructive, and accurate methods to identify and quantify nutrients in food. In this study, we propose a novel deep learning model that integrates EfficientNet, Swin Transformer, and Feature Pyramid Network (FPN) to enhance the accuracy and efficiency of food nutrient recognition. Our model combines the strengths of EfficientNet for feature extraction, Swin Transformer for capturing long-range dependencies, and FPN for multi-scale feature fusion. Experimental results demonstrate that our model significantly outperforms existing methods. On the Nutrition5k dataset, it achieves a Top-1 accuracy of 79.50% and a Mean Absolute Percentage Error (MAPE) for calorie prediction of 14.72%. On the ChinaMartFood109 dataset, the model achieves a Top-1 accuracy of 80.25% and a calorie MAPE of 15.21%. These results highlight the model's robustness and adaptability across diverse food images, providing a reliable and efficient tool for rapid, non-destructive nutrient detection. This advancement supports better dietary management and enhances the understanding of food nutrition, potentially leading to more effective health monitoring applications.

Lightweight Detection of Broccoli Heads in Complex Field Environments Based on LBDC-YOLO

Robotically selective broccoli harvesting requires precise lightweight detection models to efficiently detect broccoli heads. Therefore, this study introduces a lightweight and high-precision detection model named LBDC-YOLO (Lightweight Broccoli Detection in Complex Environment—You Look Only Once), based on the improved YOLOv8 (You Look Only Once, Version 8). The model incorporates the Slim-neck design paradigm based on GSConv to reduce computational complexity. Furthermore, Triplet Attention is integrated into the backbone network to capture cross-dimensional interactions between spatial and channel dimensions, enhancing the model’s feature extraction capability under multiple interfering factors. The original neck network structure is replaced with a BiFPN (Bidirectional Feature Pyramid Network), optimizing the cross-layer connection structure, and employing weighted fusion methods for better integration of multi-scale features. The model undergoes training and testing on a dataset constructed in real field conditions, featuring broccoli images under various influencing factors. Experimental results demonstrate that LBDC-YOLO achieves an average detection accuracy of 94.44% for broccoli. Compared to the original YOLOv8n, LBDC-YOLO achieves a 32.1% reduction in computational complexity, a 47.8% decrease in parameters, a 44.4% reduction in model size, and a 0.47 percentage point accuracy improvement. When compared to models such as YOLOv5n, YOLOv5s, and YOLOv7-tiny, LBDC-YOLO exhibits higher detection accuracy and lower computational complexity, presenting clear advantages for broccoli detection tasks in complex field environments. The results of this study provide an accurate and lightweight method for the detection of broccoli heads in complex field environments. This work aims to inspire further research in precision agriculture and to advance knowledge in model-assisted agricultural practices.

High-Resolution Reconstruction of Temperature Fields Based on Improved ResNet18.

High-precision measurement of temperature value distributions in production scenarios is of great significance for industrial production, but traditional temperature field reconstruction algorithms rely on the design of manual feature extraction methods with high computational complexity and poor generalization ability. In this paper, we propose a high-precision temperature field reconstruction algorithm based on deep learning, using an efficient adaptive feature extraction method for temperature field reconstruction. We design an improved temperature field reconstruction algorithm based on the ResNet18 neural network; introduce the CBAM attention mechanism in the model; and design a feature pyramid, using M-FPN, a multi-scale feature aggregation network fusing PAN and FPN, to make the extracted feature information propagate multi-dimensionally among different layers to improve the feature characterization ability. Finally, the mean square error is used to guide the model to optimize the training so that the model pays more attention to the data and reduces the large error to ensure that the gap between the predicted value and the real value is small. The experimental results show that the reconstruction accuracy of the improved algorithm presented in this paper is significantly better than that of the original algorithm in the case of typical peaked temperature field distributions.

Semantic Segmentation of Satellite Images for Landslide Detection Using Foreground-Aware and Multi-Scale Convolutional Attention Mechanism.

Advancements in satellite and aerial imagery technology have made it easier to obtain high-resolution remote sensing images, leading to widespread research and applications in various fields. Remote sensing image semantic segmentation is a crucial task that provides semantic and localization information for target objects. In addition to the large-scale variation issues common in most semantic segmentation datasets, aerial images present unique challenges, including high background complexity and imbalanced foreground-background ratios. However, general semantic segmentation methods primarily address scale variations in natural scenes and often neglect the specific challenges in remote sensing images, such as inadequate foreground modeling. In this paper, we present a foreground-aware remote sensing semantic segmentation model. The model introduces a multi-scale convolutional attention mechanism and utilizes a feature pyramid network architecture to extract multi-scale features, addressing the multi-scale problem. Additionally, we introduce a Foreground-Scene Relation Module to mitigate false alarms. The model enhances the foreground features by modeling the relationship between the foreground and the scene. In the loss function, a Soft Focal Loss is employed to focus on foreground samples during training, alleviating the foreground-background imbalance issue. Experimental results indicate that our proposed method outperforms current state-of-the-art general semantic segmentation methods and transformer-based methods on the LS dataset benchmark.

BEW-YOLOv8: A deep learning model for multi-scene and multi-scale flood depth estimation

As global climate change intensifies, the frequency of extreme weather events, including urban flooding, has risen, posing a significant threat to the safety of urban areas. Efficient and precise flood depth monitoring is crucial for a dynamic understanding of disaster conditions, enabling informed decision-making and reducing the potential for significant loss of life and property. Although object detection models have shown promise in estimating flood depths and have achieved notable results, enhancements are still needed in applications across multi-scene and multi-scale. This research enhances the YOLOv8 model by integrating Bidirectional Feature Pyramid Network (BiFPN), Effective Squeeze and Excitation (EffectiveSE), and Wise-IoU (WIoU), targeting submerged cars to develop the BEW-YOLOv8 model. Testing shows that the BEW-YOLOv8 model improves Precision, Recall, F1 Score, mAP@0.5, and mAP@0.5–0.95 by 14.1%, 2.2%, 7.9%, 8.7%, and 6.3%, respectively, compared to the unmodified YOLOv8 model. Compared with other existing models, BEW-YOLOv8 achieves the best performance with less than one-tenth of the parameters. Furthermore, this study verifies that the BEW-YOLOv8 model holds considerable promise for further enhancements with dataset expansions. Its precision and dependability make the BEW-YOLOv8 model well-suited for real-time flood depth assessment across multiple scenes and scales. The model’s capability for real-time processing supports urgent flood response needs, offering solid technical assistance for managing urban floods and mitigating disaster risks.

YOLO-RWY: A Novel Runway Detection Model for Vision-Based Autonomous Landing of Fixed-Wing Unmanned Aerial Vehicles

In scenarios where global navigation satellite systems (GNSSs) and radio navigation systems are denied, vision-based autonomous landing (VAL) for fixed-wing unmanned aerial vehicles (UAVs) becomes essential. Accurate and real-time runway detection in VAL is vital for providing precise positional and orientational guidance. However, existing research faces significant challenges, including insufficient accuracy, inadequate real-time performance, poor robustness, and high susceptibility to disturbances. To address these challenges, this paper introduces a novel single-stage, anchor-free, and decoupled vision-based runway detection framework, referred to as YOLO-RWY. First, an enhanced data augmentation (EDA) module is incorporated to perform various augmentations, enriching image diversity, and introducing perturbations that improve generalization and safety. Second, a large separable kernel attention (LSKA) module is integrated into the backbone structure to provide a lightweight attention mechanism with a broad receptive field, enhancing feature representation. Third, the neck structure is reorganized as a bidirectional feature pyramid network (BiFPN) module with skip connections and attention allocation, enabling efficient multi-scale and across-stage feature fusion. Finally, the regression loss and task-aligned learning (TAL) assigner are optimized using efficient intersection over union (EIoU) to improve localization evaluation, resulting in faster and more accurate convergence. Comprehensive experiments demonstrate that YOLO-RWY achieves AP50:95 scores of 0.760, 0.611, and 0.413 on synthetic, real nominal, and real edge test sets of the landing approach runway detection (LARD) dataset, respectively. Deployment experiments on an edge device show that YOLO-RWY achieves an inference speed of 154.4 FPS under FP32 quantization with an image size of 640. The results indicate that the proposed YOLO-RWY model possesses strong generalization and real-time capabilities, enabling accurate runway detection in complex and challenging visual environments, and providing support for the onboard VAL systems of fixed-wing UAVs.