Bounding Box Research Articles

Anchor-Based Multi-Scale Deep Grasp Pose Detector With Encoded Angle Regression

An intelligent robot grasping system should be able to automatically grasp a variety of objects that have never been seen, which requires accurate and efficient grasp pose detection. To this end, we propose a deep grasp detector designed for the robot equipped with a parallel gripper. The deep model consumes RGB or depth data and extracts features via a feature pyramid network (FPN), followed by multiple grasp prediction units to output grasp parameters in a single stage without refining process. Attaching grasp prediction units to different FPN stages increases the model capability to predict different-size grasps. Furthermore, in each prediction unit, the grasp parameters are regressed with the horizontal anchor as a reference to overcome the challenges posed by the various shapes of the grasp regions. We improve the accuracy and efficiency of grasp rotation estimation by regressing the angle directly and encoding the angle with a continuous Gaussian-like curve during training. This encoded angle regression strategy provides distance information of different angle predictions without introducing additional computational costs. Evaluations on three datasets prove the superior performance of our method than state of the arts. The experiments in real scenarios further validate the effectiveness of our grasping system. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper proposes a robot system that can automatically grasp novel objects with a parallel gripper and RGB-D camera. We focus on generating accurate grasp configurations for various objects using the captured color or depth image, which is the cornerstone of a successful grasp. To obtain effective and efficient grasp pose detection, we present a deep model that generates robust grasp poses represented by rotated bounding boxes for multiple novel objects. The first step of the grasp detector is to capture the image features through a feature pyramid network (FPN). Then, we attach separate grasp prediction units to each layer of the FPN stage and adopt anchors as references to make the model robust to variable grasp rectangle sizes. In each grasp prediction unit, two separate subnetworks are used to directly output the grasp rectangles and their probabilities, without using an extra second stage to refine the predicted grasp areas. For the prediction of rotation angle, we encode the rectangle angles with a continuous Gaussian-like curve during training to improve the prediction accuracy. Our grasp detector is trained and tested on three datasets and validated on real-scene grasp experiments. Comparisons with state-of-the-art methods show that our model is more accurate while maintaining high efficiency. The proposed grasp detection model can be applied to generate stable grasps for novel objects with different shapes, colors, and materials. Our grasping system is capable of working in multiple scenarios, including homes, factories, and warehouses.

React: recognize every action everywhere all at once

In the realm of computer vision, Group Activity Recognition (GAR) plays a vital role, finding applications in sports video analysis, surveillance, and social scene understanding. This paper introduces Recognize Every Action Everywhere All At Once (REACT), a novel architecture designed to model complex contextual relationships within videos. REACT leverages advanced transformer-based models for encoding intricate contextual relationships, enhancing understanding of group dynamics. Integrated Vision-Language Encoding facilitates efficient capture of spatiotemporal interactions and multi-modal information, enabling comprehensive scene understanding. The model’s precise action localization refines joint understanding of text and video data, enabling precise bounding box retrieval and enhancing semantic links between textual descriptions and visual reality. Actor-Specific Fusion strikes a balance between actor-specific details and contextual information, improving model specificity and robustness in recognizing group activities. Experimental results demonstrate REACT’s superiority over state-of-the-art GAR approaches, achieving higher accuracy in recognizing and understanding group activities across diverse datasets. This work significantly advances group activity recognition, offering a robust framework for nuanced scene comprehension.

Extraction of building footprint using MASK-RCNN for high resolution aerial imagery

Extracting individual buildings from satellite images is crucial for various urban applications, including population estimation, urban planning, and other related fields. However, Extracting building footprints from remote sensing data is a challenging task because of scale differences, complex structures and different types of building. Addressing these issues, an approach that can efficiently detect buildings in images by generating a segmentation mask for each instance is proposed in this paper. This approach incorporates the Regional Convolutional Neural Network (MASK-RCNN), which combines Faster R-CNN for object mask prediction and boundary box recognition and was evaluated against other models like YOLOv5, YOLOv7 and YOLOv8 in a comparative study to assess its effectiveness. The findings of this study reveals that our proposed method achieved the highest accuracy in building extraction. Furthermore, we performed experiments on well-established datasets like WHU and INRIA, and our method consistently outperformed other existing methods, producing reliable results.

TRAFFIC VIOLATION PREDICTION USING DEEP LEARNING BASED ON HELMETS WITH NUMBER PLATE RECOGNITION

At the moment, two-wheelers are the most widely used kind of transportation. It is highly advised that both bike riders and soldiers wear helmets. Many academics are interested in object tracking in video surveillance, which is an important application and burgeoning field of study in image processing and machine learning. Finding objects in an image with a bounding box and different categories or shapes of the objects placed is called object detection. This study reviews tracking techniques, classifies them into several categories, and concentrates on significant and practical tracking approaches. After reviewing broad strategies under a scan of the literature on various methodologies, we analyze potential research areas. This study uses the YOLOv8 algorithm, an image processing technique, to identify motorcycle riders who do not wear helmets. Additionally, put the Optical Character Recognition method into practice to identify the license plate in a picture and retrieve user information. Next, determine the fine amount. Eventually, SMS services will be able to notify consumers in order to prevent motorbike accidents. We evaluate the framework in terms of speed and accuracy.

Empowering lightweight detectors: Orientation Distillation via anti-ambiguous spatial transformation for remote sensing images

Knowledge distillation (KD) has been one of the most potential methods to implement a lightweight detector, which plays a significant role in satellite in-orbit processing and unmanned aerial vehicle tracking. However, existing distillation paradigms exhibit limited accuracy in detecting arbitrary-oriented objects represented with rotated bounding boxes in remote sensing images. This issue is attributed to two aspects: (i) boundary discontinuity localization distillation, caused by angle periodicity of rotated bounding boxes, and (ii) spatial ossified feature distillation, induced by orientation-agnostic knowledge transitive regions, both of which contribute to ambiguous orientation estimation of objects. To address these issues, we propose an effective KD method called Orientation Distillation (OD) via anti-ambiguous spatial transformation, which consists of two modules. (i) Anti-ambiguous Location Prediction (ALP) module reformulates the regression transformation between teacher–student bounding boxes as Gaussian distributions fitting procedure. These distributions with distilled potential are optimized to accurately localize objects with the aid of boundary continuity cost. (ii) Orientation-guided Feature Calibration (OFC) module employs a learnable affine matrix to augment fixed CNN sampling grid into a spatially remapped one, which bridges between the multi-scale feature of teacher and student for effectively delivering the refined oriented awareness within adaptively distillation regions. Overall, OD customizes the spatial transformation of bounding box representation and sampling grid to transfer anti-ambiguous orientation knowledge, and significantly improves the performance of lightweight detectors upon non-axially arranged objects. Extensive experiments on multiple datasets demonstrate that our plug-and-play distillation framework achieves state-of-the-art performance. Codes are available at https://github.com/Molly6/OD.

Forest fire detection utilizing ghost Swin transformer with attention and auxiliary geometric loss

Forest fires are a devastating natural disaster. Existing fire detection models face limitations in dataset availability, multi-scale feature extraction, and locating obscured or small flames and smoke. To address these issues, we develop a dataset containing real and synthetic forest fire images, sourced from a UAV (Unmanned Aerial Vehicle) perspective. Additionally, we propose the Ghost Convolution Swin Transformer (GCST) module to extract multi-scale flame and smoke features from different receptive fields by integrating parallel Ghost convolution and Swin Transformer. Subsequently, we design a lightweight reparameterized rotation attention module, which captures interactions across channel and spatial dimensions to suppress background noise and focus on obscured flames and smoke. Finally, we introduce a loss function, called Efficient Auxiliary Geometric Intersection over Union (EAGIoU), which employs an auxiliary bounding box to accelerate the model's convergence while integrating the geometrical principles of the predicted and real bounding boxes to accurately locate small flames and smoke. Extensive experimental results demonstrate that our method achieves 75.2 % mAP@0.5 and 42 % mAP@0.5:0.95 with a frame rate of 239 frames per second, indicating a significant improvement in accuracy and real-time performance compared to state-of-the-art techniques. The code and datasets are available at https://github.com/luckylil/forest-fire-detection.

A new network model for multiple object detection for autonomous vehicle detection in mining environment

AbstractConsidering the challenges of low multi‐object detection accuracy and difficulty in identifying small targets caused by challenging environmental conditions including irregular lighting patterns and ambient noise levels in the mining environment with autonomous electric locomotives. A new network model based on SOD−YOLOv5s−4L has been proposed to detect multi‐objects for autonomous electric locomotives in underground coal mines. Improvements have been applied in YOLOv5s to construct the SOD−YOLOv5s−4L model, by introducing the SIoU loss function to address the mismatch between real and predicted bounding box directions, facilitating the model to learn target position information more efficiently. This research introduces a decoupled head to enhance feature fusion and improve the positioning precision of the network model, enabling rapid capture of multi‐scale target features. Furthermore, the detection capability of the model has been increased by introducing the small target detection layer which is developed by increasing the number of detection layers from three to four. The experimental results on multiple object detection dataset show that the proposed model achieves significant improvement in mean average precision (mAP) of almost 98% for various types of targets and an average precision (AP) of nearly 99% for small targets on the other hand it achieves 5.19% (mAP) and 9.79% (AP) compared to the YOLOv5s model. Furthermore, comparative analysis with other models like YOLOv7 and YOLOv8 shows that the proposed model has superior performance in terms of object detection.

A Lightweight Rice Pest Detection Algorithm Using Improved Attention Mechanism and YOLOv8

Intelligent pest detection algorithms are capable of effectively detecting and recognizing agricultural pests, providing important recommendations for field pest control. However, existing recognition models have shortcomings such as poor accuracy or a large number of parameters. Therefore, this study proposes a lightweight and accurate rice pest detection algorithm based on improved YOLOv8. Firstly, a Multi-branch Convolutional Block Attention Module (M-CBAM) is constructed in the YOLOv8 network to enhance the feature extraction capability for pest targets, yielding better detection results. Secondly, the Minimum Points Distance Intersection over Union (MPDIoU) is introduced as a bounding box loss metric, enabling faster model convergence and improved detection results. Lastly, lightweight Ghost convolutional modules are utilized to significantly reduce model parameters while maintaining optimal detection performance. The experimental results demonstrate that the proposed method outperforms other detection models, with improvements observed in all evaluation metrics compared to the baseline model. On the test set, this method achieves a detection average precision of 95.8% and an F1-score of 94.6%, with a model parameter of 2.15 M, meeting the requirements of both accuracy and lightweightness. The efficacy of this approach is validated by the experimental findings, which provide specific solutions and technical references for intelligent pest detection.

Artificial Neural Network for Glider Detection in a Marine Environment by Improving a CNN Vision Encoder

Despite major economic and technological advances, much of the ocean remains unexplored, which has led to the use of remotely operated vehicles (ROVs) and gliders for surveying. ROVs and underwater gliders are essential for ocean data collection. Gliders, which control their own buoyancy, are particularly effective unmanned platforms for long-term observations. The traditional method of recovering the glider on a small boat is a risky operation and depends on the skill of the workers. Therefore, a safer, more efficient, and automated system is needed to recover them. In this study, we propose a lightweight artificial neural network for underwater glider detection that is efficient for learning and inference. In order to have a smaller parameter size and faster inference, a convolutional neural network (CNN) vision encoder in an artificial neural network splits an image of a glider into a number of elongated patches that overlap to better preserve the spatial information of the pixels in the horizontal and vertical directions. Global max-pooling, which computes the maximum over all the spatial locations of an input feature, was used to activate the most salient feature vectors at the end of the encoder. As a result of the inference of the glider detection models on the test dataset, the average precision (AP), which indicates the probability that an object is located within the predicted bounding box, shows that the proposed model achieves AP = 99.7%, while the EfficientDet-D2 model for comparison of detection performance achieves AP = 69.2% at an intersection over union (IOU) threshold of 0.5. Similarly, the proposed model achieves an AP of 78.9% and the EfficientDet-D2 model achieves an AP of 50.5% for an IOU threshold of 0.75. These results show that accurate prediction is possible within a wide range of recall for glider position inference in a real ocean environment.

Research on Automated Fiber Placement Surface Defect Detection Based on Improved YOLOv7

Due to the black and glossy appearance of the carbon fiber prepreg bundle surface, the accurate identification of surface defects in automated fiber placement (AFP) presents a high level of difficulty. Currently, the enhanced YOLOv7 algorithm demonstrates certain performance advantages in this detection task, yet issues with missed detections, false alarms, and low confidence levels persist. Therefore, this study proposes an improved YOLOv7 algorithm to further enhance the performance and generalization of surface defect detection in AFP. Firstly, to enhance the model’s feature extraction capability, the BiFormer attention mechanism is introduced to make the model pay more attention to small target defects, thereby improving feature discriminability. Next, the AFPN structure is used to replace the PAFPN at the neck layer to strengthen feature fusion, preserve semantic information to a greater extent, and finely integrate multi-scale features. Finally, WIoU is adopted to replace CIoU as the bounding box regression loss function, making it more sensitive to small targets, enabling more accurate prediction of object bounding boxes, and enhancing the model’s detection accuracy and generalization capability. Through a series of ablation experiments, the improved YOLOv7 shows a 10.5% increase in mAP and a 14 FPS increase in frame rate, with a maximum detection speed of 35 m/min during the AFP process, meeting the requirements of online detection and thus being able to be applied to surface defect detection in AFP operations.

Bud-YOLOv8s: A Potato Bud-Eye-Detection Algorithm Based on Improved YOLOv8s

The key to intelligent seed potato cutting technology lies in the accurate and rapid identification of potato bud eyes. Existing detection algorithms suffer from low recognition accuracy and high model complexity, resulting in an increased miss rate. To address these issues, this study proposes a potato bud-eye-detection algorithm based on an improved YOLOv8s. First, by integrating the Faster Neural Network (FasterNet) with the Efficient Multi-scale Attention (EMA) module, a novel Faster Block-EMA network structure is designed to replace the bottleneck components within the C2f module of YOLOv8s. This enhancement improves the model’s feature-extraction capability and computational efficiency for bud detection. Second, this study introduces a weighted bidirectional feature pyramid network (BiFPN) to optimize the neck network, achieving multi-scale fusion of potato bud eye features while significantly reducing the model’s parameters, computation, and size due to its flexible network topology. Finally, the Efficient Intersection over Union (EIoU) loss function is employed to optimize the bounding box regression process, further enhancing the model’s localization capability. The experimental results show that the improved model achieves a mean average precision (mAP@0.5) of 98.1% with a model size of only 11.1 MB. Compared to the baseline model, the mAP@0.5 and mAP@0.5:0.95 were improved by 3.1% and 4.5%, respectively, while the model’s parameters, size, and computation were reduced by 49.1%, 48.1%, and 31.1%, respectively. Additionally, compared to the YOLOv3, YOLOv5s, YOLOv6s, YOLOv7-tiny, and YOLOv8m algorithms, the mAP@0.5 was improved by 4.6%, 3.7%, 5.6%, 5.2%, and 3.3%, respectively. Therefore, the proposed algorithm not only significantly enhances the detection accuracy, but also greatly reduces the model complexity, providing essential technical support for the application and deployment of intelligent potato cutting technology.

Research on segmentation model of optic disc and optic cup in fundus

BackgroundGlaucoma is a worldwide eye disease that can cause irreversible vision loss. Early detection of glaucoma is important to reduce vision loss, and retinal fundus image examination is one of the most commonly used solutions for glaucoma diagnosis due to its low cost. Clinically, the cup-disc ratio of fundus images is an important indicator for glaucoma diagnosis. In recent years, there have been an increasing number of algorithms for segmentation and recognition of the optic disc (OD) and optic cup (OC), but these algorithms generally have poor universality, segmentation performance, and segmentation accuracy.MethodsBy improving the YOLOv8 algorithm for segmentation of OD and OC. Firstly, a set of algorithms was designed to adapt the REFUGE dataset’s result images to the input format of the YOLOv8 algorithm. Secondly, in order to improve segmentation performance, the network structure of YOLOv8 was improved, including adding a ROI (Region of Interest) module, modifying the bounding box regression loss function from CIOU to Focal-EIoU. Finally, by training and testing the REFUGE dataset, the improved YOLOv8 algorithm was evaluated.ResultsThe experimental results show that the improved YOLOv8 algorithm achieves good segmentation performance on the REFUGE dataset. In the OD and OC segmentation tests, the F1 score is 0.999.ConclusionsWe improved the YOLOv8 algorithm and applied the improved model to the segmentation task of OD and OC in fundus images. The results show that our improved model is far superior to the mainstream U-Net model in terms of training speed, segmentation performance, and segmentation accuracy.

Target Detection of Diamond Nanostructures Based on Improved YOLOv8 Modeling.

Boron-doped diamond thin films exhibit extensive applications in chemical sensing, in which the performance could be further enhanced by nano-structuring of the surfaces. In order to discover the relationship between diamond nanostructures and properties, this paper is dedicated to deep learning target detection methods. However, great challenges, such as noise, unclear target boundaries, and mutual occlusion between targets, are inevitable during the target detection of nanostructures. To tackle these challenges, DWS-YOLOv8 (DCN + WIoU + SA + YOLOv8n) is introduced to optimize the YOLOv8n model for the detection of diamond nanostructures. A deformable convolutional C2f (DCN_C2f) module is integrated into the backbone network, as is a shuffling attention (SA) mechanism, for adaptively tuning the perceptual field of the network and reducing the effect of noise. Finally, Wise-IoU (WIoU)v3 is utilized as a bounding box regression loss to enhance the model's ability to localize diamond nanostructures. Compared to YOLOv8n, a 9.4% higher detection accuracy is achieved for the present model with reduced computational complexity. Additionally, the enhancement of precision (P), recall (R), mAP@0.5, and mAP@0.5:0.95 is demonstrated, which validates the effectiveness of the present DWS-YOLOv8 method. These methods provide effective support for the subsequent understanding and customization of the properties of surface nanostructures.

Counting nematodes made easy: leveraging AI-powered automation for enhanced efficiency and precision.

Counting nematodes is a labor-intensive and time-consuming task, yet it is a pivotal step in various quantitative nematological studies; preparation of initial population densities and final population densities in pot, micro-plot and field trials for different objectives related to management including sampling and location of nematode infestation foci. Nematologists have long battled with the complexities of nematode counting, leading to several research initiatives aimed at automating this process. However, these research endeavors have primarily focused on identifying single-class objects within individual images. To enhance the practicality of this technology, there's a pressing need for an algorithm that cannot only detect but also classify multiple classes of objects concurrently. This study endeavors to tackle this challenge by developing a user-friendly Graphical User Interface (GUI) that comprises multiple deep learning algorithms, allowing simultaneous recognition and categorization of nematode eggs and second stage juveniles of Meloidogyne spp. In total of 650 images for eggs and 1339 images for juveniles were generated using two distinct imaging systems, resulting in 8655 eggs and 4742 Meloidogyne juveniles annotated using bounding box and segmentation, respectively. The deep-learning models were developed by leveraging the Convolutional Neural Networks (CNNs) machine learning architecture known as YOLOv8x. Our results showed that the models correctly identified eggs as eggs and Meloidogyne juveniles as Meloidogyne juveniles in 94% and 93% of instances, respectively. The model demonstrated higher than 0.70 coefficient correlation between model predictions and observations on unseen images. Our study has showcased the potential utility of these models in practical applications for the future. The GUI is made freely available to the public through the author's GitHub repository (https://github.com/bresilla/nematode_counting). While this study currently focuses on one genus, there are plans to expand the GUI's capabilities to include other economically significant genera of plant parasitic nematodes. Achieving these objectives, including enhancing the models' accuracy on different imaging systems, may necessitate collaboration among multiple nematology teams and laboratories, rather than being the work of a single entity. With the increasing interest among nematologists in harnessing machine learning, the authors are confident in the potential development of a universal automated nematode counting system accessible to all. This paper aims to serve as a framework and catalyst for initiating global collaboration toward this important goal.

Dynamic center point learning for multiple object tracking under Severe occlusions

Multiple Object Tracking (MOT) methods based on per-pixel prediction and association have achieved remarkable progress recently. These approaches prefer to select points in central region of the bounding box as positive samples and other points as negative samples. Under severe occlusions, these sample allocation methods might lead to the central region being contaminated by occluding samples, resulting in significant degradation of both detection and association performance. To address this issue, we propose a novel Dynamic-Center-Point-based Multiple Object Tracking method (DCP-MOT), which aims to self-identify the visible center region for occluded objects. Compared with the previous bounding box center point, our Dynamic-Center-Point (DCP) can better represent the visible region of the occluded object. Specifically, we design an Iterative Refinement Branch to generate dynamic center points for each occluded object. It includes two parts: Center Probability Predictor and Center Generator. Initially, we utilize the Center Probability Predictor to derive an accurate probability map for the occluded object. Subsequently, we use Center Generator to quantify the probability map by introducing a Mutual Exclusive Potential Function, yielding a dynamic center point for the occluded object that is distinct from its occluding counterpart. Finally, by joining the bounding box center points for unoccluded objects and our dynamic center points for occluded objects, our JDE branch can achieve better tracking performance. Extensive experiments demonstrate our DCP-based MOT method surpasses the bounding box center based SOTA in metrics MOTA (+0.7,+2.1,+1.3) and IDF1 (+0.7,+0.9,+1.5) on the challenging MOT16, MOT17, MOT20 datasets.

Steel surface defect detection algorithm in complex background scenarios

Detecting surface defects on steel poses a significant challenge attributed to factors such as poor contrast, diverse defect types, complex background clutter, and noise interference present in images of steel surface defects. Current detection techniques face challenges in quickly and accurately identifying defects within complex backgrounds. To address the deployment of high-precision detection models on edge devices with limited resources, particularly for identifying steel surface defects, this study introduces a Multi-Scale Adaptive Fusion (MSAF) YOLOv8n defect detection algorithm designed for complex backgrounds. This algorithm effectively balances detection speed and accuracy. Firstly, a Multi-Scale Adaptive Fusion Block (MS-AFB) is proposed for the extraction of multi-scale features. Secondly, a Dynamic Coordinate Attention Ghostconv Space Pooling Pyramid-fast Cross-stage Partial Convolutional (DCA-GSPPFCSPC) is devised to significantly improve detection accuracy. Furthermore, the detection head has been redesigned utilizing Lightweight Multi-scale Convolutional (LMSC) approach, and an Adaptive Pyramid Receptive Field Block (AP-RFB) has been introduced to improve the receptive field efficiently. Meanwhile, Normalized Weighted Distance (NWD) and Weighted Intersection over Union (WIoU) are employed as the boundary box loss functions, serving as substitutes for Complete Intersection over Union (CIoU) loss function with a ratio of 2:8. The experimental results obtained from the improved Northeastern University Defect Dataset (NEU-DET) dataset demonstrate that MSAF-YOLOv8n model, despite having 40.4 % of the parameters and 28.8 % of Floating Point Operations (FLOPs) of YOLOv8s, achieves a mAP@.5 that is 0.9 % higher than that of YOLOv8s. Additionally, MSAF-YOLOv8n demonstrates robust generalization capabilities in Pascal VOC2007, self-constructed datasets, and various other datasets. Subsequently, the model is implemented on embedded systems, namely Jeston TX2 NX and Orange Pi 5+, both of which demonstrate real-time detection capabilities.

Enhancing the Performance and Accuracy in Real-Time Football and Player Detection Using Upgraded YOLOv5 Architecture

The study presents a significantly improved version of the YOLOv5 real-time object detection model for football player recognition. The proposed technique includes feature-tuning and hyper-parameter optimization methods that have been carefully selected to enhance both speed and accuracy, resulting in a superior real-time performance of the YOLOv5 architecture. Furthermore, the YOLOv5 model incorporates a SimSPPF module that enables multi-scale feature extraction with less computational power, making it a highly efficient and effective solution. We selected the GhostNet module to reduce complexity and the Slim scale detection layer for precise bounding box prediction. Our tests, conducted with recordings of multiple football matches, demonstrate that our model accurately detects both the football and players even in complex scenarios with occlusions and dynamic illumination. The suggested method outperforms the original YOLOv5n model in terms of precision, recall, and mean average precision at 0.5 IoU. It is also more computationally efficient. This method has potential applications in live broadcasting, player monitoring, and sports analytics. The upgraded YOLOv5 model demonstrates superior accuracy and efficiency compared to previous methods that rely on traditional image processing techniques or two-stage detectors. This makes it highly suitable for practical, real-world deployments.

Weed detection and recognition in complex wheat fields based on an improved YOLOv7.

The precise detection of weeds in the field is the premise of implementing weed management. However, the similar color, morphology, and occlusion between wheat and weeds pose a challenge to the detection of weeds. In this study, a CSCW-YOLOv7 based on an improved YOLOv7 architecture was proposed to identify five types of weeds in complex wheat fields. First, a dataset was constructed for five weeds that are commonly found, namely, Descurainia sophia, thistle, golden saxifrage, shepherd's purse herb, and Artemisia argyi. Second, a wheat weed detection model called CSCW-YOLOv7 was proposed to achieve the accurate identification and classification of wheat weeds. In the CSCW-YOLOv7, the CARAFE operator was introduced as an up-sampling algorithm to improve the recognition of small targets. Then, the Squeeze-and-Excitation (SE) network was added to the Extended Latent Attention Networks (ELAN) module in the backbone network and the concatenation layer in the feature fusion module to enhance important weed features and suppress irrelevant features. In addition, the contextual transformer (CoT) module, a transformer-based architectural design, was used to capture global information and enhance self-attention by mining contextual information between neighboring keys. Finally, the Wise Intersection over Union (WIoU) loss function introducing a dynamic nonmonotonic focusing mechanism was employed to better predict the bounding boxes of the occluded weed. The ablation experiment results showed that the CSCW-YOLOv7 achieved the best performance among the other models. The accuracy, recall, and mean average precision (mAP) values of the CSCW-YOLOv7 were 97.7%, 98%, and 94.4%, respectively. Compared with the baseline YOLOv7, the improved CSCW-YOLOv7 obtained precision, recall, and mAP increases of 1.8%, 1%, and 2.1%, respectively. Meanwhile, the parameters were compressed by 10.7% with a 3.8-MB reduction, resulting in a 10% decrease in floating-point operations per second (FLOPs). The Gradient-weighted Class Activation Mapping (Grad-CAM) visualization method suggested that the CSCW-YOLOv7 can learn a more representative set of features that can help better locate the weeds of different scales in complex field environments. In addition, the performance of the CSCW-YOLOv7 was compared to the widely used deep learning models, and results indicated that the CSCW-YOLOv7 exhibits a better ability to distinguish the overlapped weeds and small-scale weeds. The overall results suggest that the CSCW-YOLOv7 is a promising tool for the detection of weeds and has great potential for field applications.

FSH-DETR: An Efficient End-to-End Fire Smoke and Human Detection Based on a Deformable DEtection TRansformer (DETR).

Fire is a significant security threat that can lead to casualties, property damage, and environmental damage. Despite the availability of object-detection algorithms, challenges persist in detecting fires, smoke, and humans. These challenges include poor performance in detecting small fires and smoke, as well as a high computational cost, which limits deployments. In this paper, we propose an end-to-end object detector for fire, smoke, and human detection based on Deformable DETR (DEtection TRansformer) called FSH-DETR. To effectively process multi-scale fire and smoke features, we propose a novel Mixed Encoder, which integrates SSFI (Separate Single-scale Feature Interaction Module) and CCFM (CNN-based Cross-scale Feature Fusion Module) for multi-scale fire, smoke, and human feature fusion. Furthermore, we enhance the convergence speed of FSH-DETR by incorporating a bounding box loss function called PIoUv2 (Powerful Intersection of Union), which improves the precision of fire, smoke, and human detection. Extensive experiments on the public dataset demonstrate that the proposed method surpasses state-of-the-art methods in terms of the mAP (mean Average Precision), with mAP and mAP50 reaching 66.7% and 84.2%, respectively.

Preventing falls from floor openings using quadrilateral detection and construction worker pose-estimation

This paper addressed safety risks in the construction industry, emphasizing the prevalent and fatal risk of falls from heights due to floor openings. Although advancements in computer vision and deep learning offer opportunities for automated safety monitoring, challenges such as inaccuracies in object localization and measuring distances to unsafe zones persist. To overcome these issues, a detection method employing convex quadrilateral bounding boxes was presented, taking into account perspective changes from the view of the camera. By leveraging a pretrained pose-estimation model and enhancing the YOLOv7 architecture, the new method precisely identified unsafe areas and generated virtual fences around floor openings. The presented approach resulted in an average precision of 80.55% and an F1-score of 86.49% in alerting dangers, outperforming existing techniques. This paper underscores the effective integration of state-of-the-art computer vision methodologies for practical safety monitoring in construction sites, highlighting its promise in accident prevention.