Mean Average Precision Research Articles

Query Expansion Using Proposed Location-Based Algorithm for Hindi–English CLIR: Analyzing Three Test Collections

The rapid growth of contents on the Web in different languages increases the demand of Cross-Lingual Information Retrieval (CLIR). The accuracy of result suffers due to many problems such as ambiguity and drift issue in query. Query Expansion (QE) offers reliable solution for obtaining suitable documents for user queries. In this paper, we proposed an architecture for Hindi–English CLIR system using QE for improving the relevancy of retrieved results. In this architecture, for the addition of term(s) at appropriate position(s), we proposed a location-based algorithm to resolve the drift query issue in QE. User queries in Hindi language have been translated into document language (i.e. English) and the accuracy of translation is improved using Back-Translation. Google search has been performed and the retrieved documents are ranked using Okapi BM25 to arrange the documents in the order of decreasing relevancy to select the most suitable terms for QE. We used term selection value (TSV) for QE and for retrieving the terms, we created three test collections namely the (i) description and narration of the Forum for Information Retrieval Evaluation (FIRE) dataset, (ii) Snippets of retrieved documents against each query and (iii) Nearest-Neighborhood (NN) words against each query word among the ranked documents. To evaluate the system, 50 queries of Hindi language are selected from the FIRE-2012 dataset. In this paper, we performed two experiments: (i) impact of the proposed location-based algorithm on the proposed architecture of CLIR; and (ii) analysis of QE using three datasets, i.e. FIRE, NN and Snippets. In the first case, result shows that the relevancy of Hindi–English CLIR is improved by performing QE using the location-based algorithm and a 12% of improvement is achieved as compared to the results of QE obtained without applying the location-based algorithm. In the second case, the location-based algorithm is applied on three datasets. The Mean Average Precision (MAP) values of retrieved documents after QE are 0.5379 (NN), 0.6018 (FIRE) and 0.6406 (Snippets) for the three test collections, whereas the MAP before QE is 0.37102. This clearly shows the significant improvement of retrieved results for all three test collections. Among the three test collections, QE has been found most effective along with Snippets as indicated by the results with the improvements of 6.48% and 19.12% over FIRE and NN test collections, respectively.

UAV and Deep Learning: Detection of selected riparian species along the Ganga River

Abstract. Environmental protection and sustainable natural resource management are being recognised worldwide as essential goals to safe guard human health and well-being. Riparian zones, that face the highest decline in freshwater biodiversity, are of prime conservation priority because they are essential for regulating climate, preserving aquatic-terrestrial biodiversity, maintaining ground water recharge and restoring rivers. In today's fast-paced data-driven environment, artificial intelligence (AI) is the precise answer to a wide range of problems including biodiversity conservation and wildlife management. Leveraging advancements like Uncrewed/Unmanned Aerial Vehicles (UAVs) and AI has resulted innovative strides in wildlife conservation. This study utilised UAV imagery to record high-resolution data of aquatic habitat and species along the Ganga River and employed deep learning algorithms to analyse the data. Through extensive field surveys in the Hastinapur Wildlife Sanctuary, 7,025 photos representing a variety of environments, including 20,000 annotated samples of aquatic animals such as turtles and gharials were generated. Vision based computing capabilities such as pattern recognition model were developed to identify these species. To enrich and enhance the dataset and model, we used different image pre-processing techniques. Slight rotation (±5 degrees), minor cropping (up to 10%), and adjustments in brightness, saturation, and shear (±15%) were applied. Controlled blur (up to 0.5%) and exposure modifications (±5%) were also implemented on the image dataset to improve accuracy. Three Convolutional Neural Network (CNN) architectures, single-stage detectors named YOLO v7, YOLO v8, and Roboflow 3.0, were used for detecting the select species. Results show YOLO v8 excels, achieving mean average precision (mAP) of 98.8% for gharial and 92.2% for turtle detection, with a rapid average detection time of 0.308 seconds per frame at 3200 × 3200 resolution. Additionally, our model demonstrates real-time species detection capability through innovative frame sampling techniques with UAVs. This methodology provides promising technique to collect scientific data on IUCN red listed turtles and critically endangered gharials, allowing detection, monitoring, and real time counting with minimal intrusion. In conclusion, the fusion of UAVs and deep learning promises to revolutionize habitat monitoring, aiding conservation decision-making.

Radar-optical fusion detection of UAV based on improved YOLOv7-tiny

This study presents a radar-optical fusion detection method for unmanned aerial vehicles (UAVs) in maritime environments. Radar and camera technologies are integrated to improve the detection capabilities of the platforms. The proposed method involves generating regions of interest (ROI) by projecting radar traces onto optical images through matrix transformation and geometric centroid registration. The generated ROI are matched with YOLO detection boxes using the intersection-over-union (IoU) algorithm, enabling radar-optical fusion detection. A modified algorithm, called SPN-YOLOv7-tiny, is developed to address the challenge of detecting small UAV targets that are easily missed in images. In this algorithm, the convolutional layers in the backbone network are replaced with a space-to-depth convolution, and a small object detection layer is added. In addition, the loss function was replaced with a normalized weighted distance loss function. Experimental results demonstrate that compared to the original YOLOv7-tiny method, SPN-YOLOv7-tiny achieves an improved mAP@0.5 (mean average precision at an IoU threshold of 0.5) from 0.852 to 0.93, while maintaining a high frame rate of 135.1 frames per second. Moreover, the proposed radar-optical fusion detection method achieves an accuracy of 96.98%, surpassing the individual detection results of the radar and camera. The proposed method effectively addresses the detection challenges posed by closely spaced overlapping targets on a radar chart.

An efficient deep learning model for tomato disease detection

Tomatoes possess significant nutritional and economic value. However, frequent diseases can detrimentally impact their quality and yield. Images of tomato diseases captured amidst intricate backgrounds are susceptible to environmental disturbances, presenting challenges in achieving precise detection and identification outcomes. This study focuses on tomato disease images within intricate settings, particularly emphasizing four prevalent diseases (late blight, gray leaf spot, brown rot, and leaf mold), alongside healthy tomatoes. It addresses challenges such as excessive interference, imprecise lesion localization for small targets, and heightened false-positive and false-negative rates in real-world tomato cultivation settings. To address these challenges, we introduce a novel method for tomato disease detection named TomatoDet. Initially, we devise a feature extraction module integrating Swin-DDETR’s self-attention mechanism to craft a backbone feature extraction network, enhancing the model’s capacity to capture details regarding small target diseases through self-attention. Subsequently, we incorporate the dynamic activation function Meta-ACON within the backbone network to further amplify the network’s ability to depict disease-related features. Finally, we propose an enhanced bidirectional weighted feature pyramid network (IBiFPN) for merging multi-scale features and feeding the feature maps extracted by the backbone network into the multi-scale feature fusion module. This enhancement elevates detection accuracy and effectively mitigates false positives and false negatives arising from overlapping and occluded disease targets within intricate backgrounds. Our approach demonstrates remarkable efficacy, achieving a mean Average Precision (mAP) of 92.3% on a curated dataset, marking an 8.7% point improvement over the baseline method. Additionally, it attains a detection speed of 46.6 frames per second (FPS), adeptly meeting the demands of agricultural scenarios.

A New Dataset and Comparative Study for Aphid Cluster Detection and Segmentation in Sorghum Fields

Aphid infestations are one of the primary causes of extensive damage to wheat and sorghum fields and are one of the most common vectors for plant viruses, resulting in significant agricultural yield losses. To address this problem, farmers often employ the inefficient use of harmful chemical pesticides that have negative health and environmental impacts. As a result, a large amount of pesticide is wasted on areas without significant pest infestation. This brings to attention the urgent need for an intelligent autonomous system that can locate and spray sufficiently large infestations selectively within the complex crop canopies. We have developed a large multi-scale dataset for aphid cluster detection and segmentation, collected from actual sorghum fields and meticulously annotated to include clusters of aphids. Our dataset comprises a total of 54,742 image patches, showcasing a variety of viewpoints, diverse lighting conditions, and multiple scales, highlighting its effectiveness for real-world applications. In this study, we trained and evaluated four real-time semantic segmentation models and three object detection models specifically for aphid cluster segmentation and detection. Considering the balance between accuracy and efficiency, Fast-SCNN delivered the most effective segmentation results, achieving 80.46% mean precision, 81.21% mean recall, and 91.66 frames per second (FPS). For object detection, RT-DETR exhibited the best overall performance with a 61.63% mean average precision (mAP), 92.6% mean recall, and 72.55 on an NVIDIA V100 GPU. Our experiments further indicate that aphid cluster segmentation is more suitable for assessing aphid infestations than using detection models.

A bearing surface defect detection method based on multi-attention mechanism Yolov8

Surface defects in bearings not only affect the appearance but also impact the service life and performance. Therefore, it is imperative for bearing manufacturers to conduct quality inspections before bearings leave the factory. However, traditional visual inspection methods exhibit shortcomings such as high omission rates, insufficient feature fusion and oversized models when dealing with multiple target defects in bearings. To address these challenges, this paper proposes a surface defect detection method for bearings based on an improved Yolov8 algorithm (G-Yolov8). Firstly, a C3Ghost convolutional module based on the Ghost module is constructed in YOLOv8 to simplify model computational costs. Secondly, a global attention mechanism module is designed at the end of the backbone network to increase sensitivity to implicit small target area features and optimize feature extraction efficiency. Subsequently, a deep deformable convolution feature pyramid network is constructed by introducing the deformable convolutional networks version 2 (DCNv2) and the lightweight content-aware reassembly of features upsampling operator to reduce sampling information loss and improve the fusion of multi-scale target defects. Finally, different attention mechanisms are embedded in the detection network to construct a multi-attention detection head to replace the decoupled head, refining classification and localization tasks, reducing feature confusion, and improving the model’s detection accuracy. Experimental results demonstrate that the improved algorithm achieves a 3.5% increase in mean average precision on a self-made small-scale train bearing surface defect dataset, with a 17.3% reduction in model size. This improvement not only enhances accuracy but also addresses the requirement for lightweight deployment in subsequent stages.

Automated Detection and Counting of Wild Boar in Camera Trap Images

Camera traps are becoming widely used for wildlife monitoring and management. However, manual analysis of the resulting image sets is labor-intensive, time-consuming and costly. This study shows that automated computer vision techniques can be extremely helpful in this regard, as they can rapidly and automatically extract valuable information from the images. Specific training with a set of 1600 images obtained from a study where wild animals approaching wild boar carcasses were monitored enabled the model to detect five different classes of animals automatically in their natural environment with a mean average precision of 98.11%, namely ‘wild boar’, ‘fox’, ‘raccoon dog’, ‘deer’ and ‘bird’. In addition, sequences of images were automatically analyzed and the number of wild boar visits and respective group sizes were determined. This study may help to improve and speed up the monitoring of the potential spread of African swine fever virus in areas where wild boar are affected.

A convolutional neural network (Seg-CNN) for recognition, classifi cation, and segmentation of apple fruit branches and stems in images

This article reports the results of research studies conducted in 2023–2024 on transfer learning of Segmentation Convolutional Neural Networks (Seg-CNN) models for classification, recognition, and segmentation of branches with apple fruits and stems in images. State-of-the-art convolutional neural network architectures, i.e., YOLOv8(n,s,m,l,x)-seg, were used for a detailed segmentation of biological objects in images of varying complexity and scale at the pixel level. An image dataset collected in the fi eld using a GoPro HERO 11 camera was marked up for transfer model training. Data augmentation was performed, producing a total of 2500 images. Image markup was performed using the polygon annotation tool. As a result, polygonal contours around objects were created, outlines of branches, apple tree fruits, and stems were outlined, and segments of objects in the images were indicated. The objects were assigned the following classes: Apple branch, Apple fruit, and Apple stem. Binary classification metrics, such as Precision and Recall, as well as Mean Average Precision (mAP), were used to evaluate the performance of the trained models in recognizing branches with apple fruits and stems in images. The YOLOv8x-seg (mAP50 0.758) and YOLOv8l-seg (mAP50 0.74) models showed high performance in terms of all metrics in recognizing branches, apple fruit, and fruit stems in images, outperforming the YOLOv8n-seg (mAP50 0.7) model due to their more complex architecture. The YOLOv8n-seg model has a faster frame processing speed (11.39 frames/s), rendering it a preferred choice for computing systems with limited resources. The results obtained confirm the prospects of using machine learning algorithms and convolutional neural networks for segmentation and pixel-by-pixel classification of branches with apple fruits and stems on RGB images for monitoring the condition of plants and determining their geometric characteristics.

Real-Time Detection Technology of Corn Kernel Breakage and Mildew Based on Improved YOLOv5s

In order to solve low recognition of corn kernel breakage degree and corn kernel mildew degree during corn kernel harvesting, this paper proposes a real-time detection method for corn kernel breakage and mildew based on improved YOlOv5s, which is referred to as the CST-YOLOv5s model algorithm in this paper. The method continuously obtains images through the discrete uniform sampling device of corn kernels and generates whole corn kernels, breakage corn kernels, and mildew corn kernel dataset samples. We aimed at the problems of high similarity of some corn kernel features in the acquired images and the low precision of corn kernel breakage and mildew recognition. Firstly, the CBAM attention mechanism is added to the backbone network of YOLOv5s to finely allocate and process the feature information, highlighting the features of corn breakage and mildew. Secondly, the pyramid pooling structure SPPCPSC, which integrates cross-stage local networks, is adopted to replace the SPPF in YOLOv5s. SPP and CPSC technologies are used to extract and fuse features of different scales, improving the precision of object detection. Finally, the original prediction head is converted into a transformer prediction head to explore the prediction potential with a multi-head attention mechanism. The experimental results show that the CST-YOLOv5s model has a significant improvement in the detection of corn kernel breakage and mildew. Compared with the original YOLOv5s model, the average precision (AP) of corn kernel breakage and mildew recognition increased by 5.2% and 7.1%, respectively, and the mean average precision (mAP) of all kinds of corn kernel recognition is 96.1%, and the frame rate is 36.7 FPS. Compared with YOLOv4-tiny, YOLOv6n, YOLOv7, YOLOv8s, and YOLOv9-E detection model algorithms, the CST-YOLOv5s model has better overall performance in terms of detection accuracy and speed. This study can provide a reference for real-time detection of breakage and mildew kernels during the harvesting process of corn kernels.

Multi-species weed detection and variable spraying system for farmland based on W-YOLOv5

Weed infestations have the potential to cause significant economic losses for farmers as a result of diminished crop yields and escalated labor and input costs linked with weed management. Traditional weed control methods often entail indiscriminate application of herbicides across entire fields, irrespective of weed density or species composition. In this investigation, we propose a hierarchical detection algorithm for identifying multi-species weeds and have devised a variable spraying system in agricultural settings. The weed classification detection algorithm comprises two components: crop seedling target detection and weed detection. The enhanced YOLOv5 algorithm was initially merged with Vision Transformer (ViT) to introduce the W-YOLOv5 crop seedling target detection algorithm. Experimental validation results indicate that the mean average precision (mAP) of the proposed W-YOLOv5 stands at 87.6%, representing a 3.2% increase over the original YOLOv5. Compared with YOLOv7, this method demonstrates a 4.4% improvement in mAP, accompanied by an 80.14% reduction in floating point operations (FLOPs). These findings underscore the effectiveness of the proposed crop seedling target detection algorithm in ensuring detection accuracy while minimizing model FLOPs. Subsequently, following the detection of crop seedlings, the Hue, Saturation, and Value (HSV) color space filtering algorithm was employed to identify the locations of all weeds in the image, thereby facilitating the detection of weeds among wheat (Triticum aestivum L.), radish (Raphanus sativus L.), cucumber (Cucumis sativus L.), soybean (Glycine max (L.) Merr.), and corn (Zea mays L.) seedlings. Finally, the severity of weed infestation in farmland was classified into five levels. Leveraging these severity levels, we developed a variable spraying system and integrated the developed algorithm into the system. Field validation experiments demonstrate that at a speed of 4 km/h, the spraying accuracy of the system can reach 90.32%, effectively enabling precise variable spraying.

A Comparative Dataset of Annotated Broccoli Heads Recorded with Depth Cameras from a Moving Vehicle

An extensive, publicly available dataset is presented—the LAR Broccoli dataset—which contains 20,000 manually annotated images of broccoli heads captured from a moving tractor at an organic farm in the UK. The dataset contains images of the same row of broccoli heads recorded at 30 frames per second (fps) with three different cameras. Two off-the-shelf, relatively low-cost depth-sensing cameras were used, with the tractor moving at a speed of around 1 km/h, in addition to a webcam, with the tractor moving twice as fast. The utility of the dataset is demonstrated in four ways. First, three different state-of-the-art detector models were trained on the dataset, achieving an overall mean Average Precision (mAP) score of over 95% for the best-performing detector. The results validate the utility of the dataset for the standard task of in-field broccoli head recognition. Second, experiments with transfer learning were conducted, initialised with a smaller pre-trained broccoli detection model, and refined with the LAR Broccoli dataset. Third, we assessed the advantages of transfer learning not only using mAP but also according to time and space requirements for training models, which provides a proxy metric for energy efficiency, a practical consideration for real-world model training. Fourth, the cross-camera generalisation among the three camera systems was compared. The results highlight that testing and training detector models using different camera systems can lead to reduced performance, unless the training set also includes some images captured in the same manner as those in the test set.

A Computer Vision Framework for Structural Analysis of Hand-Drawn Engineering Sketches.

Structural engineers are often required to draw two-dimensional engineering sketches for quick structural analysis, either by hand calculation or using analysis software. However, calculation by hand is slow and error-prone, and the manual conversion of a hand-drawn sketch into a virtual model is tedious and time-consuming. This paper presents a complete and autonomous framework for converting a hand-drawn engineering sketch into an analyzed structural model using a camera and computer vision. In this framework, a computer vision object detection stage initially extracts information about the raw features in the image of the beam diagram. Next, a computer vision number-reading model transcribes any handwritten numerals appearing in the image. Then, feature association models are applied to characterize the relationships among the detected features in order to build a comprehensive structural model. Finally, the structural model generated is analyzed using OpenSees. In the system presented, the object detection model achieves a mean average precision of 99.1%, the number-reading model achieves an accuracy of 99.0%, and the models in the feature association stage achieve accuracies ranging from 95.1% to 99.5%. Overall, the tool analyzes 45.0% of images entirely correctly and the remaining 55.0% of images partially correctly. The proposed framework holds promise for other types of structural sketches, such as trusses and frames. Moreover, it can be a valuable tool for structural engineers that is capable of improving the efficiency, safety, and sustainability of future construction projects.

Multispectral Pedestrian Detection Based on Prior-Saliency Attention and Image Fusion

Detecting pedestrians in varying illumination conditions poses a significant challenge, necessitating the development of innovative solutions. In response to this, we introduce Prior-AttentionNet, a pedestrian detection model featuring a Prior-Attention mechanism. This model leverages the stark contrast between thermal objects and their backgrounds in far-infrared (FIR) images by employing saliency attention derived from FIR images via UNet. However, extracting salient regions of diverse scales from FIR images poses a challenge for saliency attention. To address this, we integrate Simple Linear Iterative Clustering (SLIC) superpixel segmentation, embedding the segmentation feature map as prior knowledge into UNet’s decoding stage for comprehensive end-to-end training and detection. This integration enhances the extraction of focused attention regions, with the synergy of segmentation prior and saliency attention forming the core of Prior-AttentionNet. Moreover, to enrich pedestrian details and contour visibility in low-light conditions, we implement multispectral image fusion. Experimental evaluations were conducted on the KAIST and OTCBVS datasets. Applying Prior-Attention mode to FIR-RGB images significantly improves the delineation and focus on multi-scale pedestrians. Prior-AttentionNet’s general detector demonstrates the capability of detecting pedestrians with minimal computational resources. The ablation studies indicate that the FIR-RGB+ Prior-Attention mode markedly enhances detection robustness over other modes. When compared to conventional multispectral pedestrian detection models, Prior-AttentionNet consistently surpasses them by achieving higher mean average precision and lower miss rates in diverse scenarios, during both day and night.

Development and performance evaluation of an electric cabbage harvester with a precise cabbage pushing mechanism

The manual method of harvesting cabbages is common in many Asian countries, which creates a lot of biomechanical problems among farmers. The existing commercial cabbage harvesters being bigger in size are not suitable for small landholdings. To address this, a three-wheeled walk-behind type electric cabbage harvester with a precise cabbage pushing mechanism was developed. It comprised a cutting unit, pushing unit, conveying unit, propelling unit, storage bin and handle. Based on the results obtained from laboratory tests, the cutting speed, forward speed and cutting position of cabbage stem from cabbage head was chosen as 590 rpm, 0.25 m/s and 0.1 cm. For pushing the cut cabbages precisely, a YOLOv8 custom cabbage detection model (mean average precision of 0.938) was developed and deployed in Nvidia Jetson Nano that could detect the cabbages and actuated a pusher via Arduino microcontroller and motor driver to push the cut cabbages to the conveyor belt. In the power supply unit of the harvester, 48 V 12 Ah, 12 V 7 Ah, and 24 V 18 Ah batteries were used for powering the motors for cutting, pushing, and propelling and conveying, respectively. The harvester was tested in a polyhouse for three Indian cabbage varieties. The maximum instantaneous power consumption (IPC) was measured as 948.53 W. Hand vibration and noise at operators’ ear level were 2.098 m/s2 and 78 dB(A), respectively, which confirmed its safety from an ergonomic perspective. The harvester achieved a maximum cutting efficiency of 77.5 % and a minimum damage rate of 7.5 % during polyhouse testing.

A Multi-Weapon Detection Using Deep Learning

The escalating usage of light weapons in criminal and terrorist activities has necessitated the development of advanced weapon detection systems. This study centers on the application of the high-speed deep learning detection model, You Only Look Once version 7 (YOLOv7), to address the need for efficient and swift threat identification. The proposed system, trained on a self-curated dataset rich in images of various dangerous light weapons, is designed to recognize and distinguish multiple weapons simultaneously. The unique value of YOLOv7 in object and specifically weapon detection lies in its outstanding speed and accuracy, as demonstrated by certain weapon categories achieving a mean average precision (mAP) of 97%. The system’s performance can potentially be further enhanced by augmenting the image dataset for each weapon category. This study, therefore, not only validates the critical importance of YOLOv7 in advancing detection methodologies but also presents a practical solution for bolstering public safety.  

Real time electronic-waste classification algorithms using the computer vision based on Convolutional Neural Network (CNN): Enhanced environmental incentives

An innovative approach is needed to boost the economic value of e-waste by improving metal recovery and facilitating the separation of plastics and valuable metal components. Leveraging deep learning and computer vision technology offers a promising solution for automatically categorizing and sorting e-waste components like copper, printed circuit boards (PCB), steel, glass, and aluminum, presenting significant financial and environmental incentives for increased recycling efforts. In this instance, the real-time object identification algorithms YOLO 7 and 5 were used along with TensorFlow version 2.8.0. The e-waste dataset works incredibly well with TensorFlow and YOLO. For the detection of copper, PCBs, and plastic, the F1 values (F1 score is the harmonic mean of precision and recall; precision is the fraction of relevant instances among the retrieved instances whereas recall or sensitivity is the fraction of relevant instances that were retrieved) were as good as 1.0, whereas the score for steel and aluminum was 0.8. The mean average precision (mAP; for a set of queries, it is the mean of the average precision scores for each query) was 0.96 for all classes, with the highest precision for copper (0.99) followed by PCB (0.981). 240 e-waste objects were independently evaluated using the YOLO v7 model, achieving a remarkable ∼94 % prediction accuracy with a batch size of 16, ensuring robust performance. The real-time e-waste component detection was also done using video clips as well as webcam streaming. Deploying real-time e-waste object detection and sorting can significantly narrow the gap between e-waste accumulation and recycling rates, leading to reduced environmental strain and impact.

Enhancing the Safety of Autonomous Vehicles in Adverse Weather by Deep Learning-Based Object Detection

Recognizing and categorizing items in weather-adverse environments poses significant challenges for autonomous vehicles. To improve the robustness of object-detection systems, this paper introduces an innovative approach for detecting objects at different levels by leveraging sensors and deep learning-based solutions within a traffic circle. The suggested approach improves the effectiveness of single-stage object detectors, aiming to advance the performance in perceiving autonomous racing environments and minimizing instances of false detection and low recognition rates. The improved framework is based on the one-stage object-detection model, incorporating multiple lightweight backbones. Additionally, attention mechanisms are integrated to refine the object-detection process further. Our proposed model demonstrates superior performance compared to the state-of-the-art method on the DAWN dataset, achieving a mean average precision (mAP) of 99.1%, surpassing the previous result of 84.7%.

Enabling near-real-time safety glove detection through edge computing and transfer learning: comparative analysis of edge and cloud computing-based methods

PurposeComputer vision and deep learning (DL) methods have been investigated for personal protective equipment (PPE) monitoring and detection for construction workers’ safety. However, it is still challenging to implement automated safety monitoring methods in near real time or in a time-efficient manner in real construction practices. Therefore, this study developed a novel solution to enhance the time efficiency to achieve near-real-time safety glove detection and meanwhile preserve data privacy.Design/methodology/approachThe developed method comprises two primary components: (1) transfer learning methods to detect safety gloves and (2) edge computing to improve time efficiency and data privacy. To compare the developed edge computing-based method with the currently widely used cloud computing-based methods, a comprehensive comparative analysis was conducted from both the implementation and theory perspectives, providing insights into the developed approach’s performance.FindingsThree DL models achieved mean average precision (mAP) scores ranging from 74.92% to 84.31% for safety glove detection. The other two methods by combining object detection and classification achieved mAP as 89.91% for hand detection and 100% for glove classification. From both implementation and theory perspectives, the edge computing-based method detected gloves faster than the cloud computing-based method. The edge computing-based method achieved a detection latency of 36%–68% shorter than the cloud computing-based method in the implementation perspective. The findings highlight edge computing’s potential for near-real-time detection with improved data privacy.Originality/valueThis study implemented and evaluated DL-based safety monitoring methods on different computing infrastructures to investigate their time efficiency. This study contributes to existing knowledge by demonstrating how edge computing can be used with DL models (without sacrificing their performance) to improve PPE-glove monitoring in a time-efficient manner as well as maintain data privacy.

Video surveillance-based multi-task learning with swin transformer for earthwork activity classification

Bulldozers, pivotal in earthworks, traditionally undergo supervision through labor-intensive and potentially unreliable manual methods. This research proposes a vision-based method for automating the monitoring of bulldozer operations. First, this research develops a specialized dataset for deep learning, the bulldozer earthmoving activity dataset. Following this, a novel multi-task video classification network (MTVTNet), the multi-task video transformer network, utilizing a video swin transformer architecture, is proposed. This network is adept at concurrently detecting the shoveling action, state, and soil classification of a bulldozer. The effectiveness of this model is demonstrated through its application in a real-world construction setting, achieving a remarkable 99.68% mean average precision. This method not only facilitates comprehensive automated supervision of bulldozer earthmoving activities but also serves as a valuable data source for assessing the operational efficiency of these machines.

Autonomous surface crack identification for concrete structures based on the you only look once version 5 algorithm

Failure to repair roads in a timely manner may shorten their life and even cause traffic accidents. Thus, accurate crack detection and reasonable classification are crucial for road safety evaluation. In this study, an improved network model based on the You Only Look Once version 5 algorithm is presented, with three additional modules: The first module improves the data processing speed by replacing the C3 module in the original network with a lightweight network model. The second module was used to lighten network weight by reusing a simple convolution structure to equivalently represent the calculation of a convolution layer as a weighted sum of several small convolution blocks. And the third module is used to improve the detection accuracy by removing the upsampling and performing three-way splicing. The proposed model can detect different types of cracks, and an extensive ablation study is reported based on various combinations of the proposed modules. Based on training on a database of 5484 images, the results show that the improved network proposed in this study can effectively identify pavement cracks. Compared with the original network, mean Average Precision is increased by 5.98%, the inference time is reduced by 4.82%, and the model weight is decreased by 17.36%. Additionally, to comply with engineering practice, comparative experiments were conducted on the pre-rotated dataset. The results showed that compared with You Only Look Once version 8, the improved algorithm improved accuracy, recall, average accuracy, and F1 score by 3.28%, 8.46%, 3.79% and 5.89%, respectively. This study can serve as an important reference for the development of crack detection methods.