Real-time Inspection Research Articles

Tiny machine learning empowers climbing inspection robots for real-time multiobject bolt-defect detection

Ensuring the structural integrity of steel construction is critical, necessitating effective methods for detecting bolt defects. Traditional inspection methods are reliable but require significant labor and struggle with the accessibility of complex structures. This research introduces a real-time bolt-defect detection system that integrates tiny machine learning (TinyML) with a magnetic climbing robot to enhance inspection efficiency and scope. The system employs the faster objects, more objects (FOMO) algorithm optimized for edge computing on microcontrollers. This approach enables accurate bolt identification, including normal, loose, and missing bolts. The average system accuracy is 82%, with precision and recall values from 0.57 to 0.89 and 0.67 to 0.87, respectively. The system demonstrates balanced detection, evidenced by an F1 score improvement of up to 62%. The FOMO model displays compelling performance on defect detection tasks, achieving an F1 score of approximately 75%, outperforming the MobileNetV2 Single Shot Multibox Detector with Feature Pyramid Networks Lite and You Only Look Once version 5 small. The efficiency of the FOMO model is highlighted by its low hardware requirements at less than 0.1 MB of flash memory and 893.8 KB of random access memory for the 32-bit floating-point data format and is reduced for the 8-bit integer data format, with inference times of 142 ms and 86 ms, respectively. These findings contrast with the higher resource demands and slower inference time of the compared models, indicating the suitability of the FOMO model for low-capacity microcontrollers and its feasibility for real-time applications. The performance analysis across scenarios confirms the high precision (average 0.77) and recall (average 0.76), validating the robustness of the model in diverse environmental conditions. The system offers an advancement over traditional bolt-defect detection methods in accuracy and efficiency by leveraging TinyML and a magnetic climbing robot, setting new benchmarks for real-time inspection technology. The quantitative results underscore the performance of the system, presenting a scalable, cost-effective solution for improving the safety and durability of steel structures.

High-resolution imaging for in-situ non-destructive testing by quantitative lensless digital holography

Quantitative imaging technologies for in-situ non-destructive testing (NDT) demand high-resolution, wide-field, and stable metrology capabilities. Moreover, live processing and automation are vital for real-time quality control and inspection. Conventional methods use complex optical setups, resulting in large, immobile systems which can solely operate within controlled environmental conditions due to temporal instabilities, rendering them unsuitable for in-situ measurements of micro-to nano-scale physical phenomena. This article delves into the multiphysics application of lensless digital holography, emphasizing its metrological capacity for various in-situ scenarios, while acknowledging and characterizing the differing constraints imposed by various physical phenomena, both transient and steady-state. The digital reconstruction of holograms is computed in real-time, and numerical focusing capabilities allow for instantaneous retrieval of the optical phase at various working distances without the need of complex optical setups, making lensless digital holography well-suited for in-situ quantitative imaging under various types of environments. Current NDT capabilities are demonstrated, including high-resolution and real-time reconstructions, simultaneous measurements for comparative metrology, and practical applications ranging from vibrations and acoustics to thermo-mechanics. Furthermore, methodologies to enhance overall metrology capabilities are exploited, addressing the study of existing physical phenomena, thereby expanding the applicability of holographic techniques across diverse industrial sectors.

Prediction of water absorption of recycled coarse aggregate based on deep learning image segmentation

Abstract To quickly measure the water absorption (WA) of Recycled Coarse Aggregates (RCA), we utilize a detection platform designed for RCA to collect two-dimensional images. Utilizing the RCA-net network, we segment the areas of the mortar and aggregate on the RCA surface. Segmentations allow us to extract critical parameters for characterizing the quality of RCA, the proportion of mortar area (PMA). Subsequently, we construct three regression functions between PMA and WA. The experimental results demonstrate that our proposed segmentation method effectively separates both adhered particles of RCA and distinct areas of mortar and aggregate on RCA surfaces. Next, sprinkling water on RCA surfaces can enhance the accuracy of the segmentation. Notably, within particle size ranges of 5–10 mm, 10–20 mm, and 20–31.5 mm, we all observed a significant linear relationship between PMA and WA. We used those linear relationships and the equivalent mass of RCA detected by the image method in each particle size range to construct the prediction model of water absorption. According to the validation result of 24 groups RCA, this model’s maximum relative error of RCA water absorption predicted value was 10.6 %. The detection time of this method is short, and the detection time of 2 kg RCA is 3.8 min, with an average computation time per image of merely 0.659 s. This efficiency fulfills the requirements for real-time industrial inspection.

Fine-Grained Feature Perception for Unmanned Aerial Vehicle Target Detection Algorithm

Unmanned aerial vehicle (UAV) aerial images often present challenges such as small target sizes, high target density, varied shooting angles, and dynamic poses. Existing target detection algorithms exhibit a noticeable performance decline when confronted with UAV aerial images compared to general scenes. This paper proposes an outstanding small target detection algorithm for UAVs, named Fine-Grained Feature Perception YOLOv8s-P2 (FGFP-YOLOv8s-P2), based on YOLOv8s-P2 architecture. We specialize in improving inspection accuracy while meeting real-time inspection requirements. First, we enhance the targets’ pixel information by utilizing slice-assisted training and inference techniques, thereby reducing missed detections. Then, we propose a feature extraction module with deformable convolutions. Decoupling the learning process of offset and modulation scalar enables better adaptation to variations in the size and shape of diverse targets. In addition, we introduce a large kernel spatial pyramid pooling module. By cascading convolutions, we leverage the advantages of large kernels to flexibly adjust the model’s attention to various regions of high-level feature maps, better adapting to complex visual scenes and circumventing the cost drawbacks associated with large kernels. To match the excellent real-time detection performance of the baseline model, we propose an improved Random FasterNet Block. This block introduces randomness during convolution and captures spatial features of non-linear transformation channels, enriching feature representations and enhancing model efficiency. Extensive experiments and comprehensive evaluations on the VisDrone2019 and DOTA-v1.0 datasets demonstrate the effectiveness of FGFP-YOLOv8s-P2. This achievement provides robust technical support for efficient small target detection by UAVs in complex scenarios.

Real-time railroad track components inspection framework based on YOLO-NAS and edge computing

The demand for efficient track inspection systems in the rapidly evolving rail transportation field is more pronounced than ever. Hence, this study combines deep learning and edge computing for railroad track component inspection, focusing on the YOLO-NAS architecture. Our objective was twofold: to harness the advantages of YOLO-NAS for accurate and high-speed detection while addressing the computational constraints of edge devices. Consequently, YOLO-NAS-S-PTQ model achieved a remarkable balance, with 74.77% mAP and 92.20 FPS, on the NVIDIA Jetson Orin platform. By deploying this model on an edge device and utilizing a multiprocessor pipeline, we observed an inference speed of 60.468 FPS, which was nearly double the rate of its single-threaded counterpart. Field tests further confirmed the efficiency of the model, demonstrating a recall rate of 80.77% and an accuracy of 96.64%. These findings underscore the potential of YOLO-NAS in transforming traditional rail component inspection methods, significantly reducing human interventions and potential errors.

Looper, an updated closed loop feature detection platform for automated neonate cardio-respiratory measurements and data analysis

Facemask pneumotachography is a common, precise experimental approach for measuring respiratory outcomes in neonate rodents (P7-8), particularly in autoresuscitation experiments that inform upon Sudden Infant Death Syndrome where failure of autoresuscitation is hypothesized to be a common endpoint. However, current approaches rely on direct, real-time waveform inspection by an observer and their highly variable reaction times to execute responsive changes in assay gas exposures. Looper, represents a novel and significant improvement on typical observer-based interpretation of live data to initiate challenge and recovery gas applications to uncover nuanced features in neonate cardiorespiratory physiology. Leveraging our previously published Breathe Easy software, we have expanded the analytical capabilities of the software to analyze key features within a typical autoresuscitation cycle. Utilizing the software output, we have further developed machine learning approaches that enable real-time prediction of cardio-respiratory failure based distinct waveform features more than 10min in advance, setting the stage for novel experiments to identify critical interventional windows for enhancing resuscitation. R01HL161142 R01HL130249. This is the full abstract presented at the American Physiology Summit 2024 meeting and is only available in HTML format. There are no additional versions or additional content available for this abstract. Physiology was not involved in the peer review process.

Automatic lesion detection for narrow-band imaging bronchoscopy.

Early detection of cancer is crucial for lung cancer patients, as it determines disease prognosis. Lung cancer typically starts as bronchial lesions along the airway walls. Recent research has indicated that narrow-band imaging (NBI) bronchoscopy enables more effective bronchial lesion detection than other bronchoscopic modalities. Unfortunately, NBI video can be hard to interpret because physicians currently are forced to perform a time-consuming subjective visual search to detect bronchial lesions in a long airway-exam video. As a result, NBI bronchoscopy is not regularly used in practice. To alleviate this problem, we propose an automatic two-stage real-time method for bronchial lesion detection in NBI video and perform a first-of-its-kind pilot study of the method using NBI airway exam video collected at our institution. Given a patient's NBI video, the first method stage entails a deep-learning-based object detection network coupled with a multiframe abnormality measure to locate candidate lesions on each video frame. The second method stage then draws upon a Siamese network and a Kalman filter to track candidate lesions over multiple frames to arrive at final lesion decisions. Tests drawing on 23 patient NBI airway exam videos indicate that the method can process an incoming video stream at a real-time frame rate, thereby making the method viable for real-time inspection during a live bronchoscopic airway exam. Furthermore, our studies showed a 93% sensitivity and 86% specificity for lesion detection; this compares favorably to a sensitivity and specificity of 80% and 84% achieved over a series of recent pooled clinical studies using the current time-consuming subjective clinical approach. The method shows potential for robust lesion detection in NBI video at a real-time frame rate. Therefore, it could help enable more common use of NBI bronchoscopy for bronchial lesion detection.

Advancing the Decarbonization of the Construction Sector: Lifecycle Quality and Performance Assurance of Nearly Zero-Energy Buildings

Dealing with and maintaining high-quality standards in the design and construction phases is challenging, especially for on-site construction. Issues like improper implementation of building components and poor communication can widen the gap between design specifications and actual conditions. To prevent this, particularly for energy-efficient buildings, it is vital to develop resilient, sustainable strategies. These should optimize resource use, minimize environmental impact, and enhance livability, contributing to carbon neutrality by 2050 and climate change mitigation. Traditional post-occupancy evaluations, which identify defects after construction, are impractical for addressing energy performance gaps. A new, real-time inspection approach is necessary throughout the construction process. This paper suggests an innovative guideline for prefabricated buildings, emphasizing digital ‘self-instruction’ and ‘self-inspection’. These procedures ensure activities impacting quality adhere to specific instructions, drawings, and 3D models, incorporating the relevant acceptance criteria to verify completion. This methodology, promoting alignment with planned energy-efficient features, is supported by BIM-based software and Augmented Reality (AR) tools, embodying Industry 4.0 principles. BIM (Building Information Modeling) and AR bridge the gap between virtual design and actual construction, improving stakeholder communication and enabling real-time monitoring and adjustments. This integration fosters accuracy and efficiency, which are key for energy-efficient and nearly zero-energy buildings, marking a shift towards a more precise, collaborative, and environmentally sensible construction industry.

Detection of Fault in Solar Panel Using Thermal Image Processing

This paper explores the practical application of thermal cameras mounted on drones for thermal image processing in solar panel maintenance, with an emphasis on hotspot fault detection. Traditional manual inspection methods for solar panels are often time-consuming and labour-intensive, prompting the exploration of more efficient alternatives. By leveraging the capabilities of drones equipped with thermal cameras, high-resolution images and videos of solar panels can be captured swiftly and effectively. Advanced thermal image processing techniques are then employed to analyse these visuals, with a particular focus on identifying hotspot faults. The study introduces novel methodologies aimed at enhancing the accuracy and speed of anomaly detection in thermal imagery, thereby enabling proactive maintenance interventions. Real-time inspection capabilities integrated into the drone-based system provide immediate feedback to maintenance personnel, facilitating prompt decision-making and action. Through the adoption of this approach, solar panel maintenance operations are streamlined, leading to improved efficiency and cost-effectiveness. Ultimately, the implementation of drone-based thermal imaging solutions contributes to the sustainability and longevity of solar energy systems. Key Words: thermal image processing, solar panel, faults in solar panel, drones, hotspot, fault detection.

Large Span Sizes and Irregular Shapes Target Detection Methods Using Variable Convolution-Improved YOLOv8.

In this work, an object detection method using variable convolution-improved YOLOv8 is proposed to solve the problem of low accuracy and low efficiency in detecting spanning and irregularly shaped samples. Aiming at the problems of the irregular shape of a target, the low resolution of labeling frames, dense distribution, and the ease of overlap, a deformable convolution module is added to the original backbone network. This allows the model to deal flexibly with the problem of the insufficient perceptual field of the target corresponding to the detection point, and the situations of leakage and misdetection can be effectively improved. In order to solve the issue that small target detection is susceptible to image background and noise interference, the Sim-AM (simple parameter-free attention mechanism) module is added to the backbone network of YOLOv8, which enhances the attention to the underlying features and, thus, improves the detection accuracy of the model. More importantly, the Sim-AM module does not need to add parameters to the original network, which reduces the computation of the model. To address the problem of complex model structures that can lead to slower detection, the spatial pyramid pooling of the backbone network is replaced with focal modulation networks, which greatly simplifies the computation process. The experimental validation was carried out on the scrap steel dataset containing a large number of targets of multiple shapes and sizes. The results showed that the improved YOLOv8 network model improves the AP (average precision) by 2.1%, the mAP (mean average precision value) by 0.8%, and reduces the FPS (frames per second) by 5.4, which meets the performance requirements of real-time industrial inspection.

Intelligent railroad inspection and monitoring

Railways are essential to the global transportation infrastructure, providing eco-friendly and economical solutions for the movement of freight and passengers. Inspecting and maintaining extensive rail networks timely poses significant challenges. My group and collaborators have focused on automated railroad inspection technologies, emphasizing the use of deep learning and computer vision to overcome the limitations of traditional manual inspections. Our research introduces groundbreaking real-time inspection methods, leveraging a specialized dataset of railroad components for enhanced instance segmentation models, achieving unprecedented accuracy and inference speeds. The developed computer vision systems efficiently detect track components and their changes over time, and also quantify rail surface defects. Additionally, our work extends to improving railroad crossing safety, utilizing deep learning frameworks for the detection of unusual pedestrian behaviors and object identification, aimed at reducing crossing incidents and improving emergency response times. Our future research directions aim to further refine the cost-effectiveness and autonomy of railroad inspection systems. Through these innovations, we hope to aid in the inspection and maintenance of railroads, offering practical solutions for railroad and other civil engineering applications.

Real-Time Inspection of Fire Safety Equipment using Computer Vision and Deep Learning

The number of accidental fires in buildings has been significantly increased in recent years in Saudi Arabia. Fire Safety Equipment (FSE) plays a crucial role in reducing fire risks. However, this equipment is prone to defects and requires periodic checks and maintenance. Fire safety inspectors are responsible for visual inspection of safety equipment and reporting defects. As the traditional approach of manually checking each piece of equipment can be time-consuming and inaccurate, this study aims to improve the inspection processes of safety equipment. Using computer vision and deep learning techniques, a detection model was trained to visually inspect fire extinguishers and identify defects. Fire extinguisher images were collected, annotated, and augmented to create a dataset of 7,633 images with 16,092 labeled instances. Then, experiments were carried out using YOLOv5, YOLOv7, YOLOv8, and RT-DETR. Pre-trained models were used for transfer learning. A comparative analysis was performed to evaluate these models in terms of accuracy, speed, and model size. The results of YOLOv5n, YOLOv7, YOLOv8n, YOLOv8m, and RT-DETR indicated satisfactory accuracy, ranging between 83.1% and 87.2%. YOLOv8n was chosen as the most suitable due to its fastest inference time of 2.7 ms, its highest mAP0.5 of 87.2%, and its compact model size, making it ideal for real-time mobile applications.

IoT Based Pipe- inspection, and Cleaning Robot

Abstract: The "IOT Based Pipeline Inspection and Cleaning Robot" is a new addition to the infrastructure maintenance lineup! Pipelines are vital for moving necessary materials in the field of infrastructure maintenance. Our "IOT-based pipeline inspection and cleaning robot" is an inventive IoT-based rover outfitted with an ESP32 camera for real-time streaming and a motor propeller for effective exploration and operation. Its purpose is to improve maintenance procedures and tackle the difficulties associated with pipeline inspection. The major goal of the IOT-based pipeline cleaning and inspection robot is to transform pipeline maintenance by offering an adaptable, affordable, and successful solution. Real-time pipeline exploration and inspection are made possible by the rover's sophisticated features, which improve the infrastructure's functioning and integrity.

Assessment of adenosine triphosphate content in sausages stuffed in different modified casing treatments added with orange extracts, utilising hyperspectral imaging combined with multivariate analysis.

An investigation was conducted using a hyperspectral imaging (HSI) system to non-invasively estimate adenosine triphosphate (ATP) content in vacuum packaged sausages in different modified casing treatments added with orange extracts after a year of storage at 4°C. Various pre-processing combinations were applied to the spectra to enhance the performance of partial least squares regression (PLSR). PLSR models, utilising the full absorbance spectrum with pre-treatment of standard normal variate combined with 1st derivative,exhibited prediction coefficients of determination (Rp2) reaching up to 0.6629. A distribution map developed through MATLAB was employed to display the location and concentration of ATP content in these unique sausages for the first time. The integration of HSI and multivariate analysis not only quantifies but also visually represents the changes in ATP content response to the different casing treatments, demonstrating the significant potential for real-time inspection in the processed meat industry.

Validating the Use of Smart Glasses in Industrial Quality Control: A Case Study

Effective quality control is crucial in industrial manufacturing for influencing efficiency, product dependability, and customer contentment. In the constantly changing landscape of industrial production, conventional inspection methods may fall short, prompting the need for inventive approaches to enhance precision and productivity. In this study, we investigate the application of smart glasses for real-time quality inspection during assembly processes. Our key innovation involves combining smart glasses’ video feed with a server-based image recognition system, utilizing the advanced YOLOv8 model for accurate object detection. This integration seamlessly merges mixed reality (MR) with cutting-edge computer vision algorithms, offering immediate visual feedback and significantly enhancing defect detection in terms of both speed and accuracy. Carried out in a controlled environment, our research provides a thorough evaluation of the system’s functionality and identifies potential improvements. The findings highlight that MR significantly elevates the efficiency and reliability of traditional inspection methods. The synergy of MR and computer vision opens doors for future advancements in industrial quality control, paving the way for more streamlined and dependable manufacturing ecosystems.

A simple image correlation technique for imaging subsurface damage from low-velocity impacts in composite structures

A robust computer vision system is proposed to visualize subsurface barely visible impact damage (BVID) in composite structures through a simple image correlation technique together with a damage imaging condition. This system uses a digital camera to record a video of the surface motion, capturing micron-scale dynamic movement from guided waves propagating on the surface of the structure generated via a sweeping frequency excitation (chirp) up to the ultrasonic frequency range. As the excitation frequency changes during the chirp, waves become trapped within this damaged region, forming standing waves to generate local resonance at specific frequencies. This localized resonance accumulates high wave energy in the region, generating a higher transverse displacement at the damage site compared to the remaining part of the structure. In this work, a simple image correlation technique is proposed to correlate each filtered video frame with the temporal mean of the filtered wavefield video to highlight standing waves at localized damage. The proposed image correlation technique is distinct from digital image correlation (DIC) since it does not use correlation for subset matching to track the movement of surface patterns between two images (video frames). Instead, it uses correlation to directly quantify similarities between corresponding image pixels (windows). Utilizing a single camera greatly simplifies system complexity and hence enhances the practicality and potential for real-time performance over a recently developed technique that utilized 3D DIC with a stereo camera for vision-based BVID detection. To realize the aim, work was conducted in two sequential steps: (1) off-axis 2D DIC was employed rather than 3D DIC to examine the potential of employing a single camera to capture images and extract not only in-plane displacement but also a fraction of transverse displacement in which local resonance is dominant and (2) image correlation was then employed, supplanting the off-axis 2D DIC image processing involved in the first step, to highlight the damage with significantly less processing complexity. This proposed technique using a zero-mean normalized cross-correlation imaging condition, rather than the total wave energy used for DIC-based approaches, is efficient and effective for identifying regions of minute surface movement from local resonance within the damage region without the use of the computationally intensive interpolation to get the sub-pixel gray level information followed by subset matching employed in DIC-based image processing. Two geometrically identical CFRP composite honeycomb panels that had been subjected to low-velocity impacts were used for verification and validation, and two excitation location configurations were tested for each panel. Damaged images produced with correlation for a 100 mm × 100 mm field of view using a single 3-s video, or a total of 19,200 video frames, show accurate damage imaging capabilities regardless of excitation location that exceed that of DIC techniques and is comparable to benchmark damage images obtained from laser Doppler vibrometry, ultrasonic C-scans, and X-ray CT scans. This success of correlation-based imaging of subsurface BVID demonstrates substantial improvements in efficiency and practicality and shows high potential for in situ and real-time computer vision-based nondestructive inspection or structural health monitoring of subsurface BVID in composite aircraft and other critical structures.

In-situ quality inspection system of injection parts based on transfer learning

PurposeThis paper aims to realize an in-situ quality inspection system rapidly for new injection molding (IM) tasks via transfer learning (TL) approach and automation technology.Design/methodology/approachThe proposed in-situ quality inspection system consists of an injection machine, USB camera, programmable logic controller and personal computer, interconnected via OPC or USB communication interfaces. This configuration enables seamless automation of the IM process, real-time quality inspection and automated decision-making. In addition, a MobileNet-based deep learning (DL) model is proposed for quality inspection of injection parts, fine-tuned using the TL approach.FindingsUsing the TL approach, the MobileNet-based DL model demonstrates exceptional performance, achieving validation accuracy of 99.1% with the utilization of merely 50 images per category. Its detection speed and accuracy surpass those of DenseNet121-based, VGG16-based, ResNet50-based and Xception-based convolutional neural networks. Further evaluation using a random data set of 120 images, as assessed through the confusion matrix, attests to an accuracy rate of 96.67%.Originality/valueThe proposed MobileNet-based DL model achieves higher accuracy with less resource consumption using the TL approach. It is integrated with automation technologies to build the in-situ quality inspection system of injection parts, which improves the cost-efficiency by facilitating the acquisition and labeling of task-specific images, enabling automatic defect detection and decision-making online, thus holding profound significance for the IM industry and its pursuit of enhanced quality inspection measures.

A Pavement Crack Detection and Evaluation Framework for a UAV Inspection System Based on Deep Learning

Existing studies often lack a systematic solution for an Unmanned Aerial Vehicles (UAV) inspection system, which hinders their widespread application in crack detection. To enhance its substantial practicality, this study proposes a formal and systematic framework for UAV inspection systems, specifically designed for automatic crack detection and pavement distress evaluation. The framework integrates UAV data acquisition, deep-learning-based crack identification, and road damage assessment in a comprehensive and orderly manner. Firstly, a flight control strategy is presented, and road crack data are collected using DJI Mini 2 UAV imagery, establishing high-quality UAV crack image datasets with ground truth information. Secondly, a validation and comparison study is conducted to enhance the automatic crack detection capability and provide an appropriate deployment scheme for UAV inspection systems. This study develops automatic crack detection models based on mainstream deep learning algorithms (namely, Faster-RCNN, YOLOv5s, YOLOv7-tiny, and YOLOv8s) in urban road scenarios. The results demonstrate that the Faster-RCNN algorithm achieves the highest accuracy and is suitable for the online data collection of UAV and offline inspection at work stations. Meanwhile, the YOLO models, while slightly lower in accuracy, are the fastest algorithms and are suitable for the lightweight deployment of UAV with online collection and real-time inspection. Quantitative measurement methods for road cracks are presented to assess road damage, which will enhance the application of UAV inspection systems and provide factual evidence for the maintenance decisions made by road authorities.

Real-Time Object Detection for Autonomous Solar Farm Inspection via UAVs

Robotic missions for solar farm inspection demand agile and precise object detection strategies. This paper introduces an innovative keypoint-based object detection framework specifically designed for real-time solar farm inspections with UAVs. Moving away from conventional bounding box or segmentation methods, our technique focuses on detecting the vertices of solar panels, which provides a richer granularity than traditional approaches. Drawing inspiration from CenterNet, our architecture is optimized for embedded platforms like the NVIDIA AGX Jetson Orin, achieving close to 60 FPS at a resolution of 1024 ×1376 pixels, thus outperforming the camera’s operational frequency. Such a real-time capability is essential for efficient robotic operations in time-critical industrial asset inspection environments. The design of our model emphasizes reduced computational demand, positioning it as a practical solution for real-world deployment. Additionally, the integration of active learning strategies promises a considerable reduction in annotation efforts and strengthens the model’s operational feasibility. In summary, our research emphasizes the advantages of keypoint-based object detection, offering a practical and effective approach for real-time solar farm inspections with UAVs.

A railway fastener inspection method based on lightweight network

To ensure the safety of train operations, regular inspection and maintenance of railway fastener is crucial. Currently, computer vision-based fastener inspection has become the primary method. However, to improve detection accuracy, model complexity has been continuously increasing, leading to a decrease in detection efficiency and seriously affecting real-time fastener inspection task. To address this issue, we propose a fastener inspection method based on lightweight network. First, we introduce YOLOv5n-Faster, where this model selects YOLOv5n as the base network for fastener localization and uses partial convolution (PConv) from FasterNet for lightweight design, while using CoordConv to improve localization accuracy, thereby achieving efficient fastener localization task. Then, we perform lightweight design on the overall network structure and Block module of the ConvNeXt-T to propose the fastener state inspection model, Light ConvNeXt, for rapid fastener classification. Experimental results show that our proposed fastener localization model achieves a mean average precision (mAP) of 84.7%, a detection speed of 161 FPS. The fastener state inspection model achieves an accuracy of 99.7%.