Embedded lightweight detection system for road surface defect using model pruning and knowledge distillation

Railway track defects in Malaysia pose significant risks of train derailments and accidents, underscoring the urgency for early and accurate defect detection and classification. This study presents a novel approach utilizing deep learning models, VGG16 and YOLOv5, for detecting and classifying railway track defects, explicitly focusing on corrugation and squat defects. The research's uniqueness lies in its application of these specific models and the composition of a dataset collected from extensive field measurements and inspections across various railway tracks within the Track Network Maintenance Ampang Line in Malaysia. The results demonstrate that these models achieve high precision in defect classification and detection of defects by more than 80%. The proposed methodology provides the railway industry with a powerful tool to streamline maintenance planning and prioritize defect remediation efficiently. Early defect detection can prevent potential accidents and improve safety and operational efficiency. Future studies can expand on these findings by exploring the extension of the proposed techniques to address other types of rail defects. Incorporating a diverse range of scenarios and operating conditions in the dataset could further enhance the models' performance and generalization. Real-time deployment and integration with existing maintenance systems are crucial for practical adoption. This research has strengths but acknowledges limitations. Additional evaluation metrics and a diverse dataset are essential for model performance. Leveraging deep learning models offers a reliable solution for railway maintenance, enhancing safety and efficiency. Addressing these limitations will drive proactive defect management, ensuring safe and reliable railway networks.

Nowadays, wood companies are ever more interested in automatic vision systems (Li & Wu, 2009), (Astrand & Astrom, 1994), for an effective surface inspection that greatly increases the quality of the end product (Smith, 2001), (Armingol et al., 2006). The inspection process, in most visual inspection systems, pursues online defects identification, to reach optimum performance (Malamas et al., 2003), (Spinola et al., 2008). The usual wood inspection systems are visual ones, based on standard cameras and lighting (Batchelor & Whelan, 1997), (Cognex, 2011), (Parsytec, 2011), (Pham & Alcock, 2003) to operate in highly structured environments (Silven et al., 2003). The quality control in visual surface inspection systems must be robust to cope with wood variable reflectance and high speed requirements. The surface inspection methods proposed in the literature for visual inspection aim at adhoc surface inspection systems to solve each specific problem (Pham & Alcock, 1999). Usual inspection systems are based on visible lighting and few of them use diffuse components to illuminate the rough and bright surfaces. A visual wood defect detection system proposed by (Estevez et al., 2003) is composed by a colour video camera, where the standard lighting components are a mixture of two frontal halogen and ceiling fluorescent lamps. The commercial light diffusers use a light source and different components to illuminate the surface in an irregular way to eliminate shadows, but present some problems such as, short useful life, extreme sensitivity and high cost. On the other hand, one of the major drawbacks in automated inspection systems for wood defect classification is the erroneous segmentation of defects on light wood regions, (Ruz et al., 2009). Moreover, the speed of wooden boards at the manufacturing industry is at about 1 m/s, which implies high computational costs (Hall & Astrom, 1995). Current work presents a surface inspection system that uses laser diffuse lighting to cope with different type of defects and wood laminated surfaces to improve defect detection without any previous defect information. The work will not only highlight the specific requirements for a laser diffuse lighting in a visual inspection system but also those of unsupervised defect detection techniques to cope with the variability of wood laminated surfaces and defect types, leading to a heterogeneous and robust visual surface inspection system. The manuscript is organized as follows: section 2 displays images of different wood laminated surfaces, captured by a visual surface inspection system with standard lighting. In Section 3, an innovative surface inspection

WAD-YOLO (Wavelet-Based Adaptive Defect Detection with Multi-Resolution Feature Analysis) is an intelligent system for automatic steel surface defect detection. The project combines wavelet transform techniques with the YOLO deep learning model for accurate and real-time inspection. Wavelet decomposition extracts multi-resolution features to highlight defects of different sizes and textures. Adaptive enhancement improves image clarity and reduces noise effects. The processed features are fed into the YOLO network for defect localization and classification. The system detects various defects such as cracks, scratches, pits, and rolled-in scale. Multi-resolution analysis ensures better detection of both small and large surface defects. The model improves detection accuracy compared to traditional single-scale methods. It supports real-time industrial inspection on production lines. Overall, WAD-YOLO enhances quality control efficiency in steel manufacturing industries.

Metal surface defect detection models have large parameters and high computational complexity, making them difficult to deploy on industrial edge devices. To address these issues, this paper proposes a lightweight algorithm with collaborative pruning and distillation. This algorithm, based on YOLOv8, uses L1 regularization structured pruning to compress the SAF-YOLOv8 model for efficiency. The resulting pruned and unpruned models serve as the student and teacher in subsequent knowledge distillation. The BFCD-KD knowledge distillation method fuses middle-layer features with the regression and classification branches of the detection head and the original model loss. This integrated approach transfers teacher knowledge in feature representation, localization, and classification to improve optimization. Experimental results show that this algorithm reduces the model’s FLOPs by 51.0% and parameters by 41.5%, while achieving a mAP@0.50 of 70.6% on the GC10-DET dataset, balancing accuracy and efficiency. In addition, a real-time interactive metal surface defect detection system is developed for industrial sites using the lightweight SAF-YOLOv8 model. The system integrates industrial cameras, a controllable light source, and a graphical interactive interface built on the PyQt5 framework. It supports real-time visualization and interactive defect detection operations.

Due to the influence of construction quality, engineering geology and hydrological environment, defects such as dehollowing and insufficient compaction can occur in tunnels. Aiming at the problems of complex detection model, poor real-time performance and low accuracy of the current tunnel lining defect detection methods, the study proposes a lightweight defect detection algorithm of tunnel lining based on knowledge distillation. Firstly, a high-precision teacher model based on yolov5s was constructed by constructing a C3CSFM module that combines residual structure and attention mechanism, a MDFPN network structure with multi-scale feature fusion and a reweighted RWNMS re-screening mechanism. Secondly, in the distillation process, the feature and output dimension results are fused to improve the detection accuracy, and the mask feature relationship is learned in the space and channel dimension to improve the real-time detection. Tests on the tunnel lining radar defect image dataset showed that the number of parameters of the improved model was reduced from 16.03 MB to 3.20 MB, a reduction of 80%, and the average accuracy was improved from 83.4 to 86.5%, an increase of 3.1%. On the basis of maintaining the structure and detection performance of the model, the lightweight degree of the model is greatly improved, and the high-precision and real-time detection of tunnel lining defects is realized.

Automatic defect detection of metro tunnel surfaces using a vision-based inspection system

Deep neural networks (DNNs) have gained considerable attention in various real-world applications due to their strong performance in representation learning. However, running a DNN needs tremendous memory resources, which significantly restricts DNN from being applicable on resource-constrained platforms (e.g., IoT, mobile devices, etc.). Lightweight DNNs can accommodate the characteristics of mobile devices, but the hardware resources of mobile or IoT devices are extremely limited, and the resource consumption of lightweight models needs to be further reduced. However, the current neural network compression approaches (i.e., pruning, quantization, knowledge distillation, etc.) works poorly on the lightweight DNNs, which are already simplified. In this paper, we present a novel framework called Smart-DNN, which can efficiently reduce the memory requirements of running DNNs on resource-constrained platforms. Specifically, we slice a neural network into several segments and use SZ error-bounded lossy compression to compress each segment separately while keeping the network structure unchanged. When running a network, we first store the compressed network into memory and then partially decompress the corresponding part layer by layer. According to experimental results on four popular lightweight DNNs (usually used in resource-constrained platforms), Smart-DNN achieves memory saving of 1/10∼1/5, while slightly sacrificing inference accuracy and unchanging the neural network structure with accepted extra runtime overhead.

Convolutional neural networks (CNN) have a significant improvement in the accuracy of object detection. As networks become deeper, the precision of detection becomes obviously improved, and more floating-point calculations are also needed. Because of the great amount of calculation, it is inconvenient for mobile and embedded vision applications. Many researchers apply the knowledge distillation method to improve the precision of object detection by transferring knowledge from a deeper and larger teachers network to a small student one. Most methods of knowledge distillation are needed to design complex cost functions and mainly aim at the two-stage object detection algorithm. Therefore, we propose a clean and effective knowledge distillation method called Generative Adversarial Networks - Knowledge Distillation(GAN-KD) for the one-stage object detection. The feature maps generated by teacher network and student network are employed as true and fake samples respectively, and generating adversarial training for both of them to improve the performance of the student network in one-stage object detection. The experimental result shows that our approach achieves the performance gain of 5% mAP when compared with MobilenetV1 on COCO dataset.

For defect detection in composite materials, Terahertz time-domain non-destructive testing (THz NDT) remains hindered by the inherently low contrast of THz images, noise interference, and the extremely small size of minor defects compared to the background. Here we propose an autonomous defect detection model leveraging the Dynamic High-Resolution Multi-Level Attention Detection Network (DyHRMADet), to enable rapid and precise detection of minor and hidden defects in composite materials. DyHRMADet, consisting of crafted modules, employs an attention-driven multi-scale framework that simultaneously captures fine-grained spatial details and high-level semantic information from THz images, and effectively integrates features across multiple resolutions, delivering a streamlined yet highly accurate defect detector. Comparative performance evaluations indicate that DyHRMADet realises highest mean average precision (mAP) across IoU thresholds (mAP@0.5:0.95 = 0.732, mAP@0.5 = 0.993, mAP@0.75 = 0.846), while processing each image in 0.287 ms with a computational cost of 9.535 GFLOPs and 24.975 MB parameters. It confirms that the proposed DyHRMADet effectively overcomes the limitations of the low contrast and noise inherent in THz images, and realises the accurate minor defect detection. Overall, the proposed approach sets a novel benchmark of THz NDT for defect characterisation in industrial settings, addressing the critical demand for high-precision defect detection in composite materials.

Knowledge distillation (KD) enables the training of lightweight yet effective models, particularly in the visual domain. Meanwhile, reinforcement learning (RL) facilitates adaptive learning through environment-driven interactions, addressing the limitations of KD in handling dynamic and complex tasks. We propose a novel two-stage framework integrating Knowledge Distillation with Reinforcement Learning (KDRL) to enhance model adaptability to complex data distributions, such as remote sensing and medical imaging. In the first stage, supervised fine-tuning guides the student model using logit and feature-based distillation. The second stage refines the model via RL, leveraging confidence-based and cluster alignment rewards while dynamically reducing reliance on task loss. By combining the strengths of supervised knowledge distillation and reinforcement learning, KDRL provides a comprehensive approach to address the dual challenges of model efficiency and domain heterogeneity. A key innovation is the introduction of auxiliary layers within the student encoder to evaluate and reward the alignment of the characteristics with the teacher’s cluster centers, promoting robust feature learning. Our framework demonstrates superior performance and computational efficiency across diverse tasks, establishing a scalable design for efficient model training. Across remote sensing benchmarks, KDRL boosts the lightweight CLIP/ViT-B-32 student to 69.51% zero-shot accuracy on AID and 80.08% on RESISC45; achieves state-of-the-art cross-modal retrieval on RSITMD with 67.44% (I→T) and 74.76% (T→I) at R@10; and improves DIOR-RSVG visual-grounding precision to 64.21% at Pr@0.9. These gains matter in real deployments by reducing missed targets and speeding analyst search on resource-constrained platforms.

Currently, liquid crystal displays (LCDs) are the most popular type of flat panel display and are used in most applications. An LCD contains many critical optical film components that are produced in highly automated and precisely monitored facilities throughout the complex manufacturing process. However, defect detection and classification through visual inspection is very difficult during the manufacturing process. To overcome this problem, a novel framework based on machine vision known as the optical film defect detection and classification system is presented for use in the real-time inspection. First, an image acquisition system equipped with a high-resolution camera and custom-made lighting field was designed to obtain a high-quality optical film image. Second, the defects on an optical film were detected using localized cross-projection based on proposed adaptive energy analysis. Finally, the defect images were classified into four types by using the developed classification algorith—point, scratch, foreign material, and stain. The quality of the products yielded after defect detection and classification of the optical film was compared with the standard product quality of the manufacturer. Experiments were conducted using samples collected from the largest manufacturer in Taiwan to validate the performance of the proposed framework. The accurate defect detection rate is 99.6%, the classification accuracy rate is 100%, and the total operation time is short whichonly 6.129s are required on average to perform the inspection for an optical film sample. The results demonstrate that the proposed method is sound and useful for optical film inspection in industries.

On-road risk detection is one of the most important applications of Deep Neural Network (DNN). A good on-road risk detection system can accurately detect dangerous obstacles on the road and provides drivers with early warning information to avoid danger. However, current high-precision detection systems usually use a cumbersome DNN, which require a large number of computing resources. Due to the limitations of mobile devices such as limited computing power and power consumption problem, it is difficult to apply the high-precision detection model to such devices for running real-time applications. Therefore a fast and accurate model is needed to solve the on-road risk detection problem for low-cost mobile devices. In this paper, we propose a real-time on-road risk detection framework for the low computational platform. The proposed framework uses a tiny neural network model that improves accuracy by leveraging the knowledge distillation technique. By using the proposed method, we conduct our experiments in a Raspberry Pi using one Intel Neural Compute Stick-2 (NCS-2). The practical results show that our detection system can successfully detect the position and type of road obstacles in real-time while maintaining performance comparable to more complex DNN.

Surface defect detection is a core component for ensuring the quality of multi-wedge belt products, yet traditional detection methods based on Convolutional Neural Networks (CNNs) suffer from weak global feature extraction and limited multi-scale feature fusion, severely impacting detection efficiency. Meanwhile, the constrained computational resources in industrial scenarios pose critical challenges to balancing model accuracy and speed. To address these issues, we propose a lightweight defect detection model, LCT-YOLO (Lightweight CNN-Transformer YOLO), which integrates the technical advantages of CNN and Transformer while meeting the lightweight deployment requirements of industrial applications. Specifically, a dynamic channel selection feature extraction network is constructed by introducing the partial channel selection module CSP_PTB to enhance the model’s capability of capturing long-distance semantic information. A lightweight multi-scale feature fusion network is developed by designing a multi-scale path aggregation network and integrating the dynamic upsampling module Dysample in the model’s neck, effectively addressing challenges from feature scale variations. A lightweight shared convolution detection head is introduced to tackle the high computational cost of existing model detection heads, optimizing performance and reducing model complexity through parameter sharing strategies. Finally, model pruning is employed to remove redundant parameters for compression, and knowledge distillation is utilized to compensate for accuracy loss. Experimental results show that the algorithm achieves an mAP50 of 94.3% on the dataset, with 0.8 M parameters and 2.0 GFLOPs computation, outperforming the baseline model by 0.9% in mAP50, reducing parameters by 74.2%, computation by 75.3%, and featuring a model size of only 1.9 M, making it highly suitable for edge device deployment. The algorithm surpasses other advanced methods in both accuracy and speed, not only meeting the dual requirements of detection precision and real-time performance in industrial scenarios but also being successfully applied to practical production, providing robust technical support for defect detection and product quality control.

Nowadays, early defect detection plays a significant role for the railway safety warning. However, the existing methods cannot satisfy the requirements of real-time and high-precision detection. Here, a high-precision, distributed and on-line method for detecting rail defect is proposed and demonstrated. When a train goes through defects, the instantaneous elastic waves will be excited by the wheel-rail interaction, which will further propagate along railway tracks bidirectionally. Through mounting the backscattering enhanced optical fiber on the railway as sensors, the fiber optic distributed acoustic sensing system can record the propagation trace precisely. Further, the acoustic propagation fitting method is applied onto the propagation data to detect and locate defects along the long-distance railway. Especially, the dual-frequency joint-processing algorithm is proposed to improve the location accuracy. The field test proves that multiple defects along the railway can be successfully identified and located with a standard deviation of 0.314m. To the best of our knowledge, this work is the first report of distributed rail defect detection, which will bring a breakthrough for high-precision structural damage detection in the infrastructures such as the railway, pipeline and tunnel.

Motorways are a strategically important part of a country’s infrastructure. High requirements for their condition stipulate regular monitoring of quality of the road surface. The large length of motorways in Russia and the impact of weather and climate conditions (seasonal temperature fluctuations, precipitation) emphasise the relevance of searching for non-destructive testing methods for road diagnostics that ensure short terms of diagnostic work and the use of minimal resources.The considered existing solutions for detecting road surface damage include the use of ground penetrating radar, laser method, method of analysing vibration effects of road surface irregularities, detection of damage based on lidar data and mobile mapping systems.The objective of the study was to develop an algorithm for analysing the condition of the road surface that allows detecting road surface distresses based on images obtained during the diagnostics of motorways by the KP-514-RDT airfield and road measuring mobile laboratory completed by the IndorRoad and RDT-Line software packages.The development of an algorithm for detecting road surface defects was carried out using machine learning methods. The detected defects have precise georeferencing according to stationing of the measured road segment. As a result of development, a trained model was obtained that allows automatic marking of defects of different classes on the image. The developed algorithm is integrated into the software for managing the monitoring of the condition of regional and municipal roads.