Real-time Inspection Research Articles

Advanced detection of foreign objects in fresh-cut vegetables using YOLOv5

The presence of foreign objects in fresh-cut vegetables has become a significant concern for the industry in recent years. Ensuring the safety of consumers and suppliers necessitates comprehensive precautionary measures. This study introduces a novel approach for detecting foreign objects in fresh-cut vegetables using YOLOv5. The results indicate that YOLOv5s excels in identifying foreign objects with a high detection accuracy of 98.30%, a rapid inference time of 2.6 ms, and a compact model size of 13.3 MB. The model effectively identified foreign objects such as transparent and colored plastic, paper, wood, stone, insects, glass, and metal. Moreover, YOLOv5s accurately detected foreign objects with color similarities to green onions, which is particularly challenging due to their wide color variation. Hence, the foreign objects dataset was tested and generated 98.63% and 98.67% for cabbage and green onion, respectively. Additionally, YOLOv5s successfully detected small foreign objects (2–3 mm) in two types of fresh-cut vegetables. Despite its excellent performance, the YOLOv5s model struggles to identify foreign objects that overlap with vegetable samples. To address this issue, installing an automatic conveyor unit could facilitate continuous sample movement, while a feeder unit could reduce the possibility of overlap. This research demonstrates the feasibility of implementing the YOLOv5s, as a non-destructive technique for detecting foreign objects in fresh-cut vegetables. The findings contribute to the development of an accurate, fast, and efficient real-time inspection system, with potential applications in the fresh-cut vegetable industry to enhance product quality and safety.

Damage characterization of FRP embedded with different optical fibers under tensile load using acoustic emission and micrographic real-time inspection

Embedding optical fibers into fiber-reinforced plastics (FRPs) for composite health monitoring has attracted the interests of academia and industry recently though the impact of embedded optical fiber on FRP damage has not been clarified. In this study, the damage to the FRP structure caused by the embedded optical fibers was investigated. Five FRP samples incorporated with different types of optical fibers were subjected to tensile testing, and the crack initiation and propagation were in situ monitored using acoustic emission (AE) and micrographic inspection. After the AE signals were classified in terms of peak amplitude and peak frequency using K-means++ clustering, the damage progression from matrix cracking, fiber/matrix debonding, delamination, and fiber break were characterized. In-situ micrographic inspection clearly revealed the damage evolution around the embedded optical fiber. By correlating the stress-strain curves with the AE clustering and micrographic inspection results, it was verified that larger-diameter optical fibers reduce the modulus and ultimate strength of the FRP, while a good optical fiber/matrix interface mitigates these negative effects. The embedded optical fiber accelerated the crack propagation around it, leading to changes in the damage pattern. This phenomenon is the primary cause of FRP failure. Notably, polyimide-coated optical fibers with an outside diameter of 50 μm are ideal for embedded sensing as their incorporation into the matrix reduces the ultimate strength of FRP by only 1.8% with minimal impact on crack propagation.

Effective Bi-decoding networks for rail-surface defect detection by knowledge distillation

No-service rail-surface defect detection is a crucial method for assessing the quality of railroad tracks. However, the low-contrast and dark-tone characteristics of track-surface textures pose challenges to current defect-monitoring techniques. Real-time and on-site online inspections are important to ensure safe railway operation; however, most complex models for no-service inspections are difficult to deploy on mobile devices. To address these challenges and overcome the detection difficulties associated with complex scenes, we designed a knowledge distillation-based double decoding-layer refinement network (EBDNet-KD). The first decoding process is guided by a bimodal high-level semantic feature map obtained by extending the attention-based graph convolution to incrementally enhance the dual-stream features and obtain an image restoration prior. A divide-and-conquer decoder is then designed to distinguish features using different decoding layers. The prior is then used in the second decoding layer, which enables the bimodal features to interact fully and obtain the final prediction map. We introduce a knowledge distillation strategy that enables a lightweight, compact student network to learn a complex teacher network’s feature extraction process. This facilitates pixel-consistent learning of the knowledge within the bi-decoder layer, as well as bidirectional learning of the focused contextual response knowledge to optimize the model. The EBDNet-KD significantly reduces computational costs while guaranteeing performance with a parameter count of only 28 M. EBDNet-KD demonstrated superior performance over 15 state-of-the-art methods in experiments conducted on NEU RSDDS-AUG, an industrial RGB-depth dataset. We assessed the generalizability of EBDNet-KD by evaluating its performance on three additional public datasets, yielding competitive results. The source code and results can be found at https://github.com/Wuyue15/EBDNet.

Target detection of helicopter electric power inspection based on the feature embedding convolution model.

This study aims to improve the helicopter electric power inspection process by using the feature embedding convolution (FEC) model to solve the problems of small scope and poor real-time inspection. First, simulation experiments and model analysis determine the keyframe and flight trajectory. Second, an improved FEC model is proposed, extracting features from aerial images in large ranges in real time and accurately identifying and classifying electric power inspection targets. In the simulation experiment, the accuracy of the model in electric power circuit and equipment detection is improved by 30% compared with the traditional algorithm, and the inspection range is expanded by 26%. In addition, this study further optimizes the model with reinforcement learning technology, conducts a comparative analysis of different flight environments and facilities, and reveals the diversity and complexity of inspection objectives. The performance of the optimized model in fault detection is increased by more than 36%. In conclusion, the proposed model improves the accuracy and scope of inspection, provides a more scientific strategy for electric power inspection, and ensures inspection efficiency.

LiDAR-based real-time geometrical inspection for large-scale additive manufacturing

LiDAR-based real-time geometrical inspection for large-scale additive manufacturing

PSIII-4 Exploring the behavioral effects of glycerol supplementation in ruminally cannulated cows grazing wheat pasture

Abstract Energy intake and metabolism have a key role in the control of dry matter intake (DMI), fermentation processes, and grazing behavior of beef cattle, and is likely affected by intertwined relationships between forage availability and stage of maturity, offering of energy supplements and climate- and animal-related factors. Ruminally cannulated mixed-breed nonlactating cows [n = 9; initial body weight (BW) = 564.4 ± 101.6 kg] grazing wheat pasture were used in a split-plot design with the objectives of evaluating the effects crude glycerine supplementation and forage maturity on grazing behavior. Treatment was the main plot, and stage of forage maturity was the subplot. Treatments were unsupplemented control (CON) and crude glycerine supplementation (SUP). Crude glycerol (37.2% glycerol, DM basis) was offered at 0.11% of BW (DM basis) and dosed intraruminally via rumen cannula at 0700 and 1900 h daily. The stages of wheat pasture maturity were March (MAR; vegetative stage) and April (APR; early reproductive stage). All cows were fitted with LoRa WAN tracking collars, housing GPS and triaxial accelerometers and allowing real-time inspection of cow position and activity data through time. Hourly (d 10 and 23) and daily change of walking distance (m), activity index and budget for grazing, resting, and walking time (min) were analyzed with a repeated measure mixed model for the split-plot design. Walking distance (1,200 ± 13 m), activity index (10,006 ± 122), resting time (607 ± 5 min), grazing time (662 ± 8 min) and walking time (171 ± 8 min) were not affected (P ≥ 0.52) by crude glycerine supplementation or by stage of forage maturity (P ≥ 0.07), but varied (P < 0.001) by day and by hour of day, likely indicating circadian effects on meal frequency and duration associated with the hour of the day and forage depletion through time. Grazing behavior was more likely affected by climate, pasture, animal, and management variables and less affected by GLY dose. Results indicate that crude glycerine may be used in supplements for cattle grazing wheat pasture to increase energy intake and without adversely creating associative or substitution effects, grazing behavior or sacrificing pasture utilization.

Fault detection method for transmission line components based on lightweight GMPPD-YOLO

Abstract This paper designs a lightweight high-precision transmission line component detection model, named grouped dense, monotonic self-regularized, and partial faster convolution, pruning, and distillation optimized—you only look once (GMPPD-YOLO), in transmission line inspection. It addresses the issue of low detection accuracy of target detection algorithms due to the complex background, large differences in target shape, location, texture, etc, as well as diversified and smaller defects in insulator and vibration hammer images taken by unmanned aerial vehicles from multiple angles. To enhance the model’s feature extraction capabilities in complex backgrounds and across different scales, the grouped dense C3 dense feature extraction module was designed, enabling the model to more effectively handle diverse defect forms. Simultaneously, the monotonic self-regularized pyramid pooling–fast (MSPPF) module is proposed to enhance the model’s capability to process multi-scale information. Additionally, the partial-faster C3 feature awareness module is designed to improve feature fusion performance, enhancing the model’s ability to perceive features at different scales. Finally, channel pruning was used to reduce redundant parameters, and knowledge distillation was employed to compensate for the accuracy loss caused by pruning. This approach further compressed the model size while ensuring its detection performance. The experimental results demonstrate that compared to the original YOLOv5s algorithm, the proposed GMPPD-YOLO algorithm achieves a reduction in parameters by 68.4%, a decrease in Giga floating-point operations per second by 58.2%, and a reduction in the model size by 66.4%, while achieving an increase in precision by 1%, mAP50 by 1.1%, and mAP95 by 0.4%. This confirms the significant potential of the GMPPD-YOLO algorithm for deployment in real-time drone-based power transmission line inspections.

A Lightweight Insulator Defect Detection Model Based on Drone Images

With the continuous development and construction of new power systems, using drones to inspect the condition of transmission line insulators has become an inevitable trend. To facilitate the deployment of drone hardware equipment, this paper proposes IDD-YOLO (Insulator Defect Detection-YOLO), a lightweight insulator defect detection model. Initially, the backbone network of IDD-YOLO employs GhostNet for feature extraction. However, due to the limited feature extraction capability of GhostNet, we designed a lightweight attention mechanism called LCSA (Lightweight Channel-Spatial Attention), which is combined with GhostNet to capture features more comprehensively. Secondly, the neck network of IDD-YOLO utilizes PANet for feature transformation and introduces GSConv and C3Ghost convolution modules to reduce redundant parameters and lighten the network. The head network employs the YOLO detection head, incorporating the EIOU loss function and Mish activation function to optimize the speed and accuracy of insulator defect detection. Finally, the model is optimized using TensorRT and deployed on the NVIDIA Jetson TX2 NX mobile platform to test the actual inference speed of the model. The experimental results demonstrate that the model exhibits outstanding performance on both the proprietary ID-2024 insulator defect dataset and the public SFID insulator dataset. After optimization with TensorRT, the actual inference speed of the IDD-YOLO model reached 20.83 frames per second (FPS), meeting the demands for accurate and real-time inspection of insulator defects by drones.

A portable real-time concrete bridge damage detection system

A drone detection system can achieve real-time inspection of concrete bridge damage. However, it is difficult to deploy existing detection algorithms on a portable system due to their high computational cost. Therefore, a portable damage detection algorithm called CD-YOLOv8 is designed. CD-YOLOv8 is operated on Jetson Xavier NX to detect drone-captured bridge images in real-time. The multi-scale feature adaptive fusion module is used to reduce model parameters and improve detection speed. The spatial context pyramid and deformable convolutional structure are introduced to improve the detection accuracy of tiny concrete bridge damage. Compared with the YOLOv8n, the mAP50 of CD-YOLOv8 is increased by 3.6 %, while the model parameters and FLOPs are decreased by 0.05 M and 0.7 G, respectively. CD-YOLOv8 is also tested on the Jetson Xavier NX with a detection speed of 44 FPS, which is 15.79 % faster than the YOLOv8n. Therefore, CD-YOLOv8 has high accuracy and excellent real-time performance.

Automated inspection approach for OCA particles in multi-layered cover glass of display module assembly

Abstract CG (Cover Glass) is a critical component of electronic screens, and its manufacturing quality is closely relevant to the display effect. To satisfy the requirement of quality control, a machine vision system for real-time inspection of particles in the OCA (Optically Clear Adhesive) layer of CG is presented in this paper. With a brief description of the optical logic of particle imaging and the design of the vision system, our emphasis is put on the post-processing image analysis. To align particles regions in the CG under multi-mode imaging, a spatial alignment calibration algorithm is proposed with perspective distortion correction to calculate the triaxial offset. Then, a CLAHE+PM filtering is adopted to enhance the contrast of the particle. Furthermore, a Meanshift method combined with adaptive local thresholding is proposed to extract the contours of tiny particles. Finally, to distinguish between multiple layers of particles in the CG and detect OCA particles, a combination of fast background reconstruction and Averaged Stochastic Gradient Descent-Support Vector Machine is used. According to in-line experiments and tests, our system can find out a majority of the OCA particles with a PR (over-detection rate) of 1.31% and a PM (miss detection rate) of 0.33% for over 10 000 CG samples.

Laser line scanner based real-time geometry monitoring using Encoder-Decoder network during Directed Energy deposition

Directed Energy Deposition (DED) is a metal additive manufacturing technology that is gaining popularity for its ability to rapidly manufacture virtually any metal components no matter how complex the shapes and properties are. However, the current lack of real-time geometry monitoring and control is hindering the wider dissemination of DED in industries. This study developed and validated a geometry monitoring methodology which can achieve real-time inspection of the melt pool and newly solidified layer, and layer-wise inspection of the deposited layer during DED process. An encoder-decoder network was developed and applied to the profile images from the laser line scanner to obtain track profiles. A point cloud generation method was proposed to convert the obtained track profiles into 3D point cloud data using intrinsic/extrinsic calibration and printing position. Experiments have been successfully conducted to validate the proposed methodology by depositing multi-layer X-shape objects.

Automated fabric inspection system development aided with convolutional autoencoder-based defect detection

Industrial automatic fabric inspection system, a critical technology in the industry, enhances both total production quantity and quality compared to conventional inspection techniques. This study aims to create a reliable and effective real-time automated visual inspection system for fabrics, focusing on defect detection. The goals of the study can be stated as; installing a system with advanced technology for capturing and processing images swiftly, the development and deployment of a system capable of autonomously learning and scanning fabrics in use, and the creation of a smart framework for accurate fabric defect detection and classification. We focus on the development of unsupervised fabric defect detection using a convolutional autoencoder model, and defect classification using a convolutional neural network model, which takes input as the feature vector generated by the convolutional autoencoder. The experimental outcomes have displayed significant success rates in both detecting defects and classifying them, confirming the effectiveness of the framework in real-time visual inspection systems.

Enhancing Inductive IR Thermography by Using FFT-Equalization, Motion Tracking Detection and VDSR Super-resolution Processing

Among non-destructive inspection techniques, infrared thermography stands out as a promising technology that enables real-time inspection of large areas without the need for physical contact. In this study, we employed the dynamic induction thermography method, which is one of the active infrared thermography techniques, to detect defects on the back side of the S275 material specimen. This technique involves creating relative movement between the IR camera and the specimen. We acquired sequence images at different moving speeds using the induction thermography technique, and then used the FFT with Gaussian filtering to solve for non-uniform heat sources. To further enhance the resolution, we applied the VDSR technique, which is based on deep neural networks. The effectiveness of this approach was validated both qualitatively and quantitatively. Finally, we utilized the MOT algorithm to automatically detect defects in the image with the highest thermal contrast, which was captured at a speed of 15 mm/s. In this study, we demonstrate the effectiveness of thermal equalization using the GF-based FFT algorithm, as well as the superresolution conversion achieved through the VDSR-based deep neural network. Additionally, we present a mechanism for automated slit detection using the MOT algorithm.

Development and validation of a real-time vision-based automatic HDMI wire-split inspection system

Development and validation of a real-time vision-based automatic HDMI wire-split inspection system

Real-Time Object Detection from UAV Inspection Videos by Combining YOLOv5s and DeepStream.

The high-altitude real-time inspection of unmanned aerial vehicles (UAVs) has always been a very challenging task. Because high-altitude inspections are susceptible to interference from different weather conditions, interference from communication signals and a larger field of view result in a smaller object area to be identified. We adopted a method that combines a UAV system scheduling platform with artificial intelligence object detection to implement the UAV automatic inspection technology. We trained the YOLOv5s model on five different categories of vehicle data sets, in which mAP50 and mAP50-95 reached 93.2% and 71.7%, respectively. The YOLOv5s model size is only 13.76 MB, and the detection speed of a single inspection photo reaches 11.26 ms. It is a relatively lightweight model and is suitable for deployment on edge devices for real-time detection. In the original DeepStream framework, we set up the http communication protocol to start quickly to enable different users to call and use it at the same time. In addition, asynchronous sending of alarm frame interception function was added and the auxiliary services were set up to quickly resume video streaming after interruption. We deployed the trained YOLOv5s model on the improved DeepStream framework to implement automatic UAV inspection.

User-centered design of an augmented reality inspection tool for industry 4.0 operators

AbstractIn the context of Industry 4.0, the Operator 4.0 paradigm plays a crucial role in seamlessly integrating new digital technologies into smart factories. These technologies are designed to accommodate workers with varying skills, capabilities, and preferences. To promote the adoption of these innovative digital tools and ensure high user acceptance, it is essential to prioritize human factors and place workers at the heart of the development process. This can be achieved through structured design strategies, such as those employed in user-centered design approaches. Within this framework, this paper introduces an innovative Augmented Reality tool designed to facilitate and streamline real-time inspection activities performed by operators on assembled products at the workplace. The development of this tool was based on a user-centered design approach, which actively engaged end-users at different stages of design and testing phases. User studies were conducted using a real case study to evaluate the usability and user acceptance of the proposed solution. The results are encouraging, indicating a promising path forward for the effective and valid implementation of this AR tool in an industrial setting.

X-Ray Image Generation as a Method of Performance Prediction for Real-Time Inspection: a Case Study

X-Ray Image Generation as a Method of Performance Prediction for Real-Time Inspection: a Case Study

Detection of electrode misalignment and its effect on joint quality in resistance spot welding: a low-cost computer vision-based approach

Electrode misalignment in resistance spot welding can be caused by poor fitting or deformation of electrode with continuous usage. This leads to asymmetric weld nugget formation, porosity and expulsion. This paper presents a novel low-cost real-time inspection system for angular misalignment using an image processing approach. The proposed solution can effectively segment the electrode tips even from the image captured at noisy industrial background such as automotive assembly line, by using a regional convolutional neural network based object identification method. The trained model has a mean average precision and recall of 99.01% and 96.6%, respectively. A series of image processing tools and mathematical operations were used to identify the edge line contours of electrode tips accurately from the detection mask, and determine the angular misalignment with a maximum deviation of less than 0.06°. Experimental results showed that the weld nuggets exhibited porosity, shrinkage voids, and cracks when performed under angular misalignment conditions.

In situ real-time pathway to study the polyethylene long-term degradation process by a marine fungus through confocal Raman quantitative imaging

Since plastic waste has become a worldwide pollution problem, studying the ability of marine microorganisms to degrade plastic waste is important. However, conventional methods are unable to in situ real-time study the ability of microorganisms to biodegrade plastics. In recent years, Raman spectroscopy has been widely used in the characterization of plastics as well as in the study of biological metabolism due to its low cost, rapidity, label-free, non-destructive, and water-independent features, which provides us with new ideas to address the above limitations. Here, we have established a method to study the degradation ability of microorganisms on plastics using confocal Raman imaging. Alternaria alternata FB1, a recently reported polyethylene (PE) degrading marine fungus, is used as a model to perform a long-term (up to 274 days) in situ real-time nondestructive inspection of its degradation process. We can prove the degradation of PE plastics from the following two aspects, visualization and analysis of the degradation process based on depth imaging and quantification of the degradation rate by crystallinity calculations. The findings also reveal unprecedented degradation details. The method is important for realizing high-throughput screening of microorganisms with potential to degrade plastics and studying the degradation process of plastics in the future.

Evaluating Moisture Content in Immersion Vacuum-Cooled Sausages with Citrus Peel Extracts Using Hyperspectral Imaging.

The moisture content of immersion vacuum-cooled sausages with modified casings containing citrus fruit extracts under different storage conditions was studied using hyperspectral imaging (HSI) associated with chemometrics. Different pre-processing combinations were applied to improve the robustness of the model. The partial least squares regression model, employing the full reflectance spectrum with pre-treatment of the standard normal variate, showed calibration coefficients of determination (Rc2) of 0.6160 and a root mean square error of calibration (RMSEC) of 2.8130%. For the first time, prediction maps developed via HSI visualized the distribution of moisture content in the immersion vacuum-cooled sausages with unique modified casings in response to fluctuating storage conditions. The prediction maps showed exact parts with high water content, which will help us to monitor and prevent mold growth. The combination of HSI with multivariate analysis not only quantifies changes in moisture content but also visually represents them in response to various casing treatments under different storage conditions, illustrating the significant potential for real-time inspection and early mold detection in sausages within the processed meat industry.