Real-time Quality Inspection Research Articles

Validating the Use of Smart Glasses in Industrial Quality Control: A Case Study

Effective quality control is crucial in industrial manufacturing for influencing efficiency, product dependability, and customer contentment. In the constantly changing landscape of industrial production, conventional inspection methods may fall short, prompting the need for inventive approaches to enhance precision and productivity. In this study, we investigate the application of smart glasses for real-time quality inspection during assembly processes. Our key innovation involves combining smart glasses’ video feed with a server-based image recognition system, utilizing the advanced YOLOv8 model for accurate object detection. This integration seamlessly merges mixed reality (MR) with cutting-edge computer vision algorithms, offering immediate visual feedback and significantly enhancing defect detection in terms of both speed and accuracy. Carried out in a controlled environment, our research provides a thorough evaluation of the system’s functionality and identifies potential improvements. The findings highlight that MR significantly elevates the efficiency and reliability of traditional inspection methods. The synergy of MR and computer vision opens doors for future advancements in industrial quality control, paving the way for more streamlined and dependable manufacturing ecosystems.

In-situ quality inspection system of injection parts based on transfer learning

PurposeThis paper aims to realize an in-situ quality inspection system rapidly for new injection molding (IM) tasks via transfer learning (TL) approach and automation technology.Design/methodology/approachThe proposed in-situ quality inspection system consists of an injection machine, USB camera, programmable logic controller and personal computer, interconnected via OPC or USB communication interfaces. This configuration enables seamless automation of the IM process, real-time quality inspection and automated decision-making. In addition, a MobileNet-based deep learning (DL) model is proposed for quality inspection of injection parts, fine-tuned using the TL approach.FindingsUsing the TL approach, the MobileNet-based DL model demonstrates exceptional performance, achieving validation accuracy of 99.1% with the utilization of merely 50 images per category. Its detection speed and accuracy surpass those of DenseNet121-based, VGG16-based, ResNet50-based and Xception-based convolutional neural networks. Further evaluation using a random data set of 120 images, as assessed through the confusion matrix, attests to an accuracy rate of 96.67%.Originality/valueThe proposed MobileNet-based DL model achieves higher accuracy with less resource consumption using the TL approach. It is integrated with automation technologies to build the in-situ quality inspection system of injection parts, which improves the cost-efficiency by facilitating the acquisition and labeling of task-specific images, enabling automatic defect detection and decision-making online, thus holding profound significance for the IM industry and its pursuit of enhanced quality inspection measures.

Real Time Quality Detection and Automatic Recognition of Concrete Structures: Image Processing Algorithm Based on Deep Learning

The safety and durability of concrete structure is an important issue in engineering quality management. In this paper, an image processing algorithm based on deep learning is proposed to realize real-time quality inspection and automatic defect identification of concrete structures. This algorithm uses the Convolutional Neural Network (CNN) to automatically extract the features of concrete surface quality images, and then identify the existence of defects, thus improving the detection efficiency and accuracy. In this paper, for this method, data samples with different specific structures are collected and manually labeled to the data set; then, a multi-layer CNN model with convolution layer, pool layer and full connection layer is designed to train the model, and then image enhancement technology is used to reduce information noise, and data enhancement technology is used to improve the problem-solving ability of the model. In addition, the strategy of Dropout is used to close some nodes to reduce parameters and prevent over-fitting, and the learning rate is adjusted to optimize the classification effect. In addition, this study constructs an all-weather real-time detection framework, including data acquisition, preprocessing, feature extraction, classification and identification and decision-making alarm system, to ensure the rapid positioning of the detection system. To sum up, the results of this study show that the deep learning image processing algorithm has good contrast performance in the field of real-time quality inspection of concrete structures. CNN model has better performance than GAN (Generative Adversarial Network) and LSTM (Long Short-Term Memory) models in detection time, defect identification resolution and detection accuracy. The maximum detection time is 366ms and the shortest is 213 ms. The successful development of this algorithm provides a new method for automatic detection of concrete structure quality, which has important application value in engineering practice. This research has a broad development prospect.

Real-Time Quality Inspection Based on Transfer Learning and Feature Clustering for Wave Soldering

Real-Time Quality Inspection Based on Transfer Learning and Feature Clustering for Wave Soldering

Portable hyperspectral imaging coupled with multivariate analysis for real-time prediction of plant-based meat analogues quality

Growing demand for plant-based meat analogues and their quality changes during storage necessitates real-time quality monitoring and chemical mapping. This study reports the use of portable hyperspectral imaging in the visible-near-infrared (VNIR) range (400–1000 nm) coupled with multivariate analysis for the prediction of quality traits (color, moisture, pH, and hardness) in plant-based meat burgers (PBMB) of different formulations ranging from 10 % to 30 % textured vegetable protein (TVP) and 5 % to 10 % pea protein (PP) during a 14-day storage period and map the quality distribution on each pixel. PCA was applied to explore both spectral (PCA-S) and measured quality data (PCA-M), while PLSR was applied for prediction modeling. ANOVA and PCA-M results showed the instability of PBMB quality even over a short storage period. PCA-S showed successful spatial separation of PBMB samples based on storage days, TVP, and PP concentrations. The PLSR models possessed good prediction accuracies for redness (R2P = 0.92, RMSEP = 0.74), yellowness (R2P = 0.93, RMSEP = 0.44), moisture (R2P = 0.90, RMSEP = 0.81 % Wb), and hardness (R2P = 0.81, RMSEP = 0.87 N). Overall, the prediction results and distribution maps unravel the feasibility of VNIR-HSI coupled with chemometrics for real-time quality inspection in plant-based food production.

An augmented reality maintenance assistant with real-time quality inspection on handheld mobile devices

Advances of industry 4.0 are enabling augmented reality (AR) devices to be deployed across the manufacturing sector to enhance worker engagement and performance. This evolution of AR in the workspace is driving changes in how workers on the factory floor interact with the enterprise. AR is evolving to provide work instructions for assembly and maintenance. Recent demonstrations show the capability to allow for quality inspection to be integrally evaluated as part of the AR workflow. Previous integrations of quality include the ability for manual assessment to be logged into the enterprise as part of completing work instructions; however, this is largely based on the perception of the user to determine if they accurately performed the AR instructions. This study is designed to demonstrate the ability of an enterprise-level AR maintenance experience to go beyond manual inspection by embedding real-time quality assessment into the work instructions. Essentially, the goal is to enable the enterprise to perform a self-assessment of the quality of the work performed by maintenance personnel. Furthermore, the AR instructions enable additional instruction to empower maintenance personnel to correct their work prior to moving on to other tasks. The novelty of this work is focused on deployment on a mobile device with integrated vision inspection using the same device that is being used to deploy the AR work instructions. This work enumerates several essential qualities of mobile quality inspection tools and outlines some of the challenges associated with the development of such a system. Finally, results from testing the system demonstrate that handheld mobile devices can be used to capture inspection images while simultaneously deploying AR work instructions. Analysis of the image correction, timing for vision process of the quality check, and overall performance of the integrated system are presented.

A cloud-based 3D real-time inspection platform for industry: a case-study focusing automotive cast iron parts

A 3D real-time quality inspection platform that specifically focus on automotive cast iron parts was developed for the industry and is presented in this work. It is supported by a cloud-based platform, which combines recent software and hardware advances to deal with large amounts of information related to the acquisition process and the computational power needed to execute the computer vision platform algorithms (e.g., point cloud filtering, alignment, and comparison). This platform introduces changes in the current workflow through the inspection process’ digitalization. Indeed, it promotes the reduction of human-related inspection errors, as well as ergonomic issues, while simultaneously making available a solution for the automatic gathering and storing of data in a cloud-like environment, for further access and advanced data analytics.

A BIM-IoT and intelligent compaction integrated framework for advanced road compaction quality monitoring and management

The road construction industry is undergoing a continuous digital transformation due to higher construction quality control and management requirements to improve efficiency and reduce cost consumption. IC (Intelligent compaction) is an emerging and developing technology that can overcome the limitations of current quality control and quality assurance approaches, such as detection hysteresis and incompleteness. However, IC is mainly used in a vehicle system to realize real-time quality inspection, while its further potential is not fully exerted in advanced construction control and management as a comprehensive, continuous and nondestructive testing tool. This paper proposes a BIM (Building Information Model)-IoT (Internet of things) and IC integrated framework to provide a fundamental platform for advanced compaction quality monitoring and management. Specifically, BIM provides high-fidelity virtual assets of road infrastructure for data integration, visualization and construction organization management. IC is used to provide comprehensive compaction quality monitoring data and quality control methods. IoT (Internet of things) is adopted to connect physical and cyber assets by capturing real-time data from the actual operations in construction. Finally, a prototype system is developed as an implementation of the framework. The case demonstration of the system shows that the proposed framework can effectively integrate BIM and IC to realize the monitoring of road compaction quality and data in combination with IoT data, and the construction schedule can be rearranged and optimized based on the monitoring results. This study can facilitate to realize road compaction quality real-time monitoring and management, as well to offer a basic BIM-IoT platform that can be utilized as a prototype of road infrastructure digital twin to achieve more intelligent and sustainable management of the entire construction process in the future.

Fast Removal of Powerline Harmonic Noise From Surface NMR Datasets Using a Projection-Based Approach on Graphical Processing Units

Surface nuclear magnetic resonance (NMR) measurements are notorious for their low signal-to-noise ratio (SNR). Powerlines are probably the most common source of noise and give the greatest contribution to noise levels. The noise from powerlines manifests itself as sinusoidal signals oscillating at the fundamental powerline frequency (50 or 60 Hz) and at integer multiples of this frequency. Modeling and subtraction of the powerline noise have been demonstrated as a highly applicable method for improving SNR and are common practice today. However, the methods used to determine the parameters of the powerline noise are computationally expensive. Consequently, it is difficult to do real-time noise removal during the acquisition of field data and, therefore, also difficult to do a real-time quality inspection of data. Here, we demonstrate how the removal of powerline noise in surface NMR data can be significantly faster. We obtain this through two new developments. First, we apply a projection-based method to determine the powerline model, which is twice as fast as the commonly applied least-squares solution of a matrix equation. Second, we obtain a further 10–25 times speed-up by exploiting the high-performance parallel computations offered by graphical processing units (GPUs). We demonstrate the method on a noise-only field dataset with an embedded synthetic NMR signal.

Motor brush assembly inspection system based on OpenMV vision technology

Motor brush assembly inspection is an important link in the industrial field. Traditional quality inspection methods have high errors, poor efficiency and low automation. To this end, the OpenMV machine vision module is used, through the MicroPython language, combined with Canny operator and Hough Transform (Hough Transform, Hough) algorithm. A program is designed to collect the image of the motor brush and process the data by itself. After the actual application in the factory, the detection effect is good, and the accuracy rate is more than 90%, which effectively improves the detection efficiency and meets the real-time quality inspection needs of the motor brushes.

Surface finish classification using depth camera data

We propose a novel approach for surface finish classification of digitally fabricated structures using an industrial depth camera. Data collected at different viewpoints are jointly processed to derive the spatial distribution of features describing the reflectance, which is in turn related to the surface finish. The features can be used to classify the surfaces according to their finish e.g., for assessing the homogeneity or conformance. We apply the method to four sprayed plaster specimens of similar visual appearance but different roughness. Using nearest neighbor classification we achieve an accuracy of 97% for the plaster samples. The approach is a contribution towards real-time quality inspection in digital fabrication.

Implementation of surface crack detection method for nuclear fuel pellets guided by convolution neural network

ABSTRACT Crack detection is one of the important contents of the nuclear fuel pellet quality inspection. Aiming at the problem of high crack false detection rate caused by low image contrast of nuclear fuel pellets, complex image background and fine cracks on the pellet surface, a Weighted Object Variance (WOV) threshold crack detection method guided by SE-CrackNet convolutional neural network is proposed. The method first uses the sliding window scanning technology and SE-CrackNet network to locate the crack regions in the pellet image, and then uses the WOV threshold method to extract the cracks to achieve accurate identification of the cracks on the surface of the nuclear fuel pellet. The pixel-level F1-measure of the method is about 92%, which can accurately identify cracks on the surface of nuclear fuel pellets, greatly reduce the crack false detection rate, meet the real-time quality inspection requirements of nuclear fuel pellet production lines, and vastly improve the performance of traditional machine vision inspection systems. At the same time, the method can be extended to the quality inspection of other industrial products.

Structured-light three-dimensional scanning for process monitoring and quality control in precast concrete production

Quality control is a crucial step in the fabrication of precast concrete products. There are three critical quality features: the fabrication consistency of surface finish, the dimensional accuracy of the overall geometry, and the positioning accuracy of the embedded parts. Existing quality control methods rely on tape measures and inspectors’ experience, which may lead to measurement inconsistency, operator faults, and safety hazards. This paper proposes an innovative real-time quality inspection system for the three critical quality features. The system first uses a structured-light three-dimensional scanner to capture the surface and geometric data from the precast concrete parts and then calculates and visualizes the deviations by applying specially developed algorithms. In addition, all of the functions are compiled into a graphical user interface that can be easily used by operators without a data analytics background. The system has been tested on a precast concrete sample with five different surface finishes in five regions of the sample, complex geometries, and a variety of embedded parts. The experiment results show that the proposed quality inspection and data analysis system can obtain critical quality information efficiently and accurately.

Online measurement of weld penetration in robotic resistance spot welding using electrode displacement signals

Realtime quality inspection of resistance spot welding (RSW) is a promising way to actualize intelligent manufacturing of automotive body structures. Apart from nugget diameter, weld penetration is also an important quality index for weld quality assessment. To that point, a novel approach to nondestructive measurement of weld penetration for the robotic RSW process is presented. In-depth research on the electrode displacement signal is carried out based on mechanism analysis and feature extraction. The effectiveness of the signal feature is tested under not only standard but also abnormal manufacturing conditions to investigate the generalization ability. Finally, an analytical model for predicting weld penetration is established via physical modeling to achieve online measurement capability. The error of weld penetration estimation could be limited to ±9.6% (±2σ) of the sheet thickness, and the measurement accuracy is robust to varying welding parameters and abnormal manufacturing conditions.

Welding stability evaluation of 2219 aluminum alloy by double-pulse variable polarity GTAW based on empirical mode decomposition and Hilbert-Huang transform

According to the current real-time quality inspection requirements for welding processes, the electrical signals in double-pulse variable polarity gas tungsten arc welding (VP-GTAW) welding process of 2219 aluminum alloy were collected. The characteristics of steady-state and non-steady-state signals of the welding process were studied. The Hilbert-Huang transform based on empirical mode decomposition was applied to analyze the stability of double-pulse VP-GTAW. Fast Fourier Transform (FFT) spectrum and Hilbert spectrum of the intrinsic mode functions were obtained, and the stability of the welding process was studied. The relationship between the welding signal of the welding process and the weld formation was established, as the welding process becomes unstable, the time-frequency entropy of the welding process increases.

Magnetic Ring Multi-Defect Stereo Detection System Based on Multi-Camera Vision Technology.

Magnetic rings are the most widely used magnetic material product in industry. The existing manual defect detection method for magnetic rings has high cost, low efficiency and low precision. To address this issue, a magnetic ring multi-defect stereo detection system based on multi-camera vision technology is developed to complete the automatic inspection of magnetic rings. The system can detect surface defects and measure ring height simultaneously. Two image processing algorithms are proposed, namely, the image edge removal algorithm (IERA) and magnetic ring location algorithm (MRLA), separately. On the basis of these two algorithms, connected domain filtering methods for crack, fiber and large-area defects are established to complete defect inspection. This system achieves a recognition rate of 100% for defects such as crack, adhesion, hanger adhesion and pitting. Furthermore, the recognition rate for fiber and foreign matter defects attains 92.5% and 91.5%, respectively. The detection speed exceeds 120 magnetic rings per minutes, and the precision is within 0.05 mm. Both precision and speed meet the requirements of real-time quality inspection in actual production.

Enabling Real-Time Quality Inspection in Smart Manufacturing Through Wearable Smart Devices and Deep Learning

In this paper, we present a novel method for utilising wearable devices with Convolutional Neural Networks (CNN) trained on acoustic and accelerometer signals in smart manufacturing environments in order to provide real-time quality inspection during manual operations. We show through our framework how recorded or streamed sound and accelerometer data gathered from a wrist-attached device can classify certain user actions as successful or unsuccessful. The classification is designed with a Deep CNN model trained on Mel-frequency Cepstral Coefficients (MFCC) from the acoustic input signals. The wearable device provides feedback on three different modalities: audio, visual and haptic; thus ensuring the worker’s awareness at all time. We validate our findings through deployments of the complete AI-enabled device in production facilities of Mercedes-Benz AG. From the conducted experiments it is concluded that the use of acoustic and accelerometer data is valuable to train a classifier with the purpose of action examination during industrial assembly operations, and provides an intuitive interface for ensuring continued and improved quality inspection.

Image Processing for Sustainable Remodeling: Introduction to Real-time Quality Inspection System of External Wall Insulation Works

The external wall insulation method was introduced to enhance the energy efficiency of existing buildings. It does not cause a decrease of inner space and costs less in comparison to methods that insert insulation panels inside walls. However, it has been reported that external wall insulation boards are disconnecting from walls due to malfunctions of the adhesive. This causes not only repair costs, but also serious injury to pedestrians. Separation problems occur when the bonded positions are incorrect and/or the total area and thickness of the adhesive is smaller than the required amount. A challenge is that these faults can hardly be inspected after installing boards. For this reason, a real-time inspection system is necessary to detect potential failure during adhesive works. Position, area and thickness are major aspects to inspect, and thus a method to process image data of these seems efficient. This paper presents a real-time quality inspection system introducing image processing technology to detect potential errors during adhesive works of external wall insulation, and it is predicted to contribute to achieving sustainable remodeling construction by reducing squandered material and labor costs. The system consists of a graphic data creation module to capture the results of adhesive works and a quality inspection module to judge the pass or fail of works according to an algorithm. A prototype is developed and validated against 100 panels with 800 adhesive points.

A novel image processing framework to detect geometrical features of horticultural crops: case study of Anthurium flowers

An image processing framework was developed to automatically detect the geometrical features of horticultural crops via identifying theirs proximal and distal ends. The developed algorithm detects boundary of the crops and fits a suitable B-spline curve on it. Subsequently, the algorithm calculates the first and second derivatives of the curve and finds its geometrical key points as mathematical critical points. Thus, it is possible to calculate geometrical features of the crops such as perimeter, area, curvature, and especially length and width, at any desired angle that the crops are placed under the camera. The algorithm was tested on three cultivars of Anthurium flower and results showed that the reconstructed shapes of the flowers considered high adaptation to the original shapes. Geometrical key points of the flower shape were correctly detected in all experiments. In addition, geometrical features of rotated flowers were calculated with a relative error of less than 2%. The presented algorithm could be used to detect geometrical attributes of horticultural crops with irregular shapes from two dimensional (2D) images. It would be useful to develop real-time quality inspection and grading systems, as well as to collect accurate and detailed morphological data from 2D objects.

Automated Quality Classification of Wooden Parts for Flexible Manufacturing

In this paper we present a system for real-time quality inspection of wooden parts. The quality inspection system is based on computer vision and was developed for classifying wooden parts in an application with automated robotic handling operations. The quality inspection stands for evaluation of the surface quality, including image capture, and feature-based surface classification. The features are extracted from blobs and the classification algorithm relies on support vector machines. Based on our test results, the reliability of the classification is at a sufficient level. Index Terms—Wood industry, Support vector machines,