On-line Inspection Research Articles

A method for enhancing near-mirror object detection by integrating AWCS and CBAM into ResNet18

ABSTRACT Defects, such as scratches, frequently arise during the manufacturing of near-mirror workpieces, adversely affecting both their longevity and aesthetic appeal. The identification of these near-mirror defects through machine vision is fraught with challenges, including the scarcity of comprehensive datasets, variability in surface curvature, and the diminutive size of the defects. In response to these challenges, this study integrates Visual Parser and deep learning methodologies for the visual detection of near-mirror defects. A dataset conducive to deep learning was developed utilising a sliding window approach. The proposed model incorporates an Adaptive Weighted Convolution Structure (AWCS) and Convolutional Block Attention Module (CBAM) within the ResNet18 architecture. Additionally, we introduce an efficient and precise technique termed Visual Parser, which facilitates the detection and localisation of defect areas in near-mirror workpieces. When combined with a classification model, the Visual Parser allows for the online inference and detection of extensive near-mirror images. Experimental findings indicate that the performance metrics of the proposed ResNet18-AWCS-CBAM (ResNet18-ASCM) model significantly exceed those of established networks. The defect detection and localisation method based on Visual Parser is capable of fulfilling the online inspection requirements pertinent to the production of near-mirror workpieces.

On-line inspection of lattice structures and metamaterials via in-situ imaging in Additive Manufacturing

As advanced production capabilities are moving towards novel types of geometries as well as higher customization demands, a new and more efficient approach for process and part qualification is becoming an urgent need in industry. The layerwise nature of additive manufacturing (AM) potentially allows anticipating qualification tasks in-line and in-process, aiming at reducing the time and costs devoted to post-process inspections, enabling at the same time an early detection of defects since their onset stage. Such opportunity is particularly attractive in the presence of highly complex shapes like lattice structures or metamaterials, which have been increasingly investigated for industrial adoption in various sectors, aiming to achieve enhanced mechanical properties and innovative functionalities. This paper presents a novel methodology to inspect the geometry of lattice structures while the part is being built. The method is specifically designed to tackle the natural variability affecting layerwise images gathered in laser powder bed fusion. To this aim, it combines the segmentation of in-situ powder bed images of solidified layers with a data modelling approach to synthesize the 3-D shape of each unit cell into a 1-D profile representation. Such low-dimensional representation is suitable to quickly detect undesired distortions that may have a detrimental impact on final quality and performance. By using post-process X-ray computed tomography as ground truth reference, this study shows the effectiveness of the proposed approach for in-line inspection, opening a novel and cost-efficient way to address complex shape qualification for lattice structures in AM.

Effective Bi-decoding networks for rail-surface defect detection by knowledge distillation

No-service rail-surface defect detection is a crucial method for assessing the quality of railroad tracks. However, the low-contrast and dark-tone characteristics of track-surface textures pose challenges to current defect-monitoring techniques. Real-time and on-site online inspections are important to ensure safe railway operation; however, most complex models for no-service inspections are difficult to deploy on mobile devices. To address these challenges and overcome the detection difficulties associated with complex scenes, we designed a knowledge distillation-based double decoding-layer refinement network (EBDNet-KD). The first decoding process is guided by a bimodal high-level semantic feature map obtained by extending the attention-based graph convolution to incrementally enhance the dual-stream features and obtain an image restoration prior. A divide-and-conquer decoder is then designed to distinguish features using different decoding layers. The prior is then used in the second decoding layer, which enables the bimodal features to interact fully and obtain the final prediction map. We introduce a knowledge distillation strategy that enables a lightweight, compact student network to learn a complex teacher network’s feature extraction process. This facilitates pixel-consistent learning of the knowledge within the bi-decoder layer, as well as bidirectional learning of the focused contextual response knowledge to optimize the model. The EBDNet-KD significantly reduces computational costs while guaranteeing performance with a parameter count of only 28 M. EBDNet-KD demonstrated superior performance over 15 state-of-the-art methods in experiments conducted on NEU RSDDS-AUG, an industrial RGB-depth dataset. We assessed the generalizability of EBDNet-KD by evaluating its performance on three additional public datasets, yielding competitive results. The source code and results can be found at https://github.com/Wuyue15/EBDNet.

Biofuser: a multi-source data fusion platform for fusing the data of fermentation process devices.

In the past decade, the progress of traditional bioprocess optimization technique has lagged far behind the rapid development of synthetic biology, which has hindered the industrialization process of synthetic biology achievements. Recently, more and more advanced equipment and sensors have been applied for bioprocess online inspection to improve the understanding and optimization efficiency of the process. This has resulted in large amounts of process data from various sources with different communication protocols and data formats, requiring the development of techniques for integration and fusion of these heterogeneous data. Here we describe a multi-source fusion platform (Biofuser) that is designed to collect and process multi-source heterogeneous data. Biofuser integrates various data to a unique format that facilitates data visualization, further analysis, model construction, and automatic process control. Moreover, Biofuser also provides additional APIs that support machine learning or deep learning using the integrated data. We illustrate the application of Biofuser with a case study on riboflavin fermentation process development, demonstrating its ability in device faulty identification, critical process factor identification, and bioprocess prediction. Biofuser has the potential to significantly enhance the development of fermentation optimization techniques and is expected to become an important infrastructure for artificial intelligent integration into bioprocess optimization, thereby promoting the development of intelligent biomanufacturing.

Quasi-visualizable detection of deep sub-wavelength defects in patterned wafers by breaking the optical form birefringence

Abstract In integrated circuit (IC) manufacturing, fast, nondestructive, and precise detection of defects in patterned wafers, realized by bright-field microscopy, is one of the critical factors for ensuring the final performance and yields of chips. With the critical dimensions of IC nanostructures continuing to shrink, directly imaging or classifying deep-subwavelength defects by bright-field microscopy is challenging due to the well-known diffraction barrier, the weak scattering effect, and the faint correlation between the scattering cross-section and the defect morphology. Herein, we propose an optical far-field inspection method based on the form-birefringence scattering imaging of the defective nanostructure, which can identify and classify various defects without requiring optical super-resolution. The technique is built upon the principle of breaking the optical form birefringence of the original periodic nanostructures by the defect perturbation under the anisotropic illumination modes, such as the orthogonally polarized plane waves, then combined with the high-order difference of far-field images. We validated the feasibility and effectiveness of the proposed method in detecting deep subwavelength defects through rigid vector imaging modeling and optical detection experiments of various defective nanostructures based on polarization microscopy. On this basis, an intelligent classification algorithm for typical patterned defects based on a dual-channel AlexNet neural network has been proposed, stabilizing the classification accuracy of λ/16-sized defects with highly similar features at more than 90%. The strong classification capability of the two-channel network on typical patterned defects can be attributed to the high-order difference image and its transverse gradient being used as the network's input, which highlights the polarization modulation difference between different patterned defects more significantly than conventional bright-field microscopy results. This work will provide a new but easy-to-operate method for detecting and classifying deep-subwavelength defects in patterned wafers or photomasks, which thus endows current online inspection equipment with more missions in advanced IC manufacturing.

A customised ConvNeXt-SCC network: integrating improved principal component analysis with ConvNeXt to enhance tire crown defect detection

ABSTRACT Defects such as bubbles and rubber shortages occasionally occur during vehicle tire production. Tiny defects in the crown gradually expand under load, ultimately causing tire failure. However, machine-vision-based crown defect detection faces challenges like limited datasets, varying groove patterns, low colour contrast, and small defect sizes. Therefore, this article integrates principal component analysis (PCA) and deep learning techniques for the visual detection of tire crown defects. A dataset of tire crown defects suitable for deep learning was constructed using an improved-PCA method and composite sampling. We propose the ConvNeXt-Shallow Conv CBAM (ConvNeXt-SCC) model, which includes an improved convolutional block attention module (CBAM) in a shallow stacked structure. And we propose an efficient and accurate method called Improved Selective Search (ISS) to detect and locate tire crown defective regions. The ISS pre-selected box generation method, enhanced with a priori knowledge filtering module, was used alongside a classification model to achieve online inference detection of large-size tire crown images. The experimental results show that the performance indices of the proposed ConvNeXt-Shallow Conv CBAM (ConvNeXt-SCC) model significantly improve compared to the original ConvNeXt-T. The defect detection and localisation method based on ISS meets the online inspection requirements of tire production.

Online Inspection Method and System Design for Screw Threads of Rebar Head Based on Machine Vision

An online inspection method based on machine vision was proposed and validated to address the issues of high work intensity, low efficiency, low accuracy, and risk of missed inspection in traditional sampling methods for screw threads of rebar head. Firstly, an industrial camera was used to capture real-time images of the processed rebar thread heads, preprocess the images, and locate the target positions in the images to reduce the complexity and running time of subsequent algorithms. Then, the Canny operator was used to roughly extract the edge feature information of the rebar head, and the Shi–Tomasi algorithm was used for corner inspection to achieve precise optimization of sub-pixel level corners. Based on robust linear regression, the diagonal points were fitted with lines to detect the corresponding size parameters. Finally, an inspection system on screw threads of rebar head parameter was designed and developed, which consisted of an image-acquisition device, Siemens PLC controller, and inspection software. Test results show that this method can achieve online inspection without contact, with inspection accuracy reaching the micrometer level, and 8–10 rebar heads can be inspected per second.

An analytical study of TEG loss in a natural gas purification plant based on high-pressure droplet angular scattering

In the process of purifying natural gas, natural gas dehydration plants use triethylene glycol (TEG) as a solvent for absorption. However, the loss of TEGs in the form of droplets can pollute pipelines and create a technological challenge. Using the geometric optical approximation (GOA) method, a model was developed to predict the scattering of droplets at high pressure. Based on this model, a device was designed and constructed to inspect droplets in external pipelines and distillation column tailgate lines online, which are the outlets of the natural gas dehydration unit (TEG absorber tower) of four columns in stable production. This device analyzes the components of the sampled droplets and performs online inspection to ensure smooth and safe operation. The results of the experiment indicate that the TEG content in the droplet samples exceeded 80%, which helped identify the source of the loss. The amount of treatment directly affected the concentration and distribution features of the entrained droplets, whereas the impact of the gas-phase pressure change was negligible. This study investigated three different forms of filtration separation, which provided industrial data for the separation and recovery process of entrained droplets. This study provides a new testing tool and method to examine plate towers under high pressure and provides a theoretical reference for dewatering procedures in natural gas purification plants.

Live Dimension Detection During Machining of Workpiece using Smart Detection Set up on General-Purpose Lathe

In a manufacturing setting, the profile dimensions of the workpiece are key quality factors. Various automatic inspection approaches currently assess dimensions based on characteristics and variables, tool paths as OK or Not OK in CNC, and online inspection methods for other machines after manufacturing. However, continuous on-machine profile dimensions and simultaneous manufacturing were left to focus. An intelligent recognition system based on a laser lines sensor using a smart profile dimension detection algorithm (SPDA) communicates real-time measurement information to the operator. It plots the current status of the Workpiece, mapping linear distance by accurate laser reflection of the exact mid-surface line of the workpiece as profile data of a symmetric shaft. It compares the length-diameter obtained from longitudinal and lateral sensors with the master data provided. The algorithm implements Python libraries for displaying panels and organizing workpieces. The standard image changes color from red to green as it is relevant to the status. The predetermined order of manufacturing parts and gradient descent optimization feature sets benchmarks to display the status of the workpiece. The real-time live measurement is facilitated with an error of less than 10%, i.e., 0.1 mm, saving manufacturing time by 30%. It also eliminates the inspection stage, and no reduced job rejection is found.

DNN Adaptive Partitioning Strategy for Heterogeneous Online Inspection Systems of Substations

With the explosive development of power edge equipment and the continuous improvement in power inspection performance, the requirements of substations and terminal equipment, such as drones with limited resources, cannot meet the strict delay and energy consumption requirements. This paper proposes an adaptive partitioning strategy for heterogeneous substation inspection systems. First, a layer delay prediction model and layer energy consumption prediction model are established on each heterogeneous node, and nonlinear characteristics related to delay and energy consumption are trained. On this basis, a deep neural network (DNN) hybrid partitioning strategy is proposed. The DNN task is divided into synchronous cooperative reasoning between terminal devices and multi-heterogeneous edge nodes. The experimental results show that the average absolute percentage error (MAPE) of the delay model was reduced by 31.49% on average. On drones and mobile edge nodes, the energy consumption model MAPE reduced the average by 21.92%, and the DNN end-to-end latency was reduced by 31.48%. The total cost of the system was reduced and the efficiency of UAV inspection was improved.

Abnormal pattern recognition for online inspection in manufacturing process based on multi-scale time series classification

The collection of large volumes of temporal data during the production process is streamlined in a cyber manufacturing environment. The ineluctable abnormal patterns in these time series often serve as indicators of potential manufacturing faults. Consequently, the presence of effective analytical methods becomes essential for monitoring and recognizing these abnormal manufacturing patterns. However, the extensive process data may contain various minor abnormal patterns, typically reflecting changes in production status influenced by multiple anomalous causes. This study introduces an approach for recognizing abnormal manufacturing patterns through multi-scale time series classification (TSC). Long-term process signals undergo slicing using dynamically sized observation windows and subsequent classification at multiple scales employing our proposed TSC model, the distance mode profile-multi-branch dilated convolution network (DMP-MDNet). DMP-MDNet comprises two key modules aimed at bypassing complicated feature engineering and enhancing generalization capability. The first module, DMP, uses similarity measurement to encode scale- and magnitude-invariant temporal properties. Subsequently, the MDNet, equipped with multi-receptive field sizes, effectively leverages multi-granularity data for accurate classification. The effectiveness of our method is demonstrated through the analysis of a real-world body-in-white production dataset and various widely used public TSC datasets, showing promising applicability in actual manufacturing processes.

A Bayesian hypothesis testing model for quality control of mass customized cabinet manufacturing process

The demand for customization of cabinets and homes has increased, and customized product components are cumbersome in size and time-consuming. In order to realize the mass production of customized cabinet furniture, research from the custom design, optimization of factory machine production process and quality inspection, as well as the flow of the way, is to realize the fundamental method of large-scale production of cabinet furniture. Based on modern intelligent information processing functions instead of manual ordering, order splitting, and unit parts for reasonable division of classification, optimize the topology of the production line, to achieve modern intelligent production of panels. However, in the production process, there is a lack of on-line inspection of some links, and the yield of the product is affected. This thesis establishes a Bayesian hypothesis testing model based on the topological model of the production line process, and through the actual production data, the defective nodes of the manufacturing process are retraced to achieve the purpose of optimizing the customized production.

DSN-BR-Based Online Inspection Method and Application for Surface Defects of Pharmaceutical Products in Aluminum-Plastic Blister Packages

Ensuring high product quality is of paramount importance in pharmaceutical drug manufacturing, as it is subject to rigorous regulatory practices. This study presents a research focused on the development of an on-line detection method and system for identifying surface defects in pharmaceutical products packaged in aluminum-plastic blisters. Firstly, the aluminum-plastic blister packages exhibit multi-scale features and inter-class indistinction. To address this, the deep semantic network with boundary refinement (DSN-BR) model is proposed, which leverages semantic segmentation domain knowledge, to accurately segment the defects in pixel level. Additionally, a specialized image acquisition module that minimizes the impact of ambient light is established, ensuring high-quality image capture. Finally, the image acquisition module, image detection module, and data management module are designed to construct a comprehensive online surface defect detection system. To validate the effectiveness of our approach, we employ a real dataset for instance verification on the implemented system. The experimental results substantiate the outstanding performance of the DSN-BR, achieving the mean intersection over union (MIoU) of 90.5%. Furthermore, the proposed system achieves an inference speed of up to 14.12 f/s, while attaining an F1-Score of 98.25%. These results demonstrate that the system meets the actual needs of the enterprise and provides theoretical and methodological support for intelligent inspection of product surface quality. By standardizing the control process of pharmaceutical manufacturing and improving the management capability of the manufacturing process, our approach holds significant market application prospects.

Intelligent Inspection Method and System of Plastic Gear Surface Defects Based on Adaptive Sample Weighting Deep Learning Model.

After injection molding, plastic gears often exhibit surface defects, including those on end faces and tooth surfaces. These defects encompass a wide range of types and possess complex characteristics, which pose challenges for inspection. Current visual inspection systems for plastic gears suffer from limitations such as single-category defect inspection and low accuracy. There is an urgent industry need for a comprehensive and accurate method and system for inspecting defects on plastic gears, with improved inspection capability and higher accuracy. This paper presents an intelligent inspection algorithm network for plastic gear defects (PGD-net), which effectively captures subtle defect features at arbitrary locations on the surface compared to other models. An adaptive sample weighting method is proposed and integrated into an improved Focal-IoU loss function to address the issue of low inspection accuracy caused by imbalanced defect dataset distributions, thus enhancing the regression accuracy for difficult defect categories. CoordConv layers are incorporated into each inspection head to improve the model's generalization capability. Furthermore, a dataset of plastic gear surface defects comprising 16 types of defects is constructed, and our algorithm is trained and tested on this dataset. The PGD-net achieves a comprehensive mean average precision (mAP) value of 95.6% for the 16 defect types. Additionally, an online inspection system is developed based on the PGD-net algorithm, which can be integrated with plastic gear production lines to achieve online full inspection and automatic sorting of plastic gear defects. The entire system has been successfully applied in plastic gear production lines, conducting daily inspections of over 60,000 gears.

The Microstructure Characterization of a Titanium Alloy Based on a Laser Ultrasonic Random Forest Regression

The traditional microstructure detecting methods such as metallography and electron backscatter diffraction are destructive to the sample and time-consuming and they cannot meet the needs of rapid online inspection. In this paper, a random forest regression microstructure characterization method based on a laser ultrasound technique is investigated for evaluating the microstructure of a titanium alloy (Ti-6Al-4V). Based on the high correlation between the longitudinal wave velocity of ultrasonic waves, the average grain size of the primary α phase, and the volume fraction of the transformed β matrix of the titanium alloy, and with the longitudinal wave velocity as the input feature and the average grain size of the primary α phase and the volume fraction of the transformed β matrix as the output features, prediction models for the average grain size of the primary α phase and the volume fraction of the transformed β matrix were developed based on a random forest regression. The results show that the mean values of the mean relative errors of the predicted mean grain size of the native α phase and the volume fraction of the transformed β matrix for the six samples in the two prediction models were 11.55% and 10.19%, respectively, and the RMSE and MAE obtained from both prediction models were relatively small, which indicates that the two established random forest regression models have a high prediction accuracy.

Real-time grading of roasted tobacco using near infrared spectroscopy technology

Traditional tobacco quality grading methods have limitations such as high human resource consumption and poor consistency, resulting in the grading accuracy being affected. Near infrared spectroscopy (NIRS) technology, as a rapid and non-destructive analytical method, has been widely used for online inspection. This study aims to accomplish rapid, accurate and reliable real-time grading of tobacco quality by employing online near-infrared spectroscopy to detect key chemical indicators. A key indicators selection method based on information gain and Fisher criterion (IFRI) was proposed in this paper by associating tobacco chemical indicators with traditional quality grades. The key chemical indicators were selected from many indicators by two classical feature selection functions, and a chemical indicator grading table was designed based on a large number of statistical data on tobacco chemical indicators, and the table was applied to grade the tobacco quality of real data from different years. The results show that the IFRI algorithm is an effective method for selecting key indicators, and the selected reducing sugars-to-total nitrogen ratio and nicotine are the key indicators for tobacco quality grading. Moreover, we demonstrated that the designed grading table is reasonable and feasible. Compared with traditional grading, chemical indicator grading based on near infrared spectroscopy can not only be realized quickly by using an online near infrared instrument, but also the tobacco of the same grade is less discrete, better consistency and more representative after grading, and it is applicable to tobacco samples of different years, which is a convenient, rapid and promising grading method that can provide a reference for tobacco quality grading. Furthermore, the formulation design by utilizing different grades after grading can greatly improve the utilization rate of tobacco leaves. Notably, the method can also be applied to other fields, such as quality control of agricultural products, monitoring of food processing processes, etc.

Wafer Edge Metrology and Inspection Technique Using Curved-Edge Diffractive Fringe Pattern Analysis

Abstract This paper introduces a novel wafer-edge quality inspection method based on analysis of curved-edge diffractive fringe patterns, which occur when light is incident and diffracts around the wafer edge. The proposed method aims to identify various defect modes at the wafer edges, including particles, chipping, scratches, thin-film deposition, and hybrid defect cases. The diffraction patterns formed behind the wafer edge are influenced by various factors, including the edge geometry, topography, and the presence of defects. In this study, edge diffractive fringe patterns were obtained from two approaches: (1) a single photodiode collected curved-edge interferometric fringe patterns by scanning the wafer edge and (2) an imaging device coupled with an objective lens captured the fringe image. The first approach allowed the wafer apex characterization, while the second approach enabled simultaneous localization and characterization of wafer quality along two bevels and apex directions. The collected fringe patterns were analyzed by both statistical feature extraction and wavelet transform; corresponding features were also evaluated through logarithm approximation. In sum, both proposed wafer-edge inspection methods can effectively characterize various wafer-edge defect modes. Their potential lies in their applicability to online wafer metrology and inspection applications, thereby contributing to the advancement of wafer manufacturing processes.

A virtual measurement method for evaluating the gear radial composite deviation based on point cloud data

Radial composite deviation is an important index to evaluate the precision of gear transmission, and its detection method is also an important trend of gear detection in industrial development. The ability to predict radial composite deviation while obtaining geometric deviation of gears is a key challenge in advancing intelligent online inspection within gear manufacturing workshops. This study presents a virtual measurement method for determining gear radial composite deviation using point cloud data. The method involves using an optical three-dimensional scanner to capture tooth surface cloud data, determining tooth profile deviation, and applying reverse engineering technology to reconstruct the gear model, the variation curve of gear center distance is determined by simulating, allowing for the calculation of the total radial composite deviation and tooth-to-tooth radial composite deviation. In comparison to actual measurement, the virtual measurement method is generally reliable. It can effectively illustrate the overall trend of center distance variation and has good interpretability for predicting radial composite deviation results. This method reduces the dependence on the specialized gear measuring instrument and addresses the critical issue of simultaneously obtaining tooth profile deviation and radial composite deviation.

A multimodal fusion method for soldering quality online inspection

A multimodal fusion method for soldering quality online inspection

Research on a Corrosion Detection Method for Oil Tank Bottoms Based on Acoustic Emission Technology.

This paper presents an acoustic emission (AE) detection method for refined oil storage tanks which is aimed towards specialized places such as oil storage tanks with high explosion-proof requirements, such as cave oil tanks and buried oil tanks. The method utilizes an explosion-proof acoustic emission instrument to detect the floor of a refined oil storage tank. By calculating the time difference between the defective acoustic signal and the speed of acoustic wave transmission, a mathematical model is constructed to analyze the detected signals. An independent channel AE detection system is designed, which can store the collected data in a piece of independent explosion-proof equipment, and can analyze and process the data in a safe area after the detection, solving the problems of a short signal acquisition distance and the weak safety protection applied to traditional AE instruments. A location analysis of the AE sources is conducted on the bottom plate of the tank, evaluating its corrosion condition accurately. The consistency between the evaluation and subsequent open-tank tests confirms that using AE technology effectively captures corrosion signals from oil storage tanks' bottoms. The feasibility of carrying out online inspection under the condition of oil storage in vertical steel oil tanks was verified through a comparison with open inspections, which provided a guide for determining the inspection target and opening order of large-scale oil tanks.