Manufacturing Inspection Research Articles

Attention Pyramid Dilated Region-based Model for Metallurgical Defect Detection

Abstract Defect recognition is the key to realizing automatic visual inspection and quality control in intelligent manufacturing. Deep learning (DL) has been widely applied to locate damaged areas in images, characterize impurities of materials and further analyze the quality of products. However, automatic defect detection by DL in practical industrial applications is still a challenge due to the lack of sufficient datasets and appropriate recognition methods. Here, a novel Attention Pyramid Dilated Region-based Convolutional Neural Network (ADRCNN) is proposed to realize the multi-scale defect recognition and pixel-level instance segmentation of metallographic images. We also collected a dataset containing 900 images of commercial A356 aluminum alloy with different casting defects. The ADRCNN model is evaluated on our dataset with an average detection accuracy of 87.2% and 93.3% on validation and test sets respectively. To address the overfitting problem, a pretraining fine-tuning strategy is implemented by using the pretraining weights from large-scale datasets and temporarily freezing the weights of the backbone network during the training process. The experimental results show that the proposed model can achieve satisfactory defect segmentation in metallographic images, promoting the development of intelligent manufacturing in alloy industries.

PACS: Projection-driven with Adaptive CADs X-ray Scatter compensation for additive manufacturing inspection

Additive Manufacturing (AM) has revolutionised the production of custom-shaped samples through direct manufacturing from digital design models. As the internal structural integrity of these printed samples is of critical importance in diverse applications, X-ray radiography and X-ray Computer Tomography (X-CT) have emerged as widely used non-destructive imaging methods for quality control of AM samples. Unfortunately, beam hardening and X-ray scatter often degrade the quality of X-CT images, posing a significant challenge for X-ray based inspection. In addressing X-ray scatter, most of the methodologies assume fixed scanning geometries or stationary/known object characteristics, limiting their practicality in dynamic industrial scenarios where these may change over time. Simulation-based methods have been proposed that estimate and suppress scatter by accurately simulating the forward projection process. Yet, these methods assume the availability of X-CT reconstruction for simulation, thereby requiring a large number of projections (and hence scan time) for faithful X-CT reconstruction. In this work, we propose a simulation-based scatter compensation method (PACS) that eliminates the need for a prior X-CT scan. By employing few projections and nominal surface meshes of the scanned objects, the actual pose of the objects and their superficial deviation (e.g., due to printing) are estimated and used during X-ray simulations. Furthermore, as the pipeline is inherently coupled with a mesh projector, analysis of projective residuals facilitates the inspection for deformities or defects within the scanned object. To demonstrate the versatility of PACS for mitigating scatter, experiments across various inspection scenarios are conducted, and the outcomes are compared with those of a well-established scatter compensation technique. The results consistently show a higher Signal-to-Noise Ratio and Contrast-to-Noise Ratio of pore-defects, as well as lower residual errors in all examined cases.

High throughput observation of latent images on resist using laser-based photoemission electron microscopy

The rapid evolution of lithography technology necessitates faster pattern inspection methods. Here, we propose the use of laser-based photoemission electron microscopy (laser-PEEM) for high-throughput observation of latent images on an electron beam resist. We revealed that this technique can visualize latent images as chemical contrasts, and estimated the throughput millions of times higher than those of an atomic force microscope. Moreover, we estimated that throughput tens of thousands of times higher than a single-beam scanning electron microscope is achievable for post-developed resist patterns. This breakthrough highlights the potential of laser-PEEM to revolutionize a high-throughput lithographic pattern inspection in semiconductor manufacturing.

Q3D: a complete solution for quality control and inspection in additive manufacturing processes

PurposeThis paper aims to present an integrated system designed for quality control and inspection in additive manufacturing (AM) technologies.Design/methodology/approachThe study undertakes a comprehensive examination of the process in three distinct stages. First, the quality of the feedstock material is inspected during the preprocessing step. Subsequently, the main research topic of the study is directed toward the 3D printing process itself with real-time monitoring procedures using computer vision methods. Finally, an evaluation of the 3D printed parts is conducted, using measuring methods and mechanical experiments.FindingsThe main results of this technical paper are the development and presentation of an integrated solution for quality control and inspection in AM processes.Originality/valueThe proposed solution entails the development of a promising tool for the optimization of the quality in 3D prints based on machine learning algorithms.

Integration of eye-tracking and object detection in a deep learning system for quality inspection analysis

Abstract During quality inspection in manufacturing, the gaze of a worker provides pivotal information for identifying surface defects of a product. However, it is challenging to digitize the gaze information of workers in a dynamic environment where the positions and postures of the products and workers are not fixed. A robust, deep learning-based system, ISGOD (Integrated System with worker’s Gaze and Object Detection), is proposed, which analyzes data to determine which part of the object is observed by integrating object detection and eye-tracking information in dynamic environments. The ISGOD employs a six-dimensional pose estimation algorithm for object detection, considering the location, orientation, and rotation of the object. Eye-tracking data were obtained from Tobii Glasses, which enable real-time video transmission and eye-movement tracking. A latency reduction method is proposed to overcome the time delays between object detection and eye-tracking information. Three evaluation indices, namely, gaze score, accuracy score, and concentration index are suggested for comprehensive analysis. Two experiments were conducted: a robustness test to confirm the suitability for real-time object detection and eye-tracking, and a trend test to analyze the difference in gaze movement between experts and novices. In the future, the proposed method and system can transfer the expertise of experts to enhance defect detection efficiency significantly.

Deep Learning Based Security Schemes for IoT Applications: A Review

Due to its widespread perception as a crucial element of the Internet of the future, the Internet of Things (IoT) has garnered a lot of attention in recent years. The Internet of Things (IoT) is made up of billions of sentients, communicative "things" that expand the boundaries of the physical and virtual worlds. Every day, such widely used smart gadgets generate enormous amounts of data, creating an urgent need for rapid data analysis across a range of smart mobile devices. Thankfully, current developments in deep learning have made it possible for us to solve the issue tastefully. Deep models may be built to handle large amounts of sensor data and rapidly and effectively learn underlying properties for a variety of Internet of Things applications on smart devices. We review the research on applying deep learning to several Internet of Things applications in this post. Our goal is to provide insights into the many ways in which deep learning techniques may be used to support Internet of Things applications in four typical domains: smart industrial, smart home, smart healthcare, and smart transportation. One of the main goals is to seamlessly integrate deep learning and IoT, leading to a variety of novel ideas in IoT applications, including autonomous driving, manufacture inspection, intelligent control, indoor localization, health monitoring, disease analysis, and home robotics. We also go over a number of problems, difficulties, and potential avenues for future study that make use of deep learning (DL), which is turning out to be one of the most effective and appropriate methods for dealing with various IoT security concerns. The goal of recent research has been to enhance deep learning algorithms for better Internet of Things security. This study examines deep learning-based intrusion detection techniques, evaluates the effectiveness of several deep learning techniques, and determines the most effective approach for deploying intrusion detection in the Internet of Things. This study uses Deep Learning (DL) approaches to better expand intelligence and application skills by using the large quantity of data generated or acquired. The many IoT domains have drawn the attention of several academics, and both DL and IoT approaches have been explored. Because DL was designed to handle a variety of data in huge volumes and required processing in virtually real-time, it was indicated by several studies as a workable method for handling data generated by IoT.

Deep Learning for Automated Visual Inspection in Manufacturing and Maintenance: A Survey of Open- Access Papers

Quality assessment in industrial applications is often carried out through visual inspection, usually performed or supported by human domain experts. However, the manual visual inspection of processes and products is error-prone and expensive. It is therefore not surprising that the automation of visual inspection in manufacturing and maintenance is heavily researched and discussed. The use of artificial intelligence as an approach to visual inspection in industrial applications has been considered for decades. Recent successes, driven by advances in deep learning, present a possible paradigm shift and have the potential to facilitate automated visual inspection, even under complex environmental conditions. For this reason, we explore the question of to what extent deep learning is already being used in the field of automated visual inspection and which potential improvements to the state of the art could be realized utilizing concepts from academic research. By conducting an extensive review of the openly accessible literature, we provide an overview of proposed and in-use deep-learning models presented in recent years. Our survey consists of 196 open-access publications, of which 31.7% are manufacturing use cases and 68.3% are maintenance use cases. Furthermore, the survey also shows that the majority of the models currently in use are based on convolutional neural networks, the current de facto standard for image classification, object recognition, or object segmentation tasks. Nevertheless, we see the emergence of vision transformer models that seem to outperform convolutional neural networks but require more resources, which also opens up new research opportunities for the future. Another finding is that in 97% of the publications, the authors use supervised learning techniques to train their models. However, with the median dataset size consisting of 2500 samples, deep-learning models cannot be trained from scratch, so it would be beneficial to use other training paradigms, such as self-supervised learning. In addition, we identified a gap of approximately three years between approaches from deep-learning-based computer vision being published and their introduction in industrial visual inspection applications. Based on our findings, we additionally discuss potential future developments in the area of automated visual inspection.

Fault diagnosis in semiconductor manufacturing processes using a CNN-based generative adversarial network1

In the current industry, quality inspection in semiconductor manufacturing is of immense significance. Significant achievements have been made in fault diagnosis in fabricated semiconductor wafer manufacturing due to the development of machine learning. Since real-time intermediate signals are non-linear and time-varying, the signals undergo various distortions due to changes in equipment, material, and process. This leads to a drastic change in information in intermediate signals. This paper presents a fault diagnosis model for semiconductor manufacturing processes using a generative adversarial network (GAN). The study aims to address the challenges associated with efficient and accurate fault identification in these complex processes. Our approach involves the extraction of relevant components, development of a paired generator model, and implementation of a deep convolutional neural network. Experimental evaluations were conducted using a comprehensive dataset and compared against six existing models. The results demonstrate the superiority of our proposed model, showcasing higher accuracy, specificity, and sensitivity across various shift tasks. This research contributes to the field by introducing a novel approach for fault diagnosis, paving the way for improved process control and product quality in semiconductor manufacturing. Future work will focus on further optimizing the model and extending its applicability to other manufacturing domains.

Study Sheds Light on Optimization of Well-Diagnostics Investigation

_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 216484, “Optimization of Well-Diagnostics Investigation,” by A. Yugay, SPE, A. Zhukin, and S. Vorozhikhin, ADNOC, et al. The paper has not been peer reviewed. _ The complete paper defines the conditions for optimized well-diagnostics investigation, allowing preservation of resources and minimization of response time to well failure. It also describes a rigless solutions for common well-barrier failures and lessons learned from the operator’s field trials. This information is valuable for companies aiming to optimize their sustained annulus pressure (SAP) well-management process. The main objective of the paper is to identify scenarios in which the failed barrier element can be recognized from surface data alone. SAP Management Well-diagnostic activities can be divided into two classifications: surface activities and subsurface activities (well interventions). Common surface investigation activities include the following: - Bleedoff/buildup test - Sampling and analysis of the effluent - Wellhead seal test - Revision of well history - Leak-rate measurements - Echometer surveys - Pressure tests - Tracers Subsurface well diagnostics may include the following: - Pressure and temperature surveys - Corrosion logs - Leak-detection logs - Mechanical communication tests - Fiber-optic surveys - Various slickline tools - Installation of downhole gauges Root Cause Analysis (RCA) of Premature Well-Barrier Failures For wells with premature integrity failures (when the failure occurs within 10 years of the last rig entry), a new RCA process was implemented. Enhanced SAP investigation is conducted to identify exactly which well-barrier element failed. In the majority of cases, it is reverified during workover, after the completion is pulled to surface, and is available for visual and other types of inspection. The process includes third-party inspection and material testing, manufacture inspection, independent consultancy services, software simulations, and analytical brainstorming. On the basis of the confirmed direct causes, a multidisciplinary team conducts a full investigation of the failure, attempting to identify root causes. With the root causes identified, the team develops an action plan with fixed deadlines and specific actors. All findings and observations, along with the analysis and final outcome, are captured in the reports, presented to the management, and cascaded for implementation and improvement. The multidisciplinary team is staffed by representatives from drilling, production, and engineering departments. Thus, this group is no longer considered a “task force”; instead, a completely new dedicated business process has been implemented, created with a budget and resources allocated to move forward with continuous improvement.

Augmented Reality for Quality Inspection, Assembly and Remote Assistance in Manufacturing

In recent years, the manufacturing industry has witnessed the utilization of Augmented Reality (AR) technologies in their daily operations. This paper takes a deep dive into the applications of AR in the manufacturing industry, specifically focusing on quality inspection, assembly, and remote assistance. This research offers valuable insights into the technical architectures using custom and commercial software, as well as the Proof of Concept (PoC) applications for Hand-Held Device (HHD) and Head-Mounted Device (HMD). The custom application for an HHD combines the state-of-the-art You Only Look Once (YOLO) object detection algorithm to enable AR for quality inspection. We used commercial software from Pocket virtuality® to develop assembly and remote assistance applications for an HMD. An applied research approach was adopted, incorporating the collection of both quantitative and qualitative data to assess the various application scenarios. Among the key findings, it is evident that the functionality of the custom-developed app for quality inspection is contingent upon the operating system, software libraries available on the HHD, and the chosen version of the YOLO detection model. Furthermore, through quantitative and qualitative analysis, it becomes apparent that the software and hardware components of the HMD require significant improvements. These findings offer a high potential for improvement and emphasize the need for further research in the field. Moreover, this research emphasizes the fact that AR has the potential to make a huge impact on the manufacturing industry.

Experimental Validation of Reconstructed Microstructure via Deep Learning in Discontinuous Fiber Platelet Composite

Abstract A novel approach for microstructure reconstruction using artificial intelligence (MR-AI) was proposed to nondestructively measure the through-thickness average stochastic fiber orientation distribution (FOD) in a prepreg platelet molded composite (PPMC) plate. MR-AI approach uses thermal strain components on the surfaces of a PPMC plate as input to the deep learning model, which allows to predict a distribution of local through-thickness average fiber orientation state in the entire PPMC volume. The experimental setup with a heating stage and digital image correlation (DIC) was used to measure thermal strains on the surface of the PPMC plate. Optical microscopy was then used to measure FOD in the cross section of the PPMC plate. FOD measurements from optical microscopy imagery compared favorably with FOD prediction by MR-AI. The proposed methodology opens the opportunity for rapid, nondestructive inspection of manufacturing-induced FOD in molded composites.

PointSGRADE: Sparse learning with graph representation for anomaly detection by using unstructured 3D point cloud data

Surface anomaly detection by using 3D point cloud data has recently received significant attention. To completely measure the common free-form surfaces without loss of details, advanced 3D scanning technologies, such as 3D laser scanners, can be applied and will produce an unstructured point cloud. However, this irregular data structure poses challenges to anomaly detection, in that the existing methods based on regular data, e.g., 2D image, cannot be directly applied. This article proposes a sparse learning framework with a graph representation of the unstructured point cloud for anomaly detection (PointSGRADE). Specifically, the free-form surface is assumed to be smooth. Then, the associated point cloud can be represented as a graph. Subsequently, considering the sparse anomalies, we propose a sparse learning framework and formulate the anomaly detection problem as a penalized optimization problem, which is further solved by a computationally efficient majorization-minimization framework. Case studies demonstrate the accuracy and robustness of the proposed method. This article proposes a novel methodology for sparse anomaly detection on smooth free-form surfaces represented by unstructured point cloud, which is critical for quality inspection in manufacturing and other application areas.

Predictive modeling for online in-plane shape deviation inspection and compensation of additive manufacturing

PurposeLimited geometric accuracy is one of the major challenges that hinder the wider application of additive manufacturing (AM). This paper aims to predict in-plane shape deviation for online inspection and compensation to prevent error accumulation and improve shape fidelity in AM.Design/methodology/approachA sequence-to-sequence model with an attention mechanism (Seq2Seq+Attention) is proposed and implemented to predict subsequent layers or the occluded toolpath deviations after the multiresolution alignment. A shape compensation plan can be performed for the large deviation predicted.FindingsThe proposed Seq2Seq+Attention model is able to provide consistent prediction accuracy. The compensation plan proposed based on the predicted deviation can significantly improve the printing fidelity for those layers detected with large deviations.Practical implicationsBased on the experiments conducted on the knee joint samples, the proposed method outperforms the other three machine learning methods for both subsequent layer and occluded toolpath deviation prediction.Originality/valueThis work fills a research gap for predicting in-plane deviation not only for subsequent layers but also for occluded paths due to the missing scanning measurements. It is also combined with the multiresolution alignment and change point detection to determine the necessity of a compensation plan with updated G-code.

Scanning Magneto-Optic Kerr Effect Microscope for Inspection of MRAM Manufacturing

Scanning Magneto-Optic Kerr Effect Microscope for Inspection of MRAM Manufacturing

A novel image feature based self-supervised learning model for effective quality inspection in additive manufacturing

A novel image feature based self-supervised learning model for effective quality inspection in additive manufacturing

Improved YOLOv5l-based Detection of Surface Defects in Hot Rolled Steel Strips

To address the problems of complex background, different sizes and easy to miss and mis-detect in the detection of surface defects in hot-rolled strip, an improved YOLOv5l-based method for detecting surface defects in hot-rolled strip is proposed. Firstly, by adding the SimAM attention mechanism module to the aggregation network, the important information is focused with high weights to improve the recall rate of the original algorithm; secondly, by replacing all C3 modules in the YOLOv5l structure with C2F, a richer gradient of information flow is obtained to improve the accuracy rate of the original algorithm. The experimental results show that the average detection accuracy using the improved YOLOv5l improves by 5.3% and the accuracy rate by 8.3% compared to the original network, resulting in higher detection accuracy and lower error and miss detection rates, meeting the requirements of hot-rolled strip steel inspection in industrial manufacturing.

STUDI KINERJA HANDHELD PORTABLE 3D SCANNER DALAM PENGUKURAN DIAMETER DAN KEDALAMAN LUBANG PADA SPESIMEN LOGAM

A Handheld portable 3D Scanner is an optical-based tool that can be gripped by hand and moved from one place to another. This tool is used to scan part’s surface and produce a 3-dimensional representation in the form of a point cloud. The use of 3D Scanners has grown significantly in recent years, especially in applications in the metrology, industrial manufacturing inspection, and reverse engineering. Apart from the ease of use and flexibility of use, as an optical-based tool with triangulation principles, Handheld Portable 3D Scanners have limitations in scanning the geometric shape of a narrow hole or slit. The specifications published by the manufacturer do not fully describe tool’s performance in this regard. For this reason, an evaluation of the performance of the Handheld Portable 3D Scanner is carried out by determining the error of measuring holes on the workpiece with variations in depth and diameter. As a reference value, the results of the Coordinate Measuring Machine –CMM measurement are used. . The study result show that handheld 3D scanners generally have an error in measuring the diameter and measuring the depth of the hole, above the specification of 0.03 mm. In holes with a diameter of 1 mm, the handheld 3D scanner has a relationship between the error of measuring hole’s diameter and hole’s diameter with the regression equation: y=0.037x-0.54. En values categorized as "outliers" were found in holes with a diameter of 1 mm with a depth of 0.5, 1, 2, 4 mm and holes with a diameter of 2mm with a depth of 4 mm.

Unsupervised Image Anomaly Detection and Segmentation Based on Pretrained Feature Mapping

Image anomaly detection and segmentation are important for the development of automatic product quality inspection in intelligent manufacturing. Because the normal data can be collected easily and abnormal ones are rarely existent, unsupervised methods based on reconstruction and embedding have been mainly studied for anomaly detection. But the detection performance and computing time require to be further improved. This article proposes a novel framework, named as pretrained feature mapping (PFM), for unsupervised image anomaly detection and segmentation. The proposed PFM maps the image from a pretrained feature space to another one to detect the anomalies effectively. The bidirectional and multihierarchical bidirectional PFM are further proposed and studied for improving the performance. The proposed framework achieves the better results on well-known MVTec AD dataset compared with state-of-the-art methods, with the area under the receiver operating characteristic curve of 97.5% for anomaly detection and of 97.3% for anomaly segmentation over all 15 categories. The proposed framework is also superior in terms of the computing time. The extensive experiments on ablation studies are also conducted to show the effectiveness and efficiency of the proposed framework.

Artificial Intelligence-Based Smart Quality Inspection for Manufacturing.

In today's era, monitoring the health of the manufacturing environment has become essential in order to prevent unforeseen repairs, shutdowns, and to be able to detect defective products that could incur big losses. Data-driven techniques and advancements in sensor technology with Internet of the Things (IoT) have made real-time tracking of systems a reality. The health of a product can also be continuously assessed throughout the manufacturing lifecycle by using Quality Control (QC) measures. Quality inspection is one of the critical processes in which the product is evaluated and deemed acceptable or rejected. The visual inspection or final inspection process involves a human operator sensorily examining the product to ascertain its status. However, there are several factors that impact the visual inspection process resulting in an overall inspection accuracy of around 80% in the industry. With the goal of 100% inspection in advanced manufacturing systems, manual visual inspection is both time-consuming and costly. Computer Vision (CV) based algorithms have helped in automating parts of the visual inspection process, but there are still unaddressed challenges. This paper presents an Artificial Intelligence (AI) based approach to the visual inspection process by using Deep Learning (DL). The approach includes a custom Convolutional Neural Network (CNN) for inspection and a computer application that can be deployed on the shop floor to make the inspection process user-friendly. The inspection accuracy for the proposed model is 99.86% on image data of casting products.

An application of Generative Adversarial Networks to improve automatic inspection in automotive manufacturing

In manufacturing systems, the quality of inspection is a critical issue. This can be conducted by humans or by employing Computer Vision Systems (CV S), which are trained upon representative datasets of images to detect classes of defects that may occur. The construction of such datasets strongly limits the use of CV S methods, as the variety of defects has combinatorial nature. Alternatively, instead of recognizing defects, a system can be trained to detect non-defective standards, becoming appropriate for some application profiles. In automotive manufacturing, for example, parts are assembled within a reduced set of correct combinations, while the amount of possible incorrect assembling is enormous. This paper integrates a Generative Adversarial Network (GAN) within the CV S framework used by Renault/Brazil to improve the detection of defective production in its automotive assembly line. By sparing the construction of expensive defect image datasets, our solution has proved to be cost-effective and more efficient in comparison with the current CV S solution to detect defects, besides generalizing better to inspect different components without any modification in the method.