Process Monitoring Methods Research Articles

Non-invasive in-situ monitoring of deep etching processes using terahertz metasurfaces

This study presents an in-situ and non-invasive process control and monitoring (PCM) method for deep silicon etching, leveraging terahertz metasurfaces. The technique addresses the challenges for monitoring deep and high aspect ratio etching processes, which are prevalent in semiconductor microfabrication. By incorporating metasurfaces with identical geometric shapes and sizes as crucial components of targeted devices, the method enables accurate monitoring of the etching depth in the process. Continuous shifts of terahertz reflection spectra provide information on etching depth, while abrupt change in the curves highlights the etching endpoint, preventing over-etching. For the commonly used comb-finger structure, numerical simulations demonstrate a strong linear relationship between etching depth and terahertz resonant wavelengths (nonlinearity < 1%) and an abrupt resonant frequency change (> 0.6 THz) at the endpoint. Experimental validations confirm the accuracy of the PCM method, with an etching depth estimation error below 2 µm. This approach enhances the precision of PCM in microfabrication, offering the potential for widespread applications in the production of micromechanical sensors, actuators, and other microelectronic devices.

Nonlinear Dynamic Process Monitoring Based on Discriminative Denoising Autoencoder and Canonical Variate Analysis

Modern industrial processes are characterized by increasing complexity, often exhibiting pronounced dynamic behaviors and significant nonlinearity. Addressing these dynamic and nonlinear characteristics is essential for effective process monitoring. However, many existing methods for process monitoring and fault diagnosis are insufficient in handling these challenges. In this article, we present a novel process monitoring approach, CVA-DisDAE, which integrates an improved Denoising Autoencoder (DAE) with Canonical Variate Analysis (CVA) to address the challenges posed by dynamic behaviors and nonlinear relationships in industrial processes. First, CVA is employed to reduce data dimensionality and minimize information redundancy by maximizing correlations between past and future observations, thereby effectively capturing process dynamics. Following this, we introduce a discriminative DAE model (DisDAE) designed to serve as a semi-supervised denoising autoencoder for precise feature extraction. This is achieved by incorporating both between-class separability and within-class variability into the traditional DAE framework. The key distinction between the proposed DisDAE and the conventional DAE lies in the integration of a linear discriminant analysis (LDA) penalty into the DAE’s loss function, resulting in extracted features that are more conducive to fault classification. Finally, we validate the effectiveness of the proposed semi-supervised dynamic process monitoring approach through its application to the Tennessee Eastman benchmark process, demonstrating its superior performance.

Novel approach for in-line process monitoring during ultrasonic metal welding of dissimilar wire/terminal joints based on the thermoelectric effect

AbstractUltrasonic metal welding (USMW) is a manufacturing technique widely employed in the automotive and aerospace industries due to its efficiency in joining similar as well as dissimilar metals. Despite its prevalence, the lack of effective in-line process monitoring methods has resulted in high scrap rates, product recalls due to unrecognized scrap or financial losses due to pseudo-scrap, limiting its application in more sensitive industries. This paper presents a novel thermoelectric effect-based method for in-line process monitoring of USMW processes. This approach utilizes the thermoelectric properties, that manifest at the junctions of dissimilar metals during welding to accurately measure the temperature of the weld zone without the need of additional thermocouples, pyrometers or infrared cameras. An experimental setup was developed to validate the thermoelectric-based temperature measurement methodology. Key to this approach is the detection of thermoelectric voltage developed due to thermo diffusion when dissimilar materials are joined. The experiments showed a strong correlation between the thermoelectric voltage and the mechanical strength of the welds, suggesting that this parameter can effectively predict the quality of the weld. In the trials, a series of welded samples was created under controlled conditions to measure the generated thermoelectric voltage and correlate it with ultimate tensile strength tests. The data were analyzed using Spearman’s correlation coefficients to determine the correlation of the thermoelectric signals and joint strength. Results indicate that the thermoelectric voltage measurements correlate highly with the joint strength, with a Spearman’s correlation coefficient of over 0.94, thereby providing a promising predictive metric for assessing weld quality.

TEMPERATURE MONITORING FOR LASER METAL DEPOSITION USING NEAR-INFRARED SPECTROSCOPY

Temperature monitoring during laser metal deposition (LMD) aids process control to ensure high-quality production. However, the fluctuating emissivity of the melt pool limits the accuracy of the existing non-contact temperature measuring devices. Thus, this work explores a new temperature monitoring technique, where the thermal radiation emitted by the processing zone was collected with a near-infrared (NIR) spectrometer and fitted to Planck's law to determine the temperature. This work has established the optimised angle and distance of the fiber probe from the LMD processing area to maximise the spectral signal acquisition. The temperature determined from the technique was cross-validated with a thermocouple, resulted in a small deviation of 2.39%. The applicability of the spectroscopic method for continuous temperature monitoring of the LMD process has been demonstrated. The optimised placement for the fiber probe end was determined at the 45˚ angle relative to the surface of the substrate and positioned 5 cm away from the molten pool. The temperature during the LMD process decreased gradually then stabilised after approximately 17 mm track length, resembling those reports in prior studies. Our findings support the practicality of the proposed temperature monitoring approach in LMD.

KPI-oriented process monitoring based on causal-weighted partial least squares

With the increasing demand for product quality, Key performance indicator (KPI)-oriented process monitoring plays an important role in modern industrial. Partial least squares (PLS)-based methods are widely adopted for KPI-oriented process monitoring, in which the process variable space is decomposed into a principal component subspace that has a strong correlation with KPIs and a residual subspace unrelated to KPIs, and the two spaces are monitored separately. However, if a KPI-unrelated fault occurs in a variable that has no causal relation with any KPI, but has a spurious correlation with some KPIs because of the existence of confounders, PLS based methods may mis-regard the KPI-unrelated fault as a KPI-related fault. This occurs because as a correlation analysis-based method, PLS cannot discriminate whether a variable is a real cause of KPIs or has a spurious correlation with KPIs. Motivated by this, this paper proposes two causal-weighted PLS methods for KPI-oriented process monitoring by combining the LiNGAM-based causal discovery with PLS, which first calculate causal weights based on the modified LiNGAM algorithm and bootstrap strategy, and then employ the causal weights to reweight the weight vectors of PLS to enhance the influence of causal variables in the KPI-related principal subspace and reduce the influence of spurious correlations. Case studies based on a simulated dataset, Tennessee Eastman Process dataset and field data from finishing rolling mill process show that the proposed methods can significantly reduce the false alarm rate for KPI-unrelated faults (i.e., the probability that a KPI-unrelated fault sample is mis-regarded as a KPI-related fault sample) caused by spurious correlation without significantly compromising the fault detection rate of KPI-related faults.

Gradient Scale Monitoring for Federated Learning Systems

As the computational and communicational capabilities of edge and IoT devices grow, so do the opportunities for novel Machine Learning solutions. This leads to an increase in popularity of Federated Learning (FL), especially in cross-device settings. However, while there is a multitude of ongoing research works analyzing various aspects of the FL process, most of them do not focus on issues of operationalization and monitoring. For instance, there is a noticeable lack of research in the topic of effective problem diagnosis in FL systems. This work begins with a case study, in which we have intended to compare the performance of four selected approaches to the topology of FL systems. For this purpose, we have constructed and executed simulations of their training process in a controlled environment. We have analyzed the obtained results and encountered concerning periodic drops in the accuracy for some of the scenarios. We have performed a successful reexamination of the experiments, which led us to diagnose the problem as caused by exploding gradients. In view of those findings, we have formulated a potential new method for the continuous monitoring of the FL training process. The method would hinge on regular local computation of a handpicked metric - the gradient scale coefficient (GSC). We then extend our prior research to include a preliminary analysis of the effectiveness of GSC and average gradients per layer as potentially suitable for FL diagnostics metrics. In order to perform a more thorough examination of their usefulness in different FL scenarios, we simulate the occurrence of the exploding gradient problem, vanishing gradient problem and stable gradient serving as a baseline. We then evaluate the resulting visualizations based on their clarity and computational requirements. We introduce a gradient monitoring suite for the FL training process based on our results.

Comparative evaluation of VAE-based monitoring statistics for real-time anomaly detection in AIS data

ABSTRACT Real-time tracking and anomaly detection in vessel navigation using AIS (Automatic Identification System) data is a critical issue in maritime logistics. However, the substantial volume, inherent noise, and irregular temporal intervals of AIS data pose challenges in applying traditional Statistical Process Monitoring (SPM) methods. To overcome these challenges, recent research has proposed the use of deep probabilistic latent variable models, specifically variational autoencoders (VAE). However, the monitoring statistics based on VAE employed in previous studies may vary. In this study, we conduct a comprehensive comparison and evaluation of various monitoring statistics based on VAE within the context of statistical process monitoring. Furthermore, we propose a new real-time monitoring method by integrating VAE-based monitoring statistics with the CUSUM chart for monitoring AIS data. By utilizing both simulated and real-world AIS data, our proposed method demonstrates superior detection performance and robustness compared to traditional methods.

An unsupervised embedding method based on streaming videos for process monitoring in repetitive production systems

Advanced production systems, such as multi-step assembly processes, predominantly comprise of repetitive operations. The repetitive manual or human–robot integrated production operations call for new real-time process management technologies such as the increasing use of sensors and the development of system intelligence. Conventional process monitoring and management methods, which are often labor-intensive, fall short in providing immediate and actionable insights. To tackle this limitation, we develop an unsupervised embedding method to automatically delineate the process into different stages and predict real-time progress information. We propose a Contrastive Variational Autoencoder as a feature extractor to adeptly embed repetitive processes into a Gaussian Mixture Model. Based on the extracted features, we propose an adaptive change-point detection and an Iterative Dynamic Time Wrapping algorithm to identify and segment multiple standardized process stages automatically. Theoretically, we establish the asymptotic optimality of the detected change-points associated with the given precision of image and feature extractors, ensuring the high-quality process stage separation and labeling. The proposed method autonomously extracts essential features encapsulating progress information from a limited set of unlabelled process videos. Through four diverse case studies including production of an actual aircraft spoiler, our method exhibits very promising performance. Specifically, it achieves an average of 98.14% accuracy in predicting production progress and 0.9202 area under the curve in predicting progress deviation across three distinct production environments. The proposed process monitoring method in repetitive production systems has the potential to significantly improve productivity, promote standardization of repetitive operations, and predict production deviations.

Spatio-temporal feature extraction network based multi-performance indicators synergetic monitoring method for complex industrial processes

In the context of smart manufacturing, modern industrial processes are becoming increasingly complex in terms of process flows, product varieties, and performance indicators (PIs). Performance-driven process monitoring attracts extensive attentions in recent years. However, most methods require spatio-temporal correspondence between process variables and PIs, and rarely consider the correlation among various PIs. In this paper, a spatio-temporal feature extraction network-based multi-performance indicators synergetic monitoring framework is presented. Firstly, considering the missing data in PI measurements, a weighted sum of tensor nuclear norm (WSTNN) based batch-process data completion approach is developed, which can adeptly handle local missing and incomplete data issues and establish the spatio-temporal correspondence for subsequent modeling. Secondly, for a specific PI, a new canonical variate analysis embedded spatio-temporal convolutional network (CVA-STCN) is designed to extract the PI-related features with spatio-temporal dependency. Thirdly, considering the dynamic interaction of multiple PIs, a third-order feature tensor is established to perform the future fusion, and the correlations among various PI-related features are explored via tensor decomposition. Finally, a hierarchical multi-performance indicators synergetic monitoring model is developed over several subspaces. The proposed method is verified on Tennessee Eastman process and an actual hot strip mill process. Overall, the monitoring performance of the proposed method outperforms the traditional ones, in terms of higher fault detection rates and lower false alarm rates. Moreover, the information provided by the multi-subspace synergetic monitoring charts can offer valuable guidance to field engineers.

Measurements of Spontaneous and External Stimuli Molecular Release Processes from a Single Optically Trapped Poly(lactic-co-glycolic) Acid Microparticle and a Liposome Containing Gold Nanospheres.

We investigated the single particle kinetics of the molecular release processes from two types of microcapsules used as drug delivery systems (DDS): biodegradable poly(lactic-co-glycolic) acid (PLGA) and a light-triggered-degradable liposome encapsulating gold nanospheres (liposome-GNP). To optimize the design of DDS capsules, it is highly desirable to develop a method for real-time monitoring of the release process. Using a combination of optical tweezers and confocal fluorescence microspectroscopy we successfully analyzed a single optically trapped PLGA particle and liposome-GNPs in solution. From temporal decay profiles of the fluorescence intensity, we determined the time constant τ of the release processes. We demonstrated that the release rate of spontaneously degradable microcapsules (PLGA) decreased with increasing size, while conversely, the release rate of external stimuli-degradable microcapsules (liposome-GNPs) increased in proportion to their size. This result is explained by the differences in the disruption mechanisms of the capsules, with PLGA undergoing hydrolysis and the GNPs in the liposome-GNP undergoing a photoacoustic effect under nanosecond pulsed laser irradiation. The present approach offers a way forward to an alternative microanalysis system for single drug delivery nanocarriers.

Implementation of an intelligent process monitoring system for screw presses using the CRISP-DM standard

AbstractIncreasing the service life and process reliability of systems plays an important role in terms of sustainable and economical production. Especially in the field of energy-intensive bulk forming, low scrap rates and long tool lifetimes are business critical. This article describes a modular method for AI-supported process monitoring during hot forming within a screw press. With this method, the following deviations can be detected in an integrated process: the height of the semi-finished product, the positions of the die and the position of the semi-finished product. The method was developed using the CRISP-DM standard. A modular sensor concept was developed that can be used for different screw presses and dies. Subsequently a hot forming-optimized test plan was developed to examine individual and overlapping process deviations. By applying various methods of artificial intelligence, a method for process-integrated detection of process deviations was developed. The results of the investigation show the potential of the developed method and offer starting points for the investigation of further process parameters.

Ultrasonic mode conversion for in-line foam structure measurement in highly aerated batters using machine learning

An ultrasonic-based method was developed to enable in-line measurements of foam structure parameters for highly aerated batters by mode conversion. Biscuit batters were foamed to different degrees (density: 364–922 g/L) by varying the mixing head speed and pressure. Density and foam structure changes were detected by efficient offline analytics (nref measurement = 96). Ultrasonic signal data were recorded using two ultrasonic sensors attached to an industry-standard tube. Mode conversion effects in the ultrasonic signals were obtained to predict the rheological parameters of the batters. The frequency range in which surface waves are expected was particularly suitable for detecting rheological changes in highly aerated batters. An ultrasonic-based, online-capable method for process monitoring was implemented and established regarding feature selection in combination with machine learning and 5-fold cross-validation. The developed ultrasonic sensor system shows high accuracy for online density measurement (R2 = 0.98) and offers decent accuracy for measurements of foam structure parameters (Bubble count: R2 = 0.95, Relative span: R2 = 0.93, Sauter diameter: R2 = 0.83). The main benefit of this novel technique is that integrating ultrasonic signal features based on mode conversion leads to a robust foam structure analysis, which has the advantage of being retrofitable into existing processes.

The Optical Spectra of Hydrogen Plasma Smelting Reduction of Iron Ore: Application and Requirements

Due to the ever‐increasing demand for high‐quality steel and the need to reduce CO2 emissions, research and development of sustainable steelmaking processes have gained a lot of interest in the past decade. One of these processes is the hydrogen plasma smelting reduction (HPSR), which has proven to be a promising solution for iron ore reduction where water vapor is formed instead of CO2. However, due to the highly dynamic and sometimes unpredictable behavior of plasmas and their nonlinear interaction with the liquid oxides, the monitoring and control of the underlying processes must be improved. This article explores the usage of optical emission spectroscopy (OES) and image analysis for HPSR process monitoring at laboratory and pilot scale. The results cover the time evolution of the OES and camera data with the focus on the most interesting radiating species, such as atomic hydrogen, iron, and oxygen together with the FeO molecule. In addition, the advantages, disadvantages, and requirements of these methods for HPSR process monitoring are discussed.

Evaluation of hydration behavior of cement-stabilized macadam via electrochemical impedance spectroscopy

Electrochemical Impedance Spectroscopy (EIS) was an important method for continuous and non-destructive monitoring of cement hydration process. The EIS, porosity and compressive strength of cement-stabilized macadam with different structures and cement content at different hydration ages were tested. By fitting the quasi-Randles equivalent circuit model, the electrochemical parameters were obtained to analyze the hydration degree and microstructure changes of materials during the hydration process. The mathematical models of electrochemical parameters and hydration parameters in skeleton dense structure, suspension dense structure and skeleton pore structure were established. The results show that the electrochemical parameters of the quist-Randles equivalent circuits model had a fine correlation with the hydration parameters of cement-stabilized macadam materials. The impedance curves of different cement-stabilized macadam structures had the same variation laws under different cement contents. As the hydration age increased, more hydration products formed and filled the internal pores of the structure, leading to the gradual formation of a dense structure in the material, further reducing the porosity and increasing the pore solution resistance RS. Meanwhile, the increase in cement content also led to a gradual increase in the compressive strength of the sample. The research results show that Rs was inversely proportional to porosity and had a good logarithmic relationship with compressive strength.

Stain-Free Approach to Determine and Monitor Cell Heath Using Supervised and Unsupervised Image-Based Deep Learning

Cell-based medicinal products (CBMPs) are a growing class of therapeutics that promise new treatments for complex and rare diseases. Given the inherent complexity of the whole human cells comprising CBMPs, there is a need for robust and fast analytical methods for characterization, process monitoring, and quality control (QC) testing during their manufacture. Existing techniques to evaluate and monitor cell quality typically constitute labor-intensive, expensive, and highly specific staining assays. In this work, we combine image-based deep learning with flow imaging microscopy (FIM) to predict cell health metrics using cellular morphology “fingerprints” extracted from images of unstained Jurkat cells (immortalized human T-lymphocyte cells). A supervised (i.e., algorithm trained with human-generated labels for images) fingerprinting algorithm, trained on images of unstained healthy and dead cells, provides a robust stain-free, non-invasive, and non-destructive method for determining cell viability. Results from the stain-free method are in good agreement with traditional stain-based cytometric viability measurements. Additionally, when trained with images of healthy cells, dead cells and cells undergoing chemically induced apoptosis, the supervised fingerprinting algorithm is able to distinguish between the three cell states, and the results are independent of specific treatments or signaling pathways. We then show that an unsupervised variational autoencoder (VAE) algorithm trained on the same images, but without human-generated labels, is able to distinguish between samples of healthy, dead and apoptotic cells along with cellular debris based on learned morphological features and without human input. With this, we demonstrate that VAEs are a powerful exploratory technique that can be used as a process monitoring analytical tool.

Monitoring dynamic process with orthonormal subspace analysis

AbstractTraditional multivariate statistics‐based process monitoring (MSPM) methods are static algorithms, and the “time lag shift” method (TLSM) is the most commonly used approach to handle the dynamic issue. This paper proves in theory that two drawbacks exist in TLSM‐based dynamic approaches: information unrelated to the real‐time data is also analyzed, and information that can be predicted by historical data is counted repeatedly in both real‐time and historical data. This paper adopts orthonormal subspace analysis (OSA) to handle these issues. OSA can successfully separate real‐time data into information that can be predicted by historical data (the dynamic component) and cannot be predicted for process monitoring (the static component), so the detection result is not influenced by redundant information and is more sensitive to process faults than TLSM‐based dynamic methods.

In Situ Stereo Digital Image Correlation with Thermal Imaging as a Process Monitoring Method in Vacuum-Assisted Thermoforming

This experimental study probes the dynamic behaviour of a 3 mm ABS sheet during positive mould vacuum-assisted thermoforming. In this process, the sheet undergoes large and fast deformations caused by the applied vacuum and mechanical stretching by the mould. The objective is to elucidate the complexities of these large, rapid, and non-isothermal deformations. The non-isothermal conditions are caused by the radiative heating of the sheet, convective heat loss to the surrounding air, and conductive heat transfer from the sheet to the mould. By utilizing stereo digital image correlation (DIC) in tandem with thermal imaging, the present study accurately maps the occurring displacement, strain, and strain rate field in relation to real-time temperature variation in the material. The study progresses to observe the ABS material from the moment it contacts the mould until it conforms to a positive 250 mm diameter semi-sphere cast aluminum mould. The DIC methods are validated by comparing thickness values derived from DIC’s principal strain directions to ultrasonic thickness gauge readings. This knowledge not only broadens the understanding of the thermo-mechanical behaviour of the material but also aids in optimizing process parameters for improved thickness uniformity in thermoformed products.

Process monitoring and fault diagnosis method combining bagging DPCA‐ICA with moving window Kolmogorov–Smirnov test

AbstractProcess monitoring and fault diagnosis systems are essential for ensuring the quality and safety of industrial processes. Real industrial process data always has dynamic features, autocorrelation features, and non‐Gaussian distributions, making many multivariate statistical process monitoring (MSPM) methods, such as principal component analysis (PCA), not work well. To solve the problem, a new process monitoring method (named DPCA‐ICA‐moving window Kolmogorov–Smirnov test [MWKS]) is proposed in the paper, which is a combination of DPCA and independent component analysis (ICA) with adaptive weight parameters and introduces a moving window technique and two‐sample K‐S test to obtain the K‐S distance of the commonly used Hotelling's and square predictive error as new monitoring parameters. Experimental results on the Tennessee Eastman chemical process demonstrate the effectiveness and accuracy of the proposed method, which shows it has better performance than existing commonly used methods. The average fault detection rate (FDR) is 93.131%, while the average false alarm rate (FAR) of the proposed method is only 0.1%.

Distributed Process Monitoring for Multiagent Systems Through Cognitive Learning

Multi-agent systems are usually large-scaled with a growing degree of intelligence and integration. Direct applications of traditional (centralized) methods will become incompetent for effective process monitoring of multi-agent systems. It necessitates the cognitive learning strategies that determine the effective interactions among subsystems or individuals. Therefore, in order to improve monitoring performance, this paper targets the development of a new distributed process monitoring method that has the cognitive learning ability by embedding an adaptive pickup rule. The proposed cognitive learning-based method can reduce the computation loads in both off-line and online phases because only necessary information exchange (or communication topology) is involved. Furthermore, the threshold used for system monitoring is obtained by developing a fast search algorithm based on statistical learning theory. Case studies on the wastewater treatment system, which can be regarded as a typical multi-agent system, demonstrate the superiority of the proposed distributed process-monitoring method.

Influence of full penetration welding on the multispectral signal of the process radiation during laser beam welding of austenitic stainless steel

A 100% visual inspection of the welds carried out is not always possible for economic reasons. Different methods of process monitoring are therefore used for non-destructive testing of process and weld seam quality. In the following, the recorded multispectral signals of the reflective process radiation are evaluated using descriptive statistics and characteristics of a full penetration weld during laser beam welding of austenitic stainless steel are worked out. The aim is to detect and localize unintentional full penetration welding at an early stage and thus avoid rejects. For this purpose, adapted sample geometries with varying material thicknesses are prepared for the test series. As a result, the need to vary the welding process parameters to achieve the different welding states (full and partial penetration weld) is avoided. By evaluating the measured values of the wavelength-selective signals, a significant influence on the signal intensity and the signal frequency could be confirmed.