Adaptive ensemble reinforcement learning for industrial process control

Purpose: The purpose of this publication is to present the usage of Statistical Process Control (SPC) approach in Industry 4.0 conditions. Design/methodology/approach: Critical literature analysis. Analysis of international literature from main databases and polish literature and legal acts connecting with researched topic. Findings: The integration of Statistical Process Control (SPC) with Industry 4.0 signifies a transformative shift in quality management, elevating SPC from a conventional monitoring tool to a proactive force in contemporary manufacturing. Originally employed for ensuring consistent quality through process monitoring, SPC's role has been redefined in the Industry 4.0 era, utilizing data analytics, real-time monitoring, and connectivity to offer a comprehensive understanding of the manufacturing ecosystem. Enabled by the Internet of Things (IoT), SPC gains real-time insights into production processes, crucial for swift anomaly identification and issue resolution. Advanced analytics and artificial intelligence enhance SPC's predictive capabilities, enabling proactive measures to maintain product quality. This publication underscores the significance of SPC in Industry 4.0 conditions, emphasizing its roots in statistical principles, systematic process control, and the distinction between common and special cause variations. The integration of SPC with Industry 4.0 and Quality 4.0 leverages technologies for enhanced quality management, emphasizing real-time data monitoring and collaborative, data-driven approaches. While Table 2 outlines specific aspects of SPC integration, highlighting its versatility, Table 3 enumerates the advantages, emphasizing improved visibility and predictive quality management. However, challenges such as data security concerns and technology integration complexity, outlined in Table 4, necessitate strategic solutions. In conclusion, this integration represents a pivotal advancement, positioning SPC as an indispensable asset for organizations seeking quality excellence, operational efficiency, and resilience in the dynamic landscape of modern manufacturing. Originality/Value: Detailed analysis of all subjects related to the problems connected with the usage of Statistical process Control in Industry 4.0 conditions. Keywords: Industry 4.0; Quality 4.0, quality management; quality methods, SPC, Statistical Process Control. Category of the paper: literature review.

Reinforcement learning (RL) has attracted great interest from researchers in recent years. RL performs as human or better in many fields such as games and robot control. Although this technology is booming in computer science, it has not been practically applied in industrial process control. Up to now, proportional–integral–derivative (PID) control is still the most dominating and popular control method in industrial control. In this paper, we propose a combination of deep reinforcement learning (DRL) and PID control for better process control performance. The idea is generated by the following observations: for PID controller, its transient performance is not usually well enough to meet a strict requirement or in complex signal tracking tasks; For RL technology, a perfectly designed reward function is required for training. However, in practice, the reward function needs to be tested through trial and error, which will lead to a waste of computational power and time. By combining these two strategies, PID controller can help to improve the steady-state performance of RL control by its integral term, while the trained RL agent is able to improve the transient performance of PID controller. Several case studies with the water tank system are presented to demonstrate the effectiveness of the combined PID + RL control strategy.

Wireless communication technology has become another research focus in the field of industrial control. It, which is a revolutionary technology that can increase the scope of application of industrial measurement and control system, can be used not only as a supplement to the wired network, but also as a independent network. This article focuses on the wireless network communication protocol commonly used in modern industrial control, and highlights the wireless communication protocol ZigBee based on IEEE 802.15.4 and its characteristics. From the low-power, low-cost point of view the overall design of embedded wireless communication system which is based on ZigBee technology in industrial control are proposed, meanwhile it put forward the basic framework of wireless network nodes ZigBee, and the options of industrial site networking mode.

Computer technology, network technology, communication technology and control technology to construct the network structure based on the industrial control system, using the method of system engineering, analysis of industrial process control framework and networking logic hierarchical structure, the establishment of industrial control network structure of three layers, the accuracy of feature show things and main functions. Correctly build a system architecture and model that is easy to apply and has industrial control and Internet of things capabilities. The three layer Internet of things framework is of practical significance for the composition and analysis of the architecture of industrial control Internet of things. It provides the basis and reference for the structure and mode of the Internet of things of industrial control system.

<p>Increased globalisation and competition are drivers for process analytical technologies (PAT) that enable seamless process control, greater flexibility and cost efficiency in the process industries.This research aims to introduce an integrated process control approach, embedding novel sensors for monitoring in real time the critical control parameters of key processes in the minerals, ceramics, non-ferrous metals, and chemical process industries. The paper will discuss smart sensors, data fusion and process modelling and control in industrial applications with an emphasis on solutions enabling the real-time data analytics of sensor measurements that PAT demands.</p>

In light of the development of computing systems and machine learning techniques, the development of industrial control processes in production processes has become easier, more accurate, and more flexible. Machine learning techniques, after being integrated with industrial control processes, have become one of the most important tools that achieve sustainability in the field of industry. Thus, economic sustainability is achieved. Through it, production systems can be improved, costs reduced, energy consumption reduced, quality increased, and future malfunctions predicted. Thus, reducing the cost of repair and maintenance. The study aims to clarify the importance of machine learning techniques in industrial control processes, and that integrating machine learning techniques with industrial control techniques contributes to achieving sustainability in the field of industry. The study also aims to identify the obstacles and challenges facing the field of machine learning techniques in the industrial control process and how to solve them. Through a combination of description, analysis, comparison and simulation methodologies, the results indicated that 10% to 20% of the total cost was saved, 1% to 10% of the energy consumed was saved, and the response was improved by a rate ranging between 10% and 20%. The results also indicated to improve system flexibility using machine learning techniques, increase product quality, and reduce operation time. The use of machine learning techniques to improve the proposed model led to an improvement in reducing the cost by 10%, improving energy consumption by 1%, and improving the response by 1%.

Reinforcement Learning (RL) is a machine learning methodology that develops the capability to make sequential decisions in intricate issues using trial-and-error techniques. RL has become increasingly prevalent for decision-making and control tasks in diverse fields such as industrial processes, biochemical systems and energy management. This review paper presents a comprehensive examination of the development, models, algorithms and practical uses of RL, with a specific emphasis on its application in process control. The study examines the fundamental theories, methodology and applications of RL, classifying them into two categories: classical RL such as such as Markov decision processes (MDP) and deep RL viz., actor critic methods. RL is a topic of discussion in multiple process industries, such as industrial chemical process control, biochemical process control, energy systems, wastewater treatment and the oil and gas sector. Nevertheless, the paper also highlights challenges that hinder its larger acceptance, including the requirement for substantial computational resources, the complexity of simulating real-world settings and the challenge of guaranteeing the stability and resilience of RL algorithms in dynamic and unpredictable environments. RL has demonstrated significant promise, but more research is needed to fully integrate it into industrial and environmental systems in order to solve the current challenges. Abbreviations: AC: Actor critic; AI: Artificial intelligence; ANN: Artificial neural networks; A3C: Asynchronous advantage actor critic; CRL : Classical Reinforcement learning; CV : Controlled variable; DDPG : Deep deterministic policy gradient; DQN: Deep Q network; DRL: Deep reinforcement learning; DP: Dynamic programming; FOMDP: Fully observable Markov decision process; GRU: Gated recurrent unit; LQR: Linear quadratic regulator; LSTM: Long short-term memory; ML: Machine learning; MV : Manipulated variable; MC: Monte Carlo; MDP: Markov decision process; MPC: Model predictive controller; MIMO: Multi input multi output; PG: Policy gradient; PID: Proportional integral derivative; PPO: Proximal policy optimisation; RL: Reinforcement learning; PPO: Proximal policy optimisation; SAC: Soft actor critic; SISO: Single input single output; TD: Temporal difference; TRPO: Trust region policy optimisation; TD3: Twin delayed deep deterministic policy gradient.

In process control in the casting industry, the features of the product, such as diameter, thickness, density, are generally considered as quality characteristics and assignable causes affecting the process is tried to be determined by monitoring these quality characteristics in the quality control charts. However, instead of the features of the producing product as quality characteristics in the casting industry, the proportions of the elements that make up the product can also be accepted. Because the proportions of the elements that make up the product are desired to be within certain limits within the product and these generally vary. In addition, metal ratios, which can be selected as quality characteristics, can be monitored with quality control charts as in the properties of the product, but interpretation of out-of-control signals may not be sufficient. Therefore, in the solution of the problem, instead of quality control graphics, the process-oriented basis representations method in the literature can be used. As a result of the research in the literature, it has been determined that the process-oriented basis representations method has been used successfully in the modeling of geometric deviations in the manufacturing industry, but it is not applied in the process (chemistry, petro-chemistry, casting, etc.) industries, and in multivariate industrial production processes with interrelated quality characteristics. In this content, the aim of this study was is to show that metal alloy ratios can be used as quality characteristics and the process-oriented basis representations method can be applied in process control in the casting industry. The data used in the study were obtained from the production process of Brass Factory Directorate of Mechanical and Chemical Industry Company in Kirikkale province between 01 January 2015 and 31 March 2015. The module in the Minitab package program was used to create the control charts. At the end of the study, it has been determined that in the process control in the casting industry, the element ratios that make up the product produced as quality characteristics can be selected and positive results can be obtained by monitoring the quality characteristics selected in this way with the process-oriented basis representations method. It is evaluated that the results obtained in the study will contribute both to the domestic and foreign literature theoretically and to the quality control applications in terms of practicality in the casting industry.

Ontology for Enhanced Industrial Process Control

Industrial Process Coordinated and Controlled Based on Multi-Agent Technology

Non-destructive determination of β-carotene content in mango by near-infrared spectroscopy compared with colorimetric measurements

In the majority of domains such as public health, industrial safety, and weather forecasting, monitoring of the environment is essential. The legacy monitoring systems do not have real-time data capture and cannot provide structured, queryable data. Sensors, cloud storage, and semantic web technologies are all employed in IoT-based solutions to address these issues. A system for environmental monitoring based on IoT for capturing, processing, and semantically representing real-time temperature and humidity information is suggested in this research. Semantic technologies drive the transformation of IoT sensor data into an even more valuable and useful form to support smart applications in different domains. Raw sensor data is augmented with context-dependent meaning through ontologies and RDF-based representation, improving machine comprehension and interoperability. It improves data exchange, analysis, and reasoning and is particularly useful for smart cities, industrial control, healthcare, and environmental monitoring. In smart cities, semantic IoT data supports efficient resource management, in industrial automation, process control and predictive maintenance are maximized and Context-aware technologies can provide individualized patient monitoring in the medical field. Semantic web technologies simplify IoT information, making it more structured, accessible, and actionable, paving the way for complex, intelligent systems.

A practical Reinforcement Learning implementation approach for continuous process control

Industrial process control using model-based technologies is well established. These technologies are typically non-adaptive and so have limitations. Reinforcement Learning (RL) provides a model-free adaptive alternative. RL is a type of machine learning (ML) where models or data sets of the environment are not necessary before learning can start. It generates data, by exploring the environment and then learn the behavior from it. Though RL has been successfully applied for learning and playing various games such as Go, Chess, Atari; its application to continuous process control problems is not trivial. There is a need for online RL implementation to be safe, fast learning and explainable when applied to industrial control problems. Rather than adding to the extensive research on augmenting existing RL algorithms, the work focuses on developing a unique systematic method of formulating the RL problem incorporating domain-specific knowledge about process constraints and objectives, reducing dimensionality and modifying the exploration process, applicable to any model free RL algorithm supporting continuous states and actions, to enhance safety, speed and explainability of online RL implementation without requiring a simulation model. The approach is successfully implemented on two multivariable processes: a simulated distillation column and a temperature control lab setup using the Deep Deterministic Policy Gradient (DDPG) algorithm. The work demonstrates that the developed method is applicable to multivariable, noisy, non-linear processes with disturbances. It will further the potential of introducing the advances in Artificial Intelligence and ML algorithms for intelligent process control capable of enabling autonomous operation in the process industry.

PURPOSE: The work covers the development of intelligent sensors, as well as intelligent
\nmechanisms for the assembly and control of industrial processes using modern measurement
\ntechniques, process tomography, vision systems, motion and temperature sensors.