Optimal Control Actions Research Articles

Pontryagin Neural Networks for the Class of Optimal Control Problems With Integral Quadratic Cost

This work introduces Pontryagin Neural Networks (PoNNs), a specialised subset of Physics-Informed Neural Networks (PINNs) that aim to learn optimal control actions for optimal control problems (OCPs) characterised by integral quadratic cost functions. PoNNs employ the Pontryagin Minimum Principle (PMP) to establish necessary conditions for optimality, resulting in a two-point boundary value problem (TPBVP) that involves both state and costate variables within a system of ordinary differential equations (ODEs). By modelling the unknown solutions of the TPBVP with neural networks, PoNNs effectively learn the optimal control strategies. We also derive upper bounds on the generalisation error of PoNNs in solving these OCPs, taking into account the selection and quantity of training points along with the training error. To validate our theoretical analysis, we perform numerical experiments on benchmark linear and nonlinear OCPs. The results indicate that PoNNs can successfully learn open-loop control actions for the considered class of OCPs, outperforming the commercial software GPOPS-II in terms of both accuracy and computational efficiency. The reduced computational time suggests that PoNNs hold promise for real-time applications.

CNN-LSTM-Based Nonlinear Model Predictive Controller for Temperature Trajectory Tracking in a Batch Reactor.

Batch reactors are type of chemical reactors, where the reactants are loaded to process for a defined batch time and the products are removed after the polymerization reaction completion. Specialty chemicals and food processing industries widely use BRs due to their versatility and suitability for handling small- to medium-scale production, complex reactions, and varying reaction conditions. This article employs a CNN-LSTM-based nonlinear model predictive controller (NMPC) to effectively track the temperature profile of a BR. This model offers significant advantages in NMPC by leveraging convolutional neural networks (CNNs) to capture spatial features and long short-term memory (LSTM) networks to manage temporal dependencies, thus enhancing prediction accuracy and control performance. The approach involves training the CNN-LSTM model using input and output data obtained from open-loop experimentation with the BR. This model evaluates the goal of optimizing the coolant flow rate while managing the heat generated by the exothermic reaction within the reactor. Additionally, a heuristic method incorporating a sigmoidal weighting functions are utilized to improve the computational efficiency of the model. The successful implementation of this CNN-LSTM-based NMPC model demonstrates its potential for large-scale usage in industrial applications. By providing accurate temperature predictions and optimizing control actions, this approach can enhance process efficiency, reduce energy consumption, and improve safety in various pharmaceutical industries.

A comparative analysis of intelligent energy modelling approaches for a grid-tied PVT system application

Renewable energy resources are a long-term answer to the current chronic energy dilemma. Solar photovoltaic (PV) systems are the most often used form of a practical solution for power supply and energy management in buildings. A PV-thermal (PVT) system is essential to lowering the demand for grid energy by capturing electrical and thermal energy from sunlight, optimising energy usage and encouraging sustainable energy practices. The most significant difficulty faced by developing countries, such as South Africa, is a lack of power supply to meet demand while limiting grid overload. An alternative source of energy is a critical component. This research compares two intelligent energy modelling methodologies: optimal control scheme-based open loop and model predictive control (MPC) in a PVT application. The primary goal of the modelling is to limit the electricity required from the grid while stressing the system’s energy efficiency and cost-effectiveness. The open loop approach uses mathematical optimisation techniques to establish the best control strategy for the PVT system. The ideal set-points for various system parameters are found by presenting the problem as an optimisation task to reduce grid power usage. The optimal control technique delivers a global optimisation solution based on the system dynamics and constraints. The MPC technique employs a predictive control technique of the PVT system to create optimal control actions in a receding horizon manner. The MPC algorithm anticipates future system behaviour and generates control actions that minimise grid power drawn by iteratively solving an optimisation problem at each time step. This robust online control scheme enables real-time modifications and responses to system disruptions that effectively and dynamically coordinate system behaviour.

Differential-algebraic equation-based model predictive control for trajectory tracking of spacecraft-mounted continuum manipulators

Spacecraft-mounted continuum manipulators (SMCMs) exhibit great potential for performing dexterous operations in unstructured environments due to their inherent compliance and dexterity. However, the dynamic model of the rigid-flexible coupling SMCM is highly nonlinear and typically formulated as a set of implicit differential-algebraic equations (DAEs), posing significant challenges for precise trajectory tracking control. This paper proposes a novel model predictive control (MPC) framework specifically designed for generic DAEs to achieve precise trajectory tracking of the SMCM under uncertain disturbances and input limitations. The DAE model of the SMCM is discretized into a set of nonlinear algebraic equations. By performing implicit differentiation of these equations with respect to the system state, the state transition matrix (STM) for the DAE model is derived. The optimal control action for the SMCM can be further determined based on the derived STM. Additionally, nonlinear complementary functions are introduced to address the issue of input limitations, allowing the problem of determining the optimal control sequence to be equivalently transformed into a set of nonlinear algebraic equations for solving. Numerical simulations demonstrate that the proposed approach can achieve precise trajectory tracking of the SMCM while strictly adhering to input limitations.

Load Frequency Optimal Active Disturbance Rejection Control of Hybrid Power System

The widespread adoption of the power grid has led to increased attention to load frequency control (LFC) in power systems. The LFC strategy of multi-source hybrid power systems, including hydroelectric generators, Wind Turbine Generators (WTGs), and Photovoltaic Generators (PVGs), with thermal generators is more challenging. Existing methods for LFC tasks pose challenges in achieving satisfactory outcomes in hybrid power systems. In this paper, a novel method for the multi-source hybrid power system LFC task by using an optimal active disturbance rejection control (ADRC) strategy is proposed, which is based on the combination of the improved linear quadratic regulator (LQR) and the ADRC controller. Firstly, an established model of a hybrid power system is presented, which incorporates multiple regions and multiple sources. Secondly, utilizing the state space representation, a novel control strategy is developed by integrating improved LQR and ARDC. Finally, a series of comparative simulation experiments has been conducted using the Simulink model. Compared with the LQR with ESO, the maximum relative error of the maximum peaks of frequency deviation and tie-line exchanged power of the hybrid power system is reduced by 96% and 83%, respectively, by using the proposed strategy. The experimental results demonstrate that the strategy proposed in this paper exhibits a substantial enhancement in control performance.

Evolutionary method of digital twin model for building physics mechanism

ABSTRACT Building digital twin (BDT) realizes real-time monitoring, prediction, and optimization control of the building system through the synchronization between the real world and the virtual platform, and improves the efficiency of building operation and maintenance. However, the changes in physical laws caused by the decline and aging of buildings make it difficult for the BDT models to depict the dynamic evolution mechanism of buildings and improve decision-making, which brings great challenges to the prediction and maintenance of buildings. To this end, an evolutionary method of digital twin model for building physics mechanisms is proposed, which integrates the internet of things (IoT), building information physical model (BIPM), and machine learning algorithms to build a self-evolution framework of digital twins for building physics mechanism, and supports high-precision modeling of BDTs. The advanced prediction (AP) model and autonomous decision-making (ADM) model for building physics mechanisms with self-evolution capability are designed, which support the autonomous correction of model parameters and the dynamic updating of the model driven by real-time data, and the ability to learn optimal control actions through interactive trial and error with the environment. The experimental results based on the settlement prediction and control case of underground engineering show that this evolutionary method can dynamically simulate the future evolution trend of the settlement law of underground engineering, and adjust the physical parameters to the ideal range to guide the evolution of underground engineering in a better direction. Compared with the existing methods, it has better prediction accuracy, interpretability, and generalization performance.

SCR + ASC systems control by backward induction with adaptive grid and different disturbance scenarios

The purpose of this study is to enhance control strategies for selective catalytic reduction (SCR) and ammonia slip catalyst (ASC) systems, aiming to effectively reduce NOx emissions from automotive engines during realistic driving cycles. Despite the effectiveness of these after-treatment systems (ATS), their dynamic and non-linear characteristics present significant challenges in achieving precise control. Therefore, this research proposes a hybrid approach that combines backward induction (BI) as the primary optimization technique with model predictive control (MPC) framework for real-time application. The article introduces a reduced-state control-oriented model of the SCR + ASC system, which is embedded into the BI algorithm to calculate optimal control actions within a finite horizon. Additionally, it is proposed an alternative approach for adapting the grid of model states within the BI algorithm, effectively reducing the computational cost. This adjustment enables the algorithm to operate in real-time with near-optimal results, as confirmed by experimental validation. Lastly, the study explores how different degrees of knowledge regarding system disturbances impact the strategy’s performance, examining three distinct scenarios: constant prediction horizon, probabilistic description, and full knowledge of the prediction horizon.

Model Predictive Control (MPC) of a Countercurrent Flow Plate Heat Exchanger in a Virtual Environment.

This research proposes advanced model-based control strategies for a countercurrent flow plate heat exchanger in a virtual environment. A virtual environment with visual and auditory effects is designed, which requires a mathematical model describing the real dynamics of the process; this allows parallel fluid movement in different directions with hot and cold temperatures at the outlet, incorporating control monitoring interfaces as communication links between the virtual heat exchanger and control applications. A multivariable and non-linear process like the plate and countercurrent flow heat exchanger requires analysis in the controller design; therefore, this work proposes and compares two control strategies to identify the best-performing one. The first controller is based on the inverse model of the plant, with linear algebra techniques and numerical methods; the second controller is a model predictive control (MPC), which presents optimal control actions that minimize the steady-state errors and aggressive variations in the actuators, respecting the temperature constraints and the operating limits, incorporating a predictive model of the plant. The controllers are tested for different setpoint changes and disturbances, determining that they are not overshot and that the MPC controller has the shortest settling time and lowest steady-state error.

Enhancing solar furnace thermal stress testing using an adaptive model and nonlinear predictive control

Solar furnaces rely on renewable energy for thermal tests using concentrated solar direct irradiance. To address the complex dynamics and constraints of these systems, a novel control approach is introduced based on an adaptive model strategy. The proposed adaptive model representation with an Adaptive Practical Nonlinear Predictive Controller (A-PNMPC) strategy is employed in the solar furnace SF60 at Plataforma Solar de Almería (Spain), demonstrating enhanced model prediction that culminates in a finer control performance. A Recursive Least Squares solution for model adaptation adjusts the model dynamics to the actual behavior of the solar furnace. Then, the predictive control strategy uses the updated model to calculate optimal control actions that lead the sample temperature to the desired reference. Simulation studies demonstrated the superior performance of the A-PNMPC in temperature control and prediction compared to the constant parameter model PNMPC strategy. The proposed A-PNMPC was compared to the conventional PNMPC in simulations, showing up to a 5% improvement in control performance and an 80% in large model predictions. Additionally, the A-PNMPC was tested on the actual system for various temperature profiles, achieving a mean error of less than 10 °C in sample temperature ramp tracking

Reinforcement learning for the optimization and online control of emulsion polymerization reactors: Particle morphology

Online control of emulsion polymerization reactors is challenging due to the complex nature of the polymerization process, which causes difficulties not only for online measurement, but also for real-time prediction and optimization of future trajectories. In this work, the use of reinforcement learning (RL) to overcome these challenges is explored for the case of online control of particle morphology. It is shown in silico that assuming knowledge of the system dynamics in the form of a mathematical model, reinforcement learning can be used to train a neural network that can select optimal control actions in real-time to target a desired final polymer product. Moreover, it is shown that a second neural network model can be utilized as a state estimator to allow for full online control of particle morphology, which cannot be directly measured online. This work highlights the potential of machine learning for optimization and control in challenging polymerization systems.

Hybrid control strategy for efficiency enhancement of a raft-type wave energy converter

To improve the wave energy capture performance of a raft-type wave energy converter, a novel hybrid control strategy consisting of a passive control strategy and an active control strategy is proposed. In the passive control strategy, a nonlinear stiffness mechanism is introduced to generate a nonlinear passive control force. The passive control strategy has the advantage of broadening the capture width, whereas the active control strategy, which uses nonlinear model predictive control, is mainly utilized to improve the capture efficiency. Considering the high dimensions of the dynamic model, a dimension reduction strategy is given. Moreover, a mechanism for selecting the initial value is proposed to avoid the local optimum. Then, a dimension reduction nonlinear model predictive control controller is proposed to obtain the optimal active control force. Finally, the wave energy capture performance is evaluated for both regular and irregular waves. Numerical simulation results show that the wave energy capture performance can be improved using the novel hybrid control strategy.

Dynamic-Linearization-Based Predictive Control of a Voltage-Source Inverter

In pursuit of accurate and fast trajectory tracking of power converters, an explicit model is commonly used in the finite control-set model predictive control (FCS-MPC) framework to predict precise behaviors of controlled variables. In reality, however, the model mismatch is inevitable, which causes the inherent challenges of parameter sensitivity and model uncertainties of the FCS-MPC method. This article proposes a dynamic-linearization-based predictive control architecture to circumvent such model dependence while keeping the attractive features of the conventional FCS-MPC method. By integrating the data-driven feature of the dynamic-linearization approach, the detailed model used in the FCS-MPC controller is replaced by a virtual equivalent data model, creating a data-driven predictive control architecture. The suggested method selects optimal control action solely based on the input-output data, exhibiting strong rejection against parameter variations whilst inheriting the distinctive property of the conventional FCS-MPC method. Finally, the proposed design is validated through comparative simulation and experimental results on a three-level neutral-point-clamped inverter.

Математическая модель процесса СВЧ и конвективной выпечки хлеба из сбивного теста

Whipped yeast-free bakery products require effective energy supply to dough in order to optimize energy consumption, baking time, and quality. This article introduces a verified mathematical model of microwave and convective baking for whipped bread based on heat and mass exchange equations.
 A full-scale experiment to verify the calculations involved dough samples with a humidity of 56 ± 1%. The samples underwent microwave and convective processing until the temperature in the crumb center reached 98 ± 1°C.
 The mathematical model was formalized as energy and mass conservation equations, which made it possible to consider baking as a non-stationary process of heat and mass transfer of moisture in an isotropic incompressible continuous medium in the diffusion approximation. The equation took into account the unstable phase transition boundary. The practical verification showed the mean error for microwave baking as 14.5% in temperature and 18.2% in moisture content. For convective baking, the results included 12.6% in temperature and 9.7% in moisture content. The mathematical model proved adequate to the real processes of heat and mass transfer. The error in calculating the temperature and moisture content fields was sufficient tooptimize the process.
 The physical and mathematical model of the baking process made it possible to evaluate the effect of technological variables on the temperature and moisture concentration fields in the dough samples. The mathematical model and the computational experiment can be used to identify static and dynamic characteristics of baking as an object of automatic control, i.e., to identify optimal control channels and actions, as well as to adjust the automatic control system to specific quality indicators.

Fuzzy-based collective pitch control for wind turbine via deep reinforcement learning

Wind turbines (WTs) have highly nonlinear and uncertain dynamics due to aerodynamic complexity, mechanical factors, and fluctuations in wind conditions. Turbulence and wind shear add complexity to modelling, especially in constant power region (region 3). Thus, an effective control design demands a deep understanding of the nonlinearities and uncertainties. This paper suggests a novel model-free reinforcement learning (RL) collective pitch angle controller to operate efficiently in region 3. The proposed controller stabilizes generator speed, maximizes power output, and minimizes fluctuations while accommodating system uncertainties, nonlinearity, and pitch limits. The disparity between WT dynamics due to wind speed perturbations and uncertainties is measured using a gap-metric criterion. The controller design adopts a deep deterministic policy gradient (DDPG) algorithm to train six agents in a medium-fidelity WT environment at different mean wind speeds to ensure the controller's robustness. Initially, imitation learning is used for efficient sample collection to fasten training convergence. Afterwards, the agent learns by interacting with the environment. After the training, the pitch control outputs from multi-trained agents are processed by a fuzzy system to have smooth transitions under different operating conditions. The resulting fuzzy DDPG (F-DDPG) controller is deployed to obtain the optimal pitch control action. The performance of the proposed F-DDPG controller is compared to the gain-scheduled PI (GSPI), Linear-Quadratic-Regulator (LQR), and single-DDPG-agent controllers. The controllers are simulated in high-fidelity onshore and offshore 5-MW WT environments using the OpenFAST/MATLAB simulation tools. The results reveal the superiority of the proposed controller in generalizing its optimal performance in different operating conditions.

An optimal solutions-guided deep reinforcement learning approach for online energy storage control

As renewable energy becomes more prevalent in the power grid, energy storage systems (ESSs) are playing an ever-increasingly crucial role in mitigating short-term supply–demand imbalances. However, the operation and control of ESS are not straightforward, given the ever-changing electricity prices in the market environment and the stochastic and intermittent nature of renewable energy generations, which respond to real-time load variations. In this paper, we propose a deep reinforcement learning (DRL) approach to address the electricity arbitrage problem associated with optimal ESS management. First, we analyze the structure of the optimal offline ESS control problem using the mixed-integer linear programming (MILP) formulation. This formulation identifies optimal control actions to absorb excess renewable energy and perform price arbitrage strategies. To tackle the uncertainties inherent in the prediction data, we then recast the online ESS control problem into a Markov Decision Process (MDP) framework and develop the DRL approach, which involves integrating the optimal offline control solution obtained from the training data into the training process and introducing noise to the state transitions. Unlike typical offline DRL training over a long time interval, we employ the Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) algorithms with smaller neural networks training over a short time interval. Numerical studies demonstrate the promising potential of the proposed DRL-enabled approach for achieving better online control performance than the model predictive control (MPC) method under different price errors. This highlights the sample efficiency and robustness of our DRL approaches in managing ESS for electricity arbitrage.

A System-View Optimal Additional Active Power Control of Wind Turbines for Grid Frequency Support

A System-View Optimal Additional Active Power Control of Wind Turbines for Grid Frequency Support

Bellman Neural Networks for the Class of Optimal Control Problems With Integral Quadratic Cost

This work introduces Bellman Neural Networks (BeNNs) and employs them to learn the optimal control actions for the class of optimal control problems (OCPs) with integral quadratic cost. BeNNs represent a particular family of Physics-Informed Neural Networks (PINNs) specifically designed and trained to tackle OCPs via applying the Bellman Principle of Optimality (BPO). The BPO provides necessary and sufficient optimality conditions, which result in a nonlinear partial differential equation known as the Hamilton-Jacobi-Bellman (HJB) equation. BeNNs learn the optimal control actions from the unknown solution of the arising HJB equation (i.e., the value function), where the unknown solution is modeled using a Neural Network. Additionally, the paper shows how to estimate the upper bounds on the generalization error of BeNNs while learning the solutions for the OCP class under consideration. The generalization error estimate is provided in terms of the choice and number of the training points as well as the training error. Numerical studies show that BeNNs can be successfully applied to learn the feedback control actions for the class of optimal control problems considered and, after the training is completed, deployed to control the system in a closed-loop fashion. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Impact Statement</i> —The proposed research improves our understanding of how to solve optimal control problems with closed-loop solutions and has potentially a countless number of applications in several different areas. The study is at the intersection between optimal control theory and artificial intelligence connected with mathematical tools for functional interpolation. This advances the ability to implement a higher level of autonomy in decision-making for practical applications with a beneficial impact on our society.

Experimental test performance for a comparative evaluation of a voltage source inverter: Dual voltage source inverter

Abstract This article proposes an adaptive Kernel-Hebbian least mean square (KHLMS) controller for a dual voltage source inverter (VSI). The recommended topology consists of a distributed energy resource (DER) supported VSI called main VSI (MVSI) and split capacitor supported VSI termed as auxiliary VSI (AVSI). Both the MVSI and AVSI are used to serve the shunt compensation when DER is not integrated with MVSI. The DER scenario is considered to suppress the active power flow shortage in the utility grid. Here, optimal active power flow control (OAPFC) is managed by MVSI and shunt compensation is achieved by AVSI during DER operated mode. Hence, a dual VSI based distribution static compensator (DSTATCOM) facilitates the configuration merits such as reduction in system downtime cost, filter rating switching stress etc. Supremacy of both the neural network (NN) based controller and topology is presented by comparing VSI (called AVSI) in the context of harmonic reduction in source side, voltage balancing, power factor (PF) enhancement, better voltage regulation and OAPFC. The experimental results are obtained through field programmable gate array (FPGA) based hardware units which exhibit radical improvement in the power quality (PQ) conferring as per the international standard grid code (IEEE-519-2017).

Methodology for regional industrial complex management: Architecture of an agent-based model

Industry is the backbone of the economy of developed countries and individual regions. To optimize management processes in such a complex and multi-level sector, specific economic-mathematical models and practical tools have to be developed. The paper discusses the optimal architecture of the regional industrial complex management model on a modern theoretical-methodological and instrumental (program) basis. The classical management theory, optimization theory and game theory constitute the methodology of this study. Among the research methods applied are agent-based and minimax approaches. We substantiate the use of agent-based modelling to simulate administering the regional industrial complex. The paper presents a three-tiered management architecture consisting of federal, regional and company level authorities (united by type of activity). For each level, control agents are identified and a set of indicators formed, which cover the structure of the phase vector, including its attributes, key parameters, control actions used, risks, a model of the parameters’ dynamics, and a model of the data possessed by the object. We build a hierarchical structure of administration and information relationships in the model and, based on the minimax approach, create an algorithm of agents’ efforts to select optimal control actions. The proposed architecture will allow forming a flexible toolkit for assessing industrial development scenarios and producing the best step-by-step management pattern of the regional industrial complex.

Semiactive Control of Nonlinear Parametric Vibration of Super‐Long Stay Cable in Cable‐Stayed Bridge Installed with Magnetorheological Fluid Damper

As the stay cables of cable‐stayed bridges become longer, parametric resonance with a large amplitude is more easily triggered, which becomes a vibration hazard of super‐long stay cables. An increasing number of practical applications of vibration mitigation on stay cables demonstrate that vibration control strategies can effectively facilitate hazard mitigation and improve cable‐stayed bridge reliability and service life. This study proposes a semiactive control approach to reduce the parametric vibration of super‐long stay cables in cable‐stayed bridges installed with magnetorheological fluid damper (MRFD). First, using the cable’s gravity sag curve equation, an equation governing the combined stay cable‐bridge deck‐damper control system was established to consider the effect of the chordwise force of cable gravity. Subsequently, a targeted LQR‐based optimal active control law is proposed to provide the target control force in the semiactive control. The parametric influences on the performance of the LQR‐based optimal active control were analysed to provide insight into the proposed control strategy. Since the semiactive control could achieve almost the same control efficacy of the targeted optimal active control, a semiactive control strategy employing MRFD is proposed to mitigate the parametric vibration of a super‐long stay cable. Based on the proposed semiactive control strategy, the system was attached with the MRFD of the longest cable, S36, in the designed prototype long cable‐stayed bridge. The efficacy of the established semiactive control system was also analysed. The analysis results confirm that the proposed semiactive control strategy and designed semiactive control system can perform similar to the LQR‐based optimal active control. The semiactive control system attached to the MRFD can mitigate the parametric vibration of super‐long stay cables in cable‐stayed bridge engineering practice.