Learning Model Predictive Control Research Articles

Deep learning model predictive control of a high-density polyethylene reactor with a physics-guided sequence-to-sequence model with memory

In chemical process industry, the input-output data of processes display complex nonlinear dynamics and strong influences of unobserved hidden states. Their behavior must be modeled using nonlinear time series with an observer-predictor structure. To address this, a sequence-to-sequence model with a memory layer was proposed for a high-density polyethylene slurry reactor. The memory layer retains the chronological contribution of the observer and predictor and effectively captures the lengthy time response. A physics-guided approach was adopted to ensure the directional consistency between input and output variables in key control loops. In this way, a deep learning model can be obtained with historical data and there is no need for plant tests. The resulting model can be used in a nonlinear model predictive control system that not only quickly navigates different grade transitions but also provides steady-state control even though key internal state variables such as catalyst activity change.

Privacy-preserving federated machine learning modeling and predictive control of heterogeneous nonlinear systems

Privacy preservation in data-driven modeling of a heterogeneous nonlinear system with multiple subsystems has become vitally important since communication data is vulnerable to cyber-attacks. This work develops a federated learning (FL) modeling and model predictive control (MPC) method for heterogeneous nonlinear systems with multiple subsystems to address privacy preservation and heterogeneity issues. The FL framework with two different neural network structures is developed to aggregate the submodels trained locally for the subsystems into a global model without sharing the local data with each other. Subsequently, a theoretical bound on the generalization error of the FL models is established using information theory. The closed-loop stability criteria of heterogeneous nonlinear systems under the FL-based MPC scheme are developed in an information-theoretical manner. Finally, a chemical process network is applied to demonstrate the effectiveness of the proposed FL modeling and FL-based MPC methods.

Learning model predictive controller for wheeled mobile robot with less time delay

The weak spots have been examined, a solution has been suggested, the solution has been applied, and a comparison between the simulation and experimental test results has been given in this work. In order to do this, a nonlinear model predictive controller and a linear quadratic regulator controller have been utilized to control a four-wheel mobile robot during modeling and implementation. But the combination of these classic and modern controllers with machine learning can greatly help to make these controllers work more accurately; As a result, in order to increase the accuracy of the performance of these controllers, by training neural networks of multilayer perceptrons, the controllers have been made intelligent. Controllers with cost function have coefficients as weighting to the matrix of system state variables and control input, which are greatly affected by changing these two weighting matrices of problem solving and optimization. For this reason, it is necessary to extract these two matrices for each separate path in order to improve the performance of the controller by trial and error. But by applying the proposed network which is trained with a new algorithm, not only the performance accuracy has increased, but the network extracts these two matrices without the need to spend human energy. Additionally, by training additional neural networks to optimally extract the benefit of the forecast horizon, fewer calculations and faster solution times have been achieved, helping to reduce the current time delays, particularly in the implementation of the nonlinear controller on the robot in the experimental mode. In the hardware section, the solution time has been shortened by looking at and utilizing operators like the U2D2 interface and the pixy camera, which are quicker than the standard procedure. The information indicates that the proposed intelligence technology reacts 40%–50% quicker than the NMPC standard procedure. The suggested algorithm has also minimized the path tracking error since it extracts the optimal gain at each stage. Moreover, a hardware mode solving speed comparison between the recommended and conventional methods is given. In this domain, a 33% increase in solving speed has been noted.

A flexible and deep peak shaving scheme for combined heat and power plant under full operating conditions

Improving the flexible and deep peak shaving capacity of combined heat and power (CHP) plant under full operating conditions to facilitate renewable energy consumption is the main choice of novel power system. Accordingly, a dry/wet state automatic conversion control scheme fuses the precise mechanism structure, reinforcement learning and multi-objective model predictive control (MPC) algorithms is designed to promote the deep peak shaving ability of CHP plant in this paper. Firstly, the detailed mechanism models of once-through boiler at dry and wet state are presented by analyzing the dynamic characteristics of steam-water flow. Secondly, the large-scale unknown parameters in mechanism models are identified via the Takagi-Sugeno fuzzy modeling, reinforcement learning algorithms and actual dry/wet state conversion data. Thanks to the proposed hybrid modeling strategy, the high-precision boiler-steam unit models at dry and wet state are rapid obtained. Then, to meet different peak shaving demands, the multi-objective MPC algorithm is respectively constructed under dry and wet state with the comprehensive consideration of operational constraints, load tracking error, algorithm stability, power generation cost, CO2 and NOx emission costs. Aiming at maximizing the peak shaving efficiency and flexible operation ability of plant, a dry/wet automatic control scheme combined the designed multi-objective MPC algorithms and identified dry/wet state models is proposed. Finally, the load variation rate of 5 % and 2 % RCM/min is respectively achieved in the dry/wet state conversion tests on a 350 MW CHP plant based on the proposed control scheme. Thus, the rapid and thorough dry/wet state conversion performance of once-through boiler has successfully improved the operational flexibility of CHP plant under full operating conditions.

An integrated tube robust iterative learning model predictive control strategy based on dynamic partial least squares algorithm for batch processes

AbstractIn this paper, an integrated tube robust iterative learning model predictive control (Tube‐RILMPC) strategy based on the dynamic partial least squares (DyPLS) identified algorithm is proposed. The problems of large amount of online data calculation and input and output variable dimensions disaster in the original variable space are solved. This integrated Tube‐RILMPC strategy enhances the tracking performance and robustness of two‐dimensional (2D) control system. The output trajectories of system are located in the tube the nominal trajectory, which reduces the modelling error caused by the uncertain model. Based on the worst‐case performance index of ellipsoidal uncertainty and polytopic uncertainty, a robust iterative learning control (ILC) strategy is designed. Finally, the superiority of the proposed control algorithm is verified by comparative simulation.

Value Approximator-based Learning Model Predictive Control for Iterative Tasks

Value Approximator-based Learning Model Predictive Control for Iterative Tasks

Gaussian Process-Based Learning Model Predictive Control With Application to USV

Gaussian Process-Based Learning Model Predictive Control With Application to USV

Quantitative comparison of reinforcement learning and data-driven model predictive control for chemical and biological processes

As manufacturing processes transition towards digitalization, data-driven process control is emerging as a key area of interest in future artificial intelligence technology. A crucial aspect in implementing data-driven process control is “What should we learn from the data?”. In general, the data-driven control method can be categorized into two main approaches: Learning the model and learning the value. To assist in selecting the more suitable approach, this paper applies six different control methods, with three falling under each approach, to three distinct manufacturing process systems. The simulation results indicate that the model-learning approaches display higher data efficiency and exhibit lower variance in total cost. These methods prove to be particularly advantageous for addressing the regulation problems. Conversely, value-learning approaches show competitive potential in closed-loop identification and in managing economic cost problems. The remaining challenges associated with each technique are discussed, along with practical considerations for their implementation.

Event-Based Switching Iterative Learning Model Predictive Control for Batch Processes With Randomly Varying Trial Lengths.

Iterative learning model predictive control (ILMPC) has been recognized as an excellent batch process control strategy for progressively improving tracking performance along trials. However, as a typical learning-based control method, ILMPC generally requires the strict identity of trial lengths to implement 2-D receding horizon optimization. The randomly varying trial lengths extensively existing in practice can result in the insufficiency of learning prior information, and even the suspension of control update. Regarding this issue, this article embeds a novel prediction-based modification mechanism into ILMPC, to adjust the process data of each trial into the same length by compensating the data of absent running periods with the predictive sequences at the end point. Under this modification scheme, it is proved that the convergence of the classical ILMPC is guaranteed by an inequality condition relative with the probability distribution of trial lengths. Considering the practical batch process with complex nonlinearity, a 2-D neural-network predictive model with parameter adaptability along trials is established to generate highly matched compensation data for the prediction-based modification. To best utilize the real process information of multiple past trials while guaranteeing the learning priority of the latest trials, an event-based switching learning structure is proposed in ILMPC to determine different learning orders according to the probability event with respect to the trial length variation direction. The convergence of the nonlinear event-based switching ILMPC system is analyzed theoretically under two situations divided by the switching condition. The simulations on a numerical example and the injection molding process verify the superiority of the proposed control methods.

Iterative Learning Model Predictive Control With Fuzzy Neural Network for Nonlinear Systems

Due to the existence of strong nonlinearity and external disturbances, the controller design of complex nonlinear systems is a challenging problem. Therefore, it is necessary to design an effective robust predictive controller for this issue. In this paper, based on a fuzzy neural network, an iterative learning model predictive control (FNN-ILMPC) is designed for complex nonlinear systems. Firstly, a dynamic linearization technique (DIT) is used to establish a data-driven model, which only relies on input and output data. Since the established model contains an unknown disturbance term that may have an impact on the control performance, an FNN is used to evaluate the disturbance so that the uncertainty of the system is captured. Subsequently, based on the above data-driven model, an FNN-ILMPC strategy, considering the impact of external disturbances, is developed to eliminate the influence of disturbances. Then, it is proved that the designed controller can make both modeling error and tracking error decrease gradually and ensure the closed-loop system stability. Finally, experimental results verify the effectiveness and superiority of the designed controller.

Deep Learning Model Predictive Control Frameworks: Application to a Fluid Catalytic Cracker–Fractionator Process

Deep Learning Model Predictive Control Frameworks: Application to a Fluid Catalytic Cracker–Fractionator Process

Time-optimal coordination of multiple entering and departing ferries in terminals based on decentralized learning model predictive control

In large cities with harbors or rivers, ferry services have been an important part of public transport. In practice, many ferry terminals are subject to geographical and environmental conditions, limited space resources of the terminal pool, and restricted navigable waters of ships, which leads to constrained operations and increases the difficulty of docking when multiple ferries enter and depart from the terminal. Because of the distributed, real-time and dynamic characteristics of multi-ferry maneuvering, this paper proposes a decentralized, time-optimal control strategy to coordinate the maneuvering of multiple entering and departing ferries in restricted terminals. It combines nonlinear model predictive control (MPC) with an iterative learning control (ILC) framework. Based on the states and maneuver data of previously successfully performed trajectories, the coupled constraints among multiple ferries are decomposed with support vector machine (SVM). Therefore, it also does not require real-time information exchange during ferry operations and has the merits of low communication demand, as only local information is required in each ferry controller. Simulations are carried out to evaluate the effectiveness of the proposed method.

High‐order iterative learning model predictive control for batch chemical processes

AbstractIn order to address two‐dimensional (2D) control issue for a class of batch chemical processes, we propose a novel high‐order iterative learning model predictive control (HILMPC) method in this paper. A set of local state‐space models are first constructed to represent the batch chemical processes by adopting the just‐in‐time learning (JITL) technique. Meanwhile, a pre‐clustered strategy is used to lessen the computational burden of the modelling process and improve the modelling efficiency. Then, a two‐stage 2D controller is designed to achieve integrated control by combining high‐order iterative learning control (HILC) on the batch domain with model predictive control (MPC) on the time domain. The resulting HILMPC controller can not only guarantee the convergence of the system on the batch domain, but also guarantee the closed‐loop stability of the system on the time domain. The convergence of the HILMPC method is ensured by rigorous analysis. Two examples are presented in the end to demonstrate that the developed method provides better control performance than its previous counterpart.

A General Framework for Learning-Based Distributionally Robust MPC of Markov Jump Systems

We present a learning model predictive control (MPC) scheme for chance-constrained Markov jump systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are estimated which include the true conditional probability distributions with high probability. These sets are updated online and used to formulate a time-varying, risk-averse optimal control problem. We prove recursive feasibility of the resulting MPC scheme and show that the original chance constraints remain satisfied at every time step. Furthermore, we show that under sufficient decrease of the confidence levels, the resulting MPC scheme renders the closed-loop system mean-square stable with respect to the true-but-unknown distributions, while remaining less conservative than a fully robust approach. Finally, we show that the value function of the learning MPC converges from above to its nominal counterpart as the sample size grows to infinity. We illustrate our approach on a numerical example.

Real-Time Neural MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms

Model Predictive Control (MPC) has become a popular framework in embedded control for high-performance autonomous systems. However, to achieve good control performance using MPC, an accurate dynamics model is key. To maintain real-time operation, the dynamics models used on embedded systems have been limited to simple first-principle models, which substantially limits their representative power. In contrast to such simple models, machine learning approaches, specifically neural networks, have been shown to accurately model even complex dynamic effects, but their large computational complexity hindered combination with fast real-time iteration loops. With this work, we present Real-time Neural MPC, a framework to efficiently integrate large, complex neural network architectures as dynamics models within a model-predictive control pipeline. Our experiments, performed in simulation and the real world onboard a highly agile quadrotor platform, demonstrate the capabilities of the described system to run learned models with, previously infeasible, large modeling capacity using gradient-based online optimization MPC. Compared to prior implementations of neural networks in online optimization MPC we can leverage models of over 4000 times larger parametric capacity in a 50Hz real-time window on an embedded platform. Further, we show the feasibility of our framework on real-world problems by reducing the positional tracking error by up to 82% when compared to state-of-the-art MPC approaches without neural network dynamics.

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

This work presents a framework that allows Unmanned Surface Vehicles (USVs) to avoid dynamic obstacles through initial training on an Unmanned Ground Vehicle (UGV) and cross-domain retraining on a USV. This is achieved by integrating a Deep Reinforcement Learning (DRL) agent that generates high-level control commands and leveraging a neural network based model predictive controller (NN-MPC) to reach target waypoints and reject disturbances. A Deep Q Network (DQN) utilized in this framework is trained in a ground environment using a Turtlebot robot and retrained in a water environment using the BREAM USV in the Gazebo simulator to avoid dynamic obstacles. The network is then validated in both simulation and real-world tests. The cross-domain learning largely decreases the training time (28%) and increases the obstacle avoidance performance (70 more reward points) compared to pure water domain training. This methodology shows that it is possible to leverage the data-rich and accessible ground environments to train DRL agent in data-poor and difficult-to-access marine environments. This will allow rapid and iterative agent development without further training due to the change in environment or vehicle dynamics.

Data-driven docking control of autonomous double-ended ferries based on iterative learning model predictive control

In large cities with harbors or rivers, ferry services have been a crucial part of public transport. Docking refers to the process of maneuvering a ferry to the designated mooring position by the quayside. To facilitate the transition towards increased autonomy in ferries, this paper proposes an iterative learning-based model predictive control (IL-MPC) strategy for the automatic docking of over-actuated double-ended ferries, in situations in which the hydrodynamic coefficients of a ferry are unknown. It uses the collected data from actual ferry docking operations or reliable simulations to build ferry states prediction model in a linear state-space form based on kernel regression, which reduces the model-solving complexity in MPC. Moreover, it improves its closed-loop control performance and modeling accuracy via an iterative learning scheme. The excitation control input signals have been optimized to create suitable initial data sets. Simulations are carried out considering three docking position types and results show that the open-loop prediction accuracy has been improved with the optimized excitation input signal and that closed-loop control performance has been enhanced with the increase of iterations.

Adaptive iterative extended state observer-based data-driven iterative learning model predictive control for semiconductor silicon single crystal batch process

Aiming at the problems of unstable batch control of key crystal quality parameters and susceptibility to batch-to-batch non-repetitive disturbances during repeated operation of single crystal furnaces, this paper proposes a data-driven iterative learning model predictive control method based on an adaptive iterative extended state observer (IESO) for designing melt temperature and crystal diameter learning controllers with disturbance suppression. By applying dynamic linearization techniques and model predictive control strategies along the iterative axis, an ILMPC scheme with disturbance compensation terms using only input and output data of the system is designed. Among them, adaptive IESO is used to estimate the disturbance compensation terms. Then, the theoretical analysis shows that the tracking error of the ILMPC scheme can converge to a bounded range as the number of iterations increases. The experimental results verify the effectiveness of the proposed control method, which not only ensures that the control system has learning ability, but also achieves stable and accurate control of crystal quality parameters.

Learning Model Predictive Control for Quadrotors Minimum-Time Flight in Autonomous Racing Scenarios

In this paper, we design a Learning Model Predictive Control (LMPC) algorithm for quadrotors autonomous racing. The proposed algorithm allows to define a highly customizable 3D race track, in which multiple types of obstacles can be inserted. The controller is then able to autonomously find the best trajectory minimizing the quadrotor lap time, by learning from data coming from previous flights within the track, ensuring also the avoidance of all the obstacles therein. We also present novel relaxation approaches for the LMPC optimization problem, that allow to reduce it from a mixed-integer nonlinear program to a quadratic program. The LMPC algorithm is tested via several software-in-the-loop simulations, showing that the algorithm has learned to fly the quadrotor aggressively and dexterously, managing to both find the minimum-time trajectory and avoid the obstacles inside the track.

On the Optimality and Convergence Properties of the Iterative Learning Model Predictive Controller

In this technical article, we analyze the performance improvement and optimality properties of the learning model predictive control (LMPC) strategy for linear deterministic systems. The LMPC framework is a policy iteration scheme where closed-loop trajectories are used to update the control policy for the next execution of the control task. We show that, when a linear independence constraint qualification (LICQ) condition holds, the LMPC scheme guarantees strict iterative performance improvement and optimality, meaning that the closed-loop cost evaluated over the entire task converges asymptotically to the optimal cost of the infinite-horizon control problem. Compared to previous works, this sufficient LICQ condition can be easily checked, it holds for a larger class of systems and it can be used to adaptively select the prediction horizon of the controller, as demonstrated by a numerical example.