Offset-free Model Predictive Control Research Articles

Machine learning-based predictive control of an electrically-heated steam methane reforming process

Hydrogen plays a crucial role in improving sustainability and offering a clean and efficient energy carrier that significantly reduces greenhouse gas emissions. However, the primary method of industrial hydrogen production, steam methane reforming (SMR), relies on the combustion of hydrocarbons as the heating source for the reforming reactions, resulting in significant carbon emissions. To address this issue, an experimental setup of an electrically-heated steam methane reformer (e-SMR) has been constructed at UCLA, and a lumped first-principle dynamic process model was built based on parameters estimated from the experimental data in a previous study. Subsequently, the first-principle dynamic process model was implemented into the computational model predictive control (MPC) scheme, successfully driving the hydrogen production rate to the desired setpoint. While these works are important and pave the way for developing MPC for large-scale e-SMR processes, the first-principle process model may not accurately reflect the actual process behavior, particularly as the process behavior changes with time. Therefore, the development and establishment of an adaptive data-driven approach for implementing model predictive control in the e-SMR process is necessary. To address this need, the present work investigates the construction of recurrent neural network (RNN) models for an e-SMR process in-depth, utilizing data from an experimentally-validated first-principle model. Specifically, a long short-term memory (LSTM) layer was utilized in the RNN model to effectively capture the complex correlations present in long-term sequential data. Subsequently, this LSTM-based RNN process model was employed to design an MPC, and its performance was evaluated through comparison with proportional–integral (PI) control. To address potential disturbances and variability in a typical e-SMR process, three distinct approaches were developed: MPC with an integrator, MPC with real-time online retraining (transfer learning), and offset-free MPC. These approaches effectively eliminated the offset caused by disturbances. Overall, this study underscores the effectiveness of utilizing RNN models to capture process dynamics in an experimental e-SMR process. It also outlines strategies for employing RNN-based control and multiple approaches to address disturbances in general processes with partially infrequent and delayed measurement feedback. This approach is particularly valuable in scenarios where developing first-principle models for a new process may be challenging.

Offset-Free Koopman Model Predictive Control of Thermal Comfort Regulation for a Variable Refrigerant Flow-Dedicated Outdoor Air System-Combined System

Abstract Variable refrigerant flow (VRF) system has been an appealing solution of air conditioning for residential and commercial buildings, due to its flexibility and cost effectiveness, while lack of ventilation capability is a major drawback. Incorporation of dedicated outdoor air system (DOAS) is a typical practice. However, good coordination between DOAS and VRF is critical for achieving desired thermal comfort is challenging due to the possible complexity of mixed sensible and latent heat exchanges. In this paper, to handle the nonlinear dynamic characteristics of VRF-DOAS system, we propose an offset-free Koopman model predictive control (MPC) strategy for thermal comfort regulation, in which the MPC design is computationally more efficient due to the convex problem formulation and the use of reduced-order Koopman models, and the offset-free MPC structure enhances the robustness to model uncertainties and unmeasured disturbances. A control-oriented model is obtained by hybridizing the first-principle and data-driven modeling approach. The proposed controls strategy is evaluated with a Modelica simulation model of a VRF-DOAS system. A Dymola-Python cosimulation platform is developed via the functional mockup interface (FMI), for which the MPC algorithms are implemented in Python. Simulation results show significantly better performance of the offset-free Koopman MPC in thermal comfort regulation.

Run-indexed time-varying Bayesian optimization with positional encoding for auto-tuning of controllers: Application to a plasma-assisted deposition process with run-to-run drifts

Bayesian optimization (BO) has emerged as a useful paradigm for automatic calibration (aka auto-tuning) of advanced optimization- and learning-based controllers whose closed-loop performance depends on the choice of several tuning parameters in highly nonlinear and nonconvex ways. However, BO approaches to controller auto-tuning commonly rely on the assumption that system dynamics remain constant, which does not hold for systems with time-varying dynamics, for example, due to gradual aging or persistent environmental drifts. This challenge can be further compounded when gradual and persistent system drifts occur over a series of process runs. Existing time-varying BO (TVBO) approaches with spatio-temporal kernels fall short of effectively handling an integer run index, which is imperative for capturing run-to-run changes in the system behavior. To this end, this paper presents a run-indexed TVBO (RI-TVBO) approach that can systematically account for run-to-run process drifts as the system is queried over sequential process runs. The proposed approach relies on adapting the non-stationary Wiener process kernel to accommodate an integer run index, instead of time. This is done via positional encoding that incorporates the integer run index and, thus, enables describing run-to-run variations in system dynamics. The positional embedding vector associated with each run index is then mapped onto a scalar value to leverage the relationships between different process runs within the probabilistic surrogate model of the objective function in RI-TVBO. The performance of RI-TVBO is evaluated for auto-tuning of an offset-free model predictive controller for a low-temperature plasma-assisted process for thin film deposition. Simulation results demonstrate the superior performance of RI-TVBO over standard BO and TVBO under different scenarios of run-to-run process drifts encountered in plasma-assisted deposition processes in semiconductor manufacturing.

Practical Offset-Free Model Predictive Control and Its Embedded Application to Aeroengines

Practical Offset-Free Model Predictive Control and Its Embedded Application to Aeroengines

A practically implementable reinforcement learning control approach by leveraging offset-free model predictive control

This work addresses the problem of designing an offset-free implementable reinforcement learning (RL) controller for nonlinear processes. RL-based controllers can update the control policy using observed data obtained online based on the controller-process interactions. This allows to alleviate the regular model maintenance step that is essential in advanced control techniques such as model predictive control (MPC). However, random explorations are required for an RL agent to find optimal state–action regions so as to finally reach the optimal policy. This is not implementable in practical situations due to safety concerns and economic objectives. To address this issue, a pre-training strategy is proposed to provide a secure platform for online implementations of the RL controllers. To this end, an offset-free MPC (representative industrial MPC) optimization problem is leveraged to train the RL agent offline. Having obtained similar performance to the offset-free MPC, the RL agent is utilized for online control to interact with the actual process. The efficacy of the proposed approach to handle nonlinearity and changes in plant operating conditions (due to unmeasured disturbances) are demonstrated through simulations on a chemical reactor example for a pH neutralization process. The results show that the proposed RL controller can significantly improve the oscillatory closed-loop responses, obtained by running the offset-free MPC due to the plant-model mismatch and unmeasured disturbances.

Observer-based offset-free model predictive control for fractional-order systems

Observer-based offset-free model predictive control for fractional-order systems

An industrial case study on the combined identification and offset-free control of a chemical process

For three decades, model predictive control (MPC) has been the flagship advanced control method in the chemical process industries. However, most implementations still use heuristic methods for designing MPC estimators, especially for offset-free MPC implementations. In this paper, we present a recently developed maximum likelihood-based method for the identification of linear augmented disturbance models for use in offset-free MPC. This method provides noise covariances that are used to derive Kalman filters and moving horizon estimators, forgoing the need for manual design and tuning of the estimator. The method is extended to handle closed-loop plant data. The proposed identification method and estimator design are evaluated in industrial-scale, real-world case study of a process at Eastman Chemical’s Kingsport plant. Using this identified model, we reduced the mean stage cost by 38% compared to the performance of the existing, hand-tuned MPC model.

Offset-Free Model Predictive Control of Fuel Cell DC–DC Boost Converter With Low-Complexity and High-Robustness

This paper proposes an easily implementable and computationally effective model predictive control (MPC) scheme for fuel cell DC-DC boost converter. The proposed control scheme contains an explicit MPC method just with a single-step predictive horizon. Thus, the computational burden can be greatly reduced. To further enhance its anti-disturbance ability, the observer technique is adopted to respectively estimate the load impedance and input voltage for the predictive models. Since the estimated variables can be directly sent to the cost function, the satisfactory disturbance rejection performance can be guaranteed with optimal control. Besides, an additional reference current generation module based on the flatness theory is built to deal with the inevitable parameter uncertainties and the estimation errors. Thus, the nominal system model can be directly adopted to further reduce the design complexity. Some simulation and experimental tests based on a multi-phase interleaved fuel cell DC-DC boost converter are conducted to verify the superiority of the proposed control scheme. Results show that, with the proposed control scheme, both the dynamic performance and robustness of the closed-loop system are satisfactory, which can help to contribute to the fuel cell applications in transportation electrification.

Self-Evolving Offset-Free Model Predictive Control with Model–Plant Mismatch for Dynamic Working-Point Change Tasks in Industrial Processes

This study proposes a self-evolving offset-free model predictive control (MPC) algorithm for dynamic working-point change (DWPC) tasks in industrial processes. The algorithm mitigates disturbance impacts caused by model–plant mismatch (MPM) and enhances the dynamic performance of MPC by locating sequences (scenarios) similar to the current operational scenario from historical DWPC tasks and using them for multistep-ahead disturbance prediction. First, a disturbance-augmented state–space model guarantees the basic offset-free control behavior of MPC with MPM. Next, to enhance the MPC performance, a direct multistep-ahead disturbance prediction approach is proposed by combining historically similar DWPC task scenarios. Specifically, a dynamic autoencoder is constructed to extract private features from process scenarios and locate similar scenarios from historical DWPC tasks. Based on the located scenarios, the multistep-ahead disturbance and its uncertainty are directly predicted through multioutput Gaussian process regression. Finally, the obtained disturbance results are incorporated into the MPC framework, which continuously enhances MPC performance in DWPC tasks. Two case studies demonstrate the effectiveness of the proposed MPC.

Data-driven worst case model predictive control algorithm for propylene distillation column under uncertainty of top composition

Composition soft sensors have wide application in the distillation process. In this study, considering the limitation of the prediction ability of the composition soft sensor, a data-driven worst case model predictive control of propylene distillation column is proposed to hedge against the uncertainty of top composition. Firstly, based on the compartmental method and the dynamic mechanism model, a linear state space model of the distillation column is constructed. Aiming at dealing with the uncertainty of top propylene content caused by composition soft sensor, the data-driven uncertainty set is constructed by the combination of principal component analysis and kernel density estimation based on the historical data. Then, the certainty equivalent, traditional worst case, data-driven worst case, set-point tracking and offset-free model predictive control algorithm are designed. Finally, a case study of composition control in a propylene distillation column is carried out. Compared with other strategies, the proposed algorithm ensures the quality of the product while achieving small quality surplus and low operating cost.

On offset-free Model Predictive Control with Long Short-Term Memory Networks

This paper addresses offset-free reference tracking of asymptotically constant reference signals and disturbances using Model Predictive Control with a Long Short-Term Memory Network model. Basically, an error free output estimation and an error free nominal tracking for time-varying references are proposed for an Incremental Input-to-State Stable Long Short-Term Memory Network. This implies an offset-free Model Predictive Control algorithm.

An Effort towards Offset-free Model Predictive Control of Artificial Pancreas Systems*

Model predictive control (MPC) is one of the most commonly adopted algorithms for artificial pancreas (AP) systems. One of the unsolved issues is to achieve offset-free tracking during fasting periods given the mismatch in basal rate profiles. In this work, we introduce an MPC based on the extended state observer (ESO) to enable offset-free tracking behavior of AP. The proposed controller builds on a classical MPC structure but adds an ESO for total disturbance rejection. Specifically, ESO is added to estimate the uncertainties in the provided model and the mismatch of the insulin basal rate for the patient. Then we use the estimation to adaptively compensate for insulin injection and set dynamic reference values for the predictive model to isolate its effect on the control variables. This adaptive law has more flexibility to deal with changes in blood glucose in time. The performance of the proposed controller is evaluated through the 10-adult cohort of the US Food and Drug Administration (FDA) accepted Universities of Virginia (UVA)/Padova T1DM simulator. Compared with the classical MPC, the proposed controller achieves improved performance against basal rate mismatches.

Integrated approach to AUV docking based on nonlinear offset-free model predictive control

This study proposes an integrated docking approach for autonomous underwater vehicle (AUV), which directly generate control force and torque from visual guidance features to drive precise AUV docking. Accurate docking control has been constructed as nonlinear offset-free model predictive control (OFNMPC) problem to effectively deal with external disturbances, model mismatch, and various other constraints. For the problem of state and disturbance estimation required by OFNMPC, observers have been presented based on the extended Kalman filter (EKF) and moving horizon estimation (MHE). A visual observation model is proposed so that the observer can directly use visual measurements. OFNMPC was integrated with EKF and MHE, and the proposed OFNMPC-EKF and OFNMPC-MHE docking approaches were applied to an actual AUV model. The simulation results show that the proposed approaches could drive the AUV to complete accurate docking along a reasonable trajectory based on visual measurements. The navigation, guidance, and control functions of docking were effectively integrated. It was observed that in the presence of external disturbances and measurement noise, OFNMPC-EKF has better robustness.

Thermal control of a mechanically pumped two-phase loop

With integration and miniaturization of electronic devices, the heat dissipation has grown exponentially, necessitating feasible thermal management systems and cooling alternatives. Among them, two-phase loop cooling is deemed to be highly effective, in which the minichannel evaporator is rendered as a mechanically pumped two-phase loop (MPTL), ensuring the precise thermal control. Herein, the temperature control accuracy of MPTL has been particularly emphasized by proposing the feed-forward offset-free model predictive controller (MPC), and compared with the PI controller that has also been structurally modified (namely, improved PI controller). MPC is designed, optimized and tuned based on the unsteady-state nonlinear analysis, addressing the challenges, such as frequent heat load fluctuations and various nonlinearities/uncertainties mainly encountered during the minichannel boiling. Results depict the superiority of MPC in terms of shorter transient time (measured by set-point tracking), higher temperature control accuracy and lower absolute error (IAE) index (both determined by disturbance rejection), mitigating the effect of flow-boiling uncertainties. Especially, the average transient time of MPC on MPTL has been found at least four times shorter than that of the improved PI controller. Also, IAE index of MPC on MPTL is better than that of the improved PI controller. Meanwhile, the temperature control accuracy of MPC on MPTL during disturbance suppression is as little as 0.2 °C, while that of the improved PI is higher (1.1 °C). The temperature control of the proposed controller (MPC) is therefore satisfactory based on which it can be engaged in MPTL technology for further development of electronic applications.

Motion Control of Autonomous Vehicles Based on Offset Free Model Predictive Control Methods

Abstract Model predictive control (MPC) is the mainstream method in the motion control of autonomous vehicles. However, due to the complex and changeable driving environment, the perturbation of vehicle parameters will cause the steady-state error problem, which will lead to the degradation of controller performance. In this paper, the offset-free MPC control method is proposed to solve the steady-state error problem systematically. The core idea of this method is to model the model mismatch, control input offset, and external disturbances as disturbance terms, then use filters to observe these disturbances and finally eliminate the influence of these disturbances on the steady-state error in the MPC solution stage. This paper uses the Kalman filter as an observer, which is integrated into our latest designed MPC solver. Based on state-of-the-art sparse quadratic programming (QP) solver operator splitting solver for quadratic programs (OSQP), an offset free model predictive control (OF-MPC) framework based on disturbance observation and MPC is formed. The proposed OF-MPC solver can efficiently deal with common model mismatch problems such as tire stiffness mismatch, steering angle offset, lateral slope disturbance, and so on. This framework is very efficient and completes all calculations in less than 7 ms when the horizon length is 50. The efficiency and robustness of the algorithm are verified on our newly designed robot operating system (ROS)-Unreal4-carsim real-time cosimulation platform and real vehicle experiments.

Offset-free model predictive control of a soft manipulator using the Koopman operator

The modeling and control of soft manipulators remain challenging due to their inherent complex kinematics and dynamics. This paper presents an offset-free Koopman operator-based model predictive control (OK-MPC) scheme that offers a practical data-driven approach to model and control the soft manipulator in the task space. In this scheme, the Koopman operator is used to derive a straightforward linear model to describe the complex dynamics of a soft manipulator. Then, an offset-free control strategy is incorporated with the Koopman operator to minimize the effect of modeling uncertainties and external disturbance, allowing the precise control of the soft manipulator. The proposed OK-MPC scheme was examined on both a soft pneumatic manipulator and a cable-driven origami manipulator through trajectory tracking experiments. The experimental results showed that the tracking errors decreased by 30.0% in free space and 45.4% in confined space, indicating that the OK-MPC has the capability to improve the control performance of a normal Koopman-based controller. Besides, the OK-MPC is readily available for the two different soft robotic prototypes, demonstrating its general applicability for the modeling and control in the field of soft robotics.

Learning of model-plant mismatch map via neural network modeling and its application to offset-free model predictive control

We propose an improved offset-free model predictive control (MPC) framework, which learns and utilizes the intrinsic model-plant mismatch map, to effectively exploit the advantages of model-based and data-driven control strategies and overcome the limitation of each approach. In this study, the model-plant mismatch map on steady-state manifold is approximated via artificial neural network (ANN) modeling based on steady-state data from the process. Though the learned model-plant mismatch map can provide the information at the equilibrium point (i.e., setpoint), it cannot provide model-plant mismatch information during transient state. To handle this, we additionally apply a supplementary disturbance variable which is updated from a revised disturbance estimator considering the disturbance value obtained from the learned model-plant mismatch map. Then, the learned and supplementary disturbance variables are applied to the target problem and finite-horizon optimal control problem of the offset-free MPC framework. By this, the control system can utilize both the learned model-plant mismatch information and the stabilizing property of the nominal disturbance estimator. The closed-loop simulation results demonstrate that the proposed offset-free MPC scheme utilizing the model-plant mismatch map learned via ANN modeling efficiently improves the closed-loop reference tracking performance of the control system. Additionally, the zero-offset tracking condition of the developed framework is mathematically examined.

Data-driven model predictive control design for offset-free tracking of nonlinear systems

We propose a design of data-driven Model Predictive Control (MPC) using a suboptimal trajectory and the linear time-varying (LTV) models from data-driven trajectory optimisation that achieves offset-free tracking. Data-driven constrained differential dynamic programming (CDDP) is exploited to improve the trajectory iteratively without the knowledge of the nonlinear model. A trajectory is divided to the transient and steady state regions, controlled by the Linear time-varying MPC (LTVMPC) and the offset-free linear MPC (LMPC), respectively. We prove the feasibility of the proposed LTVMPC in the transient region, and the offset-free tracking property of LMPC. The proposed scheme is validated to a continuous stirred tank reactor (CSTR) process. Simulation studies show that the suboptimal trajectory and LTV models are generated by CDDP, and the proposed MPC achieves offset-free tracking and disturbance rejection for a set of initial conditions and set points in the operating region.

Towards longitudinal and lateral coupling control of autonomous vehicles using offset free MPC

Model predictive control (MPC) is widely used in the motion control of autonomous vehicles. However, the conventional MPC relies on an accurate model and cannot achieve offset-free tracking in steady-state when the autonomous vehicle encounters external disturbance and parameter perturbation. Based on our recently designed C++ offset-free MPC (OF-MPC) solver, this paper uses MPC to solve the longitudinal and lateral coupling control problem directly. In the modeling stage, the constraints of the tire friction circle are approximated by the polygon. And then the polygon constraints are mapped into the state and control space. Finally, the tire friction circle constraints are approximated as convex linear inequality constraints, which are used as the vehicle stability boundary condition. Meanwhile, a convex optimization method is used to solve the reference value problem of MPC, and a systematic method that can prevent the reference value from being outside the feasible region is proposed. The real-time simulation and actual vehicle results show that the algorithm can solve both lateral and longitudinal disturbances problems. The algorithm achieves better tracking accuracy and high-speed stability due to consideration of the longitudinal and lateral coupling constraints. Benefiting from the proposed efficient solver, the average time to complete all calculations, including logic processing and input–output processing, is only about 5ms when the horizon length is 50.

Offset-Free Nonzero Tracking for Nonlinear Impulsive Systems With Application to Biomedical Processes

Impulsive control systems have shown strong potential to represent and address challenging problems, especially in the biomedical field. In recent research, these problems have been tackled with advances in linear impulsive control systems. However, many biomedical applications are better described by nonlinear impulsive models, and therefore, it is necessary to develop analysis tools and control strategies in this context. In the literature, the main properties of nonlinear impulsive control systems have been fully understood, but there is no major development of control strategies. Particularly, there is no substantiation of model predictive control (MPC) strategies maintaining convexity of the optimization problem and closed-loop stability, and there is no control strategy to reduce the offset problem when there are parameter variations, which is a common situation in biological processes. Therefore, the main novelties of this paper are: (i) an MPC formulation extended to nonlinear impulsive systems that addresses non-zero tracking, (ii) the sufficient and necessary conditions to guarantee the stability of the closed-loop system at an equilibrium target, (iii) a comprehensive description of an offset-free MPC to handle low to moderate plant-model mismatches, (iv) the conditions to guarantee offset-free control. Finally, the MPC and offset-free MPC are tested to address the drug administration problem in two biomedical applications: oncolytic virus therapy, to regulate tumor dynamics using doses of oncolytic, and type 1 diabetes treatment, to regulate glycemia using insulin injections. Satisfactory results were obtained in simulation scenarios including parameter variations in nonlinear models that represent the corresponding dynamics.