Learning-based Model Predictive Control Research Articles

Active Learning-Based Model Predictive Coverage Control

Active Learning-Based Model Predictive Coverage Control

A learning-based model predictive control scheme for injection speed tracking in injection molding process

A learning-based model predictive control scheme for injection speed tracking in injection molding process

Model predictive control of switched nonlinear systems using online machine learning

This work introduces an online learning-based model predictive control (MPC) approach designed for the modeling and control of switched nonlinear systems featuring scheduled mode transitions. Initially, recurrent neural network (RNN) models are constructed offline, utilizing sufficient historical operational data to capture the nominal system dynamics. Subsequently, we employ real-time process data to develop online learning RNN models, aiming to approximate the dynamics of switched nonlinear systems under the influence of bounded disturbances. In cases where the initial RNN models are unavailable for a specific switching mode due to limited historical data, we use real-time data from closed-loop operations under a proportional-integral (PI) controller to build online learning RNN models. To evaluate the predictive performance of online learning RNNs, a theoretical analysis on their generalization error bound is developed using statistical machine learning theory. Additionally, considering the presence or absence of initial RNN models, two MPC schemes are developed. These schemes employ RNNs as prediction models to stabilize switched nonlinear systems, ensuring closed-loop stability by accounting for the generalization error bound derived for online learning RNNs. Finally, the effectiveness of the proposed MPC schemes is demonstrated through a nonlinear process example with two switching modes.

Learning-based model predictive control for safe path planning and control

Learning-based model predictive control for safe path planning and control

Real-time deep learning-based model predictive control of a 3-DOF biped robot leg

Our research utilized deep learning to enhance the control of a 3 Degrees of Freedom biped robot leg. We created a dynamic model based on a detailed joint angles and actuator torques dataset. This model was then integrated into a Model Predictive Control (MPC) framework, allowing for precise trajectory tracking without the need for traditional analytical dynamic models. By incorporating specific constraints within the MPC, we met operational and safety standards. The experimental results demonstrate the effectiveness of deep learning models in improving robotic control, leading to precise trajectory tracking and suggesting potential for further integration of deep learning into robotic system control. This approach not only outperforms traditional control methods in accuracy and efficiency but also opens the way for new research in robotics, highlighting the potential of utilizing deep learning models in predictive control techniques.

Model Predictive Control with Variational Autoencoders for Signal Temporal Logic Specifications.

This paper presents a control strategy synthesis method for dynamical systems with differential constraints, emphasizing the prioritization of specific rules. Special attention is given to scenarios where not all rules can be simultaneously satisfied to complete a given task, necessitating decisions on the extent to which each rule is satisfied, including which rules must be upheld or disregarded. We propose a learning-based Model Predictive Control (MPC) method designed to address these challenges. Our approach integrates a learning method with a traditional control scheme, enabling the controller to emulate human expert behavior. Rules are represented as Signal Temporal Logic (STL) formulas. A robustness margin, quantifying the degree of rule satisfaction, is learned from expert demonstrations using a Conditional Variational Autoencoder (CVAE). This learned margin is then applied in the MPC process to guide the prioritization or exclusion of rules. In a track driving simulation, our method demonstrates the ability to generate behavior resembling that of human experts and effectively manage rule-based dilemmas.

Robust Learning-Based Model Predictive Control for Wave Energy Converters

Robust Learning-Based Model Predictive Control for Wave Energy Converters

Model predictive control for multi-period portfolio optimization: a trading-oriented learning approach

Portfolio optimization aims at finding the optimal allocation of a given wealth among different assets to maximize an investor's utility function. A major critical issue concerns the prediction of future asset returns to forecast market evolution, due to the non-stationarity and volatility of asset prices. In this work, a learning-based Model Predictive Control (MPC) strategy for multi-period portfolio optimization is proposed, where the return prediction model is estimated via a novel trading-oriented learning paradigm. According to such a perspective, the model parameters are not the ones minimizing the prediction error but those that directly maximize the investor's utility. An extensive experimental study carried out on real-market data shows the potential improvements introduced by the proposed methodology compared to benchmark financial strategies.

Learning-Based Model Reduction and Predictive Control of an Ammonia Synthesis Process

Learning-Based Model Reduction and Predictive Control of an Ammonia Synthesis Process

Deep Learning based Model Predictive Controller on a Magnetic Levitation Ball System

The magnetic levitation (maglev) ball system is a prototypical Single-Input-Single-Output (SISO) system, characterized by its pronounced nonlinearity, rapid response, and open-loop instability. It serves as the basis for many industrial devices. For describing the dynamics of the maglev ball system precisely in the pseudo linear model, the long short-term memory (LSTM) based auto-regressive model with exogenous input variables (LSTM-ARX) is proposed. Firstly, the LSTM network is modified by incorporating the auto-regressive structure with respect to sequence input, allowing it to deduce a locally linearized model without the need for Taylor expansion. Then, the LSTM-ARX model is transformed into a linear parameter varying (LPV) state space model, and upon this foundation, a model predictive controller (MPC) is proposed. Specifically, when deducing the MPC, the deep learning-based model is linearized by fixing its state input at the current state, so that the nonlinear, non-convex optimization problem can be converted to a finite-horizon quadratic programming problem, thereby deriving the explicit form of MPC. To further enhance the efficiency of the controller in real-time control tasks, a predictive functional controller (PFC) is proposed. It employs multiple nonlinear functions to fit the control sequence, thereby reducing the number of decision variables of the on-line optimization problem in MPC. The proposed controller was successfully applied to the real-time control of the maglev ball system. Simulation and real-time control experiments have validated the improvement in transient performance and efficiency of the LSTM-ARX model-based PFC (LSTM-ARX-PFC).

Online Learning-Based Model Predictive Trajectory Control for Connected and Autonomous Vehicles: Modeling and Physical Tests

Online Learning-Based Model Predictive Trajectory Control for Connected and Autonomous Vehicles: Modeling and Physical Tests

Nonlinear sparse variational Bayesian learning based model predictive control with application to PEMFC temperature control

The accuracy of the underlying model predictions is crucial for the success of model predictive control (MPC) applications. If the model is unable to accurately analyze the dynamics of the controlled system, the performance and stability guarantees provided by MPC may not be achieved. Learning-based MPC can learn models from data, improving the applicability and reliability of MPC. This study develops a nonlinear sparse variational Bayesian learning based MPC (NSVB-MPC) for nonlinear systems, where the model is learned by the developed NSVB method. Variational inference is used by NSVB-MPC to assess the predictive accuracy and make the necessary corrections to quantify system uncertainty. The suggested approach ensures input-to-state stability (ISS) and the feasibility of recursive constraints in accordance with the concept of an invariant terminal region. Finally, a PEMFC temperature control model experiment confirms the effectiveness of the NSVB-MPC method.

Learning-based model predictive secondary frequency control of PV-ESS-EV microgrid

Microgrids comprising photovoltaic (PV), energy-storage systems and electric vehicles have received increasing attention. However, the increase in renewable energy sources, such as PVs, will introduce uncertainty factors to the frequency response, resulting in frequency fluctuations that can reduce power quality. Hence, we propose an outstanding learning-based control approach that combines a model of physical principles with a Gaussian Process (GP) learning approach. The learning model corrects the physical principles model and estimates its uncertainty. Simulations are performed by comparing the proposed controller with conventional model-predictive controllers. The results indicate that the learning-based controller proposed can adjust the frequency with less overshooting and a lower root mean square error compared to the conventional MPC control method.

Optimization of energy efficiency for offshore wind farms via wake modeling-free NMPC

This paper presents an energy efficiency improvement scheme for offshore wind farms. This scheme uses learning-based nonlinear model predictive control (NMPC) and does not require wake modeling. The main feature is that each wind turbine can independently complete all optimal control, without the need for wind farm wake model calculation and iterative optimization. Specifically, this paper studies the critical factors for increasing the active power of wind farms and reducing the structural load of wind turbines, and reconstructs the optimal control objectives of wind turbines. Furthermore, combined with the nonlinear model of wind turbines containing uncertainty, a nonlinear model predictive control algorithm based on deep neural networks was designed, which can achieve multi-objective optimal control at the wind turbine level. The simulation results show that the uncertainty estimation method can effectively improve the nonlinear control performance, achieve an increase in the active power of the wind farm without wake iterative optimization, and at the same time suppress the structural loads of all wind turbines.

Safety enhancement for nonlinear systems via learning-based model predictive control with Gaussian process regression

This paper proposes a novel safe Gaussian process model predictive control scheme for discrete-time nonlinear systems subject to additive uncertainties. The scheme is implemented using an online learning framework that provides safety guarantees based on a nominal model, and employs Gaussian process regression (GPR) to learn the uncertainties to improve the control performance. The advantage of this framework lies in the ability to decouple safety and performance in the robust controller design. Furthermore, the inaccuracy measure given by GPR can be used to adaptively tighten the constraints, leading to the less conservative behaviors. A rigorous analysis of the constraint satisfaction, recursive feasibility and closed-loop stability is also presented. Two simulation examples, including a regulation problem of a nominally linear system and a tracking problem of a car whose model is nonlinear, are performed to verify the effectiveness of the proposed scheme.

Explicit machine learning-based model predictive control of nonlinear processes via multi-parametric programming

Machine learning-based model predictive control (ML-MPC) has been developed to control nonlinear processes with unknown first-principles models. While ML models can capture nonlinear dynamics of complex systems, the complexity of ML models leads to increased computation time for real-time implementation of ML-MPC. To address this issue, in this work, we propose an explicit ML-MPC framework for nonlinear processes using multi-parametric programming. Specifically, a self-adaptive approximation algorithm is first developed to obtain a piecewise linear affine function that approximates the behaviors of ML models. Then, multi-parametric quadratic programming (mpQP) problems are formulated to generate the solution map for states in discretized state–space. Furthermore, to accelerate the implementation of explicit ML-MPC, a neighbor-first search algorithm is developed. Finally, an example of a chemical reactor is used to demonstrate the effectiveness of the explicit ML-MPC.

Improving safety in mixed traffic: A learning-based model predictive control for autonomous and human-driven vehicle platooning

As autonomous vehicles (AVs) become more common on public roads, their interaction with human-driven vehicles (HVs) in mixed traffic is inevitable. This requires new control strategies for AVs to handle the unpredictable nature of HVs. This study focused on safe control in mixed-vehicle platoons consisting of both AVs and HVs, particularly during longitudinal car-following scenarios. We introduce a novel model that combines a conventional first-principles model with a Gaussian process (GP) machine learning-based model to better predict HV behavior. Our results showed a significant improvement in predicting HV speed, with a 35.64% reduction in the root mean square error compared with the use of the first-principles model alone. We developed a new control strategy called GP-MPC, which uses the proposed HV model for safer distance management between vehicles in the mixed platoon. The GP-MPC strategy effectively utilizes the capacity of the GP model to assess uncertainties, thereby significantly enhancing safety in challenging traffic scenarios, such as emergency braking scenarios. In simulations, the GP-MPC strategy outperformed the baseline MPC method, offering better safety and more efficient vehicle movement in mixed traffic.

Reinforcement Learning-Based Model Predictive Control for Discrete-Time Systems.

This article proposes a novel reinforcement learning-based model predictive control (RLMPC) scheme for discrete-time systems. The scheme integrates model predictive control (MPC) and reinforcement learning (RL) through policy iteration (PI), where MPC is a policy generator and the RL technique is employed to evaluate the policy. Then the obtained value function is taken as the terminal cost of MPC, thus improving the generated policy. The advantage of doing so is that it rules out the need for the offline design paradigm of the terminal cost, the auxiliary controller, and the terminal constraint in traditional MPC. Moreover, RLMPC proposed in this article enables a more flexible choice of prediction horizon due to the elimination of the terminal constraint, which has great potential in reducing the computational burden. We provide a rigorous analysis of the convergence, feasibility, and stability properties of RLMPC. Simulation results show that RLMPC achieves nearly the same performance as traditional MPC in the control of linear systems and exhibits superiority over traditional MPC for nonlinear ones.

Robust Learning-Based Model Predictive Control for Intelligent Vehicles With Unknown Dynamics and Unbounded Disturbances

Robust Learning-Based Model Predictive Control for Intelligent Vehicles With Unknown Dynamics and Unbounded Disturbances

Real-time Learning-based Nonlinear Model Predictive Control of a virtual motorcycle employing grey-box modeling through Gaussian processes

Virtual prototyping tools simulations are nowadays largely adopted methods in the automotive industry. However, to achieve effective simulative tests for two-wheeled vehicles, a virtual rider, i.e. a controller for virtual motorcycles, is typically needed due to the inherent instability of the system. Different control strategies have been employed to deal with this control task, and promising results have been obtained using Nonlinear Model Predictive Control (NMPC). Yet, performance of NMPC highly relies on the plant characterization, that can be highly complex to obtain analytically for 2-wheel vehicles. To improve it, learning dynamics approaches can be used within a Learning-based Nonlinear Model Predictive Control (LbNMPC) framework, exploiting the combination of data-driven techniques and NMPC strategy. In this manuscript, we present a tailored real-time capable LbNMPC for a virtual motorcycle that relies on a continuous grey-box model based on Gaussian Processes (GPs), method that has proven to be effective for 4-wheel vehicle applications. To cope with the real-time requirement, a feature selection procedure and sparse GP approximations have been adopted. A comparison with a physics-based model NMPC implementation taken from the literature has been carried out, analyzing the impact of different sparse GP representations. Remarkable improvements in both model accuracy and tracking performance have been obtained, and the generalization properties of the grey-box model have been assessed.