Learning-based Model Predictive Control Research Articles

Robust Learning-Based Model Predictive Control for Intelligent Vehicles With Unknown Dynamics and Unbounded Disturbances

Robust Learning-Based Model Predictive Control for Intelligent Vehicles With Unknown Dynamics and Unbounded Disturbances

Real-time Learning-based Nonlinear Model Predictive Control of a virtual motorcycle employing grey-box modeling through Gaussian processes

Virtual prototyping tools simulations are nowadays largely adopted methods in the automotive industry. However, to achieve effective simulative tests for two-wheeled vehicles, a virtual rider, i.e. a controller for virtual motorcycles, is typically needed due to the inherent instability of the system. Different control strategies have been employed to deal with this control task, and promising results have been obtained using Nonlinear Model Predictive Control (NMPC). Yet, performance of NMPC highly relies on the plant characterization, that can be highly complex to obtain analytically for 2-wheel vehicles. To improve it, learning dynamics approaches can be used within a Learning-based Nonlinear Model Predictive Control (LbNMPC) framework, exploiting the combination of data-driven techniques and NMPC strategy. In this manuscript, we present a tailored real-time capable LbNMPC for a virtual motorcycle that relies on a continuous grey-box model based on Gaussian Processes (GPs), method that has proven to be effective for 4-wheel vehicle applications. To cope with the real-time requirement, a feature selection procedure and sparse GP approximations have been adopted. A comparison with a physics-based model NMPC implementation taken from the literature has been carried out, analyzing the impact of different sparse GP representations. Remarkable improvements in both model accuracy and tracking performance have been obtained, and the generalization properties of the grey-box model have been assessed.

Compound Learning-Based Model Predictive Control Approach for Ducted-Fan Aerial Vehicles.

Designing an efficient learning-based model predictive control (MPC) framework for ducted-fan unmanned aerial vehicles (DFUAVs) is a difficult task due to several factors involving uncertain dynamics, coupled motion, and unorthodox aerodynamic configuration. Existing control techniques are either developed from largely known physics-informed models or are made for specific goals. In this regard, this article proposes a compound learning-based MPC approach for DFUAVs to construct a suitable framework that exhibits efficient dynamics learning capability with adequate disturbance rejection characteristics. At the start, a nominal model from a largely unknown DFUAV model is achieved offline through sparse identification. Afterward, a reinforcement learning (RL) mechanism is deployed online to learn a policy to facilitate the initial guesses for the control input sequence. Thereafter, an MPC-driven optimization problem is developed, where the obtained nominal (learned) system is updated by the real system, yielding improved computational efficiency for the overall control framework. Under appropriate assumptions, stability and recursive feasibility are compactly ensured. Finally, a comparative study is conducted to illustrate the efficacy of the designed scheme.

Path-Tracking and Lateral Stabilization for Automated Vehicles via Learning-based Robust Model Predictive Control

Path-Tracking and Lateral Stabilization for Automated Vehicles via Learning-based Robust Model Predictive Control

Online Learning for Safe Model Predictive Control with the Compatible Models Approach

In this paper, we introduce an online safe learning-based Model Predictive Control (MPC) approach. This approach, which we call the ”compatible model approach”, consists of building two models of the system. The first is a piecewise interval-valued over-approximation of the system, and the second is a single-valued piecewise multi-affine estimation of the system’s dynamics. The first model is used to find the set of safe actions at each state, whereas the second is used to choose -out of those safe actions- the input that minimizes a given cost function. For the first model, we use the -assumed known-bounds on the derivative of the dynamics to update the model. The second model should be contained in the first to ensure the feasibility of the MPC scheme everywhere (Hence, the name compatible). Both models can be updated online. We are able to do that because each new transition updates the models locally. We present a test case where we train a mobile robot at low speeds, then navigate it in an environment while avoiding obstacles and collecting new data to learn its dynamics at high speeds.

Learning and Outperforming the Model Predictive Control With a Linear ϵ-Support Vector Machine for Power Converter

This paper presents a linear ϵ error insensitive support vector machine (ϵ-SVM) learned from the model predictive control (MPC) with very limited training samples. Unlike the existing learning-based MPC which takes the same inputs as the MPC, the ϵ-SVM learns from historical data as well. Such structure is found to have both feedforward and feedback elements that improve performance. Besides, using the ϵ error-insensitive tube also increases the noise cancellation capability. The experiment is carried out on a three-phase inverter with an L-C filter and shows that the linear ϵ-SVM-MPC not only has better performance than the artificial neural networks (ANN) MPC, but also outperforms the online MPC in both low THD values and small tracking errors regardless the load conditions. Moreover, the use of the linear kernel significantly reduces the time complexity to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1), which is much less than <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (2 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ⁿ</i> ) in either online MPC or ANN-MPC. Therefore, the linear ϵ-SVM-MPC has provided an effective way to implement the MPC and it is preferable for the DSP. This article is accompanied by a video demonstrating the real-time operation.

Neural Network-Based Hammerstein Model Identification of a Lab-Scale Batch Reactor.

This paper focuses on two types of neural network-based Hammerstein model identification methods for the acrylamide polymerization reaction of a batch reactor process. The first neural-based identification type formulates the weights of the multilayer network directly as parameters of the nonlinear static and linear dynamic blocks of the Hammerstein model and trains the weights using a gradient-based backpropagation algorithm. In the second identification type, the nonlinear static block of the Hammerstein model is framed as a single hidden-layer feedforward network and both nonlinear and linear block parameters are trained using an extreme learning machine, where the training procedure is exempted from gradient calculation. The primary focus of the paper is neural-based model identification of a complex nonlinear system, which facilitates ease of linear/nonlinear controller design with good learning speed and less computations. A future work toward the machine learning-based nonlinear model predictive controller implementation using the Jetson Orin Nano board is also described.

Learning‐based model predictive control under value iteration with finite approximation errors

AbstractThis paper proposes a novel learning‐based model predictive control (LMPC) scheme for discrete‐time nonlinear systems. It overcomes the challenge of manually designing the terminal conditions for traditional MPC and enhances the control performance. The scheme employs the value iteration (VI) in reinforcement learning (RL), and autonomously designs the terminal cost by iteratively performing value function learning and policy update under known dynamics and constraints. In contrast to the existing schemes that combine RL with MPC, the proposed scheme explicitly considers the approximation errors in each iteration. Further, a rigorous theoretical analysis is provided, including the convergence of VI, the stability and performance of the closed‐loop system. In addition, the influences of the prediction horizon and the initial terminal cost on performance are also investigated. Simulation results of a linear system verify the theoretical properties of the LMPC and show that it achieves (near‐)optimal performance. Moreover, its unique superiority over traditional MPC is fully demonstrated in a nonholonomic vehicle regulation example.

Learning-based model predictive energy management for fuel cell hybrid electric bus with health-aware control

Advanced energy management strategy (EMS) can ensure healthy, stable, and efficient operation of the on-board energy systems. Model Predictive Control (MPC) and Deep Reinforcement Learning (DRL) are two powerful control methods that have been extensively researched in the field of vehicle energy management. However, there are some problems with both approaches. On the one hand, MPC is difficult to cope with the complex systems and the excessive computational load caused by the non-linear solving over long prediction horizon, on the other hand, DRL lacks adaptability to different driving conditions and is poorly interpretable. Therefore, this paper innovatively proposes a learning-based model predictive (L-MPC) EMS for fuel cell hybrid electric bus (FCHEB) with health-aware control. This method effectively merges the advantages of control theory and machine learning. Specifically, firstly, the precise aging models for vehicular energy systems are established and incorporated into the optimization framework along with hydrogen consumption related metrics. Secondly, the principles of the learning-based MPC algorithm are thoroughly elucidated. In addition, to ensure driving details under future conditions, a speed predictor based on a double-layer Bi-directional Long Short-Term Memory (BiLSTM) is proposed at the strategy supervision layer. Finally, the superiority of the proposed strategy in prolonging the lifespan of the energy systems and reducing overall vehicle operating cost is verified by comprehensive comparisons with state-of-the-art MPC and DRL methods under real-world collected driving condition.

LSTM-MPC: A Deep Learning Based Predictive Control Method for Multimode Process Control

Modern industrial processes often operate under different modes, which brings challenges to model predictive control (MPC). Recently, most MPC related methods would establish prediction models independently for different modes, which results in their control effect highly relying on switching strategies. Inspired by the powerful representation capabilities of deep learning, this paper proposed a deep learning based MPC method. Specifically, the LSTM network is applied to predict behaviours of controlled system, which can automatically match different operation modes without switching strategy. Then combined with MPC framework, an adaptive gradient descent method is introduced to handle optimization problem and its constraints. In addition, stability and feasibility analysis have been conducted from the aspect of theory to ensure practical application of the proposed method. Experiments on a numerical simulation process and an industrial process platform show the strength and reliability of the proposed method, which reduces the overshoot by about 10 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> compared to common learning-based MPC methods and improves the control accuracy effectively.

Adaptive stochastic model predictive control via network ensemble learning

This paper proposes a novel ensemble learning-based adaptive stochastic model predictive control (SMPC) algorithm for constrained linear systems with unknown nonlinear terms and random disturbances. The ensemble network combining a feedforward neural network and a Bayesian network is used to offline learn the nonlinear dynamics and disturbance distribution parameters. Then, the mixed-tube scheme is designed to cope with input constraints and state chance constraints while decreasing computational demands and conservativeness. The reliability of the stochastic tube is guaranteed using the Hoeffding inequality-based verification mechanism, which results in a chance constraint with double probabilities. The feasibility and exponential stability of the SMPC are rigorously proven. A numerical example verifies the merits of the proposed algorithm in terms of the control performance and the feasible domain.

Data-driven learning-based Model Predictive Control for energy-intensive systems

Efficient control of energy-intensive systems is essential for reducing energy consumption and realizing sustainable development. However, considering the complex inter-dependent energy-consumption devices, numerous control parameters, and dynamic environments, the energy-efficient control of energy-intensive system is always challenging. To address such problems, this paper proposes a data-driven learning-based Model Predictive Control (MPC) method for the integrated control of various devices in energy-intensive systems. Specifically, a hybrid prediction model based on two variants of RNN is integrated with the MPC scheme to learn and predict the system dynamics based on massive time-series sensing data. Then an efficient tree-based prioritized group control model for heterogeneous devices is developed with a rolling optimization and feedback correction mode. A real-life case study is provided to evaluate the performance of the proposed method, which demonstrates its superiority over existing methods on saving the energy consumption.

Learning-based model predictive control for two-point incremental sheet forming

Two-point incremental sheet forming (TPIF) is known to provide higher geometric accuracy compared to single point incremental sheet forming (SPIF). However, it may not always ensure that the maximum allowable absolute geometric error for complex industrial parts is below 0.75 mm. To overcome this challenge, we propose an on-line learning-based model predictive control (MPC) approach to enhance accuracy in TPIF. In this proposed design, the controller parameters can be updated adaptively and effectively on-line, based on historical data, current state, and system inputs, even with unknown system dynamics in future steps. The developed controller is experimentally validated using three benchmark shapes, namely a dog-bone shape, a hemispherical shape, and a water-drop shape, each with different geometric features, in TPIF. The experimental findings demonstrated that the percentage of errors below 0.75 mm can be significantly increased from less than 42 % without control to over 92 % with control for all the tested shapes, while maintaining the safety of the manufacturing process. This highlights the effectiveness and robustness of the developed MPC controller in achieving higher precision for complex parts. This will unlock TPIF capability to form custom high-accuracy parts, thereby enabling ISF as a commercially viable process with increased robustness and accuracy in the future.

Deep learning based model predictive control of active filter inverter as interface for photovoltaic generation

AbstractBy increasing the photovoltaic (PV) systems capacity worldwide, the requirement for a fast, reliable, and efficient control system is becoming more crucial. To this end, model predictive control (MPC) is known as one of the potential solutions. Although MPC is an easily implemented control system, it needs a high computational complexity due to the dependency on solving an iterative optimization problem. To overcome this problem, this study develops an artificial intelligence‐based on one‐dimensional convolutional neural network (1D‐CNN) based MPCs. While 1D‐CNN benefits from the inherent strong feature extraction/selection capability and lower computational complexity than other deep methods, it still cannot properly track the dynamic changes due to fixed weights during the training process. Thus, this paper integrates the dynamic weighting training process and proposed dynamic weighing 1D‐CNN for the implementation of intelligent MPC for the PVs. The numerical results based on different load types show an efficient performance of the proposed system and verify the superiority of the proposed method in comparison with the conventional MPC and several state‐of‐the‐arts shallow and deep based MPC for the PVs in terms of the total harmonic distortion (THD) and frequency switching.

Deep learning-based model predictive control for real-time supply chain optimization

This paper presents a deep learning-based model predictive control (MPC) method for operational supply chain optimization in real time. The method follows an offline-online procedure. In the offline phase, the state-space model of a supply chain system is developed and the MPC problem for supply chain operation is formulated. Then, the MPC problem is solved for a set of initial states to obtain the corresponding optimal inputs. A deep neural network (DNN) is built and trained by using the optimal state-input pairs as training data to approximate the optimal MPC law. In the online phase, the DNN controller is employed to provide real-time decisions. In this paper, the MPC problem for supply chain operation is formulated as a mixed-integer linear program. A deep learning-based MPC method is proposed to accommodate time delays in the system. Moreover, a heuristic method is proposed for feasibility recovery with the binary decision variables taken into account. The training set for the DNN controller contains two subsets, one formed from MPC solutions corresponding to random initial states, and the other formed from optimal state-input pairs in closed-loop simulations. The deep learning-based MPC is validated via two case studies through closed-loop simulation. The first case study involves a linear MPC, and the second case study involves a more complicated mixed-integer linear MPC. Results show that deep learning-based MPC can achieve a high accuracy in approximating the MPC decisions and a significant reduction in the online computation time. Compared with MPC, the average performance loss of using deep learning-based MPC in the two cases is 0.43% and 1.8%, respectively.

A learning-based model predictive strategy for pH control in raceway photobioreactors with freshwater and wastewater cultivation media

This article presents a study on the pH control of raceway photobioreactors (PBRs) using a learning-based model predictive control (LBMPC) approach. The primary objective is to enhance the control performance and improve microalgae biomass production. The research builds upon existing control solutions for PBRs found in the literature by integrating the robust and adaptive capabilities of the LBMPC. Specifically, the control algorithm is modified to include a pre-established Linear Quadratic Tracking with a Feedforward controller, facilitating the formulation of the conventional LBMPC. To validate the proposed controller, implementation is performed in an existing facility at the IFAPA center, located near the University of Almería in Southern Spain. The experiment involves two parallel PBRs operating under the same meteorological conditions with different microalgae strains and media (freshwater and wastewater). The LBMPC demonstrates satisfactory results and is compared with the conventional nominal MPC strategy through an analogous experiment. The LBMPC outperforms the conventional approach, achieving up to four times superior performance in terms of the average error index. The results highlight the importance of employing robust adaptive control strategies for highly nonlinear and multi-disturbed systems like the variant biological-chemical microalgae process in PBRs since the complexity associated with adapting the system’s model can negatively impact the performance of conventional model-based controllers.

Machine learning-based model predictive controller design for cell culture processes.

The biopharmaceutical industry continuously seeks to optimize the critical quality attributes to maintain the reliability and cost-effectiveness of its products. Such optimization demands a scalable and optimal control strategy to meet the process constraints and objectives. This work uses a model predictive controller (MPC) to compute an optimal feeding strategy leading to maximized cell growth and metabolite production in fed-batch cell culture processes. The lack of high-fidelity physics-based models and the high complexity of cell culture processes motivated us to use machine learning algorithms in the forecast model to aid our development. We took advantage of linear regression, the Gaussian process and neural network models in the MPC design to maximize the daily protein production for each batch. The control scheme of the cell culture process solves an optimization problem while maintaining all metabolites and cell culture process variables within the specification. The linear and nonlinear models are developed based on real cell culture process data, and the performance of the designed controllers is evaluated by running several real-time experiments.

Machine learning based model predictive control for grid connected enhanced switched capacitor cross‐connected switched multi‐level inverter (ESC3SMLI)

AbstractThis article describes an enhanced switched capacitor cross‐connected switched multilevel inverter (ESC3SMLI) with a machine learningbased model‐predictive control method (ML‐MPCM). The proposed ESC3SMLI produces nine levels using eight switches, including two bidirectional switching devices, a single DC source, and a capacitor. Additionally, the design generates a negative level without the use of extra circuitry like an H‐bridge, which implies that switches in ESC3SMLI are less subject to voltage stress. In comparison to a conventional H‐Bridge setup, only 3 switches conduct for each operating mode, leading to fewer switching transitions, reduced switching and conduction losses, and better efficiency. The exponential rise in computational complexity required to perform the optimisation, which consumes an unacceptably high quantity of computing resources, is the most important drawback of MPCMs. This article examines inverters static and dynamic behaviour since grid‐connected utility is intended. In specific, ESC3SMLI is controlled with high accuracy using the artificial neural network (ANN) model that was trained offline using the information gathered from the conventional MPC method. The rapid and accurate reaction, as well as the superior function, of the control scheme is demonstrated by its dynamic performance during sudden shifts in current and PF.

A practical approach based on learning-based model predictive control with minimal prior knowledge of patients for artificial pancreas

Background and objectivesComplete identification of the glucose dynamics for a patient generally requires prior clinical procedures and several measurements for the patient. However, these steps may not be always feasible. To address this limitation, we propose a practical approach integrating learning-based model predictive control (MPC), adaptive basal and bolus injections, and suspension with minimal requirements of prior knowledge of the patient. MethodsThe glucose dynamic system matrices were periodically updated using only input values, without any pretrained models. The optimal insulin dose was calculated based on a learning-based MPC algorithm. Meal detection and estimation modules were also introduced. The basal and bolus insulin injections were fine-tuned using the performance of glucose control from the previous day. To validate the proposed method, evaluations with 20 virtual patients from a type 1 diabetes metabolic simulator were employed. ResultsTime-in-range (TIR) and time-below-range (TBR) were 90.8% (84.1% – 95.6%) and 0.3% (0% – 0.8%), as represented by the median, first (Q1), and third quartiles (Q3), respectively, when meal intakes were fully announced. When one out of three meal intake announcements was missing, TIR and TBR were 85.2% (75.0% – 88.9%) and 0.9% (0.4% – 1.1%), respectively. ConclusionsThe proposed approach obviates the need for prior tests from patients and shows effective regulation of blood glucose levels. From the perspective of practical implementation in clinical environments, to deal with minimal prior information of the patient, our study demonstrates how essential clinical knowledge and learning-based modules can be integrated into a control framework for an artificial pancreas.

Learning-Based Model Predictive Control for Autonomous Racing

In this paper, we present the adaptation of the terminal component learning-based model predictive control (TC-LMPC) architecture for autonomous racing to the Formula Student Driverless (FSD) context. We test the TC-LMPC architecture, a reference-free controller that is able to learn from previous iterations by building an appropriate terminal safe set and terminal cost from collected trajectories and input sequences, in a vehicle simulator dedicated to the FSD competition. One major problem in autonomous racing is the difficulty in obtaining accurate highly nonlinear vehicle models that cover the entire performance envelope. This is more severe as the controller pushes for incrementally more aggressive behavior. To address this problem, we use offline and online measurements and machine learning (ML) techniques for the online adaptation of the vehicle model. We test two sparse Gaussian process regression (GPR) approximations for model learning. The novelty in the model learning segment is the use of a selection method for the initial training dataset that maximizes the information gain criterion. The TC-LMPC with model learning achieves a 5.9 s reduction (3%) in the total 10-lap FSD race time.