Abstract
Torque control of electric drives is a challenging task, as high dynamics need to be achieved despite different input and state constraints while also pursuing secondary objectives, e.g., maximizing power efficiency. Whereas most state-of-the-art methods generally necessitate thorough knowledge about the system model, a model-free deep reinforcement learning torque controller is proposed. In particular, the deep Q-learning algorithm is utilized which has been successfully used in different application scenarios with a finite action set in the recent past. This nicely fits the considered system, a permanent magnet synchronous motor supplied by a two-level voltage source inverter, since the latter is a power supply unit with a limited amount of distinct switching states. This contribution investigates the deep Q-learning finite control set framework and its design, including the conception of a reward function that incorporates the demands concerning torque tracking, efficiency maximization and compliance with operation limits. In addition, a comprehensive hyperparameter optimization is presented, which addresses the many degrees of freedom of the deep Q-learning algorithm striving for an optimal controller configuration. Advantages and remaining challenges of the proposed algorithm are disclosed through an extensive validation, which includes a direct comparison with a state-of-the-art model predictive direct torque controller.
Highlights
The utilization of sophisticated and highly tailored modelbased controller designs has been a standard in electric drive control for the past fifty years
The available AC drive control methods range from linear proportional-integral (PI) controllers based on the concept of field-oriented control (FOC) [1] and direct torque control (DTC) [2] to model predictive control (MPC) [3]
The suggested deep Q direct torque control (DQ-DTC) approach can be optimized in terms of the deep Q-network (DQN) degrees of freedom
Summary
The utilization of sophisticated and highly tailored modelbased controller designs has been a standard in electric drive control for the past fifty years. In continuouscontrol-set (CCS) problems, the optimization problem in (1) has to be solved considering real-valued, continuous input voltages u This applies whenever the drive voltage source inverter receives its input signals from an intermediate modulator. This contribution focuses finite-control-set (FCS) problems where the control output is limited to discrete actions. While first promising CCS-RL approaches for the current control problem of drives are already available [18]–[20], additional control tasks, such as torque or speed control, and learning FCS approaches have not yet been investigated. The question arises if learning, model-free controllers can be successfully applied to further, challenging control tasks in drive and power electronic applications
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have