Deep reinforcement learning for active control of flow over a circular cylinder with rotational oscillations

Abstract

An active control strategy for flow over a circular cylinder with rotational oscillations is automatically learned based on a deep reinforcement learning algorithm. Specifically, the proximal policy optimization is used to minimize the averaged drag coefficient of cylinder by properly selecting the rotational angular velocity. The data for training is sampled by direct numerical simulation of flow past a rotational circular cylinder, which is recognized by the immersed boundary method and the active control is applied to the flow in real time. The flow configurations are observed by the detecting probes located inside them and modified by the agent of deep reinforcement learning through changing the angular velocity with time. After several episodes of training, the reinforcement learning agent is shown to discover a robust and convergent control strategy, by which the cylinder is able to choose an appropriate rotational angular velocity to modify the wake based on its observation of the instantaneous flow field, resulting in a substantial drag reduction. The network architecture is tested at different Reynolds numbers, i.e. 100, 200 and 300, and drag reduction rates of 10%, 19% and 21% are obtained, respectively. Effects of initial flow field and angular velocity, position and number of detecting probes and hyper-parameters of the deep reinforcement learning architecture are also examined.