Bio-Inspired Rapid Escape and Tight Body Flip on an At-Scale Flapping Wing Hummingbird Robot Via Reinforcement Learning

Abstract

Insects and hummingbirds are capable of acrobatic maneuvers such as rapid turns and tight 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> body flips. It is challenging for bio-inspired flapping wing micro aerial vehicles to achieve animal-like performance during such maneuvers due to their limitation in mechanism sophistication and flight control. Besides being significantly underactuated compared to their natural counterparts, flight dynamics is highly nonlinear with rapidly changing, unsteady aerodynamics which remains largely unknown during aggressive maneuvers. As a result, conventional model-based control methods are inadequate to address such maneuvers effectively due to the lack of control references and aerodynamic models. In addition, during acrobatic maneuvers such as body flips, conventional control methods with underlying stabilization mechanisms would temporarily contradict the maneuvering requirements when the vehicle undergoes a full body flip including turning upside-down. In this article, reinforcement learning (RL) has been used to complement model-based control to enable animal-like maneuverability. The learned control policy serves in two different ways to either aid or even completely takeover the conventional stabilization controller in certain cases. We experimentally demonstrate animal-like maneuverability on an at-scale, dual-motor actuated flapping wing hummingbird robot. Two test cases have been performed to demonstrate the effectiveness of such integrated control methods: 1) a rapid escape maneuver recorded from hummingbirds, 2) a tight 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> body flip inspired by houseflies. By leveraging RL, the hummingbird robot demonstrated a shorter completion time in escape maneuvers compared to the traditional control-based method. It also performed 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> body flip successfully within only one wingspan vertical displacement.