DNN Patching: Progressive Fixing and Augmenting the Functionalities of DNNs for Autonomous Vehicles

Abstract

Autonomous vehicles have become one of the most-awaited technologies of the century. In the recent years the advances in the area of deep learning and artificial intelligence enabled rapid development of the autonomous driving sector. Some car companies have already deployed autonomous driving software into their vehicles and millions of customers are using them. Although the autonomous driving modules are typically extensively tested prior to its release on the market, the customers can either still encounter certain previously overlooked failure cases or identify missing functionalities of the model. In this letter we focus on developing the methodology for repairing the deep neural network (DNN) steering the self-driving vehicle in response to these two scenarios via patching such that the following conditions are satisfied: i) the patch is added on the top of the original DNN to extend or fix its functionality for the particular failure case, ii) the patched model keeps the original functionalities of the DNN unchanged, and iii) the size of the patch is significantly smaller than the size of the original DNN. Our approach avoids training and evaluating the network from the scratch to improve or fix it, effectively enabling fast response of the autonomous vehicle manufacturers to the customer needs. We demonstrate the plausibility of the proposed approach on the task of autonomous driving in the Carla simulator A. Dosovitskiy <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al</i> .