Adverse weather simulation for training neural networks

Abstract

Convolutional neural networks generally require considerable amount of data for training to perform adequately well in all real-world scenarios. Many times, the data for all scenarios is hard to collect and ground truth annotation is also a challenge. Similar problem exists in training networks for the autonomous vehicles given the diverse weather conditions where these cars are expected to be driven. Thus, a synthetic data generation model is imperative and we go about building a weather simulation framework. This framework is intended to generate weather conditions over different driving scenarios. To start with, we go about implementing a completely configurable rain/fog/windshield simulation model. The scope of this framework, however is much more than these three models. Apart from refining these models further as and when need, we intend to build in mechanisms to simulate more diverse weather conditions within this framework. There are multiple challenges in the implementation of these models. To begin with, we need a mechanism to simulate diverse weather conditions in a driving environment. One method could be to simulate the entire 3D environment, with the roads, automobiles, and an artificial world, but this approach would be extremely challenging both in terms of the realism that can be achieved, and in terms of the time it would take for the implementation. Another method is to overlay the rain/fog effect on top of pre-rendered videos. This 2D overlaying technique is a practical solution, since there exist many driving videos at our disposal. In this paper, we outline methods to implement this effectively, and showcase the results obtained in training a Neural Network with this approach.