We present a quantitative performance analysis of a wide range of state-of-the-art object detection models, such as Mask R-CNN He et al.(2017)[8], RetinaNet Lin et al.(2017)[17] and EfficinetDet Tan et al.(2019)[28] in haze affected environments. This work uses two key performance metrics (Mean Average Precision and Localised Recall Precision) to provide a nuanced view of real world performance of these models in an on-road driving application. Our findings show that the presence of haze further exacerbates the performance differences between single-stage and multi-stage detection models. In addition, not all aspects of the model performance are affected equally. The inclusion of Local Recall Precision (LRP) Oksuz et al.(2018)[21] suggests that more recent models have much improved localisation performance even with similar false negative and false positive results. We also highlight some of the inherent limitations of Neural Network based approaches that could be addressed by Bayesian Neural Networks in the future.
Read full abstract