Disengagement Cause-and-Effect Relationships Extraction Using an NLP Pipeline

Abstract

The advancement in machine learning and artificial intelligence promotes the testing and deployment of autonomous vehicles (AVs) on public roads. The California Department of Motor Vehicles (CA DMV) has launched the Autonomous Vehicle Tester Program, which collects and releases reports related to Autonomous Vehicle Disengagement (AVD) from autonomous driving. Understanding the causes of AVD is critical to improving the AV system’s safety and stability and providing guidance for AV testing and deployment. In this work, we built a scalable end-to-end pipeline to collect, process, model, and analyze the disengagement reports released from 2014 to 2020 using natural language processing and deep transfer learning. The analysis of disengagement data using taxonomy, visualization, and statistical tests revealed the trends of AV testing, cause frequency, and significant relationships between causes and effects of AVD. We found that (1) manufacturers tested AVs intensively during the Spring and/or Winter, (2) test drivers initiated more than 80% of the disengagement while more than 75% of the disengagement were because of errors in perception, localization & mapping, planning and control of the AV system, and (3) there was a significant relationship between the initiator of AVD and the cause category. This study serves as a successful practice of deep transfer learning using pre-trained models and generates a consolidated disengagement database allowing further investigation for other researchers. The related code and data are available on github. <xref ref-type="fn" rid="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</xref>