Abstract
The paper proposes a formal approach for describing and evaluating the datasets that are used in automotive applications for machine learning, testing, and validation purposes. Proper, that is, qualitative and quantitative characterization of the datasets can simplify the analysis, evaluation, and comparison of perception-based algorithms designed for highly automated vehicles. Such formalism is also needed to achieve compliance with the automotive industry safety standards that have been recently introduced. Characterization in the form of size or type of raw data, number of recognized and classified objects, and environmental parameters is not perfectly suitable for describing both the static and dynamic aspects of automotive datasets; therefore, another approach is required. In this paper, an efficient method based on an object tracking mechanism, grid representation of the sensor field of view, heatmap concept, and Wasserstein metric is proposed. The efficiency of the method is demonstrated by its ability to handle both the size, properties, and diversity of the dataset, including static and time-varying aspects. The presented description can also be used to compare different datasets and to define the amount of data to be collected.
Highlights
As special skills and the ability to properly perceive the surrounding environment are required by human beings driving cars, artificial intelligence (AI) together with machine Learning learning (ML) are essential to achieve higher levels of autonomy
An additional benefit of this approach is that the same seed is sufficient to generate variations in real-world user profiles (RWUPs) data for a given distribution, for example, the same set of scenes rendered for different countries, the same set of scenes rendered for different sun heights, and the same set of scenes rendered for different weather conditions and light illumination
In this paper, an efficient method to formally characterize automotive datasets used for perception-based system development and verification is presented
Summary
The development of new generation cars with higher level of automation will require solving many research problems. One of the most challenging problems in the development of automotive control systems is defining the performance indicators to be used to evaluate and compare different algorithms on a given dataset. This is important for solutions that are based on machine learning techniques where the data are categorized into learning, test, and validation subsets. Evaluation of active driving assistance systems in newest cars performed by researchers at AAA shows that on over 4000 miles of real-world driving on average some type of issue occurs every eight miles [4] This discrepancy shows that validation datasets do not reflect reality correctly. The pipeline of the proposed methodology is depicted in figure 1
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
