Comparative evaluation for alternative variable importance rankings for pedestrian injury severities

Abstract

Little research is dedicated to evaluating the performance difference of various metrics in ranking predictor importance in the traffic safety field. To this end, the main objective of the current paper is to evaluate and quantify different methods for sorting the variable importance related to crash severity. A comprehensive database for pedestrian-related crashes in the state of California was developed. Four popular measurement metrics used in the past were chosen for evaluation purpose: Mean Decrease Accuracy (MDA), Mean Decrease Gini (MDG), log-likelihood ratio test associated with multinomial logit model, and Principal Component Analysis (PCA). The former two metrics come under the same umbrella of the Random Forest (RF) technique, while the latter two are methods belonging to different domains. The results show the alternative methods yield different variable importance rankings with PCA being isolated from others. The two methods under the same domain of the random forest, or MDG and MDA, have the most common results, but still reveal a 17% ranking difference. It is anticipated that the results could raise more awareness of the importance of selecting the appropriate metrics to evaluate the predictor importance from different perspectives.