Validation of Double Transition Model by Analyzing Reward Distributions

Abstract

To keep up with the world’s increasing complexity, we need ever-more sophisticated computational models that can help to make wiser decisions and accurately predict potential consequences. Validating and explaining these models is difficult, but very crucial in understanding their effectiveness and applicability. In this paper, we validate a computational model called the Double Transition Model by using data from the SCOUT dataset that captures human behaviors from pre-scripted scenarios controlling unmanned aerial vehicles. By applying the Double Transition Model on collected raw data, we gain insights about complex human decision-making processes after comparing the model’s behavior to reward distributions that are extracted using inverse reinforcement learning. We examine the reward distributions by addressing the following questions: 1) How are reward distributions different between individuals? 2) Do reward distributions follow the normal distribution? If not, how far do they deviate? 3) What information do these reward distributions represent? To capture different facets of model complexities, we compute both Kolmogorov Complexity and Information Entropy. From our empirical analysis, we obtained high correlations between the reward distributions and the complexities, thus confirming the validity of the Double Transition Models in capturing the dynamics of the SCOUT dataset correctly.