A randomized controlled trial on the nomenclature of scientific computing

Abstract

ABSTRACT Background and Context Data science and statistics are used across a broad spectrum of professions, experience levels and programming languages. The popular scientific computing languages, such as Matlab, Python and R, were organized without using empirical methods to show evidence for or against their design choices, resulting in them feeling eclectic or esoteric in their design. Objective To meaningfully organize scientific computing based on evidence gathered through user feedback, build a statistical package based on the findings and provide a replication packet to run similar studies on people with different backgrounds. Method A randomized controlled trial using a weighted, ranked choice survey (n = 118) with between-subjects design having two independent variables: Language Group (Matlab, Python and R) and Method Name options. Our dependent variable was a normalized preference rating. Findings There was a very small interaction between Language Group and Method Name. Language Group did not have a statistically significant effect, but Method Name did (F(4, 27037) = 2211.23, p < .001)( η p 2 = .247). Finally, many names in Matlab, Python and R were ranked so poorly that they were not statistically significantly different from a random word in 63.0%, 62.2% and 30.4% of concepts respectively. Implications We found organized and structured names were ranked by a large margin, suggesting statistical programming today likely needs considerable improvement. Finally, we outline a statistical package built using these principles, provide comparison scripts and describe some of the challenges from going from simple surveys to in-practice libraries.