Evaluating the usability of open source frameworks in energy system modelling

Abstract

Usability is considered one of the key factors in determining the success of Open Source Software, but is it sufficiently addressed within the development process as of yet? Thus far, there are no studies on record that explicitly examine the usability of Open Source Software in the field of Energy System Modelling. In this paper, we publish a novel method, the Energy System Modelling Usability Testing (ESMUT), including a step-by-step guide on how to apply the method and the corresponding usability questionnaire (ESMUQ). The questionnaire is based on the Post Study System Usability Questionnaire and adopted for quantitative usability testing of Open Source Energy System Modelling frameworks. To illustrate its usage and show its applicability, we apply the method in a case study with five frameworks (Balmorel, GENeSYS-MOD, GENESYS-2, oemof, and urbs) and within a group of eight framework developers. Based on the case study results, we find that the participants were largely satisfied in working with the frameworks analysed, and identify correct handling of input data and error messages as the most frequently mentioned problems when working with the frameworks. Consequently, we find that the usability of the frameworks analysed in the case study requires further improvement. Due to the fact that only the developer perspective was taken into account, and the number of participants involved in the study was limited, further research is required to assess the usability of Open Source Energy System Modelling frameworks.