Fast and curious: A model for building efficient monitoring- and decision-making frameworks based on quantitative data

Abstract

Nowadays, the hype around artificial intelligence is at its absolute peak. Large amounts of data are collected every second of the day and a variety of tools exists to enable easy analysis of data. In practice, however, making meaningful use of it is way more challenging. For instance, affected stakeholders often struggle to specify their information needs and to interpret the results of such analyses. In this study we investigate how to enable continuous monitoring of information needs, and the generation of knowledge and insights for various stakeholders involved in the lifecycle of software-intensive products. The overarching goal is to support their decision making by providing relevant insights related to their area of responsibility. We implement multiple monitoring- and decision-making frameworks for six individual, real-world cases selected from three different platforms and covering four types of stakeholders. We compare the individual procedures to derive a generic process for instantiating such frameworks as well as a model to scale it up for multiple stakeholders. For one, we discovered that information needs of stakeholders are often related to a limited subset of data sources and should be specified in stages. For another, stakeholders often benefit from sharing and reusing existing components among themselves in later phases. Specifically, we identify three types of reuse: (1) Data and knowledge, (2) tools and methods, and (3) concepts. As a result, key aspects of our model are iterative feedback and specification cycles as well as the reuse of appropriate components to speed up the instantiation process and maximize the efficiency of the model. Our results indicate that knowledge and insights can be generated much faster and stakeholders feel the benefits of the analysis very early on by iteratively specifying information needs and by systematically sharing and reusing knowledge, tools and concepts.