Discussion of ‘Is designed data collection still relevant in the Big Data era?'

Abstract

AbstractGiven the popularity of Big Data (BD), there can be an impression that fields such as design of experiments (DOE) are now irrelevant. We would like to thank the authors for starting the conversation about the possible relationship between these two fields. A key contribution of this paper is in showing how DOE principles, as summarized under the name designed data collection (DDC), can be applied throughout the BD process. This name is quite appropriate, demonstrating that these principles apply not just to designed experiments, but to any form of data collection. This is especially important for situations where designed experiments are either impossible (i.e., assessing how a country's economy may impact certain responses) or unethical (i.e., certain sensitive types of medical studies). It shows that DOE is more than a particular choice of design type, but is rather a methodology for approaching data collection. One that seeks to extract the most relevant information from the data while also taking into account the various nuances and constraints of physical and social processes, which are ever present, even in massive datasets. The paper divides BD efforts into three general phases: Before BD, During BD, and After BD. As such, we have grouped our discussion accordingly, with general comments provided for the suggested contributions of DDC in each phase. We then close with some additional thoughts.