Is designed data collection still relevant in the big data era?

Abstract

AbstractThe excitement and buzz around big data (BD) has been so prevalent that it seems BD has become thought of as a panacea for solving all challenges. But there are in fact limitations to what BD can do, which has led to missed opportunities and potentially suboptimal problem‐solving and decision‐making. Hence the answer to the question posed in the title is an emphatic ‘Yes!’ The intentional strategic use of designed data collection (DDC) represents an opportunity to elevate how studies involving BD are planned, executed, and interpreted to deliver more satisfying solutions. In this paper, we outline eleven opportunities for integrating DDC strategies, including statistical design of experiments (DoE) and sampling techniques, with BD at all stages of the study. There are opportunities for building background understanding of the problem and improving the quality of the data via strategic planning before acquiring BD. In addition, incorporating DDC into the collection and processing of the actual BD, and strengthening conclusions or augmenting with supplementary designed data after BD also represent key opportunities to increase the impact of BD in the era of Industry 4.0 and Quality 4.0.