Dynamic data science and official statistics

Abstract

AbstractMany of the challenges and opportunities of data science have to do with dynamic factors: a growing volume of administrative and commercial data on individuals and establishments, continuous flows of data and the capacity to analyze and summarize them in real time, and the necessity for resources to maintain them. With its emphasis on data quality and supportable results, the practice of Official Statistics faces a variety of statistical and data science issues. This article discusses the importance of population frames and their maintenance; the potential for use of multi‐frame methods and linkages; how the use of large scale non‐survey data may shape the objects of inference; the complexity of models for large data sets; the importance of recursive methods and regularization; and the benefits of sophisticated spatial visualization tools in capturing spatial variation and temporal change. The Canadian Journal of Statistics 46: 10–23; 2018 © 2017 Statistical Society of Canada