Official Statistics and Big Data

Abstract

More than five years ago Eurostat started a project with the aim to 'tame' sources of Big Data in a way that they can be incoporated into official statistical systems. In order to solve the problems a statistician might be faced with during the official statistical application of Big Data, first of all, we give an overview of traditional data collection, and then point to the differences one has to face when dealing with Big Data. We introduce common sources of data (traditional, administrative) and highlight the ways huge sets of data are different compared to them. Next, we discuss characteristics of Big Data versus traditional statistical methods based on the qualitative criteria of official statistics, and we also elaborate on the problems of analysing Big Data. Finally, we provide a list of use cases for Big Data in official statistical data collections.