Stats NZ Integrated Data Infrastructure (IDI) – Back to the Future: How a prototype design developed into a world leading tool for critical decision making

Abstract

IntroductionStats NZ maintains and provides access to the Integrated Data Infrastructure (IDI). This is a critical tool in the government's efforts to reduce poverty, improve health, and support a growing, inclusive economy. Insights are used at all levels of government decision making.
 Objectives and ApproachSince the IDI prototype was created in 2011, it has grown rapidly. The use cases for the IDI continue to develop, with more complex projects that extend the boundaries of what might be considered traditional research. The Ministry of Education is now running a key school funding model, the Equity Index, out of the IDI. We are seeing an increased interest for the IDI to support fast responses to policy questions. Statistical production areas in and outside Stats NZ increasingly need administrative data for production, and the IDI is the best source of relevant linked administrative data.
 ResultsStats NZ has received one-off funding to continue sustainable expansion of the IDI and improve the data and services it provides. This ‘IDI 2’ project will enhance the IDI, making it easier, quicker and more stable for our customers to engage with data. We have focussed on strengthening, and futureproofing, our core offering. The IDI 2 project is currently looking to replace the systems that are used to build our databases and streamline production to provide the technical capability to cope with a much higher volume of data.
 Conclusion / ImplicationsWhile the IDI 2 project looks to shore up the infrastructure and lay a platform for future development, Stats NZ seeks to undertake customer focussed improvements to be responsive to other customer needs. There are opportunities to grow the integrated data community and provide more ways for researchers to connect with each other.