ABSTRACT The popularity of OECD’s PISA tests as benchmarks for shaping global and national education policies is premised on its reputation for developing technically robust surveys by engaging leading experts who are untouched by politics. Drawing on the Science and Technology Studies (STS) subfield of the sociology of measurement, we illustrate how the view of these surveys as a-political and purely technical achievements is a myth. Using detailed documentary analysis supported by interviews with PISA insiders, we describe the interplay between technical expertise and political expertise in the development of the 2018 surveys of global competence. This paper highlights the challenge for the PISA insider group who must negotiate how to maintain scientific authority whilst gaining political backing and trust. We refer to the key strategy of ‘tinkering’ that is required to keep various actors aligned to the project through stages of development and moments of uncertainty.