How can tobacco policy models quality be assessed: a systematic review

Abstract

Abstract Policy simulation models (PSMs) have been used extensively, both to shape health policies before real-world implementation and to evaluate post-implementation impact. However, an accepted quality assessment framework for simulation models is lacking. This systematic review aimed to develop a novel quality assessment framework for tobacco control PSMs. We searched five databases to identify peer-reviewed tobacco control PSMs that projected tobacco-use and tobacco-related outcomes from smoking policy scenarios. We extracted information on modelling inputs, structure and outputs. Using our proposed quality assessment framework, we scored models on nine dimensions: representativeness of population, policy effectiveness evidence, simulated smoking histories, simulated smoking-related diseases, exposure-outcome lag time, transparency, sensitivity analysis, validation and equity. We then compared the model score with the number of cited papers using each model. The results were narratively presented. After screening 5046 candidate papers; 145 papers were included and categorised into 25 PSMs. After scoring the models using our proposed framework, we observed that seven models were given seven and more points. The higher-scored models were generally those with a higher number of publications. While all included models had been subject to sensitivity analysis, other best practices were often lacking. Nine models did not explicitly consider smoking-related diseases. Smoking histories were commonly collapsed into crude smoking status categories rather than reflecting smoking intensity or quitting history. Furthermore, only four models estimated policy equity impact. Our systematic review revealed a variety of modelling techniques used in tobacco control. Our novel quality assessment framework offers a potential quality measure for tobacco control policy simulation models. It may guide health decision modellers and inform health policymaking. Key messages All nine framework dimensions were observed in two tobacco control PSM, suggesting the framework relevance and feasibility. However, few models achieved high scores in all dimensions. This novel quality assessment framework aims to support the development and sharing of good modelling practice and thus promote better health policy decision-making.