Sample representativeness and influence of attrition on longitudinal data collected as part of a national medical career tracking project

Abstract

BackgroundThe Medical Schools Outcomes Database and Longitudinal Tracking Project (MSOD) in New Zealand is one example of a national survey-based resource of medical student experiences and career outcomes. Longitudinal studies of medical students are valuable for evaluating the outcomes of medical programs against workforce objectives. As a prospective longitudinal multiple-cohort study, survey response rates at each collection point of MSOD vary. This paper assesses the effects of participant non-response rates on MSOD data.MethodsDemographic variables of MSOD respondents between 2012 and 2018 were compared to the distribution of the demographic variables in the population of all NZ medical graduates to ascertain whether respondent samples at multiple survey collection points were representative of the population. Analysis using logistic regression assessed the impact of participant non-response on variables at collection points throughout MSOD.Results2874 out of a total population of 2939 domestic medical students graduating between 2012 and 2018 responded to MSOD surveys. Entry and exit surveys achieved response rates around 80% and were broadly representative of the total population on demographic variables. Post-graduation survey response rates were around 50% of the total population of graduates and underrepresented graduates from the University of Auckland. Between the entry and exit and the exit and postgraduation year three samples, there was a significant impact of non-response on ascribed variables, including age at graduation, university, gender and ethnic identity. Between the exit and postgraduation year one sample, non-response significantly impacted ascribed and non-ascribed variables, including future practice intentions.ConclusionSamples collected from MSOD at entry and exit are representative, and findings from cross-sectional studies using these datasets are likely generalisable to the wider population of NZ medical graduates. Samples collected one and three years post-graduation are less representative. Researchers should be aware of this bias when utilizing these data. When using MSOD data in a longitudinal manner, e.g. comparing the change in career intentions from one collection point to the next, researchers should appropriately control for bias due to non-response between collection points. This study highlights the value of longitudinal career-tracking studies for answering questions relevant to medical education and workforce development.