Sample sizes, scoops and educational science

Abstract

It seems that lately I am being consumed by an obsession with philosophy of science, and the nature (epistemology) of research in medical education. Perhaps it is a natural consequence of a sabbatical (just finished), which often leaves you with too much time on your hands to think deep thoughts. So this is the third editorial in the series to examine an aspect of our research game. Hopefully, not too much repetition. Three seemingly unrelated personal anecdotes, both recent and ancient: (1) Recently I received an e-mail from a medical student along the lines of: ‘‘Dear Dr. Norman. I have a study I want to do. I’ve met with the chair of the undergraduate program to discuss it, and he suggested I meet with you to do a sample size calculation’’. I wrote him back indicating that, while I can meet with him, meetings of this sort may start out as sample size calculations but rarely end up that way, as I may have a number of suggestions about how to do the study differently. What I wanted to say, but didn’t, was A) I’m not your neighbourhood sample size monkey. Go and Google something to get a sample size, why don’t you? B) Why do you think that your question, as yet unmentioned, deserves the resources needed to conduct a study? Have you looked at the literature? Have you read papers on it? Why do you think your bright idea deserves any attention until you do this? On reflection I realized that I was being overly sensitive, and this was not the way I would have gone about it a few years ago. I would have welcomed him into my office; thrilled that he showed the initiative to get interested in education research. We would have talked about sample selection, research design, measures, analysis approaches and so on. We would NOT have talked about whether the question was worth asking. My role was to encourage research in education—ALL research in education. I may still do that, now that I’ve calmed down. (2) Last week, I was reviewing study questions generated by students in our introductory epidemiology course. One student wanted to solve all the problems of childbirth in the developing world; another wanted to do a horrendously complex study of remissions in