Foreword: The Moment of Discovery: How Do You Know When You Hit a Question That's Pure Gold?

Abstract

When asked about the breakthrough that led to the polio vaccine, Jonas Salk once commented, “What people think of as the moment of discovery is really the discovery of the question.” So how do scholars “discover” questions, and, perhaps more important, how do scholars know when they land on a question worth pursuing? The members of the 2019 Research in Medical Education (RIME) Program Planning Committee selected research questions as a theme for the year and hope that through the commentaries and papers presented in this supplement and the presentations at the RIME sessions of the Learn Serve Lead meeting, our community will illuminate questions to guide us toward breakthroughs on some of our gnarliest problems in health professions education. In this foreword to the RIME supplement, we—the past, present, and future RIME chairs—offer a few overarching thoughts about research questions based on our own experiences of teaching, reading, reviewing, generating, and discussing such questions with colleagues in the field. What Makes a Research Question Different From Other Questions? We ask all sorts of questions that serve a variety of purposes. In medical school, students learn to ask questions to establish rapport with a patient, elicit information to guide diagnosis, and check understanding of, or commitment to, a plan. In faculty development sessions, educators learn to ask questions that push a learner’s thinking in a supportive way, set the stage for feedback, and help them identify a learner’s goals, needs, and abilities. In quality improvement, health professionals learn to ask Why 5 times to get to the core of a problem.1 While all these types of questions are valuable and share some common features (e.g., open-ended, probing the unknown), they are not research questions. A research question straddles the realm of known and unknown. It recognizes a base of knowledge but finds a crack in the foundation or offers a different perspective on current interpretations that opens up new territory for exploration. The question cannot be answered by a single person or by merely looking up information. Rather, it requires a systematic process of collecting and analyzing new information. Typically the question is novel—not something that has been asked before or at least not in this way or in this context. A research question strikes a balance between a general idea and specific construct, meaning it must identify a problem and define a piece that can be studied in a reasonable amount of time given available resources, knowledge, and principles or methodologies identified as appropriate within the specific research community. Yet, it must not be so specific that it makes too minimal of a contribution or has limited relevance or generalizability. A research question is not the same as interview questions or survey items. While the difference may seem obvious, it is not uncommon for investigators to move back and forth between specific interview or survey questions and the broader research question in an attempt to find the optimal level and scope. What initially seems like an important research question may ultimately become a single question in an interview guide that explores a larger phenomenon. Alternatively, an observation might prompt a question that seems straightforward and perhaps not worthy of a whole study, but the initial investigation unearths complexities that warrant further pursuit. Who Generates the Best Research Questions? The “Aha! moment” appeals to us all, but such moments of inspiration rarely happen without the 99% perspiration expended in preparation. For the 2019 RIME plenary, we invited members of the RIME community who we know have invested substantial portions of their careers in tackling some of the most challenging problems in health professions education. Their deep knowledge of faculty development, feedback, systems of assessment, and equity pedagogy, coupled with their creativity and passion as educators and researchers, makes them well poised to guide the RIME community in identifying key questions to push the frontiers of research in health professions education. That said, our choice of panelists is not meant to suggest that all the best questions come from experts. By contrast, many of the papers selected for RIME come from newer investigators and teams of investigators who bring novel observations, fresh perspectives, and infinite curiosity to bear in ways that spark excellent research questions. Ultimately, crafting a promising research question takes preparation, patience, perseverance, and passion—regardless of who you are. How Do You Know a Promising Research Question When You See It? Scholars can find countless resources to help them write a good, effective research question.2–5 These resources emphasize criteria such as relevance, interest, originality, clarity, feasibility, answerability, and generalizability. Many also underscore the importance of situating the question in a theoretical or conceptual framework. While these criteria offer reasonable guidance, each must be interpreted in context, and this makes it far more difficult to evaluate a research question’s prospects than those suggested by criteria. A research question deemed promising in one field, by a particular audience, or for a certain purpose, might fall flat when presented in another field, to a different audience, for another purpose. For example, a cognitive psychologist may consider questions about how undergraduate psychology students integrate and retain basic science information under 2 different instructional strategies highly relevant to medical education, while a medical educator might view the question with skepticism, largely due to generalizability and, correspondingly, relevance. Or, as another example, 1 reviewer might doubt the originality of a question about learners struggling with transitions in training while another reviewer appreciates the novel context or approach to the question (e.g., applying a different theoretical lens, asking the question at a structural level rather than at an individual level, or comparing struggles in 2 different types of programs). As such, it is also the job of a scholar to clearly articulate the motivation and relevance of the research question—what are the gaps, and how does this question add to the scholarly discussions and advances in the field? A promising research question would explain these facets in a way that is appreciated by the reviewer and the audience. The type and level of research question also factor into the promise it holds. One commonly used typology characterizes 4 types of questions based on their function: exploratory, explanatory, descriptive, emancipatory.6 Some problems may be newly emerging and thus better suited to descriptive and exploratory questions, while others are better understood and ready for explanatory questions or empowerment and transformation through emancipatory questions. Additionally, as discussed in greater detail in the RIME commentary, the level or focus of a question can range from micro (individual level; highly contextualized view of a phenomenon) to meso (interpersonal or interactional level; bridging theory and practice to study a phenomenon) to macro (cultural, societal, or system level; philosophical or theoretical aspects of a phenomenon).7,8 Many topics can be studied from any of these levels, which makes it all the more important for investigators to specify the position of their question relative to appropriate landmarks. If the terrain is well mapped at 1 level (e.g., we have much evidence of individual biases in various circumstances), the optimal questions may need to begin exploring other levels (e.g., we have little evidence of ways in which structural and cultural changes affect bias). Health professions education is an interdisciplinary field, with scholars who have varying backgrounds in research paradigms, theoretical frameworks, and methodologies. This diversity can make it even more challenging for our community to decide what constitutes a promising research question. While some members of our community may not be particularly bothered by the “epistemological anarchy” Eva uses to characterize our field,9(p1100) others see this as a signal that we need standards and expectations for research questions along with the supporting methodologies that can address the questions in an appropriate manner, acceptable to the interdisciplinary health professions education community. For example, Ringsted and colleagues orient readers to different types of research questions as they discuss different research purposes and approaches, along with the questions and methodologies suited to each.5 Of note—published research questions look neat and tidy. When presented this way, readers often assume that the investigators started out with a clear, insightful question. In some cases, this is true; in many, it is not. The research question can be a fluid entity rather than a fixed one. In qualitative research and certain types of reviews, the research question may morph over time as the investigator interacts with participants and/or literature, develops a better understanding of the problem, and reflects on their own assumptions and interpretations.2 Even in research that begins with hypotheses or specific questions for a meta-analysis or other types of reviews, the question presented in the final manuscript might differ from the one posed at the outset. The scope of the question might be adjusted, or questions might be added or modified to accommodate an unexpected finding. In this regard, the scale and specificity of research questions also depend on the state of the supporting literature. Some questions can be more nuanced and specific to address a particular, yet significant, gap in knowledge, whereas in other contexts, the question can also be broader in scope perhaps for topics that are still in earlier stages, prompting the study findings to yield additional questions and a research agenda for the community. A Few Parting Thoughts Many scholars find it easy enough to generate questions; far fewer find it easy to choose the ones worth pursuing. Investing time and effort in the research questions, both upfront and throughout the research process, matters. Agee astutely notes, “Good research questions do not necessarily produce good research, but poorly conceived or constructed questions will likely create problems that affect all subsequent stages of a study.”2(p431) And yet, our purpose is not to set such high standards that scholars feel all research questions must be pure gold. Such standards would likely squelch many good prospects for research. Rather, our intent is to minimize the risk of mistaking glitter for gold by encouraging careful consideration of the purpose, value, and substance of our questions. One promising approach may be to think about the research question as part of a program of research rather than an isolated study. Such an approach pushes scholars and the community as a whole to think of questions as layers or links in a chain that contribute to a larger purpose and build value over time. It also can enrich the connections within our community by illuminating opportunities to work together on a common research agenda that may benefit from the contributions that different levels of focus, theoretical orientations, and methodologies have to offer. Another, related approach might be to reconsider how we read and write the “suggested directions for future research” sections of papers. How often do we, as readers, actually use these suggestions as a starting point for a study? And, when we write these recommendations, how often do we actually write them with an expectation that others might take them up and build on them? As a starting point, we offer this as an opportunity for scholars to engage in the thought experiment, asking “What’s the next question?” as a way to deepen and refine their questions. We hope you enjoy delving into the many questions posed throughout the RIME supplement and the RIME meeting. May they inspire many promising insights and fruitful questions to enrich our field!