Rigorous Qualitative Research Research Articles

Enhancing Themes and Strengths Assessment: Leveraging Academic-Led Qualitative Inquiry in Community Health Assessment to Uncover Roots of Community Health Inequities.

Rigorous qualitative research can enhance local health departments' efforts to gain a deeper insight into residents' perceived community health inequities necessary for productive community health assessments (CHAs) and community health improvement plans (CHIPs). The Chicago Department of Public Health and the Partnership for Healthy Chicago used the National Association of County & City Health Officials' Mobilizing for Action through Planning and Partnerships (MAPP) model to conduct its CHA/CHIP, Healthy Chicago 2.0 (HC 2.0). Public health graduate students conducted qualitative research for part of the Community Themes and Strengths Assessment (CTSA), one of the 4 MAPP assessments. Using a health equity lens, this qualitative component included focus groups and oral histories with residents in Chicago Community Areas with the highest social and economic hardship to better understand how residents perceive health inequities in their respective neighborhoods. Community-based organizations in 6 Chicago neighborhoods with the highest quartile of social and economic hardship. Forty-eight Chicago residents from 5 community areas participated in focus groups, and 6 residents of a Mexican ethnic enclave shared oral histories. Residents' perceptions of community needs and assets. Needs identified include inaccessible resources and opportunities, economic instability, and safety. Assets include the efficacy and agency of resilient residents, as well as faith and spirituality. Systemic and institutional discrimination was identified at the roots of community health inequities. Through qualitative inquiry, the more nuanced understanding of how residents perceive health inequities better positioned HC 2.0 to develop upstream strategies in line with advanced health equity practice. Engaging qualitative academic researchers in CTSA brings academic expertise to enrich the CHA while providing real-time learning experiences to prepare future public health practitioners to work on upstream structural determinants of health.

Developing rigor in qualitative research: problems and opportunities within sport and exercise psychology

ABSTRACTQualitative research has grown within sport and exercise psychology and is now widely conducted. The purpose of this review is to discuss three commonly used ways to demonstrate rigor when conducting or judging qualitative research in sport and exercise psychology. These are the method of member checking, the method of inter-rater reliability, and the notion of universal criteria. Problems with each method are first highlighted. Member checking and inter-rater reliability are shown to be ineffective for verification, trustworthiness, or reliability purposes. Next, universal criteria within the context of Tracy’s, heavily drawn on paper within sport and exercise psychology is problematized. Throughout the discussion of each method and universal criteria more suitable possibilities for conducting rigorous qualitative research are offered. The paper concludes that to support high-quality qualitative research, scholars – including journal editors and reviewers – need to change how rigor is developed and judged, rather than perpetuate the problems with how it has been commonly evaluated in the past. Recommendations for developing rigor when conducting and/or judging qualitative research within sport and exercise psychology are also offered.

A Guide to Field Notes for Qualitative Research: Context and Conversation.

Field notes are widely recommended in qualitative research as a means of documenting needed contextual information. With growing use of data sharing, secondary analysis, and metasynthesis, field notes ensure rich context persists beyond the original research team. However, while widely regarded as essential, there is not a guide to field note collection within the literature to guide researchers. Using the qualitative literature and previous research experience, we provide a concise guide to collection, incorporation, and dissemination of field notes. We provide a description of field note content for contextualization of an entire study as well as individual interviews and focus groups. In addition, we provide two "sketch note" guides, one for study context and one for individual interviews or focus groups for use in the field. Our guides are congruent with many qualitative and mixed methodologies and ensure contextual information is collected, stored, and disseminated as an essential component of ethical, rigorous qualitative research.

Qualitative Assessment of the Symptoms and Impact of Pancreatic Exocrine Insufficiency (PEI) to Inform the Development of a Patient-Reported Outcome (PRO) Instrument.

BackgroundPancreatic exocrine insufficiency (PEI) affects patients with chronic pancreatitis (CP) and cystic fibrosis (CF) who produce insufficient digestive pancreatic enzymes. Common symptoms include steatorrhoea, diarrhea, and abdominal pain.ObjectiveThe objective of the study was to develop and test the content validity of a patient-reported outcome (PRO) instrument assessing PEI symptoms and their impact on health-related quality of life.MethodsInstrument development was supported by a literature review, expert physician interviews (n = 10: Germany 4, UK 3, France 3), and exploratory, qualitative, concept-elicitation interviews with patients with CF and CP with PEI (n = 61: UK 29, Germany 18, France 14) and expert physicians (n = 10). Cognitive debriefing of the draft instrument was then performed with patients with PEI (n = 37: UK 24, Germany 8, France 5), and feasibility was assessed with physicians (n = 3). For all interviews, verbatim transcripts were qualitatively analysed using thematic analysis methods and Atlas.ti computerized qualitative software. All themes were data driven rather than a priori.ResultsPatient interviews elicited symptoms and impacts not reported in the literature. Six symptom concepts emerged: pain, bloating, bowel symptoms, nausea/vomiting, eating problems, and tiredness/fatigue. Six impact domains were also identified. A 45-item instrument was developed in English, French, and German for testing in cognitive debriefing patient interviews. Following cognitive debriefing, 18 items were deleted.ConclusionRigorous qualitative patient research and expert clinical input supported development of a PEI-specific PRO with the potential to aid management and monitoring of unmet needs among patients with PEI. The next step is to perform psychometric evaluation of the resulting instrument.

A comparative method for themes saturation (CoMeTS) in qualitative interviews

Saturation of data has been acknowledged by many social scientists as evidence of rigor in qualitative research. Though there is a consensus about its definition and usefulness in qualitative research, the methods for achieving saturation are scarce while the social scientists who mention data saturation in their research methodologies do not adequately describe how this is achieved. This paper describes a new method for achieving saturation in studies that use interviews. The method is called Comparative Method for Themes Saturation (CoMeTS). CoMeTS is comparative in two ways. First, all themes from all interviews are compared with each other. Second, the sequence of interviews is reordered multiple times in order to check saturation again. This is because the sequence of interviews during the check makes saturation vary and, therefore, reordering helps confirming saturation. This paper concludes that CoMeTS is a simple, comparative and holistic method for achieving saturation and that it can be used in complicated qualitative studies.

The 10-Item Sickle Cell Disease Severity Measure (SCDSM-10): A Novel Measure of Daily SCD Symptom Severity Developed to Assess Benefit of GBT440, an Experimental HbS Polymerization Inhibitor

Introduction and Objective: GBT440 inhibits HbS polymerization and reduces hemolysis, and is a promising treatment for Sickle Cell Disease (SCD). The traditional clinical endpoint, vaso-occlusive crisis (VOC) requiring healthcare utilization, is not a sensitive measure of the daily burden of symptoms in patients (the "tip of the iceberg" of severe symptom exacerbations) and poses a barrier to development of new treatments for SCD.The objective was to develop a sensitive measure to detect treatment-related improvements in daily symptoms and exacerbations that define the disease. Existing SCD measures focus on distal impact, not on the core signs and symptoms of disease, and therefore not appropriate for meeting the regulatory standard to prove treatment benefit. Beginning with a hypothesized conceptual framework focused on pain and fatigue, rigorous qualitative and quantitative research was used to develop the content of the Sickle Cell Disease Severity Measure (SCDSM).Methods: The content of the new measure was derived from 1-on-1 open-ended concept elicitation interviews in 56 adults and 10 adolescents with SCD in the US and UK. Participants were aged 12-63 years, 64% female, 44% currently on hydroxyurea treatment, 86% had 1 to 42 VOC in the previous year requiring urgent care. During the interview, 74% reported having a typical or "good" day versus 26% having a crisis or "bad" day.Interview transcripts were coded to identify the most bothersome and frequently mentioned concepts related to both "good" and "bad" days. As hypothesized, patients reported having crisis days more frequently than those crises requiring healthcare utilization (traditional VOC endpoint). Interviews continued until saturation was demonstrated. How items worked individually and together were explored to quantify the full range of SCD severity. Both Classical Test Theory and Rasch Measurement Theory quantitative analyses supported the qualitative data to refine item content. To document content validity, the SCDSM content in its final electronic daily diary format will be evaluated in cognitive debriefing studies. Subsequently, longitudinal evaluation will occur in future clinical trials with GBT440. Translation and cultural adaptation is planned.Results: After eliciting an initial pool of items using open-ended interviews, the items were administered to a small cohort of SCD patients. After analyses of the response data and re-examination of the qualitative data, revisions were made. The revised set was re-administered to a cohort of 50 SCD patients. Qualitative and quantitative analyses provided evidence that a set of 10-items demonstrated content validity in adult and adolescent SCD patients to produce a score which can measure the full range of sickle cell disease severity. The mixed methods process enabled streamlined decision-making concerning measure content. The final content includes items specific to pain, fatigue, concentration and breathlessness. Evidence supports the derived estimate as a reliable and valid measure of SCD symptom severity covering a wide range of experience and providing the ability to differentiate baseline symptoms from severe symptoms. Patient responses to the 10 SCDSM items are combined to generate a single daily estimate of SCD symptom severity. Two "scores" can be computed: an ordinal-level total score can be computed by summing item scores; an interval-level (linear) measure can be derived from item responses using RMT analysis.Conclusions: SCD patients have a high burden of daily symptoms, with severe or "crisis" days occurring more frequently than traditionally defined VOC. The 10-item SCDSM is a promising new measure of daily SCD symptom severity developed to be fit for purpose in clinical trials to determine the treatment effect of new drugs on potential improvement in baseline symptoms and/or prevention of symptom exacerbations. SCDSM will soon be tested in a clinical trial setting to determine its longitudinal measurement characteristics, generate guidelines for measure interpretability, and to explore the clinical benefit of GBT440 in SCD. Qualitative and psychometric findings based on the SCD patient voice will be presented at the meeting to provide evidence of SCDSM validity, reliability, and usefulness in assessing symptom severity in a disease that has heretofore been poorly characterized on a daily basis. DisclosuresBurke:Lora Group: Employment; Global Blood Therapeutics: Consultancy. Hobart:Lora Group: Research Funding. Fox:Lora Group: Research Funding. Lehrer-Graiwer:Global Blood Therapeutics: Employment, Equity Ownership. Bridges:Global Blood Therapetics: Employment, Equity Ownership. Kraus:Global Blood Therapeutics: Employment, Equity Ownership. Ramos:Global Blood Therapeutics: Employment, Equity Ownership.

Telling the difficult things: Creating spaces for disclosure, rapport and ‘collusion’ in qualitative interviews

Qualitative interviews continue to offer an established way to collect rich data about everyday experiences of the social world. It is also recognised that data collected during face-to-face interviews are the product of a social interaction with co-constructive elements. Reflection on the research process and methodological transparency, have become mainstays of rigorous qualitative research practice, facilitating critical assessment of research findings. But in what ways can and do researchers co-construct interview accounts and what happens once data are collected? This paper focuses on what happens during the interview, for example the creation of spaces and endurance of silences, or supportive comments made in order to invite and allow disclosures, and what happens around the interview encounter. Do ‘permissions’ to voice difficult, challenging experiences amount to collusion or just good, effective interviewing technique? How/do research relationships – including experiences of power – shift within and around the interview and when does ‘rapport’ cease?

When Assessment Data Are Words: Validity Evidence for Qualitative Educational Assessments.

Quantitative scores fail to capture all important features of learner performance. This awareness has led to increased use of qualitative data when assessing health professionals. Yet the use of qualitative assessments is hampered by incomplete understanding of their role in forming judgments, and lack of consensus in how to appraise the rigor of judgments therein derived. The authors articulate the role of qualitative assessment as part of a comprehensive program of assessment, and translate the concept of validity to apply to judgments arising from qualitative assessments. They first identify standards for rigor in qualitative research, and then use two contemporary assessment validity frameworks to reorganize these standards for application to qualitative assessment.Standards for rigor in qualitative research include responsiveness, reflexivity, purposive sampling, thick description, triangulation, transparency, and transferability. These standards can be reframed using Messick's five sources of validity evidence (content, response process, internal structure, relationships with other variables, and consequences) and Kane's four inferences in validation (scoring, generalization, extrapolation, and implications). Evidence can be collected and evaluated for each evidence source or inference. The authors illustrate this approach using published research on learning portfolios.The authors advocate a "methods-neutral" approach to assessment, in which a clearly stated purpose determines the nature of and approach to data collection and analysis. Increased use of qualitative assessments will necessitate more rigorous judgments of the defensibility (validity) of inferences and decisions. Evidence should be strategically sought to inform a coherent validity argument.

School Psychology as a Relational Enterprise: the Role and Process of Qualitative Methodology

The purpose of this article is to explore the application of qualitative research to establishing a more complete understanding of relational processes inherent in school psychology practice. We identify the building blocks of rigorous qualitative research design through a conceptual overview of qualitative paradigms, methodologies, methods (i.e., for data collection, analysis, and interpretation), and for the integrity standard of trustworthiness. This review may then serve as a resource for those interested in conducting qualitative or mixed methods research. Practical illustrations each of essential features of qualitative research were garnered from four studies on school-based consultation included in a recent qualitative metasynthesis (a meta-analytic process applied to qualitative findings). Together with these examples, we offer direct consideration of the relevance of qualitative methods to the future study of relational processes in school psychology practice.

The Importance Of Integrating Narrative Into Health Care Decision Making.

When making health care decisions, patients and consumers use data but also gather stories from family and friends. When advising patients, clinicians consult the medical evidence but also use professional judgment. These stories and judgments, as well as other forms of narrative, shape decision making but remain poorly understood. Furthermore, qualitative research methods to examine narrative are rarely included in health science research. We illustrate how narratives shape decision making and explain why it is difficult but necessary to integrate qualitative research on narrative into the health sciences. We draw on social-scientific insights on rigorous qualitative research and our ongoing studies of decision making by patients with cancer, and we describe new tools and approaches that link qualitative research findings with the predominantly quantitative health science scholarship. Finally, we highlight the benefits of more fully integrating qualitative research and narrative analysis into the medical evidence base and into evidence-based medical practice.

Ensuring rigour and trustworthiness of qualitative research in clinical pharmacy.

The use of qualitative research methodology is well established for data generation within healthcare research generally and clinical pharmacy research specifically. In the past, qualitative research methodology has been criticized for lacking rigour, transparency, justification of data collection and analysis methods being used, and hence the integrity of findings. Demonstrating rigour in qualitative studies is essential so that the research findings have the "integrity" to make an impact on practice, policy or both. Unlike other healthcare disciplines, the issue of "quality" of qualitative research has not been discussed much in the clinical pharmacy discipline. The aim of this paper is to highlight the importance of rigour in qualitative research, present different philosophical standpoints on the issue of quality in qualitative research and to discuss briefly strategies to ensure rigour in qualitative research. Finally, a mini review of recent research is presented to illustrate the strategies reported by clinical pharmacy researchers to ensure rigour in their qualitative research studies.

Choosing a Qualitative Research Approach.

Educators often pose questions about qualitative research. For example, a program director might say: “I collect data from my residents about their learning experiences in a new longitudinal clinical rotation. If I want to know about their learning experiences, should I use qualitative methods? I have been told that there are many approaches from which to choose. Someone suggested that I use grounded theory, but how do I know this is the best approach? Are there others?”Qualitative research is the systematic inquiry into social phenomena in natural settings. These phenomena can include, but are not limited to, how people experience aspects of their lives, how individuals and/or groups behave, how organizations function, and how interactions shape relationships. In qualitative research, the researcher is the main data collection instrument. The researcher examines why events occur, what happens, and what those events mean to the participants studied.1,2Qualitative research starts from a fundamentally different set of beliefs—or paradigms—than those that underpin quantitative research. Quantitative research is based on positivist beliefs that there is a singular reality that can be discovered with the appropriate experimental methods. Post-positivist researchers agree with the positivist paradigm, but believe that environmental and individual differences, such as the learning culture or the learners' capacity to learn, influence this reality, and that these differences are important. Constructivist researchers believe that there is no single reality, but that the researcher elicits participants' views of reality.3 Qualitative research generally draws on post-positivist or constructivist beliefs.Qualitative scholars develop their work from these beliefs—usually post-positivist or constructivist—using different approaches to conduct their research. In this Rip Out, we describe 3 different qualitative research approaches commonly used in medical education: grounded theory, ethnography, and phenomenology. Each acts as a pivotal frame that shapes the research question(s), the method(s) of data collection, and how data are analyzed.4,5Before engaging in any qualitative study, consider how your views about what is possible to study will affect your approach. Then select an appropriate approach within which to work. Alignment between the belief system underpinning the research approach, the research question, and the research approach itself is a prerequisite for rigorous qualitative research. To enhance the understanding of how different approaches frame qualitative research, we use this introductory challenge as an illustrative example.The clinic rotation in a program director's training program was recently redesigned as a longitudinal clinical experience. Resident satisfaction with this rotation improved significantly following implementation of the new longitudinal experience. The program director wants to understand how the changes made in the clinic rotation translated into changes in learning experiences for the residents.Qualitative research can support this program director's efforts. Qualitative research focuses on the events that transpire and on outcomes of those events from the perspectives of those involved. In this case, the program director can use qualitative research to understand the impact of the new clinic rotation on the learning experiences of residents. The next step is to decide which approach to use as a frame for the study.The table lists the purpose of 3 commonly used approaches to frame qualitative research. For each frame, we provide an example of a research question that could direct the study and delineate what outcomes might be gained by using that particular approach.

Moving Beyond Data Transcription

Rigor in qualitative research has been based upon criteria of credibility, dependability, confirmability, and transferability. Drawing upon articles published during our editorship of the Journal of Adolescent & Adult Literacy, we illustrate how the use of digital data in research study reporting may enhance these areas of rigor, and can provide more detailed and nuanced representations that move beyond print-based transcriptions. Data shared demonstrate how including digitally rich artifacts in research articles deepens readers' access to the content, forces authors to show and not just tell of the connections to practice, and ultimately improves the rigor of the study. We also address challenges and solutions to potential problems of using digital data in research reports.

Complicating methodological transparency

A historical indicator of the quality, validity, and rigor of qualitative research has been the documentation and disclosure of the behind-the-scenes work of the researcher. In this paper, we use what we call methodological data as a tool to complicate the possibility and desirability of such transparency. Specifically, we draw on our disparate attempts to address calls for transparency about methodological processes in our respective dissertation studies in order to examine how novice researchers can explore transparency as a situated, ongoing, and philosophically informed series of decisions about how, when, and if to be transparent about our work. This work contributes to conversations about how qualitative researchers in education can understand, discuss, and teach qualitative inquiry while continuing to push the boundaries of the field.

Assessing the FACTS: A Mnemonic for Teaching and Learning the Rapid Assessment of Rigor in Qualitative Research Studies

Teaching and learning research appraisal strategies is a challenge in undergraduate education and for practitioners alike. The appraisal of rigor in qualitative research papers is particularly complex and sophisticated work for many undergraduate research students and practitioners who want to develop their critical reading skills. The mnemonic strategy (The FACTS) explained in this paper is one pedagogical strategy for establishing a simplified approach to teaching and learning the appraisal of rigor in qualitative research. While not a comprehensive tool, the FACTS are a useful introduction to the complex challenge of qualitative research appraisal.

Promoting and evaluating scientific rigour in qualitative research.

This article explores perspectives on qualitative research and the variety of views concerning rigour in the research process. Evaluating and ensuring the quality of research are essential considerations for practitioners who are appraising evidence to inform their practice or research. Several criteria and principles for evaluating quality in qualitative research are presented, recognising that their application in practice is influenced by the qualitative methodology used. The article examines a range of techniques that a qualitative researcher can use to promote rigour and apply it to practice.

Rigor Makes a Comeback: In Praise ofLa rigeur du qualitatifBy Jean-Pierre Olivier De Sardan

La rigueur du qualitatif, a collection of essays spanning many years of research and writing, signals that the question of rigor in qualitative research may very well be making a come-back. Olivier de Sardan's book is not a nostalgic return to the foundations of rational thought and scientific validity, nor is it just another critique of post-modernism. Indeed, the critical epistemological approach he takes enables us to go beyond the now very tired distinction between objectivity and subjectivity. It presents a number of theoretical questions from an original point of view (interpretation, validity, observation, populism) and it bridges the sometimes impossible gap between francophone and anglophone thinking and literatures. In this volume, Olivier de Sardan shows clearly and coherently how qualitative research constitutes a system for producing knowledge and how, if we are attentive to these questions, we might make it more rigorous but also more socially responsible.

A Pragmatic, Step-by-Step Guide for Qualitative Methods: Capturing the Disaster and Long-Term Recovery Stories of Katrina and Rita

This paper commences with candid warnings about the difficulties of publishing and funding qualitative research along with a brief discussion regarding why these difficulties persist. The paper then provides a methodological tour of the qualitative portion of the Louisiana Healthy Aging Study (LHAS), a mixed-method study of psychological health, coping, and adjustment during the immediate and longer term aftermath of the 2005 Hurricanes Katrina and Rita. Next, I provide a proven, pragmatic guide to navigate through a qualitative research project from development to completion. A guide for doing solo qualitative (“monk/nun in a cell”) work is provided (Table 1) but the article focuses on the Four-Phase, Team-Based (or “Quad-Squad”) Method. This method, originally designed for solo researchers, has been adapted for team-based application (including the extensive utilization of undergraduate and graduate students). The method is presented as a pragmatic and productive approach to publishing more valid, reliable, and rigorous qualitative research.

Qualitative Inquiry and Research Design: Choosing Among Five Approaches

Qualitative Inquiry and Research Design provides an overview of the five main traditions of qualitative research. The author explains the uniqueness of each approach and its applicability to different types of inquiry. Illustrative examples from public health and social science fields are provided. The book details study design, question development, data collection and analysis, and summarizing and interpreting results, and how the research process differs according to each approach. This resource can serve as a useful guide for public health practitioners and graduate-level students interested in the theory and practice of rigorous qualitative research.

A Discussion of Obser-View as a Method of Generating Data in the Construction Industry

This study demonstrates the paradigmatic position of obser-view and argues for the incorporation of reflectivity in obser-view to foster rigorous data generation. Aimed at introducing obser-view to the construction industry as a method of generating data, this study critically examines obser-view, exploring its application to construction research. Obser-view is an emerging data collection technique developed by Kragelund in 2006, where a non-scripted interview immediately post-observation is conducted. This helps to gain a deeper understanding of the phenomenon. Although researchers adopt various strategies to ensure rigor in qualitative research, there are still concerns in relation to validity, reliability, bias and objectivity in qualitative research. However, developing strategies that will help in gaining a deeper understanding of the phenomenon in study and tackling the aforementioned challenges will help in ensuring transparency in qualitative research. A review of literature is presented, the limitations and benefits of obser-view are also presented.