‘Unsatisfactory Saturation’: a critical exploration of the notion of saturated sample sizes in qualitative research

Purpose Qualitative researchers have been criticised for not justifying sample size decisions in their research. This short paper addresses the issue of which sample sizes are appropriate and valid within different approaches to qualitative research. Design/methodology/approach The sparse literature on sample sizes in qualitative research is reviewed and discussed. This examination is informed by the personal experience of the author in terms of assessing, as an editor, reviewer comments as they relate to sample size in qualitative research. Also, the discussion is informed by the author’s own experience of undertaking commercial and academic qualitative research over the last 31 years. Findings In qualitative research, the determination of sample size is contextual and partially dependent upon the scientific paradigm under which investigation is taking place. For example, qualitative research which is oriented towards positivism, will require larger samples than in-depth qualitative research does, so that a representative picture of the whole population under review can be gained. Nonetheless, the paper also concludes that sample sizes involving one single case can be highly informative and meaningful as demonstrated in examples from management and medical research. Unique examples of research using a single sample or case but involving new areas or findings that are potentially highly relevant, can be worthy of publication. Theoretical saturation can also be useful as a guide in designing qualitative research, with practical research illustrating that samples of 12 may be cases where data saturation occurs among a relatively homogeneous population. Practical implications Sample sizes as low as one can be justified. Researchers and reviewers may find the discussion in this paper to be a useful guide to determining and critiquing sample size in qualitative research. Originality/value Sample size in qualitative research is always mentioned by reviewers of qualitative papers but discussion tends to be simplistic and relatively uninformed. The current paper draws attention to how sample sizes, at both ends of the size continuum, can be justified by researchers. This will also aid reviewers in their making of comments about the appropriateness of sample sizes in qualitative research.

Quantitative approaches offer important insights into human behaviour, yet they may not adequately reflect the contextual and subjective nuances inherent in such behaviour. While quantitative research methods are useful for explaining relationships between variables, they may have limitations when it comes to gaining in-depth understanding of a specific context. Quantitative research has generally accepted criteria regarding validity, reliability, sampling methods, and sample size; however, these aspects remain a topic of debate for qualitative research. Quantitative research has established standards for validity, reliability, sampling methods, and sample size. In contrast, these criteria are often debated in qualitative research, especially when qualitative approaches are evaluated using frameworks derived from quantitative traditions. Qualitative research relies on distinct methodologies that are tailored to explore complex phenomena in context. This study addresses the debates on the validity and reliability of qualitative research and provides information about the validity, reliability, sampling methods, and sample size in qualitative studies. Within this framework, the study aims to clarify ongoing debates regarding the validity, reliability, sampling methods, and sample size in qualitative research, thereby offering a structured overview that may guide researchers in designing and evaluating qualitative studies.

I explore the sample size in qualitative research that is required to reach theoretical saturation. I conceptualize a population as consisting of sub-populations that contain different types of information sources that hold a number of codes. Theoretical saturation is reached after all the codes in the population have been observed once in the sample. I delineate three different scenarios to sample information sources: “random chance,” which is based on probability sampling, “minimal information,” which yields at least one new code per sampling step, and “maximum information,” which yields the largest number of new codes per sampling step. Next, I use simulations to assess the minimum sample size for each scenario for systematically varying hypothetical populations. I show that theoretical saturation is more dependent on the mean probability of observing codes than on the number of codes in a population. Moreover, the minimal and maximal information scenarios are significantly more efficient than random chance, but yield fewer repetitions per code to validate the findings. I formulate guidelines for purposive sampling and recommend that researchers follow a minimum information scenario.

A common misconception about sampling in qualitative research is that numbers are unimportant in ensuring the adequacy of a sampling strategy. Yet, simple sizes may be too small to support claims of having achieved either informational redundancy or theoretical saturation, or too large to permit the deep, case-oriented analysis that is the raison-d'être of qualitative inquiry. Determining adequate sample size in qualitative research is ultimately a matter of judgment and experience in evaluating the quality of the information collected against the uses to which it will be put, the particular research method and purposeful sampling strategy employed, and the research product intended.

Qualitative Research in CKD: How to Appraise and Interpret the Evidence

Questions about how to determine and justify sample size remain among the most contested issues in qualitative research. This article advances the debate by contrasting data saturation, which emphasizes the recurrence of redundant information, with theoretical saturation, which prioritizes explanatory sufficiency of emerging theory. Clarifying these forms of saturation matters because reliance on data saturation alone often leads to premature closure and weak theorization while treating data saturation as an interim milestone and continuing until theoretical saturation is reached arguably better serves the explanatory goals of qualitative research. Noteworthily, theoretical saturation often requires twice as many interviews as data saturation. Positioning saturation as more than a procedural benchmark, therefore, enables qualitative researchers to justify sample size decisions in ways that strengthen both methodological rigor and theoretical contribution. Thus, this article contributes by drawing a clear boundary between descriptive sufficiency and explanatory sufficiency, reorganizing dispersed rules of thumb into an approach-versus-method range table with a governing choice rule, operationalizing an auditable five-step procedure that blends information power with threshold monitoring of new information, and translating these advances into concrete interview and focus group heuristics that can be readily implemented.

PurposeThe lack of a definite standard for determining the sample size in qualitative research leaves the research process to the initiative of the researcher, and this situation overshadows the scientificity of the research. The primary purpose of this research is to propose a model by questioning the problem of determining the sample size, which is one of the essential issues in qualitative research. The fuzzy logic model is proposed to determine the sample size in qualitative research.Design/methodology/approachConsidering the structure of the problem in the present study, the proposed fuzzy logic model will benefit and contribute to the literature and practical applications. In this context, ten variables, namely scope of research, data quality, participant genuineness, duration of the interview, number of interviews, homogeneity, information strength, drilling ability, triangulation and research design, are used as inputs. A total of 20 different scenarios were created to demonstrate the applicability of the model proposed in the research and how the model works.FindingsThe authors reflected the results of each scenario in the table and showed the values for the sample size in qualitative studies in Table 4. The research results show that the proposed model's results are of a quality that will support the literature. The research findings show that it is possible to develop a model using the laws of fuzzy logic to determine the sample size in qualitative research.Originality/valueThe model developed in this research can contribute to the literature, and in any case, it can be argued that determining the sample volume is a much more effective and functional model than leaving it to the initiative of the researcher.

This review aimed to answer the question of how many interviews are enough for one qualitative research? The question ‘how many interviews are enough for one qualitative research is persistently controversial among qualitative researchers in social science. For this frequently occurring question especially among novice practitioners, the majority of scholars are opted to say no universally guiding rule to decide on a required number of sample for qualitative research rather ‘it depends’. But, this also raises another insight among researchers urging them to look for different things, i.e., on what circumstance would be making a decision about the required number of respondents depend? Though we lack one guideline dictating researchers how to decide on the number of sample size, the majority of researchers agree on one reasonable answer this is ‘it depends’. Indeed, in our work, we endeavor to identify, on what it depends? We also attempted to figure out or indicate the commonly referred range of sample size in qualitative research. Generally when we sum up our review work, the decision on “How many” is depended on several factors among which the following are some; the focus of the research, the type of research question, available resource and time, institutional committee requirements, the judgments of epistemic community in which a researcher is located, the nature of the selected group, the domain of inquiry, the experience of the researcher with qualitative research, and so on. Specific to number 20-60 is the most frequently observed range of sample size in qualitative research which of course is determined by the aforementioned factors.

A number of issues can affect sample size in qualitative research; however, the guiding principle should be the concept of saturation. This has been explored in detail by a number of authors but is still hotly debated, and some say little understood. A sample of PhD studies using qualitative approaches, and qualitative interviews as the method of data collection was taken from theses.com and contents analysed for their sample sizes. Five hundred and sixty studies were identified that fitted the inclusion criteria. Results showed that the mean sample size was 31; however, the distribution was non-random, with a statistically significant proportion of studies, presenting sample sizes that were multiples of ten. These results are discussed in relation to saturation. They suggest a pre-meditated approach that is not wholly congruent with the principles of qualitative research.

BackgroundChoosing a suitable sample size in qualitative research is an area of conceptual debate and practical uncertainty. That sample size principles, guidelines and tools have been developed to enable researchers to set, and justify the acceptability of, their sample size is an indication that the issue constitutes an important marker of the quality of qualitative research. Nevertheless, research shows that sample size sufficiency reporting is often poor, if not absent, across a range of disciplinary fields.MethodsA systematic analysis of single-interview-per-participant designs within three health-related journals from the disciplines of psychology, sociology and medicine, over a 15-year period, was conducted to examine whether and how sample sizes were justified and how sample size was characterised and discussed by authors. Data pertinent to sample size were extracted and analysed using qualitative and quantitative analytic techniques.ResultsOur findings demonstrate that provision of sample size justifications in qualitative health research is limited; is not contingent on the number of interviews; and relates to the journal of publication. Defence of sample size was most frequently supported across all three journals with reference to the principle of saturation and to pragmatic considerations. Qualitative sample sizes were predominantly – and often without justification – characterised as insufficient (i.e., ‘small’) and discussed in the context of study limitations. Sample size insufficiency was seen to threaten the validity and generalizability of studies’ results, with the latter being frequently conceived in nomothetic terms.ConclusionsWe recommend, firstly, that qualitative health researchers be more transparent about evaluations of their sample size sufficiency, situating these within broader and more encompassing assessments of data adequacy. Secondly, we invite researchers critically to consider how saturation parameters found in prior methodological studies and sample size community norms might best inform, and apply to, their own project and encourage that data adequacy is best appraised with reference to features that are intrinsic to the study at hand. Finally, those reviewing papers have a vital role in supporting and encouraging transparent study-specific reporting.

Guest editorial: What's common with qualitative nursing research these days?

In this paper I explore the sample size in qualitative research that is required to reach theoretical saturation. I conceptualize a population as consisting of sub-populations that contain differen...

As our editorial board pondered the focus for a special issue, which our journal has not had previously, we were struck by a dominant feature of the top ten list of most downloaded articles. Seven or eight of the ten most downloaded articles in the past 3 years have been papers on research methods. Eureka! We decided that if researchers of nursing and health turn to our journal for strong methods articles, we would oblige them with an entire issue. Readers and researchers alike may not know that until recently, we published research reports or research methods papers only. In 2018, we added a new category: research protocols. Research methods papers, however, have always been the “Little Engine that Could” of our journal, averaging about one paper per issue, tucked in the back after the empirical standards. To find that the infrequent and last in the table of contents papers are truly the hidden gems of our journal was a revelation. I believe our readers would be interested to know what those most popular papers are and what we have to offer in this special issue. First, a few words about what I now refer to as “RINAH's qualitative gems.” All but one of the most downloaded methods papers focused on qualitative research. As a quantitative researcher, I am proud that our journal serves the vital role of providing well-regarded resources on qualitative research methods. In both 2017 and 2018, a core set of six papers dominated the top 10 list, comprising over 70% of all downloads each year. For reference, the top 10 downloads were each downloaded from 2000 to nearly 9000 times. What is perhaps more notable than the qualitative approach of these popular papers is the vintage status of some of the perennial winners. The legacy paper of this set is Sandelowski's “Sample Size in Qualitative Research” (Sandelowski, 1995), but Margarete Sandelowski's contributions to qualitative research methods classics only begin there. Also on the top download list is her work on qualitative description (Sandelowski, 2000b, 2010) and mixed methods (Sandelowski, 2000a). More recently, Dempsey, Dowling, Larkin, and Murphy (2016) presented a framework of essential elements when planning and conducting qualitative interviews on sensitive topics. The sixth gem, a newcomer, is Kim, Sefcik, and Bradway (2017) contribution, which details the characteristics of methods in 55 qualitative descriptive studies. I infer from the download volume that these gems are required reading in many a qualitative research methods course. Two other research methods papers deserve mention because they were top 10 in 1 year and top 15 in the other. Novick's (2008) article explores the apparent bias against phone interviews as a data collection modality in qualitative research and recommends research to support guidelines for optimizing interview data. Hertzog's 2008 article, the lone one on quantitative methods, presents guidance on determining sample size for pilot studies. In contrast to our top downloads, which are nearly exclusively qualitative papers, the special issue is decidedly quantitative in character. This issue covers quantitative research methodology from innovative designs to sample recruitment to data analysis. Here is what you will discover in this issue. The lead paper in the issue describes SMART designs for nursing research, in which adaptive randomized controlled trials are conducted (Doorenbos, Haozous, Kyeong, & Langford, 2019). SMART designs represent a shift to trials that mirror clinical practice and thereby support evidence-based practice better than traditional RCTs. Next in the issue is a paper presenting propensity score matching using doses (Ji, Cui, & Liu, 2019). Propensity score matching designs overcome a limitation of observational research, by statistically matching an exposure group to a nonexposed group on observed covariates. This exemplar examines physical abuse and sleep quality among adolescents in China. The researchers utilize levels of exposure to physical abuse as doses to achieve better matches between abused and nonabused adolescents than traditional propensity score matching. The next two papers focus on recruitment. The first paper presents the challenges of recruiting participants in community-based studies (Odoh et al., 2019). This study focused on the impact of coal ash from a power plant on the health of children living nearby. The paper presents approaches, challenges, and solutions to recruiting the sample of 300 children. The recruitment paper by Hutchinson and Sutherland (2019) addresses the challenges of surveying health care providers who work in unusual practice settings, including the lack of appropriate sampling frames. The researchers report on their study of multidisciplinary college health providers. Accordingly, these researchers present the three approaches to sampling, recruitment, and data collection they utilized and compare the researcher effort, response rates, and sample characteristics. The next paper in our special issue addresses how geospatial data and spatial methodologies may be used to better understand how the environment relates to health (DePriest, Shields, & Curriero, 2019). The researchers measured neighborhood greenspace and neighborhood violence to measure their association with children's asthma control. This paper has vivid maps displaying the neighborhood variables. LaFave et al. (2019) paper considers the content and outcomes of the attention control group, which often is taken for granted given a study's focus on the intervention of interest. The researchers tested an aging-in-place intervention for community-dwelling older adults. The attention control participants chose activities with a lay visitor and completed a survey on the perceived benefit of the visits. Mobile health (mHealth) interventions are the focus of the paper by Phillips et al. (2019). These researchers focus on how to develop and refine an mHealth app for health behavior change. They present the five step mHealth framework to guide mHealth development and testing as well as a coding framework for qualitative analysis. The researchers illustrate these frameworks with an app for self-management for children with sickle cell disease. A research methods paper that presents a data analysis software is the contribution of Stiglic, Watson, and Cilar (2019). The researchers present R, a package in the public domain, and provide the code in R for a confirmatory factor analysis. Our last special issue paper is not a research methods paper. Nor is it any of our traditional paper types. This paper, by Messer, Sousa, and Cook (2019), is an applied theory paper. These researchers present two minds theory for changing health behavior. The theory addresses a gap between intentions and behaviors. They apply the theory to type 1 diabetes self-management. Nurses can use the theory to develop behavior change interventions. Collectively, these special issue papers showcase the advances in methods (and theory) that enable nursing and health researchers to address innovative questions or increase the rigor of familiar research elements such as attention control groups. To conclude, be sure to check out our perennial research methods gems, in addition to this new crop of contenders. I am confident that your research methods will be enriched by the offerings from one or more of these papers.

The value that qualitative research brings to the occupational therapy body of evidence is widely accepted. Including evidence relating to individuals’ lived experiences through qualitative studies parallels occupational therapy's practice focus and theoretical underpinnings of the person in context. Engaging with consumers of occupational therapy services, for example, ensures that our practices, clinical and academic, remain current and reflective of the experience of people we work with. Qualitative research, where participants provided rich and in-depth description of their personal experience, is thus akin to client centred practice which is our modus operandi (World Federation of Occupational Therapists, 2016). Qualitative research can be considered a natural extension of occupational therapy practice experience and expertise. After all, occupational therapists who are guided by a person-centred approach, will inevitably explore and consider their client's unique values and experiences in order to understand the context and meanings underlying occupational participation. We are comfortable interacting with clients and, consequently, occupational therapists should have prerequisite skills necessary for meaningful data collection with research participants using qualitative approaches. There is, however, a difference as the primary purpose of interaction is different – one aims to assist the other to enquire. Different techniques are used in these interactions reflecting the different methodological traditions from which they emerge. Clinicians know that ‘one size does not fit all’ in therapy. We thus interpret findings of reliable, standardised and valid assessments in the context of unique occupational performance needs, goals, roles and environments of the client, in order to collaboratively develop and intervention plan with our client. So it is with qualitative research data and methodology. The information collected will only ‘make sense’ if the assessment/enquiry methods used are rigorous, take account of the unique context and experiences of people involved, and ‘make sense’ from a range of perspectives. How do we know if qualitative research methodologies are rigorous? Can something that is so unique even incorporate attributes comparable across study questions and contexts? Just as therapy uses guidelines and methodological research to enhance the consistency and quality of practice processes, what tools can qualitative researchers in occupational therapy use to develop and implement robust research questions and procedures? Do we apply the same level of precision used to generate qualitative evidence in practice to the data collection, analysis and interpretation that we do in qualitative research? Is there a risk that qualitative findings from research is less rigorous than what we would use in practice? Are we taking the appropriate measures to ensure qualitative research produce findings that will have the potential to advance occupational therapy practice, research, and education? When it comes to the ‘quality’ of qualitative research, evidence suggests that ‘sample size’ is both a matter of contention and a meaningful guide to issues that need consideration. In the first instance, while sample size has been identified as a factor in study design, and it can be difficult to determine with precision, the size of the sample should be guided by ‘data saturation’ rather than sampling of people per se (Creswell, 2013). At the same time, the concept of data saturation has been challenged as a means to determine the quality of qualitative studies (O'Reiley & Parker, 2013). Other authors have proposed sample size can be determined on the number of participants rather than ‘data’, depending on the qualitative methodologies. For example, phenomenological studies may use between five and 20 participants (Creswell, 2013; Green & Thorogood, 2014) while three to 15 participants is acceptable in studies using interpretative phenomenological analysis (Smith, Flowers & Larkin, 2009). Still further, some qualitative researchers avoid suggestion of any particular approach, instead providing resources to researchers to help align their study design, procedures and sample size as best as possible with the question they are seeking to answer (for example, the Rosalind Franklin-Qualitative Research Appraisal Instrument and the Consolidated criteria for reporting qualitative research.) These resources have been developed precisely to help qualitative researchers design, conduct, and report comprehensive research that is credible, dependable and transferable to other contexts. The Australian Occupational Therapy Journal through its Editorial Board and Reviewers, stays abreast of discussions and developments in qualitative methodologies and approaches. Authors need to remember that, the journal will publish ‘papers that have a sound theoretical basis, methodological rigour with sufficient scope and scale to make important new contributions to the occupational therapy body of knowledge’ (Wiley & Sons, 2018). It will maintain its commitment to ‘disseminate scholarship and evidence to substantiate, influence and shape policy and occupational therapy practice locally and globally’ (Wiley & Sons). To achieve this commitment, authors of qualitative research studies must provide strong justification to support their methodological choices and clearly describe data collection, analysis and interpretation steps that will demonstrate trustworthiness of findings and design rigour.

According to most researchers carrying out qualitative researches, adequacy of sample size is a key marker for the research’s quality. However, there is no consensus with respect to the exact size of a proper sample. For some authors, the count of investigated units is irrelevant when they assess the sample size’s adequacy, as they emphasize the abundance of data submitted by the units included in the sample. Other researchers deem the sample size all-important in order to reach reliable outputs and to ensure the reliability of qualitative researches. No clear methods and rules are given for qualitative investigation in order to guide researches in establishing the sample’s proper size. Size determination is a matter of consideration, as the researchers follow various guidelines in order to assess whether their own research sample is proper or not. This paper aims to identify the main external guidelines for a qualitative research project allowing researchers to determine the proper sample size in qualitative research..