Sample Size Justification Research Articles

Evaluating the impact of malaria rapid diagnostic tests on patient-important outcomes in sub-Saharan Africa: a systematic review of study methods to guide effective implementation

ObjectiveTo perform critical methodological assessments on designs, outcomes, quality and implementation limitations of studies evaluating the impact of malaria rapid diagnostic tests (mRDTs) on patient-important outcomes in sub-Saharan Africa.DesignA systematic...

Acupuncture for the Treatment of Diabetes Related Distal Sensory Neuropathy: A Review of Study Designs and Quality of Randomized Controlled Trials

Diabetes affects nearly every organ system in the body, including the peripheral nervous system, and leads to the development of distal sensory peripheral neuropathy (DSP) in roughly 50% of patients. Between 30% to 50% of patients with DSP experience pain and impaired quality of life. Dissatisfaction with currently available pharmacological treatments for neuropathic pain is common, primarily due to limited effectiveness and potential for adverse side effects. Acupuncture, a non-pharmacologic intervention, is often used in the management of chronic pain and is widely available and acceptable to many patients. In this narrative review, we describe clinical research related to acupuncture and diabetes related DSP management including meta-analyses, systematic reviews, and randomized controlled clinical trials (RCTs). Data from the few clinical trials available support the use of acupuncture to reduce neuropathic pain including DSP pain; however, available studies suffer from methodological weaknesses including inadequate blinding, sample size justification, lack of well-designed controls, and gaps in reporting. These limitations highlight a critical gap and an unmet need for more rigorous and robust clinical research requiring the use of Standards for Reporting Clinical Trials of Acupuncture (STRICTA) guidelines as a design, implementation, and reporting template.

ADNEX risk prediction model for diagnosis of ovarian cancer: systematic review and meta-analysis of external validation studies

ObjectivesTo conduct a systematic review of studies externally validating the ADNEX (Assessment of Different Neoplasias in the adnexa) model for diagnosis of ovarian cancer and to present a meta-analysis of...

Magnitude of effect and sample size justification in trials supporting anti-cancer drug approval by the US Food and Drug Administration

Approval of drugs is based on randomized trials observing statistically significant superiority of an experimental agent over a standard. Statistical significance results from a combination of effect size and sampling, with larger effect size more likely to translate to population effectiveness. We assess sample size justification in trials supporting cancer drug approvals. We identified US FDA anti-cancer drug approvals for solid tumors from 2015 to 2019. We extracted data on study characteristics, statistical plan, accrual, and outcomes. Observed power (Pobs) was calculated based on completed study characteristics and observed hazard ratio (HRobs). Studies were considered over-sampled if Pobs > expected with HRobs similar or worse than expected or if Pobs was similar to expected with HRobs worse than expected. We explored associations with over-sampling using logistic regression. Of 75 drug approvals (reporting 94 endpoints), 21% (20/94) were over-sampled. Over-sampling was associated with immunotherapy (OR: 5.5; p = 0.04) and associated quantitatively but not statistically with targeted therapy (OR: 3.0), open-label trials (OR: 2.5), and melanoma (OR: 4.6) and lung cancer (OR: 2.17) relative to breast cancer. Most cancer drug approvals are supported by trials with justified sample sizes. Approximately 1 in 5 endpoints are over-sampled; benefit observed may not translate to clinically meaningful real-world outcomes.

Less is More? Review and Recommendations for Qualitative Sampling Strategy using the S.C.A.D.E Approach

The determination of sample size in qualitative research introduces a unique and multifaceted challenge, setting it apart from the more structured methodology of quantitative research. Contrary to sampling methods in quantitative research, which primarily aim to secure random and statistically representative samples that facilitate the generalisation of findings to broader populations, sampling in qualitative research requires a distinct set of considerations in its pursuit of a deeper understanding of specific phenomena. The objective of this editorial is to provide qualitative researchers with clear and foundational guidance for effectively communicating the methodological aspects of their research papers, particularly pertaining to sample size justification. Building on this, we present S.C.A.D.E, an acronym comprising five key actionable elements—Selecting, Clarifying, Aligning, Deploying and Evaluating—to guide researchers in determining the appropriate sample size and ensuring that data saturation is achieved as they plan their qualitative exploration.

Sample size determination for external pilot cluster randomised trials with binary feasibility outcomes: a tutorial

Justifying sample size for a pilot trial is a reporting requirement, but few pilot trials report a clear rationale for their chosen sample size. Unlike full-scale trials, pilot trials should not be designed to test effectiveness, and so, conventional sample size justification approaches do not apply. Rather, pilot trials typically specify a range of primary and secondary feasibility objectives. Often, these objectives relate to estimation of parameters that inform the sample size justification for the full-scale trial, many of which are binary. These binary outcomes are referred to as “feasibility outcomes” and include expected prevalence of the primary trial outcome, primary outcome availability, or recruitment or retention proportions.For pilot cluster trials, sample size calculations depend on the number of clusters, the cluster sizes, the anticipated intra-cluster correlation coefficient for the feasibility outcome and the anticipated proportion for that outcome. Of key importance is the intra-cluster correlation coefficient for the feasibility outcome. It has been suggested that correlations for feasibility outcomes are larger than for clinical outcomes measuring effectiveness. Yet, there is a dearth of information on realised values for these correlations.In this tutorial, we demonstrate how to justify sample size in external pilot cluster trials where the objective is to estimate a binary feasibility outcome. We provide sample size calculation formulae for a variety of scenarios, make available an R Shiny app for implementation, and compile a report of intra-cluster correlations for feasibility outcomes from a convenience sample. We demonstrate that unless correlations are very low, external pilot cluster trials can be made more efficient by including more clusters and fewer observations per cluster.

Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis.

Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis.

Sample size requirements are not being considered in studies developing prediction models for binary outcomes: a systematic review

BackgroundHaving an appropriate sample size is important when developing a clinical prediction model. We aimed to review how sample size is considered in studies developing a prediction model for a binary outcome.MethodsWe searched PubMed for studies published between 01/07/2020 and 30/07/2020 and reviewed the sample size calculations used to develop the prediction models. Using the available information, we calculated the minimum sample size that would be needed to estimate overall risk and minimise overfitting in each study and summarised the difference between the calculated and used sample size.ResultsA total of 119 studies were included, of which nine studies provided sample size justification (8%). The recommended minimum sample size could be calculated for 94 studies: 73% (95% CI: 63–82%) used sample sizes lower than required to estimate overall risk and minimise overfitting including 26% studies that used sample sizes lower than required to estimate overall risk only. A similar number of studies did not meet the ≥ 10EPV criteria (75%, 95% CI: 66–84%). The median deficit of the number of events used to develop a model was 75 [IQR: 234 lower to 7 higher]) which reduced to 63 if the total available data (before any data splitting) was used [IQR:225 lower to 7 higher]. Studies that met the minimum required sample size had a median c-statistic of 0.84 (IQR:0.80 to 0.9) and studies where the minimum sample size was not met had a median c-statistic of 0.83 (IQR: 0.75 to 0.9). Studies that met the ≥ 10 EPP criteria had a median c-statistic of 0.80 (IQR: 0.73 to 0.84).ConclusionsPrediction models are often developed with no sample size calculation, as a consequence many are too small to precisely estimate the overall risk. We encourage researchers to justify, perform and report sample size calculations when developing a prediction model.

Statistical guidance provided to authors by clinical neurology and neuroscience journals.

Transparent reporting and appropriate interpretation of statistical methods and results is important to facilitate scientific evaluation and enable future replication. The goal of this study was to describe statistical reporting guidance provided to authors by Clinical Neurology and Neuroscience journals. For first-quartile journals in each discipline (per Clarivate InCites), information collected from Instructions to Authors website sections included: whether journals required presentation of sample size justification, estimates of precision, and method of checking assumptions; and guidance for interpretation of p-values and appropriate presentation of descriptive statistics and graphs. Journal endorsement of common but statistically non-specific published transparent reporting guidelines for human and animal research was also collected, to capture the select statistical reporting items included in each guideline. Journals (n=85) frequently did not require/recommend: sample size justifications (15% not required; 62% only required per external transparent reporting guideline), estimates of precision (15% not required; 41% only required per external guidelines), or disclosure of method of checking assumptions (46%); nor provide guidance for reporting/interpretation of p-values (71%), reporting of descriptive statistics (75%), nor use of appropriate graphs (92%). Endorsement of statistically non-specific standalone reporting guidelines ranged between 52% and 68%, depending on guideline. There is opportunity for journals to facilitate improvement in transparency of statistical methods and results for Clinical Neurology and Neuroscience studies by providing guidelines and advice to authors at manuscript submission.

Sample size, replicability, and pre-test likelihoods - essential, overlooked, and critical components of statistical inference - A Journal of Neurotrauma Guide to Statistical Methods and Study Design.

Perhaps one of the most overlooked components of statistical inference is the sample size. While in randomized controlled trials, power analysis is common and sample size justification is an integral component of the core statistical analysis plan, observational and laboratory research studies often rely on convenience samples or underpowered analyses. This increases uncertainty associated with the results and limits interpretability. Moreover, it increases the likelihood that the findings might be disproved in future replication studies. A scientific study can be compared to a diagnostic test for the "truth", i.e. whether a certain effect exists or whether a relationship is actually true. In this diagnostic analogy, the positive predictive value is dependent not only on the statistical power of the study in question, but also on the pre-test likelihood that any true relationship exists at all. The concept of the using an estimate of the pre-test likelihood to interpret observed results is another critical and often overlooked component of statistical inference. Even if a statistically significant relationship or an effect is found, however, the finding in itself may be insufficient. It often must be replicated, ideally in a more generalizable setting. Furthermore, if the effect size is small, replication may require sample sizes that are substantially larger than the original study. As an extreme example, previous research into brain-wide associations where large numbers of comparisons are being performed teaches us that the effects measured are usually small and often require thousands of patients in order to be reliably estimated. For most neurotrauma research, thousands of subjects are usually not required, but many studies do require substantially larger sample sizes than are typically presented in order to increase replicability. In this methodological tutorial, choice of sample size, pre-test probability, and the concept of positive predictive value for scientific findings will be discussed, together with suggestions to improve replicability of neurotrauma research in the future.

Intraindividual variability in sleep among athletes: A systematic review of definitions, operationalizations, and key correlates.

Via systematic review with narrative synthesis of findings, we aimed to document the ways by which researchers have defined, operationalized, and examined sleep variability among athletes. We identified studies in which scholars examined intraperson variability in sleep among athletes via a search of six databases (Web of Science, Embase, Medline, PsycINFO, CINHAL Plus, and ProQuest Dissertations and Theses Global) using a protocol that included keywords for the target outcome (sleep*), population (athlet* OR sport*), and outcome operationalization (variability OR variation OR "standard deviation" OR fluctuate OR fluctuation OR stability OR instability OR reactivity OR IIV OR intraindividual). We complemented this primary search with citation searching of eligible articles. Assessments of study quality captured eight core elements, namely aims/hypotheses, sample size justification, sample representativeness, number of days sleep assessed, measures of sleep and its correlates, missing data, and inferences and conclusions. From a total of 1209 potentially relevant papers, we identified 16 studies as meeting our eligibility criteria. Concept definitions of variability were notably absent from this work and where available were vague. Quantitative deviations from one's typical level of target sleep metrics reflected the essence by which all but one of the research teams operationalized sleep variability. We assessed the overall quality of empirical work as moderate in nature. We propose a working definition of sleep variability that can inform knowledge generation on the temporal, day-to-day dynamics of sleep functioning that is required for personalized interventions for optimizing sleep health.

Pericapsular Nerve Group Block in Hip Surgery: A Narrative Review

Study Objective: This narrative review discusses the anatomy, mechanism of action, techniques, indications and complications of the pericapsular nerve group block in a hip surgery setting. Interventions: The MEDLINE, EMBASE and Google Scholar databases (inception to the first week of March 2023) were searched. For anatomy, mechanism of action, techniques, and complications, cadaveric research, randomized trials, retrospective studies and case series were considered. Nonetheless, for surgery indications, to highlight the best evidence available, only randomized trials without major discrepancies with their prospective registration, blinded assessment and sample size justification were retained for analysis. Main Results: The anatomical studies suggest that pericapsular nerve group block may work through a combination of different mechanisms (i.e., blocking lateral capsule nerves, local anesthetic spread to the femoral nerve, spread towards medial capsule nerves). Compared to alternatives, except for the periarticular local anesthetic infiltration, pericapsular nerves group block results in similar or improved postoperative analgesia in total hip arthroplasty. It should be noted that the motor blockade has not been completely circumvented and the scarcity of adequate studies on other surgical procedures prevents from obtaining further conclusions about its indications. Conclusions: The pericapsular nerve group block has become very popular after its first description as an analgesic motor-sparing technique for hip fractures. However, without an absolute motor-sparing effect, its current indication is better supported in postoperative analgesia for total hip arthroplasty. Therefore, further investigation is required to find the optimal motor-sparing analgesic block for hip surgery.

Nivolumab with chemotherapy in pleural mesothelioma after surgery: The NICITA trial.

TPS8601 Background: Pleural mesothelioma (PM) is a highly aggressive cancer of the pleura, predominantly caused by prior asbestos exposure. Currently, there is no approved standard therapy for the treatment of early-stage PM. In most cases a multimodal therapy is recommended consisting of locoregional treatment by surgical cytoreduction via extended pleurectomy/decortication (eP/D), which, if feasible, can be combined with hyperthermic intrathoracic chemoperfusion (HITOC), together with inductive or adjuvant chemotherapy. Considering the immunogenic effects of chemotherapy on the tumor microenvironment, synergistic effects are expected when such a treatment is combined with immune checkpoint inhibitor therapy. In addition, interactions between immune infiltrates and mesothelioma cells play a role in the advent of PM, also implying a beneficial role for immunotherapy in this entity. This is also supported by recent clinical data that demonstrated beneficial effects of immune checkpoint inhibitors in patients with advanced PM. The NICITA trial is an investigator-initiated trial, investigating the combination of adjuvant chemotherapy with immune checkpoint inhibitor compared to chemotherapy alone in radically resected patients with early stage PM. Methods: The NICITA trial is a randomized, open-label, phase II clinical trial that is conducted in 14 centers across Germany. Eligible patients have been diagnosed with PM in tumor stages I-III (UICC 8th edition) and epithelioid subtype, and must have undergone cytoreductive surgery by eP/D with or without HITOC. Patients will be randomized 1:1 to receive either a combination of 4 cycles of pemetrexed/platinum-based adjuvant chemotherapy and nivolumab (480 mg q4w) followed by nivolumab maintenance therapy (12 cycles, 480 mg q4w) or 4 cycles of adjuvant chemotherapy only. Stratification will take place according to previous HITOC treatment (yes vs. no), ECOG status (0,1 vs. 2), and achievement of macroscopic complete resection (yes vs. no). The primary endpoint of this trial is time-to-next-treatment. Secondary endpoints include additional measures of efficacy (progression-free survival, overall survival, measures of treatment-beyond-progression) and quality of life, as well as the assessment of safety. Furthermore, a comprehensive longitudinal collection of biomarker samples, including tumor tissue, blood, and stool samples, for an accompanying translational research project is implemented in this clinical trial. Sample size justification: the recruitment of 46 patients to each arm with a low drop-out rate of 13% appears feasible resulting in 40 patients to be analyzed per arm. This sample size will permit a descriptive comparison and adequately describe the tested treatment options as deduced from the precision of the median TNT confidence interval estimate. As of February 2nd 2023, 85 of planned 92 patients have been enrolled into the NICITA trial. Clinical trial information: NCT04177953 .

Critical appraisal of knowledge, attitude and practice studies for blood donation in India.

A critical appraisal of the literature helps to assess the strength and weakness of the research and suggests ways to improve future research. Our aim was to critically appraise the knowledge, attitude, and practice (KAP) studies conducted in India for blood donation. Of 70 articles identified in our search on PubMed, Scopus and Google Scholar, 32 were assessed for quality using an appraisal tool for cross-sectional studies (AXIS) and questionnaire items. Quality assessment revealed that only 6 of 32 studies had acceptable reporting (≥80% score on the AXIS tool). The most frequently identified shortcomings were failure to address the non-responders, lack of justification for sample size, assessment of outcome variables and demographic results for the survey. Our evaluation of questionnaires revealed that knowledge for need for blood donation, its benefits and site/place for blood donation were assessed by very few studies. With this, issues such as parental/family consent, religious beliefs, and indifference to blood donation process were amongst the common reasons for non-donation. Many studies also failed to ask questions related to procedural information/instructions, which are necessary for promoting voluntary blood donations. Most published KAP studies for blood donation in India were not appropriately described, especially the methodology and result section. These deficiencies could have led to suboptimal interpretation of the prevalent issues. Use of an open-ended and validated KAP questionnaire with a problem-based approach and inclusion of participants from various socio-cultural backgrounds is required for good quality of evidence.

Evaluating the validity of game-based assessments measuring cognitive function among children and adolescents: A systematic review and meta-analysis.

Games offer advantages over traditional methods of assessing cognitive function among children and adolescents. However, the validity of game-based assessments has not been systematically evaluated. We conducted a systematic review and meta-analysis to assess the validity of game-based assessments measuring cognitive function among children and adolescents. We systematically searched several databases using pre-defined inclusion criteria. For papers that met the criteria, we extracted and analyzed the cognitive functions measured by each study, the correlation coefficients between game-based and traditional assessments, and factors that could influence the validity of game-based assessments. Our review identified 19 articles featuring 20 studies, 18 games, and 378 unique correlations between game-based and traditional assessments of cognitive function. Game-based assessments yielded significant correlations (n=282, 75%) with traditional assessments, over half of which were in the low to medium range in strength (r=0.3-0.69, n=227, 80%). Factors related to the child, such as age, gender, and prior gaming experience, may influence the validity of game-based assessments by modifying performance on game-based assessments. In addition, we found that game-based assessments that measured cognitive functions across more than one neurocognitive domain and used a prediction model for scoring were more likely to yield significant correlations. In contrast, including a narrative storyline in a game-based assessment was less likely to yield significant correlations. Most studies were of good quality, although the lack of sample size justification was a limiting factor. Further research is needed to elucidate the influence of identified factors on the validity of game-based assessment to justify the wide adoption of game-based assessments of cognitive function among children and adolescents.

Salivary Metabolomics for Systemic Cancer Diagnosis: A Systematic Review.

Cancers are the leading cause of death worldwide. The most common cancers include breast, lung, and colorectum. Salivary metabolome profiling is a novel non-invasive method in oncological diagnosis. This systematic review was designed to answer the question "Are salivary metabolites reliable for the diagnosis of systemic cancers?". Following the inclusion and exclusion criteria, nineteen studies were included (according to PRISMA statement guidelines). Changes in salivary metabolome were most commonly determined in patients with breast cancer, gastrointestinal cancers, and lung cancer. Most studies involved unstimulated whole saliva as the diagnostic material, evaluated by different spectroscopic methods. Among the found saliva metabolites, the alterations in the metabolic pathways of amino acids and polyamines were most frequently observed, which showed significant predictive values in oncological diagnostics. The most frequently encountered risks of bias were the absence of data regarding blinding, sample size justification, and randomisation. In conclusion, salivary metabolites seem to be potentially reliable for detecting the most common systemic cancers. However, further research is desirable to confirm these outcomes and to detect new potential metabolic biomarkers in saliva.

TRANSPARENCY, DOCUMENTATION, AND OPEN SCIENCE

Abstract Some GSA journals are especially interested in promoting transparency and open science practices, reflecting how some subdisciplines in aging are moving toward open science practices faster than others. In this talk, I will consider the transparency and open science practices that seem most relevant to aging researchers, such as preregistration, open data, open materials and code, sample size justification and analytic tools for considering null effects. I will also discuss potential challenges to implementing these practices as well as reasons why it is important to do so despite these challenges. The focus will be on pragmatic suggestions for researchers planning and conducting studies now that they hope to publish later.

Power to the People: A Beginner’s Tutorial to Power Analysis using jamovi

Authors have highlighted for decades that sample size justification through power analysis is the exception rather than the rule. Even when authors do report a power analysis, there is often no justification for the smallest effect size of interest, or they do not provide enough information for the analysis to be reproducible. We argue one potential reason for these omissions is the lack of a truly accessible introduction to the key concepts and decisions behind power analysis. In this tutorial targeted at complete beginners, we demonstrate a priori and sensitivity power analysis using jamovi for two independent samples and two dependent samples. Respectively, these power analyses allow you to ask the questions: “How many participants do I need to detect a given effect size?”, and “What effect sizes can I detect with a given sample size?”. We emphasise how power analysis is most effective as a reflective process during the planning phase of research to balance your inferential goals with your resources. By the end of the tutorial, you will be able to understand the fundamental concepts behind power analysis and extend them to more advanced statistical models.

Statistical power and sample size calculations for time-to-event analysis

ObjectiveTo provide thoracic and cardiovascular surgeons with the necessary tools for performing sample size and power calculations for studies with time-to-event outcomes. MethodsPower and sample size calculations enhance the overall quality of research studies by providing readers with assurance and insight into the number of patients included in the study. A 5-step approach is presented for performing sample size calculations in comparing groups on time-to-event endpoints. The steps are as follows: (1) identify the primary outcome of interest, (2) define size of the effect and desired power, (3) determine the appropriate statistical test, (4) perform calculations of the required sample size, and (5) write formal power and sample size statement. This approach is demonstrated with 5 clinical examples for time-to-event studies in cardiovascular surgery, featuring Cox regression, 2-sample log-rank test, 1-sample log-rank test, and competing risks analysis. ConclusionsStatistical power is an essential element for designing studies to ensure sufficient sample sizes for detecting treatment effects or group differences in time-to-event patient outcomes. Power and sample size justification not only adds statistical rigor and credibility to research manuscripts, but also provides the reader with assurance that the findings and conclusions are valid and based on a sufficient number of patients.

A descriptive study of samples sizes used in agreement studies published in the PubMed repository

IntroductionA sample size justification is required for all studies and should give the minimum number of subjects to be recruited for the study to achieve its primary objective. The aim of this review is to describe sample sizes from agreement studies with continuous or categorical endpoints and different methods of assessing agreement, and to determine whether sample size justification was provided.MethodsData were gathered from the PubMed repository with a time interval of 28th September 2018 to 28th September 2020. The search returned 5257 studies of which 82 studies were eligible for final assessment after duplicates and ineligible studies were excluded.ResultsWe observed a wide range of sample sizes. Forty-six studies (56%) used a continuous outcome measure, 28 (34%) used categorical and eight (10%) used both. Median sample sizes were 50 (IQR 25 to 100) for continuous endpoints and 119 (IQR 50 to 271) for categorical endpoints. Bland–Altman limits of agreement (median sample size 65; IQR 35 to 124) were the most common method of statistical analysis for continuous variables and Kappa coefficients for categorical variables (median sample size 71; IQR 50 to 233). Of the 82 studies assessed, only 27 (33%) gave justification for their sample size.ConclusionsDespite the importance of a sample size justification, we found that two-thirds of agreement studies did not provide one. We recommend that all agreement studies provide rationale for their sample size even if they do not include a formal sample size calculation.