Sample Size Calculations For Trials Research Articles

Sample size calculation in clinical trials with two co-primary endpoints including overdispersed count and continuous outcomes.

Count outcomes are collected in clinical trials for new drug development in several therapeutic areas and the event rate is commonly used as a single primary endpoint. Count outcomes that are greater than the mean value are termed overdispersion; thus, count outcomes are assumed to have a negative binomial distribution. However, in clinical trials for treating asthma and chronic obstructive pulmonary disease (COPD), a regulatory agency has suggested that a continuous endpoint related to lung function must be evaluated as a primary endpoint in addition to the event rate. The two co-primary endpoints that need to be evaluated include overdispersed count and continuous outcomes. Some researchers have proposed sample size calculation methods in the context of co-primary endpoints for various outcome types. However, methodologies for sample size calculation in trials with two co-primary endpoints, including overdispersed count and continuous outcomes, required when planning clinical trials for treating asthma and COPD, remain to be proposed. In this study, we aimed to develop a hypothesis-testing method and a corresponding sample size calculation method with two co-primary endpoints including overdispersed count and continuous outcomes. In a simulation, we demonstrated that the proposed sample size calculation method has adequate power accuracy. In addition, we illustrated an application of the proposed sample size calculation method to a placebo-controlled Phase 3 trial for patients with COPD.

Potential of ChatGPT in facilitating research in radiation oncology?

PurposeTo evaluate the potential of the artificial intelligence (AI) chatbot ChatGPT in supporting young clinical scientists with scientific tasks in radio oncological research. Materials and MethodsSeven scientific tasks were to be completed in 3 h by 8 radiation oncologists with different scientific experience working at a university hospital: creation of a scientific synopsis, creation of a research question and corresponding clinical trial hypotheses, writing of the first paragraph of a manuscript introduction, clinical trial sample size calculation, and clinical data analyses (multivariate analysis, boxplot and survival curve). No participant had prior experience with an AI chatbot. All participants were instructed in ChatGPT v3.5 and its use was provided for all tasks. Answers were scored independently by two blinded experts. The subjective value of ChatGPT was rated by each participant. Data were analyzed with regression-, t-test and Spearman correlation (p < 0.05). ResultsParticipants completed tasks 1–3 with an average score of 50% and 4–7 with 56%. Scientific experience, number of original publications and of first/last authorships showed a positive correlation with overall scoring (p = 0.01–0.04). Participants with little to moderate scientific experience scored ChatGPT to be more helpful in solving tasks 4–7 compared to more experienced participants (p = 0.04), with simultaneously presenting lower scorings (p = 0.03). ConclusionsChatGPT did not compensate for differences in scientific experience of young clinical scientists, with less experienced researchers believing false AI-generated scientific results.

Do sample size calculations in longitudinal orthodontic trials use the advantages of this study design?

To examine whether optimal calculations of the sample size are being used in longitudinal orthodontic trials. Longitudinal orthodontic trials with a minimum of three time points of outcome assessment published between January 1, 2017, and December 30, 2020, were sourced from a single electronic database. Study characteristics at the level of each trial were undertaken independently and in duplicate. Descriptive statistics and summary values were calculated. Inferential statistics (Fisher's exact test and logistic regression) were applied to detect associations between reporting of a sample size calculation and the study characteristics. A total of 147 trials were analyzed; 75.5% of these trials reported a sample size calculation with none reporting optimal sample size calculation for longitudinal trials. Most of the longitudinal orthodontic trials did not report the correlation and the number of longitudinal measurements in calculating the sample size. An association between reporting of a sample size calculation (yes or no) and the type of journal (orthodontic and non-orthodontic) was detected with higher odds of reporting a sample size calculation in orthodontic journals than in non-orthodontic journals (3.04; 95% confidence interval, 1.4-6.59; P < .01). The findings of this study highlighted that the undertaking of optimal sample size calculations in longitudinal orthodontic trials is being underused. Greater awareness of the variables required for undertaking the correct sample size calculation in these trials is required to reduce suboptimal research practices.

Power and sample-size calculations for trials that compare slopes over time: Introducing the slopepower command.

Trials of interventions that aim to slow disease progression may analyze a continuous outcome by comparing its change over time-its slope-between the treated and the untreated group using a linear mixed model. To perform a sample-size calculation for such a trial, one must have estimates of the parameters that govern the between- and within-subject variability in the outcome, which are often unknown. The algebra needed for the sample-size calculation can also be complex for such trial designs. We have written a new user-friendly command, slopepower, that performs sample-size or power calculations for trials that compare slope outcomes. The package is based on linear mixed-model methodology, described for this setting by Frost, Kenward, and Fox (2008, Statistics in Medicine 27: 3717-3731). In the first stage of this approach, slopepower obtains estimates of mean slopes together with variances and covariances from a linear mixed model fit to previously collected user-supplied data. In the second stage, these estimates are combined with user input about the target effectiveness of the treatment and design of the future trial to give an estimate of either a sample size or a statistical power. In this article, we present the slopepower command, briefly explain the methodology behind it, and demonstrate how it can be used to help plan a trial and compare the sample sizes needed for different trial designs.

Test-Retest Reliability and Consistency of HVPG and Impact on Trial Design: A Study in 289 Patients from 20 Randomized Controlled Trials.

Portal hypertension (PH) is a major driver for cirrhosis complications. Portal pressure is estimated in practice by the HVPG. The assessment of HVPG changes has been used for drug development in PH. This study aimed at quantifying the test-retest reliability and consistency of HVPG in the specific context of randomized controlled trials (RCTs) for the treatment of PH in cirrhosis and its impact on power calculations for trial design. We conducted a search of published RCTs in patients with cirrhosis reporting individual patient-level data of HVPG at baseline and after an intervention, which included a placebo or an untreated control arm. Baseline and follow-up HVPGs in the control groups were extracted after digitizing the plots. We assessed reliability and consistency and the potential impact of study characteristics. We retrieved a total of 289 before and after HVPG measurements in the placebo/untreated groups from 20 RCTs. The time span between the two HVPG measurements ranged between 20minutes and 730days. Pre-/post-HVPG variability was lower in studies including only compensated patients; therefore, modeled sample size calculations for trials in compensated cirrhosis were lower than for decompensated cirrhosis. A higher proportion of alcohol-associated cirrhosis and unicentric trials was associated with lower differences between baseline and follow-up measurements. The smallest detectable difference in an individual was 26% and 30% in compensated and decompensated patients, respectively. The test-retest reliability of HVPG is overall excellent. Within-individual variance was higher in studies including higher proportions of decompensated patients. These findings should be taken into account when performing power analysis for trials based on the effects on HVPG or when considering HVPG as a tool to guide therapy of PH.

Sample size calculation using Markov chains for a one-arm study of heroin administration routes

ABSTRACT Sample size calculations for trials with time-to-event outcomes are usually based on the assumption that an event – prototypically death in survival analysis – occurs only once per sample unit. However, events like changes in disease status or switches between treatment modalities may repeat over time. In trials with such outcomes, standard sample size formulae derived from the classical survival time models are not applicable. Instead, modeling the repeating transition events must precede the actual sample size calculation. Markov chains are an obvious choice to model transitions. Accordingly, in order to determine the sample size for a one-arm feasibility and acceptability study of a new drug intake route, we model switches of administration routes by a homogeneous finite-state, higher-order Markov chain. Assumptions about its transition matrix translate into multinomial distributions of the preferred administration routes at given points in time. From these distributions, the required sample size can then be calculated according to the study’s specific question. In this manuscript, we first introduce the method for the case of drug intake preferences, before we briefly discuss how the proposed method can also be used for power-based sample size calculation in multi-arm trials.

Clinically relevant sample size calculations for trials.

An orthodontist attends the scientific orthodontic congress and passing by the company exhibitions receives a free sample of a novel orthodontic archwire introduced in the market for initial alignment/levelling with favourable performance over conventional superelastic nickel-titanium archwires. This novel archwire should increase according to the manufacturer’s claims alignment efficacy and halve the pain for the patient. After using the free sample wire on a patient and observing that the patient felt little to no pain during the alignment/levelling phase, the orthodontist decides to venture into unknown territory and find out if this holds true for most of her patients by doing a randomised clinical trial in her practice.

Are non-constant rates and non-proportional treatment effects accounted for in the design and analysis of randomised controlled trials? A review of current practice

BackgroundMost clinical trials with time-to-event primary outcomes are designed assuming constant event rates and proportional hazards over time. Non-constant event rates and non-proportional hazards are seen increasingly frequently in trials. The objectives of this review were firstly to identify whether non-constant event rates and time-dependent treatment effects were allowed for in sample size calculations of trials, and secondly to assess the methods used for the analysis and reporting of time-to-event outcomes including how researchers accounted for non-proportional treatment effects.MethodsWe reviewed all original reports published between January and June 2017 in four high impact medical journals for trials for which the primary outcome involved time-to-event analysis. We recorded the methods used to analyse and present the main outcomes of the trial and assessed the reporting of assumptions underlying these methods. The sample size calculation was reviewed to see if the effect of either non-constant hazard rates or anticipated non-proportionality of the treatment effect was allowed for during the trial design.ResultsFrom 446 original reports we identified 66 trials with a time-to-event primary outcome encompassing trial start dates from July 1995 to November 2014. The majority of these trials (73%) had sample size calculations that used standard formulae with a minority of trials (11%) using simulation for anticipated changing event rates and/or non-proportional hazards. Well-established analytical methods, Kaplan-Meier curves (98%), the log rank test (88%) and the Cox proportional hazards model (97%), were used almost exclusively for the main outcome. Parametric regression models were considered in 11% of the reports. Of the trials reporting inference from the Cox model, only 11% reported any results of testing the assumption of proportional hazards.ConclusionsOur review confirmed that when designing trials with time-to-event primary outcomes, methodologies assuming constant event rates and proportional hazards were predominantly used despite potential efficiencies in sample size needed or power achieved using alternative methods. The Cox proportional hazards model was used almost exclusively to present inferential results, yet testing and reporting of the pivotal assumption underpinning this estimation method was lacking.

Accounting for twin births in sample size calculations for randomised trials.

Including twins in randomised trials leads to non-independence or clustering in the data. Clustering has important implications for sample size calculations, yet few trials take this into account. Estimates of the intracluster correlation coefficient (ICC), or the correlation between outcomes of twins, are needed to assist with sample size planning. Our aims were to provide ICC estimates for infant outcomes, describe the information that must be specified in order to account for clustering due to twins in sample size calculations, and develop a simple tool for performing sample size calculations for trials including twins. ICCs were estimated for infant outcomes collected in four randomised trials that included twins. The information required to account for clustering due to twins in sample size calculations is described. A tool that calculates the sample size based on this information was developed in Microsoft Excel and in R as a Shiny web app. ICC estimates ranged between -0.12, indicating a weak negative relationship, and 0.98, indicating a strong positive relationship between outcomes of twins. Example calculations illustrate how the ICC estimates and sample size calculator can be used to determine the target sample size for trials including twins. Clustering among outcomes measured on twins should be taken into account in sample size calculations to obtain the desired power. Our ICC estimates and sample size calculator will be useful for designing future trials that include twins. Publication of additional ICCs is needed to further assist with sample size planning for future trials.

Current practice in methodology and reporting of the sample size calculation in randomised trials of hip and knee osteoarthritis: a systematic review

Purpose: This study aims to review the methodology and reporting of sample size calculation in a contemporary sample of hip and knee osteoarthritis trials. The choice of sample size is a key aspect of designing a clinical trial, ensuring that the trial is powered to detect clinically important treatment effects, is ethically sound and does not overuse resources. Accurate and complete reporting of the sample size calculation allows the reader to interpret the trial results alongside the aims and assumptions made when the trial was designed. Methods: This study is a systematic review of randomised controlled trials in hip and/or knee osteoarthritis published in 2016. Studies were identified by searching MEDLINE, Cochrane library, CINAHL, EMBASE, PsycINFO, PEDro and AMED. Data was extracted on study characteristics, methods used to calculate the sample size, and reporting and justification of individual components used in the sample size calculation. The reported information was used to replicate the sample size calculation. The standard deviation assumed in the sample size calculation was compared to the corresponding value in the study results. The results were synthesised using the number and proportion of studies for categorical outcomes and the median and interquartile range for continuous outcomes. Subgroup analysis was conducted to test for differences in reporting quality based on funding source, type of intervention and control treatment, and number of study centres. Results: We identified 116 eligible trials for inclusion in this review. The majority were parallel-group, superiority, single-centre trials of knee osteoarthritis funded by non-industry sources. The number of randomised participants ranged from 20 to 633 (median 73). Of these, 78/116 (67%) reported a power calculation. Less than a quarter of studies reported all core components of the sample size calculation (21%, 16/78). The power and level of statistical significance were reported in almost all trials (96%, 75/78). The level of attrition assumed was not reported or unclear in a quarter of trials (27% n = 21/78). Of trials powered on a single continuous outcome, two-thirds of studies reported the standard deviation (67%, 51/76) and less than half provided a justification for the value used (46%, 35/76). In trials with continuous outcomes, the justification for the mean difference was provided in 40 trials (53%, 40/76). The mean difference was most commonly based on the results of a previously published trial (29%, 22/76) or a published minimum clinically important difference (17%, 13/76). Of the 78 trials which reported a power calculation, the sample size was only reproducible in half of the trials (53%, n = 41/78) and assumptions were often needed when information was ambiguous or not reported. A quarter of trials did not provide sufficient information for the sample size calculation to be replicated (28%, n = 22/78). The replicated calculation produced a sample size over 10% larger than the reported value in 12% of studies (n = 9/78). There were 4 studies where the difference between the reproduced and reported sample size was greater than 50 participants. When comparing the standard deviation of the primary outcome, the estimate used in the sample size calculation was accurate (within 10%) in only a third of studies where they could be compared (24%, n = 9/29). In 6 studies, the follow-up standard deviation was over 30% larger than the value assumed in the sample size calculation leading to a reduction in power. Conclusions: Sample size calculations in trials of hip and knee osteoarthritis are not reported adequately. Even where there is sufficient information, the calculation cannot be accurately reproduced. Furthermore, the assumptions made can also be insufficient. This raises questions about the robustness of the findings, and whether these trials are able to address their stated principal aim.

Current practice in methodology and reporting of the sample size calculation in randomised trials of hip and knee osteoarthritis: a protocol for a systematic review

BackgroundA key aspect of the design of randomised controlled trials (RCTs) is determining the sample size. It is important that the trial sample size is appropriately calculated. The required sample size will differ by clinical area, for instance, due to the prevalence of the condition and the choice of primary outcome. Additionally, it will depend upon the choice of target difference assumed in the calculation. Focussing upon the hip and knee osteoarthritis population, this study aims to systematically review how the trial size was determined for trials of osteoarthritis, on what basis, and how well these aspects are reported.MethodsSeveral electronic databases (Medline, Cochrane library, CINAHL, EMBASE, PsycINFO, PEDro and AMED) will be searched to identify articles on RCTs of hip and knee osteoarthritis published in 2016. Articles will be screened for eligibility and data extracted independently by two reviewers. Data will be extracted on study characteristics (design, population, intervention and control treatments), primary outcome, chosen sample size and justification, parameters used to calculate the sample size (including treatment effect in control arm, level of variability in primary outcome, loss to follow-up rates). Data will be summarised across the studies using appropriate summary statistics (e.g. n and %, median and interquartile range). The proportion of studies which report each key component of the sample size calculation will be presented. The reproducibility of the sample size calculation will be tested.DiscussionThe findings of this systematic review will summarise the current practice for sample size calculation in trials of hip and knee osteoarthritis. It will also provide evidence on the completeness of the reporting of the sample size calculation, reproducibility of the chosen sample size and the basis for the values used in the calculation.Trial registrationAs this review was not eligible to be registered on PROSPERO, the summary information was uploaded to Figshare to make it publicly accessible in order to avoid unnecessary duplication amongst other benefits (https://doi.org/10.6084/m9.figshare.5009027.v1); Registered January 17, 2017.

The predictive distribution of the residual variability in the linear-fixed effects model for clinical cross-over trials.

In the linear model for cross-over trials, with fixed subject effects and normal i.i.d. random errors, the residual variability corresponds to the intraindividual variability. While population variances are in general unknown, an estimate can be derived that follows a gamma distribution, where the scale parameter is based on the true unknown variability. This gamma distribution is often used for the sample size calculation for trial planning with the precision approach, where the aim is to achieve in the next trial a predefined precision with a given probability. But then the imprecision in the estimated residual variability or, from a Bayesian perspective, the uncertainty of the unknown variability is not taken into account. Here, we present the predictive distribution for the residual variability, and we investigate a link to the F distribution. The consequence is that in the precision approach more subjects will be necessary than with the conventional calculation. For values of the intraindividual variability that are typical of human pharmacokinetics, that is a gCV of 17-36%, we would need approximately a sixth more subjects.

The Sleep Apnea cardioVascular Endpoints (SAVE) Trial: Rationale, Ethics, Design, and Progress.

The Sleep Apnea cardioVascular Endpoints (SAVE) study is an ongoing investigator-initiated and conducted, international, multicenter, open, blinded endpoint, randomized controlled trial that was designed to determine whether treatment of obstructive sleep apnea (OSA) with continuous positive airways pressure (CPAP) can reduce the risk of serious cardiovascular (CV) events in patients with established CV disease (clinical trial registration NCT00738179). The results of this study will have important implications for the provision of health care to patients with sleep apnea around the world. The SAVE study has brought together respiratory, sleep, CV and stroke clinicians-scientists in an interdisciplinary collaboration with industry and government sponsorship to conduct an ambitious clinical trial. Following its launch in Australia and China in late 2008, the recruitment network expanded across 89 sites that included New Zealand, India, Spain, USA, and Brazil for a total of 2,717 patients randomized by December 2013. These patients are being followed until December 2015 so that the average length of follow-up of the cohort will be over 4 y. This article describes the rationale for the SAVE study, considerations given to the design including how various cultural and ethical challenges were addressed, and progress in establishing and maintaining the recruitment network, patient follow-up, and adherence to CPAP and procedures. The assumptions underlying the original trial sample size calculation and why this was revised downward in 2012 are also discussed. NCT00738179. ACTRN12608000409370.

How big should the pilot study for my cluster randomised trial be?

There is currently a lot of interest in pilot studies conducted in preparation for randomised controlled trials. This paper focuses on sample size requirements for external pilot studies for cluster randomised trials. We consider how large an external pilot study needs to be to assess key parameters for input to the main trial sample size calculation when the primary outcome is continuous, and to estimate rates, for example recruitment rates, with reasonable precision. We used simulation to provide the distribution of the expected number of clusters for the main trial under different assumptions about the natural cluster size, intra-cluster correlation, eventual cluster size in the main trial, and various decisions made at the piloting stage. We chose intra-cluster correlation values and pilot study size to reflect those commonly reported in the literature. Our results show that estimates of sample size required for the main trial are likely to be biased downwards and very imprecise unless the pilot study includes large numbers of clusters and individual participants. We conclude that pilot studies will usually be too small to estimate parameters required for estimating a sample size for a main cluster randomised trial (e.g. the intra-cluster correlation coefficient) with sufficient precision and too small to provide reliable estimates of rates for process measures such as recruitment or follow-up rates.

Abstract T P73: Overcoming Framing Bias in Stroke Neurologists’ Assessments of the Minimally Clinically Important Difference for Novel Acute Ischemic Stroke Therapies

Background: The determination of the minimally clinically important difference (MCID) worth detecting is an essential prerequisite for clinical trial sample size calculation, funding decisions by NIH and other sponsors, and drug or device approval by regulatory agency. However, prior surveys to establish expert judgment of MCID have had a cognitive framing bias by offering response options with midpoints in the 5-15% range, well above the treatment effect actually accepted in practice and national guidelines as worthwhile (1% for acute aspirin). Methods: A 5 item internet-based survey was administered to US academic stroke neurologists. Respondents were asked to indicate, for an acute neuroprotective drug free of side effects, how many patients, per 1000 treated, would need to benefit by achieving freedom from disability at 90 days (mRS 0-2) for the agent to be worthwhile to use in clinical practice? Results: Survey responses were received from 122 of 332 academic stroke neurologists. Among the participants, 38.0% were professors, 24.0% associate professors, 33.1% assistant professors, and 5.0% neurology residents or fellows. All respondents were engaged in stroke clinical care, with cerebrovascular disease accounting for more than half of practice time in over 75%. Categorized responses to the value of the MCID were: 1 per 1000 by 3.3% of respondents, 2-5 in 7.4%, 6-10 by 19.7%, 11-15 by 22.1%, 16-20 by 15.6%, and greater than 20 by 32.0%. The median MCID was 13 (IQR 8 to &gt; 20). Converted to natural base 100 values, the MCID was 1.3% (IQR 0.8% - &gt;2%). Conclusion: When assessed with framing based on clinical practice rather than convenience, vascular neurologists indicated the minimally clinical important difference for a safe agent to be worthwhile to use in acute ischemic stroke is about 1%. Drug and device agencies should consider this value to be the expert opinion MCID for acute ischemic stroke treatments when making regulatory decisions.

Specifying the target difference in the primary outcome for a randomised controlled trial: guidance for researchers.

BackgroundCentral to the design of a randomised controlled trial is the calculation of the number of participants needed. This is typically achieved by specifying a target difference and calculating the corresponding sample size, which provides reassurance that the trial will have the required statistical power (at the planned statistical significance level) to identify whether a difference of a particular magnitude exists. Beyond pure statistical or scientific concerns, it is ethically imperative that an appropriate number of participants should be recruited. Despite the critical role of the target difference for the primary outcome in the design of randomised controlled trials, its determination has received surprisingly little attention. This article provides guidance on the specification of the target difference for the primary outcome in a sample size calculation for a two parallel group randomised controlled trial with a superiority question.MethodsThis work was part of the DELTA (Difference ELicitation in TriAls) project. Draft guidance was developed by the project steering and advisory groups utilising the results of the systematic review and surveys. Findings were circulated and presented to members of the combined group at a face-to-face meeting, along with a proposed outline of the guidance document structure, containing recommendations and reporting items for a trial protocol and report. The guidance and was subsequently drafted and circulated for further comment before finalisation.ResultsGuidance on specification of a target difference in the primary outcome for a two group parallel randomised controlled trial was produced. Additionally, a list of reporting items for protocols and trial reports was generated.ConclusionsSpecification of the target difference for the primary outcome is a key component of a randomized controlled trial sample size calculation. There is a need for better justification of the target difference and reporting of its specification.Electronic supplementary materialThe online version of this article (doi:10.1186/s13063-014-0526-8) contains supplementary material, which is available to authorized users.

Improved repeatability of nasal potential difference with a larger surface catheter

ObjectiveTo increase the power of nasal potential difference (NPD) as a biomarker of CFTR function, improvement of its repeatability is needed. We evaluated the improvement in repeatability resulting from measuring NPD (1) over a larger surface area and (2) at a fixed location. MethodsTo assess repeatability, NPD was measured on two occasions with a new method using a larger surface catheter at fixed locations on the nasal floor (LSC-floor5cm and LSC-floor3cm) or at the most negative basal potential (LSC-floormax); with a sidehole catheter on the nasal floor at 5cm from the nasal margin (SHC-floor5cm) or at the most negative potential (SHC-floormax); and with an endhole catheter below the inferior surface of the lower turbinate at the most negative potential (EHC-turbmax). ResultsThe within-subject standard deviation (Sw) for repeated measurements of the total chloride response in the controls was smallest with the LSC-floor at a fixed location (LSC-floor5cm 3.1mV; 95% CI 2.3–4.6mV) and highest with the SHC-floor (SHC-floormax 14.6mV; 95% CI 10.9–22.2mV) or the EHC-turbinate (EHC-turbmax 12.5mV; 95% CI 10.7–23.0mV) at the most negative basal potential. Measuring with the LSC-floor at the maximal potential increased the Sw (LSC-floormax 8.8mV, 95% CI 6.0–16.1mV, p=0.009 vs LSC-floor5cm), while measuring with the SHC-floor at a fixed location slightly decreased the Sw (SHC-floor5cm 9.8mV, 95% CI 8.9–20.6mV, p=0.06 vs SHC-floormax). In patients with cystic fibrosis, the Sw was comparable, between 2.2mV and 4.3mV. Sample size calculations for trials using NPD to assess changes in ion transport showed that the number of subjects to be included could be approximately halved measuring with the larger surface catheter at a fixed location vs SHC or EHC at fixed locations. ConclusionMeasuring the NPD at a fixed location and over a larger surface resulted in increased repeatability and thereby also power as a biomarker of CFTR modulation.

Sample size calculation in trials of public health interventions: a discussion of implications for health economists

BackgroundStatistical analysis enables ascertainment of whether or not there are interesting differences in effects between two or more groups and allows inferences to be made about the population from which a sample comes. For those of us interested in these differences, to be able to report on their statistical significance is useful to help us remark on the confidence we have in our results. However, careful consideration should be given to the choice of sample size. We investigated recent developments in the methodological considerations surrounding issues of identification of an appropriate sample size for a study, investigating first the issue in the context of clinical trials before going on to discuss the issue in terms of public health. MethodsIn this discussion piece, we highlight recent developments in the area of sample size calculation. We then offer two case studies that provide examples of sample size estimation under different scenarios: when no power or sample size calculations have been mentioned and when power or sample size calculations have been done properly. The first was done by the UK prospective diabetes study group (1998) in which the sample size was 1148, although there was no explicit mention of how this sample size was calculated. The second case study was an example by Briggs and Gray (1998) on the study of intracranial aneurysms. They plotted the sample size requirements as a function of the maximum cost-effectiveness ratio. One can then, for a given level of cost effectiveness, work out the sample size needed for different levels of power. FindingsAlthough clinical trials calculate sample sizes on the basis of clinical outcomes, we show that these sample sizes might not provide enough power for any economic assessment we might want to undertake, because economic assessments deal with both costs and treatment effects. Usually, economic assessment relates to estimation rather than hypothesis testing. Therefore, calculations of power and sample size in economic assessment are done in relation to some value of maximum willingness to pay (WTP) for a unit of treatment effect. The upper confidence limit of the incremental cost-effectiveness ratio of a cost-effective treatment must fall below the value of the maximum WTP and the sample size must provide enough power for this to be possible. Further, in economic assessment of public health interventions, the effective sample size might be less than the actual sample size used because of intra-cluster correlation and so the sample size must be corrected for this factor. InterpretationWe argue that a systematic review of the published work is needed to highlight the state of play with regard to sample size estimation, especially in economic assessment of public health interventions. By methodically collating the available evidence, the case for best practice in choosing the appropriate sample size can be put forward and progress can be made in increasing the number of studies that are sufficiently powered. FundingNone.

Sample size in orthodontic randomized controlled trials: are numbers justified?

Sample size calculations are advocated by the Consolidated Standards of Reporting Trials (CONSORT) group to justify sample sizes in randomized controlled trials (RCTs). This study aimed to analyse the reporting of sample size calculations in trials published as RCTs in orthodontic speciality journals. The performance of sample size calculations was assessed and calculations verified where possible. Related aspects, including number of authors; parallel, split-mouth, or other design; single- or multi-centre study; region of publication; type of data analysis (intention-to-treat or per-protocol basis); and number of participants recruited and lost to follow-up, were considered. Of 139 RCTs identified, complete sample size calculations were reported in 41 studies (29.5 per cent). Parallel designs were typically adopted (n = 113; 81 per cent), with 80 per cent (n = 111) involving two arms and 16 per cent having three arms. Data analysis was conducted on an intention-to-treat (ITT) basis in a small minority of studies (n = 18; 13 per cent). According to the calculations presented, overall, a median of 46 participants were required to demonstrate sufficient power to highlight meaningful differences (typically at a power of 80 per cent). The median number of participants recruited was 60, with a median of 4 participants being lost to follow-up. Our finding indicates good agreement between projected numbers required and those verified (median discrepancy: 5.3 per cent), although only a minority of trials (29.5 per cent) could be examined. Although sample size calculations are often reported in trials published as RCTs in orthodontic speciality journals, presentation is suboptimal and in need of significant improvement.

Systematic Review of Measures of Clinical Significance Employed in Randomized Controlled Trials of Drugs for Dementia

The ability to define thresholds for the clinical significance (clinical importance) of outcome measures in dementia drug research is critical to determining the changes on outcome measures that patients and families would consider worth the cost and worth risking the side effects of medications (i.e., clinical importance is central to informed consent when starting such medications). Thresholds for clinical significance are also required for drug trial sample size calculation (if the studies are to be powered to detect clinically important difference) and for decisions regarding whether medications should be accepted on formularies and should be funded. To better understand what measures of clinical significance have been employed in dementia drug research, a systematic review was performed of double-blind randomized controlled trials (RCTs) of cholinesterase inhibitors and memantine in subjects with Alzheimer's disease, vascular dementia, mixed dementia, and mild cognitive impairment. Of the 57 dementia drug RCTs reviewed, only 46% discussed the clinical significance of their results. The most commonly cited measures of clinical significance were a 4-point change in the Alzheimer's Disease Assessment Scale--Cognitive Subscale and changes on global scales. The majority of measures of clinical significance were opinion based. Only one study empirically measured patient perspective regarding thresholds for clinical significance. Despite being central to the interpretation of trial results and to decisions regarding whether to employ trial findings in clinical practice, patient- and caregiver-centered measures of clinical significance have not been adequately studied and integrated into dementia drug RCTs. It is recommended that discussions of clinical importance receive greater emphasis in research standards published by organizations such as the CONSORT group (http://www.consort-statement.org). Drug formulary review committees and licensing agencies (e.g., U.S. Food and Drug Administration, European Agency for the Evaluation of Medicinal Products, Health Canada) should consider requiring an assessment of clinical significance of the drugs they review. To move this field forward, funding agencies should consider initiating requests for proposals focused on the determination of the minimally clinically important difference (MCID) of outcome measures employed in dementia research. Once empirical data on MCIDs are available, then these funding agencies should consider supporting a consensus conference to review and select the optimal measures of clinical importance in dementia research. A preliminary organizational framework of measures of clinical significance is presented in this article to facilitate the work of such a consensus forum.