Two-group Studies Research Articles

Exceedance Probabilities of Log-normal Distributions for One Group, Two Groups, and Meta-analysis of Multiple Two-group Studies, With Application to Analyses of Prolonged Times to Tracheal Extubation

Many randomized trials measure means and standard deviations of anesthesia recovery time (e.g., times to tracheal extubation). We show how to use generalized pivotal methods to compare the probabilities of exceeding a tolerance limit (e.g., > 15min, prolonged times to tracheal extubation). The topic isimportant because the economic benefits of faster anesthesia emergence depend onreducing variability, not means, especially prevention of very long recovery times. Generalized pivotal methods are applied using computer simulation (e.g., using two Excel formulas for one group and three formulas for two group comparisons). The endpoint for each study with two groups is the ratio between groups of the probabilities of times exceeding a threshold or the ratio of the standard deviations. Confidence intervals and variances for the incremental risk ratio of the exceedance probabilities and for ratios of standard deviations are calculated using studies' sample sizes, sample means in the time scale of recovery times, and sample standard deviations in the time scale. Ratios are combined among studies using the DerSimonian-Laird estimate of the heterogeneity variance estimate, with Knapp-Hartung adjustment for the relatively small (N = 15) numbers of studies in the meta-analysis. We show larger absolute variability among studies' results when analyzed based on exceedance probabilities rather than standard deviations. Therefore, if an investigator's primary goal is to quantify reductions in the variability of recovery times (e.g., times until patients are ready for post-anesthesia care unit discharge), we recommend analyzing the standard deviations. When exceedance probabilities themselves are relevant, they can be analyzed from the original studies' summary measures.

SampleSizePlanner: A Tool to Estimate and Justify Sample Size for Two-Group Studies

Planning sample size often requires researchers to identify a statistical technique and to make several choices during their calculations. Currently, there is a lack of clear guidelines for researchers to find and use the applicable procedure. In the present tutorial, we introduce a web app and R package that offer nine different procedures to determine and justify the sample size for independent two-group study designs. The application highlights the most important decision points for each procedure and suggests example justifications for them. The resulting sample-size report can serve as a template for preregistrations and manuscripts.

Synchronous distance education vs traditional education for health science students: A systematic review and meta-analysis.

Synchronous distance education (SDE) has been widely used for health science students in recent years. This study examined the effectiveness and acceptance of SDE compared with traditional education for health science students and explored the potential moderators that could impact the pooled results. A systematic review and meta-analysis was conducted of randomised controlled trials (RCTs) from January 2000 to March 2020 searched on nine electronic databases, including Web of Science, PubMed, Cochrane Library, Scopus, EMBASE, CINAHL, ERIC, PsycINFO, and ProQuest Dissertations and Theses. The outcomes measured were knowledge, skills with objective assessments and overall satisfaction with subjective evaluations. The pooled results were calculated using random-model effects, and moderators were explored through meta-regression. A total of seven RCTs with 594 participants were included. At the post-test level, the pooled effect size of knowledge acquisitions (SMD 0.12, 95% CI -0.07-0.32) showed insignificant difference between the SDE and traditional education groups (P=.207), with low heterogeneity (I2 =17.6%). Subgroup analyses observed no factors that significantly impacted the pooled results of knowledge acquisition at the post-test levels (P for interaction>0.05). Knowledge gains from pretest to post-test in SDE groups also did not differ significantly between groups (SMD 0.15, 95% CI -0.22-0.53; P=.428). The pooled effect size of skills (SMD 0.02, 95% CI -0.24-0.28; P=.735) was similarly insignificant. The pooled effect size of overall satisfaction (SMD 0.60, 95% CI 0.38-0.83; P<.001) significantly favoured SDE over traditional education. Incorporating two-group studies without randomisations did not significantly change the overall results of knowledge acquisition at the post-test level (SMD -0.002, 95% CI -0.11-0.10; P=.994), with moderate heterogeneity (I2 =61.9%). Synchronous distance education was not significantly different from traditional education in effectiveness and had higher satisfaction ratings. Our findings might provide indications for adoptions of online remote education in health science education centres.

Complementary hypotheses in safety surveillance

Postmarketing safety surveillance studies address two actionable questions: (1) Is the test product riskier than a standard? (2) Is the risk associated with the test product within some tolerable margin by comparison to the standard? Established techniques, not commonly applied to the setting of such complementary one-sided hypotheses, lead to useful conclusions in practice. For two-group studies, a search over possible one-sided binomial test results yields sample sizes that guarantee that the confidence bounds exclude one or the other of the hypotheses. With continuous monitoring, simple curtailment reduces the sample size. Point and interval estimates follow from the binomial distribution of events at the end of the study or from component negative binomials for crossing a bound of simple curtailment with continuous monitoring and earlier stopping. An asymptotic derivation corresponds to the problem of constructing a confidence interval that is smaller than the distance between the parameter values for tolerable excess and the absence of excess risk. Studies with guaranteed rejection of one of the pair of complementary hypotheses are somewhat larger than corresponding studies of a single hypothesis under usual power requirements, but the increase may be tolerable in return for certainty that there will be an actionable conclusion.

Meta-analysis of the difference of medians.

We consider the problem of meta-analyzing two-group studies that report the median of the outcome. Often, these studies are excluded from meta-analysis because there are no well-established statistical methods to pool the difference of medians. To include these studies in meta-analysis, several authors have recently proposed methods to estimate the sample mean and standard deviation from the median, sample size, and several commonly reported measures of spread. Researchers frequently apply these methods to estimate the difference of means and its variance for each primary study and pool the difference of means using inverse variance weighting. In this work, we develop several methods to directly meta-analyze the difference of medians. We conduct a simulation study evaluating the performance of the proposed median-based methods and the competing transformation-based methods. The simulation results show that the median-based methods outperform the transformation-based methods when meta-analyzing studies that report the median of the outcome, especially when the outcome is skewed. Moreover, we illustrate the various methods on a real-life dataset.

Interventions to Prevent and Reduce Burnout Among Undergraduate and Graduate Medical Education Trainees: a Systematic Review.

The authors conducted a systematic review of the published literature to identify interventions to prevent and/or reduce burnout among medical students and residents. The authors searched 10 databases (from the start of each through September 21, 2016) using keywords related to burnout, medical education, and prevention. Teams of two authors independently reviewed the search results to select peer-reviewed, English language articles describing educational interventions to prevent and/or reduce burnout among medical students and/or residents that were evaluated using validated burnout measures. They assessed study quality using the Medical Education Research Study Quality Instrument and the Cochrane Risk of Bias Tool. Fourteen studies met inclusion criteria and all used the Maslach Burnout Inventory as at least one measure of burnout. Four were single group pre-post studies, 6 non-randomized two-group studies, and 4 randomized controlled trials. None of the studies were designed specifically to target burnout prevention. In 12 studies, residents were the targeted learners. Six of the 14 studies reported statistically significant changes in burnout scores: 5 reported improvement and 1 reported worsening of burnout. Of the 5 studies that reported statistically significant benefit, 1 studied a complementary and alternative medicine elective, 1 studied the Respiratory One Meditation method, and 3 studied duty hour changes. This review highlights the need for rigorously designed studies in burnout prevention and reduction among residents and especially medical students.

The impact of a two or three-group RCT design on blinding of patients

In RCTs using subjective outcomes, blinding of patients to treatment allocation is recommended. It is unknown whether two or three-group trial designs influence blinding success. We examined the success of blinding patients within a three or two-group pilot RCT in surgery. Cancer centres randomised patients to a three or two-group trial comparing: (i) standard open surgery; (ii) combination open and keyhole surgery, or (in one centre); (iii) totally keyhole surgery. Feasibility of blinding patients for seven days post-surgery to minimise bias in pain assessments was explored by using identical wound dressings covering all incisions. On days two and six, patients completed the Bang Blinding Index (BBI). This measures blinding success by asking patients to guess their treatment allocation. Scores range from -1 (more wrong guesses than expected) to 1 (more correct guesses), with 0 indicating perfect blinding (random guessing). Results were compared between the three and two-group studies. The study recruited 70 patients (42 and 28 three and two-group respectively). Data indicated successful blinding in the three-group study, with fewer patients than expected guessing correctly (day two BBI scores by group (i)0.00, (ii)0.14, (iii)-0.13; day six: (i)-0.13, (ii)0.38, (iii)0.04). In the two-group study, slightly more patients became unblinded, with more than expected guessing they had combination surgery (day two: (i) -0.73, (ii) 0.35; day six: (i) - 0.25, (ii) 0.57). This pilot study successfully blinded patients in a three-group study. However, in the two-group study more patients had become unblinded. This suggests that blinding is more successful in multi-arm studies.

Outcomes of gait trainer use in home and school settings for children with motor impairments: a systematic review

Objective: To summarize and critically appraise evidence regarding use of gait trainers (walkers providing trunk and pelvic support) at home or school with children who are unable to walk independently or with hand-held walkers. Data sources: Searches were performed in seven electronic databases including EBM Reviews, CINAHL, Medline and EMBASE for publications in English from database inception to November 2014. Review methods: Included studies involved at least one child with a mobility limitation and measured an outcome related to gait trainer use. Articles were appraised using American Academy of Cerebral Palsy and Developmental Medicine criteria for group and single-subject designs and quality ratings completed for studies rated levels I-III. The PRISMA statement was followed with inclusion criteria set a priori. Two reviewers independently screened titles, abstracts and full-text articles. Results: Seventeen studies involving 182 children were included. Evidence from one small randomized controlled trial suggests a non-significant trend toward increased walking distance while the other evidence level II study (concurrent multiple baseline design) reports increased number of steps. Two level III studies (non-randomized two-group studies) report statistically significant impact on mobility level with one finding significant impact on bowel function and an association between increased intervention time and bone mineral density. Remaining descriptive level evidence provides support for positive impact on a range of activity outcomes, with some studies reporting impact on affect, motivation and participation with others. Conclusions: Evidence supporting outcomes for children using gait trainers is primarily descriptive and, while mainly positive, is insufficient to draw firm conclusions.

Randomized controlled trials 5: Determining the sample size and power for clinical trials and cohort studies.

Performing well-powered randomized controlled trials is of fundamental importance in clinical research. The goal of sample size calculations is to assure that statistical power is acceptable while maintaining a small probability of a type I error. This chapter overviews the fundamentals of sample size calculation for standard types of outcomes for two-group studies. It considers (1) the problems of determining the size of the treatment effect that the studies will be designed to detect, (2) the modifications to sample size calculations to account for loss to follow-up and nonadherence, (3) the options when initial calculations indicate that the feasible sample size is insufficient to provide adequate power, and (4) the implication of using multiple primary endpoints. Sample size estimates for longitudinal cohort studies must take account of confounding by baseline factors.

Exercise programs to improve gait performance in people with lower limb amputation: A systematic review.

Few studies have explored the effects of exercise on gait performance in people with lower limb amputations. To (1) summarize the effects of exercise programs on gait performance and (2) assess the overall quality of the evidence for adults ambulating with leg prostheses. Systematic review. Six databases were searched for one- and two-group studies published through June 2013 reporting effects of exercise on gait speed, a universal measure of performance in lower limb prosthetic users. The search adhered to a predetermined protocol following Cochrane Collaboration guidelines. In all, 623 citations were reviewed and eight studies included. The quality level of the combined evidence was low with few randomized control trials and multiple sources of bias evident within the heterogeneous group of studies. The 11 exercise programs, including three control conditions, demonstrated small to large effect size improvements in self-selected gait speed. Use of exercise to improve gait speed was supported by low-quality level evidence, with low-moderate quality evidence to suggest that specific functional exercise programs were more effective than supervised walking. Using exercise to improve gait speed in people with lower limb amputation received a B grade recommendation. Future high-quality research is required. Supervised walking, muscle strengthening, balance exercises, gait training, and functional training programs demonstrated small to large effect size gait performance improvements in people with lower limb amputation. Self-selected gait speed was the most consistent outcome measure. Exercise programs emphasizing resisted gait and functional training were more effective than supervised walking.

On the importance of social integration for minority targeting effectiveness

Research into the influence of model ethnicity on advertising effectiveness usually compares a dominant or mainstream group with a single, ethnically distinctive minority group. Such two-group studies implicitly assume that all minorities react similarly to distinctive and non-distinctive models portrayed in ads. However, minorities differ in their level of integration into the dominant societal group, which likely influences their responses. This research studies target/non-target market effects for minorities whose levels of integration into the host society vary. In contrast with relatively less integrated minorities, minority groups better integrated into the dominant society exhibit target/non-target market effects unlike those previously reported. This study also contrasts two mediation routes for the observed effects of ethnic models: ad model–self-perceived similarity and perceived ad model credibility. The results show that the two routes are confounded for less integrated minorities but dissociated for better integrated minorities. The findings offer clear applications for the advertising industry.

Power Comparison of Summary Measure, Mixed Model, and Survival Analysis Methods for Analysis of Repeated-Measures Trials

In two-group repeated-measures studies, a traditional statistical approach is to base analysis directly on the observed continuous measurements, using either summary measures or a mixed linear model. In some medical studies, however, an alternate approach has been taken: Declare the occurrence of an “event” when the sequence of measurements crosses a prespecified threshold, and compare the groups with respect to time to event using the log-rank test. This approach is appealing to clinicians, but clearly involves a loss of information and therefore statistical efficiency. The aim of this article is to quantify the degree of power loss in the context of the random line model. We also compare the summary measures approach to the random line approach. In regard to the efficiency loss with the survival analysis approach, we find that the loss ranges, depending on the location of the threshold, from moderate to dramatic. Using an optimally weighted log-rank test in place of the standard log-rank test leads to minimal gain in efficiency. In regard to analysis based on the original continuous measurements, for testing the slope a weighted summary measure appears to be the best overall choice, whereas for testing the intercept the maximum likelihood (ML) approach is typically much more efficient than the summary measures approach, although the efficiency of the ML approach can be compromised in studies with a small number of observation timepoints. These results have obvious implications for the choice of study design and analysis.

Method and reporting quality in health professions education research: a systematic review

Studies evaluating reporting quality in health professions education (HPE) research have demonstrated deficiencies, but none have used comprehensive reporting standards. Additionally, the relationship between study methods and effect size (ES) in HPE research is unknown. This review aimed to evaluate, in a sample of experimental studies of Internet-based instruction, the quality of reporting, the relationship between reporting and methodological quality, and associations between ES and study methods. We conducted a systematic search of databases including MEDLINE, Scopus, CINAHL, EMBASE and ERIC, for articles published during 1990-2008. Studies (in any language) quantifying the effect of Internet-based instruction in HPE compared with no intervention or other instruction were included. Working independently and in duplicate, we coded reporting quality using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement, and coded study methods using a modified Newcastle-Ottawa Scale (m-NOS), the Medical Education Research Study Quality Instrument (MERSQI), and the Best Evidence in Medical Education (BEME) global scale. For reporting quality, articles scored a mean±standard deviation (SD) of 51±25% of STROBE elements for the Introduction, 58±20% for the Methods, 50±18% for the Results and 41±26% for the Discussion sections. We found positive associations (all p<0.0001) between reporting quality and MERSQI (ρ=0.64), m-NOS (ρ=0.57) and BEME (ρ=0.58) scores. We explored associations between study methods and knowledge ES by subtracting each study's ES from the pooled ES for studies using that method and comparing these differences between subgroups. Effect sizes in single-group pretest/post-test studies differed from the pooled estimate more than ESs in two-group studies (p=0.013). No difference was found between other study methods (yes/no: representative sample, comparison group from same community, randomised, allocation concealed, participants blinded, assessor blinded, objective assessment, high follow-up). Information is missing from all sections of reports of HPE experiments. Single-group pre-/post-test studies may overestimate ES compared with two-group designs. Other methodological variations did not bias study results in this sample.

Outcomes of Group Care for Youth: A Review of Comparative Studies

Objective: The purpose of this study was to review empirical evidence of the effects of placement in group care compared to other interventions. Method: Two-group empirical studies were identified and effect sizes for all reported outcomes were calculated. Results: Nineteen two-group studies were found that compared group care with family foster care, treatment foster care, no placement, and different group care models. Most effects were small to moderate, with strongest effects for shorter lengths of stay for family-centered group care models and decreased delinquency for Multidimensional Treatment Foster Care (MTFC) compared to group care. Conclusion: Based on the limited number of two-group studies of varying quality, outcomes of group care are often worse than alternative interventions. However, some group care models seem more promising than others.

Meta‐analysis of skewed data: Combining results reported on log‐transformed or raw scales

When literature-based meta-analyses involve outcomes with skewed distributions, the best available data can sometimes be a mixture of results presented on the raw scale and results presented on the logarithmic scale. We review and develop methods for transforming between these results for two-group studies, such as clinical trials and prospective or cross-sectional epidemiological studies. These allow meta-analyses to be conducted using all studies and on a common scale. The methods can also be used to produce a meta-analysis of ratios of geometric means when skewed data are reported on the raw scale for every study. We compare three methods, two of which have alternative standard error formulae, in an application and in a series of simulation studies. We conclude that an approach based on a log-normal assumption for the raw data is reasonably robust to different true distributions; and we provide new standard error approximations for this method. An assumption can be made that the standard deviations in the two groups are equal. This increases precision of the estimates, but if incorrect can lead to very misleading results. Copyright © 2008 John Wiley & Sons, Ltd.

Improved accuracy when screening for human growth disorders by likelihood ratios

The standard deviation score (SDS) is a powerful tool for screening for growth-related problems. However, referral rules of the type 'if SDS(Y)<d, then refer' (for some constant d) are not optimal for answering the question: 'Does this child with measurement Y belong to the reference or to the diseased population?'. If the growth standard for the diseased population is known, then the likelihood ratio (LR) and the log-likelihood ratio (LLR) can be calculated for individual measurements. Rules of the type 'if LLR(Y)<e, then refer' are uniformly the most powerful test for any constant e, implying that their receiver operating characteristic curves are above those for all other possible tests based on Y. As an empirical demonstration, both types of rules are applied to longitudinal growth data comparing a group with diagnosed Turner syndrome and a reference group from birth to 10 years of age. Conforming with theory, the LR rules were found to be superior to the SDS rules in terms of sensitivity and specificity. We conclude that the LR is the natural measure for two-group studies that can be easily calculated for individual measurements. The LR is firmly rooted within both statistical and decision theory and can be used to estimate the absolute probability of disease.