Statistical Power Analysis Research Articles

Simulated versus traditional occupational therapy placements: A randomised controlled trial.

Professional practise placements in occupational therapy education are critical to ensuring graduate competence. Australian occupational therapy accreditation standards allow up to 200 of a mandated 1000 placement hours to include simulation-based learning. There is, however, minimal evidence about the effectiveness of simulation-based placements compared to traditional placements in occupational therapy. We evaluated whether occupational therapy students completing a 40hour (one week block) Simulated Clinical Placement (SCP) attained non-inferior learning outcomes to students attending a 40 hour Traditional Clinical Placement (TCP). A pragmatic, non-inferiority, assessor-blinded, multicentre, randomised controlled trial involving students from six Australian universities was conducted. Statistical power analysis estimated a required sample of 425. Concealed random allocation was undertaken with a 1:1 ratio within each university. Students were assigned to SCP or TCP in one of three settings: vocational rehabilitation, mental health or physical rehabilitation. SCP materials were developed, manualised and staff training provided. TCPs were in equivalent practice areas. Outcomes were assessed using a standardised examination, unit grades, the Student Practice Evaluation Form-Revised and student confidence survey. A generalised estimating equationapproach was used to assess non-inferiority of the SCP to the TCP. Of 570 randomised students (84% female), 275 attended the SCP and 265 the TCP (n=540, 94.7% retention). There were no significant differences between the TCP and SCP on (i) examination results (marginal mean difference 1.85, 95% CI: 0.46-3.24; P=0.087); (ii) unit score (mean (SD) SCP: 71.9 (8.8), TCP: 70.34 (9.1); P=0.066); or (iii) placement fail rate, assessed using the Student Practice Evaluation Form-Revised (100% passed both groups). Students can achieve equivalent learning outcomes in a 40 hour simulated placement to those achieved in a 40 hour traditional placement. These findings provide assurance to students, educators and professional accreditation bodies that simulation can be embedded in occupational therapy education with good effect.

On the design of stimulation trials in the oil field: A power analysis of intervention studies

An all too often question when designing field trials aimed at the validation of EOR techniques is how long the stimulation should last. In these trials, the stimulation period is proportional to the trial variable cost, hence the question related to period of trial execution. This article uses statistical power analysis to answer such a question. It demonstrates how statistical power of a trial design plays out as a function of its variables: total trial duration, its proportion under stimulation, the magnitude of the stimulation effect, and the level of variability and autocorrelation typically found in well production data.

Unscheduled absences in a cohort of nurse anesthetists during a 3-year period: Statistical implications for the identification of outlier personnel

Study objectiveTo estimate the prevalence of unscheduled absences in a cohort of certified registered nurse anesthetists (CRNAs) over a 3-year period, for purposes of critiquing statistical review of individual providers relative to potential identification of patterns of such absences. DesignRetrospective, observational study. SettingUniversity hospital. Subjects99 CRNAs performing clinical assignments in the operating rooms. InterventionsNone. MeasurementsCRNA daily clinical assignments and unscheduled absences were retrieved from the department's staff assignment software package. Data were extracted and analyzed to estimate the prevalence of unscheduled absences by CRNAs by day of the week, and whether each absence occurred on the workday before or after either a holiday or a personal vacation. A statistical power analysis was performed to determine the number of workdays of data required to identify outlier personnel above the 95th percentile among all CRNAs while controlling for a family-wise error rate of 5%. Main resultsThe overall incidence of unscheduled absences pooled by days was 1.7%, with small differences among days of the week, and before or after vacations. A year of data would be required to detect outliers for unscheduled absences exceeding the 95% upper confidence limit among all CRNAs. Attempting to identify patterns of absences being on specific days of the week or as related to holidays and vacations would require multiple years of data. ConclusionsOR managers can detect CRNAs with excessive numbers of unscheduled absences, but at least a year of data is required. Detecting apparent “patterns” of absences would require multiple years of data and is thus impractical.

Empirically Based Mean Effect Size Distributions for Universal Prevention Programs Targeting School-Aged Youth: A Review of Meta-Analyses.

This review of reviews presents an empirically based set of mean effect size distributions for judging the relative impact of the effects of universal mental health promotion and prevention programs for school-age youth (ages 5 through 18) across a range of program targets and types of outcomes. Mean effect size distributions were established by examining the findings from 74 meta-analyses of universal prevention and promotion programs that included more than 1100 controlled outcome studies involving over 490,000 school-age youth. The distributions of mean effect sizes from these meta-analyses indicated considerable variability across program targets and outcomes that differed substantially from Cohen's (1988, Statistical power analysis for the behavioral sciences (2nd ed.)) widely used set of conventions for assessing if effects are small, medium, or large. These updated mean effect size distributions will provide researchers, practitioners, and funders with more appropriate evidence-based standards for judging the relative effects of universal prevention programs for youth. Limitations in current data and directions for future work are also discussed.

Twenty-Five Cohorts of Major Depressive Disorder in China: Evidence for Reduced But Not Enhanced Default Mode Network Functional Connectivity

Background: Major Depressive Disorder (MDD) is common and disabling, but its neural pathophysiology remains unclear. Functional brain network studies in MDD have largely had limited statistical power and data analysis approaches have varied widely. The REST-meta-MDD Project of resting-state fMRI (R-fMRI) addresses these issues. Methods: The 25 research groups in China composing the REST-meta-MDD Project contributed R-fMRI data of 1,300 patients with MDD and 1,128 normal controls (NCs). The data were preprocessed locally with a standardized protocol prior to aggregated group analyses. We focused on functional connectivity (FC) within the default mode network (DMN), frequently reported to show increased FC in MDD. Outcomes: We found decreased instead of increased DMN FC in MDD compared to NCs. We found FC reduction only in recurrent MDD, not in first-episode drug-naive MDD. Decreased DMN FC was associated with medication usage but not with MDD duration or severity. Exploratory analyses also revealed alterations of local intrinsic activity in MDD. Interpretation: We confirmed the key role of DMN in MDD but found reduced rather than increased FC within the DMN. Future studies should test whether decreased DMN FC mediates treatment response. All the R-fMRI indices have been made publicly available via the R-fMRI Maps Project. The REST-meta-MDD model can be generalized to longitudinal MDD studies and to other psychiatric disorders. Funding Statement: This work was supported by the National Key R&D Program of China (2017YFC1309902), the National Natural Science Foundation of China (81671774, 81630031 and 81371488), the Hundred Talents Program of the Chinese Academy of Sciences, Beijing Municipal Science & Technology Commission (Z161100000216152 and Z171100000117016), Department of Science and Technology, Zhejiang Province (2015C03037). Declaration of Interests: All the authors declare no competing financial interests. Ethics Approval Statement: All contributed data were from studies approved by local Institutional Review Boards. Data submitted to the consortium were fully deidentified and anonymized.

Physical and cognitive doping in university students using the unrelated question model (UQM): Assessing the influence of the probability of receiving the sensitive question on prevalence estimation.

Study objectivesIn order to increase the value of randomized response techniques (RRTs) as tools for studying sensitive issues, the present study investigated whether the prevalence estimate for a sensitive item assessed with the unrelated questionnaire method (UQM) is influenced by changing the probability of receiving the sensitive question p.Material and methodsA short paper-and-pencil questionnaire was distributed to 1.243 university students assessing the 12-month prevalence of physical and cognitive doping using two versions of the UQM with different probabilities for receiving the sensitive question (p ≈ 1/3 and p ≈ 2/3). Likelihood ratio tests were used to assess whether the prevalence estimates for physical and cognitive doping differed significantly between p ≈ 1/3 and p ≈ 2/3. The order of questions (physical doping and cognitive doping) as well as the probability of receiving the sensitive question (p ≈ 1/3 or p ≈ 2/3) were counterbalanced across participants. Statistical power analyses were performed to determine sample size.ResultsThe prevalence estimate for physical doping with p ≈ 1/3 was 22.5% (95% CI: 10.8–34.1), and 12.8% (95% CI: 7.6–18.0) with p ≈ 2/3. For cognitive doping with p ≈ 1/3, the estimated prevalence was 22.5% (95% CI: 11.0–34.1), whereas it was 18.0% (95% CI: 12.5–23.5) with p ≈ 2/3. Likelihood-ratio tests revealed that prevalence estimates for both physical and cognitive doping, respectively, did not differ significantly under p ≈ 1/3 and p ≈ 2/3 (physical doping: χ2 = 2.25, df = 1, p = 0.13; cognitive doping: χ2 = 0.49, df = 1, p = 0.48). Bayes factors computed with the Savage-Dickey method favored the null (“the prevalence estimates are identical under p ≈ 1/3 and p ≈ 2/3”) over the alternative (“the prevalence estimates differ under p ≈ 1/3 and p ≈ 2/3”) hypothesis for both physical doping (BF = 2.3) and cognitive doping (BF = 5.3).ConclusionThe present results suggest that prevalence estimates for physical and cognitive doping assessed by the UQM are largely unaffected by the probability for receiving the sensitive question p.

Precision, Reliability, and Effect Size of Slope Variance in Latent Growth Curve Models: Implications for Statistical Power Analysis.

Latent Growth Curve Models (LGCM) have become a standard technique to model change over time. Prediction and explanation of inter-individual differences in change are major goals in lifespan research. The major determinants of statistical power to detect individual differences in change are the magnitude of true inter-individual differences in linear change (LGCM slope variance), design precision, alpha level, and sample size. Here, we show that design precision can be expressed as the inverse of effective error. Effective error is determined by instrument reliability and the temporal arrangement of measurement occasions. However, it also depends on another central LGCM component, the variance of the latent intercept and its covariance with the latent slope. We derive a new reliability index for LGCM slope variance—effective curve reliability (ECR)—by scaling slope variance against effective error. ECR is interpretable as a standardized effect size index. We demonstrate how effective error, ECR, and statistical power for a likelihood ratio test of zero slope variance formally relate to each other and how they function as indices of statistical power. We also provide a computational approach to derive ECR for arbitrary intercept-slope covariance. With practical use cases, we argue for the complementary utility of the proposed indices of a study's sensitivity to detect slope variance when making a priori longitudinal design decisions or communicating study designs.

Determining Sample Size Using Fast and Slow Thinking

ABSTRACTUsing SPSS's bootstrapping procedures, this article demonstrates an approach to determining sample size that combines fast (heuristics or rules-of-thumb) and slow (statistical power analysis) thinking to balance statistical power, precision, and practicality. Sample size is determined for six commonly used statistical procedures: independent groups t-test, one-way ANOVA, one-way MANOVA, Pearson's r correlation, linear regression, and logistic regression. Overall, findings suggest that both approaches may under or over-estimate sample size. Both approaches yielded similar parameter and confidence interval estimates, but varied, sometimes by a factor of two, in their sample size requirements. It is hoped that this study's procedure and results will provide beginning reference points for sample size determination, and encourage researchers continue to search for resolutions for often difficult sample-size decisions.

Editorial: Improving Scientific Practices and Increasing Access

Editorial: Improving Scientific Practices and Increasing Access

The evaluation of endpoint variability and implications for study statistical power and sample size in conscious instrumented dogs

IntroductionThe sensitivity of a given test to detect a treatment-induced effect in a variable of interest is intrinsically related to the variability of that variable observed without treatment and the number of observations made in the study (i.e. number of animals). To evaluate test sensitivity to detect drug-induced changes in myocardial contractility using the variable LVdP/dtmax, a HESI-supported consortium designed and conducted studies in chronically instrumented, conscious dogs using telemetry. This paper evaluated the inherent variability of the primary endpoint, LVdP/dtmax, over time in individual animals as well as the variability between animals for a given laboratory. An approach is described to evaluate test system variability and thereby test sensitivity which may be used to support the selection of the number of animals for a given study, based on the desired test sensitivity. MethodsA double 4 × 4 Latin square study design where eight animals each received a vehicle control and three dose levels of a test compound was conducted at six independent laboratories. LVdP/dtmax was assessed via implanted telemetry systems in Beagle dogs (N = 8) using the same protocol and each of the six laboratories conducted between two and four studies. Vehicle data from each study was used to evaluate the between-animal and within-animal variability in different time averaging windows. Simulations were conducted to evaluate statistical power and type I error for LVdP/dtmax based on the estimated variability and assumed treatment effects in hourly-interval, bi-hourly interval, or drug-specific super interval. ResultsWe observe that the within-animal variability can be reduced by as much as 30% through the use of a larger time averaging window. Laboratory is a significant source of animal-to-animal variability as between-animal variability is laboratory-dependent and is less impacted by the use of different time averaging windows. The statistical power analysis shows that with N = 8, the double Latin square design has over 90% power to detect a minimal time profile with a maximum change of up to 15% or approximately 450 mm Hg/s in LVdP/dtmax. With N = 4, the single Latin square design has over 80% power to detect a minimal time profile with a maximum change of up to 20% or approximately 600 mm Hg/s in LVdP/dtmax. DiscussionWe describe a statistical procedure to quantitatively evaluate the acute cardiac effects from studies conducted across six sites and objectively examine the variability and sensitivity that were difficult or impossible to calculate consistently based on previous works. Although this report focuses on the evaluation on LVdP/dtmax, this approach is appropriate for other variables such as heart rate, arterial blood pressure, or variables derived from the ECG.

A Comparison of Two Treatment Approaches for Agrammatic Broca's Aphasia: Script Therapy vs. Verb Network Strengthening Treatment

A Comparison of Two Treatment Approaches for Agrammatic Broca's Aphasia: Script Therapy vs. Verb Network Strengthening Treatment

How to disentangle psychobiological stress reactivity and recovery: A comparison of model-based and non-compartmental analyses of cortisol concentrations

This article seeks to address the prevailing issue of how to measure specific process components of psychobiological stress responses. Particularly the change of cortisol secretion due to stress exposure has been discussed as an endophenotype of many psychosomatic health outcomes. To assess its process components, a large variety of non-compartmental parameters (i.e., composite measures of substance concentrations at different points in time) like the area under the concentration-time curve (AUC) are commonly utilized. However, a systematic evaluation and validation of these parameters based on a physiologically plausible model of cortisol secretion has not been performed so far.Thus, a population pharmacokinetic (mixed-effects stochastic differential equation) model was developed and fitted to densely sampled salivary cortisol data of 10 males from Montreal, Canada, and sparsely sampled data of 200 mixed-sex participants from Dresden, Germany, who completed the Trier Social Stress Test (TSST). Besides the two major process components representing (1) stress-related cortisol secretion (reactivity) and (2) cortisol elimination (recovery), the model incorporates two additional, often disregarded components: (3) the secretory delay after stress onset, and (4) deviations from the projected steady-state concentration due to stress-unrelated fluctuations of cortisol secretion.The fitted model (R2 = 99%) was thereafter used to investigate the correlation structure of the four individually varying, and readily interpretable model parameters and eleven popular non-compartmental parameters. Based on these analyses, we recommend to use the minimum-maximum cortisol difference and the minimum concentration as proxy measures of reactivity and recovery, respectively. Finally, statistical power analyses of the reactivity-related sex effect illustrate the consequences of using impure non-compartmental measures of the different process components that underlie the cortisol stress response.

Characterization of genome-wide microsatellite markers in rabbitfishes, an important resource for artisanal fisheries in the Indo-West Pacific.

Rabbitfishes are reef-associated fishes that support local fisheries throughout the Indo-West Pacific region. Sound management of the resource requires the development of molecular tools for appropriate stock delimitation of the different species in the family. Microsatellite markers were developed for thecordonnier, Siganus sutor, and their potential for cross-amplification was investigated in 12 congeneric species. A library of 792 repeat-containing sequences was built. Nineteen sets of newly developed primers, and 14 universal finfish microsatellites were tested in S. sutor. Amplification success of the 19 Siganus-specific markers ranged from 32 to 79% in the 12 other Siganus species, slightly decreasing when the genetic distance of the target species to S. sutor increased. Seventeen of these markers were polymorphic in S. sutor and were further assayed in S. luridus, S. rivulatus, and S. spinus, of which respectively 9, 10 and 8 were polymorphic. Statistical power analysis and an analysis of molecular variance showed that subtle genetic differentiation can be detected using these markers, highlighting their utility for the study of genetic diversity and population genetic structure in rabbitfishes.

Usability of Functional MRI in Clinical Studies for Fast and Reliable Assessment of Renal Perfusion and Quantification of Hemodynamic Effects on the Kidney.

The purpose of this study was to evaluate contrast-media-free arterial spin labeling, a technique of functional magnetic resonance imaging (MRI), for assessment of kidney perfusion in a clinical study. We examined renal perfusion by arterial spin labeling in 15 healthy adults using a clinical 1.5-T MRI system, twice under baseline conditions and 60 minutes after a single oral dose of 50 mg captopril. Data evaluation included assessment of interstudy and interrater reproducibility in addition to the pharmacological effect of captopril on kidney perfusion and a sample size calculation for potential application of the technique in pharmacological intervention studies. Interstudy reproducibility of cortical and medullary kidney perfusion was excellent (intraclass correlation coefficients 0.77 and 0.83, respectively). Interrater reproducibility was excellent in the cortex and good in the medulla (intraclass correlation coefficients 0.97 and 0.66, respectively). Ingestion of 50 mg captopril was associated with an 11% drop of systolic blood pressure and a rise in kidney perfusion by 22% in the cortex (369 ± 48 vs 452 ± 56 mL/[min·100 g], P < .001) and 26% in the medulla (157 ± 39 to 198 ± 45 ml/[min·100 g]; P < .01). Statistical power analysis revealed that a small sample size of only 6 participants is needed in a clinical trial to capture an equal change in kidney perfusion to the one induced by 50 mg captopril with a statistical power of 82% and an α error of 0.05. In conclusion, funtional MRI with arterial spin labeling is a reliable method for quantification of kidney perfusion and for fast assessment of pharmacologically induced renal perfusion changes, allowing low case numbers.

Reporting of Effect Size and Confidence Intervals in JAMA Otolaryngology–Head & Neck Surgery

ImportanceEffect sizes and confidence intervals (CIs) are critical for the interpretation of the results for any outcome of interest.ObjectiveTo evaluate the frequency of reporting effect sizes and CIs in the results of analytical studies.Design, Setting, and ParticipantsDescriptive review of analytical studies published from January 2012 to December 2015 in JAMA Otolaryngology–Head & Neck Surgery.MethodsA random sample of 121 articles was reviewed in this study. Descriptive studies were excluded from the analysis. Seven independent reviewers participated in the evaluation of the articles, with 2 reviewers assigned per article. The review process was standardized for each article; the Methods and Results sections were reviewed for the outcomes of interest. Descriptive statistics for each outcome were calculated and reported accordingly.Main Outcomes and MeasuresPrimary outcomes of interest included the presence of effect size and associated CIs. Secondary outcomes of interest included a priori descriptions of statistical methodology, power analysis, and expectation of effect size.ResultsThere were 107 articles included for analysis. The majority of the articles were retrospective cohort studies (n = 36 [36%]) followed by cross-sectional studies (n = 18 [17%]). A total of 58 articles (55%) reported an effect size for an outcome of interest. The most common effect size used was difference of mean, followed by odds ratio and correlation coefficient, which were reported 17 (16%), 15 (13%), and 12 times (11%), respectively. Confidence intervals were associated with 29 of these effect sizes (27%), and 9 of these articles (8%) included interpretation of the CI. A description of the statistical methodology was provided in 97 articles (91%), while 5 (5%) provided an a priori power analysis and 8 (7%) provided a description of expected effect size finding.Conclusions and RelevanceImproving results reporting is necessary to enhance the reader’s ability to interpret the results of any given study. This can only be achieved through increasing the reporting of effect sizes and CIs rather than relying on P values for both statistical significance and clinically meaningful results.

A comparison of inferential analysis methods for multilevel studies: Implications for drawing conclusions in animal welfare science

Investigations comparing the behaviour and welfare of animals in different environments have led to mixed and often conflicting results. These could arise from genuine differences in welfare, poor validity of indicators, low statistical power, publication bias, or inappropriate statistical analysis. Our aim was to investigate the effects of using four approaches for inferential analysis of datasets of varying size on model outcomes and potential conclusions. We considered aggression in 864 growing pigs over six weeks as measured by ear and body injury score and relationships with: less and more enriched environments, pig's relative weight, and sex. Pigs were housed in groups of 18 in one of four pens, replicating the experiment 12 times. We applied four inferential models that either used a summary statistic approach, or else fully or partially accounted for complexities in study design. We tested models using both the full dataset (n=864) and also using small sample sizes (n=72).The most appropriate inferential model was a mixed effects, repeated measures model to compare ear and body score. Statistical models that did not account for the correlation between repeated measures and/or the random effects from replications and pens led to spurious associations between environmental factors and indicators of aggression, which were not supported by the initial exploratory analysis. For analyses on smaller datasets (n=72), due to the effect size and number of independent factors, there was insufficient power to determine statistically significant associations.Based on the mixed effects, repeated measures models, higher body injury scores were associated with more enrichment (coef. est.=0.09, p=0.02); weight (coef. est.=0.05, p<0.001); pen location on the right side (coef. est.=0.08, p=0.03) and at the front of the experimental room (coef. est.=0.11, p=0.003). By comparison, lower ear injury scores were associated with more enrichment (coef. est.=−0.51, p=0.005) and pen location at the front of the experimental room (coef. est.=−0.4, p=0.02). These observed differences support the hypothesis that injuries to the body and ears arise from different risk factors. Although calculation of the minimum required sample size prior to conducting an experiment and selection of the inferential analysis method will contribute to the validity of the study results, conflict between the outcomes will require further investigation via different methods such as sensitivity and specificity analysis.

Can we successfully monitor a population density decline of elusive invertebrates? A statistical power analysis on Lucanus cervus

Can we successfully monitor a population density decline of elusive invertebrates? A statistical power analysis on Lucanus cervus

Safety Assessment of Food and Feed from GM Crops in Europe: Evaluating EFSA's Alternative Framework for the Rat 90-day Feeding Study.

Regulatory-compliant rodent subchronic feeding studies are compulsory regardless of a hypothesis to test, according to recent EU legislation for the safety assessment of whole food/feed produced from genetically modified (GM) crops containing a single genetic transformation event (European Union Commission Implementing Regulation No. 503/2013). The Implementing Regulation refers to guidelines set forth by the European Food Safety Authority (EFSA) for the design, conduct, and analysis of rodent subchronic feeding studies. The set of EFSA recommendations was rigorously applied to a 90-day feeding study in Sprague-Dawley rats. After study completion, the appropriateness and applicability of these recommendations were assessed using a battery of statistical analysis approaches including both retrospective and prospective statistical power analyses as well as variance-covariance decomposition. In the interest of animal welfare considerations, alternative experimental designs were investigated and evaluated in the context of informing the health risk assessment of food/feed from GM crops.

Some factors affecting statistical power of approximate tests in the linear mixed model for longitudinal data

ABSTRACTDifferent longitudinal study designs require different statistical analysis methods and different methods of sample size determination. Statistical power analysis is a flexible approach to sample size determination for longitudinal studies. However, different power analyses are required for different statistical tests which arises from the difference between different statistical methods. In this paper, the simulation-based power calculations of F-tests with Containment, Kenward-Roger or Satterthwaite approximation of degrees of freedom are examined for sample size determination in the context of a special case of linear mixed models (LMMs), which is frequently used in the analysis of longitudinal data. Essentially, the roles of some factors, such as variance–covariance structure of random effects [unstructured UN or factor analytic FA0], autocorrelation structure among errors over time [independent IND, first-order autoregressive AR1 or first-order moving average MA1], parameter estimation methods [maximum likelihood ML and restricted maximum likelihood REML] and iterative algorithms [ridge-stabilized Newton-Raphson and Quasi-Newton] on statistical power of approximate F-tests in the LMM are examined together, which has not been considered previously. The greatest factor affecting statistical power is found to be the variance–covariance structure of random effects in the LMM. It appears that the simulation-based analysis in this study gives an interesting insight into statistical power of approximate F-tests for fixed effects in LMMs for longitudinal data.

Methodological rigor of corporate governance studies: A review and recommendations for future studies

AbstractManuscript TypeArticle/ReviewResearch Question/IssueThis study presents the results of a longitudinal content analysis of governance articles published in both CGIR and other leading management outlets. Our aim with this review is to identify problematic areas, as well as opportunities to improve the methodological rigor of governance studies.Research Findings/InsightsWe examine key design aspects of corporate governance articles. We identify and compare trends (1) over time and (2) across publishing outlets. Specifically examined are the geographic scope of samples, design characteristics (i.e., single‐ versus multi‐year data, the use of archival, survey, experimental, and qualitative designs, data sources, and reliance on concurrent versus longitudinal data), survey response rates and the use of tests for non‐response bias, the statistical analysis techniques used, reporting of results (number and proportion of hypotheses supported, non‐significant, and counter‐significant, use of statistical power analysis, reporting of descriptive statistics and correlations, use and significance of control variables, and endogeneity controls), and the sophistication of the models tests (i.e., mediation and moderation).Theoretical/Academic ImplicationsBased on the results of the review, recommendations for academic researchers are offered in the following areas: (1) designing theoretical tests which both incorporate and go beyond main effects; (2) evaluating a research design's analytical parameters on an a priori basis to maximize reliability, validity, and the potential for publication; (3) incorporating assessment of sensitivity and robustness checks into the analysis; and (4) comprehensive reporting of results including post hoc power analysis, complete and expanded correlation matrix(es), reporting and assessing counter‐results, and avoiding HARKing.