Meta-analysis Of Effect Sizes Research Articles

Reasons to believe: A systematic review and meta-analytic synthesis of the motives associated with conspiracy beliefs.

Belief in conspiracy theories has been linked to harmful consequences for individuals and societies. In an effort to understand and mitigate these effects, researchers have sought to explain the psychological appeal of conspiracy theories. This article presents a wide-ranging systematic review and meta-analysis of the literature on conspiracy beliefs. We analyzed 971 effect sizes from 279 independent studies (Nparticipants = 137,406) to examine the relationships between psychological motives and conspiracy beliefs. Results indicated that these relationships were significant for all three analyzed classes of motivation: epistemic (k = 114, r = .14), existential (k = 121, r = .16), and social motivations related to the individual, relational, and collective selves (k = 100, r = .16). For all motives examined, we observed considerable heterogeneity. Moderation analyses suggest that the relationships were weaker, albeit still significant, when experimental (vs. correlational) designs were used, and differed depending on the conspiracy measure used. We statistically compare the absolute meta-analytic effect size magnitudes against each other and discuss limitations and future avenues for research, including interventions to reduce susceptibility to conspiracy theories. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

The effectiveness of non-invasive brain stimulation in treatment of major depressive disorder (MDD): a systematic review and transfer analysis.

This study aimed to analyze the transferability of non-invasive brain stimulation (NIBS) interventions in individuals with major depressive disorder (MDD) based on the FIELD model (Function, Implementation, Ecology, Level, and Duration), encompassing function, implement, ecology, level, and duration. A systematic search of electronic databases yielded a total of 21 eligible studies, comprising 12 transcranial direct current stimulation (tDCS) and 9 transcranial magnetic stimulation (TMS) trials, involving 1029 individuals with MDD. The meta-analysis of effect sizes revealed positive transfer effects across all domains of the FIELD model, suggesting that NIBS interventions have potential efficacy in improving various facets of MDD. The subgroup analysis highlighted that bilateral dlPFC stimulation exhibited the highest effect size for transferability, indicating greater transferability for rTMS, a higher dose of stimulation, and the integration of additional interventions. Additionally, the study discusses the implications of bilateral dorsolateral prefrontal cortex (dlPFC) stimulation and the integration of complementary therapies for optimizing treatment efficacy.

Lifespan trajectories of negative and positive affect: A coordinated analysis of 14 longitudinal studies

Lifespan developmental theories suggest age-related shifts in motivation, cognition, emotion regulation, and stressor experience lead to changes in mean levels of negative and positive affect across the lifespan. The present research used coordinated data analysis to examine mean-level affective trajectories in 186,752 participants ranging from 11–104 years old across 14 longitudinal studies. Random-effects models were used to estimate meta-analytic effect sizes. On average, negative affect decreased until early older adulthood, and then remained stable throughout older adulthood. Meanwhile, positive affect remained stable across most of the younger and middle-aged adult lifespan, before starting its descent in later middle-aged adulthood and continuing to decline throughout older adulthood. Studies with older samples showed a clearer flattening effect of negative affect and steeper decline of positive affect in late-life relative to younger samples. These findings suggest that lifespan developmental affect trajectories are nuanced and not a direct inverse of each other.

The effect of different types of water sources on dental unit waterline contamination: A systematic review and meta analysis

To systematically review the effect of different types of water sources on dental unit waterline (DUWL) contamination. 5 databases were searched from their inception to December 23, 2023. Two reviewers independently extracted the data and assessed the quality of the literature. The risk ratio (RR) was used as measure of effect size in meta-analysis. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) was used for evaluating quality of the evidence. Meta-analysis was completed by RevMan 5.4.5 studies involving 561 water samples were quantified for meta-analysis. The results indicated that no significant differences were found in view of contamination rate (RR = 1.01; 95 % CI, 0.72–1.41; P = 0.96, I2 = 62 %; GRADE low)and detection rate of Pseudomonas aeruginosa (RR = 0.78; 95 % CI, 0.15–4.13; P = 0.77; I2 = 83 %; GRADE very low) between using purified water and tap water as water sources of DUWL. The available evidence suggests that there is no significant difference between purified water and tap water in controlling DUWL contamination. However, the conclusions need to be further validated through more randomized controlled trials with robust design and a large sample size.

Addressing Endogeneity in Meta-Analysis: Instrumental Variable Based Meta-Analytic Structural Equation Modeling

In management research, meta-analysis is often used to aggregate findings from observational studies that lack random assignment to predictors (e.g., surveys), which may pose challenges in making accurate inferences due to the correlational nature of effect sizes. To improve inferential accuracy, we show how instrumental variable (IV) methods can be integrated into meta-analysis to help researchers obtain unbiased estimates. Our IV-based meta-analytic structural equation modeling (IV-MASEM) method relies on the fact that IVs can be incorporated into SEM, and meta-analytic effect sizes from correlational research can be used for MASEM. Conveniently, IV-MASEM does not require that each primary study measures all relevant variables, and it can address typical types of endogeneity, such as omitted variable bias. We clarify how the principles of IV-SEM can be applied to MASEM and then conduct three simulations to study the validity of IV-MASEM versus Univariate Meta-Analyses (UMA) and MASEMs that exclude IVs when the instruments were appropriate, inappropriate, and missing from a subset of primary studies. We also offer an illustrative study to demonstrate how to apply IV-MASEM to address endogeneity concerns in meta-analysis, which includes a new R function to test the qualifying conditions for IVs. We conclude with limitations and future directions for IV-MASEM.

Systematic Review and Meta-Analysis of Lifestyle and Reproductive Factors Associated with Risk of Breast Cancer in Asian Women.

Assessing breast cancer risks from lifestyle and reproductive factors is critical for developing population-specific risk prediction tools. However, limited studies have evaluated these risks in recent Asian birth cohorts. We systematically reviewed articles published from January 2010 to December 2023, examining breast cancer risk factors in Asian women. Data were described narratively, estimates pooled, and prevalence and attributable proportions compared across Asian populations. Of the 128 studies reviewed, 103 reported adjusted effect sizes for meta-analysis. Lifestyle and reproductive factors were predictive of breast cancer risk in Asian women, with varying impacts on premenopausal and postmenopausal women. Relative risks were similar within Asian populations and in comparison to European populations, except for menarche, menopause, and hormone receptor therapy. However, risk factor distributions differed across populations. While alcohol intake (21%) and oral contraceptive use (20%) emerged as the most attributable modifiable risk factors in Europeans, passive smoking (24%) and higher BMI (17%, ≥24 kg/m2 among postmenopausal women) were predominant in Asians. Our study shows that while the effects of lifestyle and reproductive breast cancer risk factors are largely similar across different populations, their distributions vary. Our analysis underscores the importance of considering population-specific risk factor distributions when developing risk prediction tools for Asian populations.

General mental ability testing and adverse impact in the United Kingdom: a meta-analysis with more than two million observations

ABSTRACT We review ethnic group differences on high-stakes General Mental Ability (GMA) tests based on 21st century UK data. Thereafter, we meta-analyse scores on 23 occupational, public sector, educational, military, or general-public selection tests, with a sample size exceeding two million. Relative to White GMA, the grand meta-analytic effect sizes (Cohen’s d) by major ethnic groups were: Mixed d = .14 (k = 24, N = 67,114), Blacks d = .65 (k = 32, N = 112,975), Asians d = .33 (k = 32, N = 311,695), and Other d = .49 (k = 24, N = 42,846). Further, although Chinese residents outscored White British residents (d = −.15, k = 20, N = 18,897), all other Asian ethnic groups scored slightly to substantially lower. For example, South Asians as a whole averaged d = .37; k = 13, N = 67,566. By subgroups, these averages were: Indians (d = .17, k = 10, N = 28,236), Pakistanis (d = .49, k = 9, N = 19,371), and Bangladeshi (d = .55, k = 7, N = 19,772). Implications for practice and theory are discussed.

Effect size in meta-analysis

Effect size in meta-analysis

The link between digital gaming and creativity: a meta analysis

ABSTRACT This study explored the relationship between digital gaming and creativity by conducting a meta-analysis of effect sizes using data from 78 effect sizes in 19 studies encompassing 3,172 samples. We employed a two-level multivariate model with a robust variance estimator and conducted separate analyses for between-subject and within-subject designs. For the between-subject designs, the mean effect size was r = .109, 95% [−0.036, 0.249], indicating non-significance. We conducted moderator analyses to explore potential sources of variation in the observed effects, including creativity test modality, creativity test type, divergent thinking index, analytical approach, age, gender, and year of publication. None of the examined moderators were found to be statistically significant. In contrast, we observed a significant mean effect size for within-subject designs based on 29 effects from seven studies (r = .772, 95% CI [0.595, 0.878]). Variation in mean effects by study design warrants caution when making inferences about the impact of digital gaming on creativity. The implications and limitations of the findings are discussed.

How Much Have We Learned about Consumer Research? A Meta-Meta-Analysis

Abstract This meta-meta-analysis study quantifies the development of scientific knowledge in consumer research by summarizing the findings of 222 meta-analyses that together include 2481 meta-analytic effect sizes. The results provide an overview of how much we know and how knowledge has developed in consumer research over time. By explaining 7.8% variance (r = 0.28) in consumer-relevant dependent variables, the findings show that consumer research, a comparatively young discipline, is relatively effective at knowledge development compared to other disciplines. Furthermore, the accumulation of knowledge is significantly increasing, suggesting that our discipline is still in the growing phase of its life cycle and generating continuously improving explanations of consumer-related phenomena. The development of knowledge varies across consumer-relevant dependent variables, with strong explanations for relationships but significantly weaker ones for memory, affect, and attitudes. Moreover, the knowledge synthesized in meta-analyses is fairly—though not fully—representative of the content of primary research on consumers overall. The findings convey a future research agenda by identifying under-researched areas, advising on the selection of dependent variables, providing indicators for the expected contributions of future studies, suggesting implications for career strategies of consumer researchers, and discussing explanations for the observed knowledge growth effects.

Hospitality Front-line Employees: A Meta-Analysis of Emotion Management at Work

The main purpose of this study was to summarize extant empirical findings on how front-line service employees experience and manage emotions at work. When consolidated, the research on front-line employee emotions revealed mixed and contradictory results and knowledge gaps on workplace challenges of service-oriented employees. The PRISMA were followed for study selection, and the appraisal theory of emotions was used to classify the direct antecedents and outcomes of emotion management. To calculate meta-analytic effect sizes, comprehensive meta-analysis methodology wasapplied to analyze 79 studies (N=28,332). The summary effects of known antecedents, affective events (conflict, customer interactions), stimuli (personality characteristics), and workplace environment (support and display rules) on front-line employees’ emotion management were more modest than previously indicated. Organization, supervisor, and coworker support were found to be a moderator between conflict and employees’ emotion management. The most-researched outcomes of emotion management (burnout and job performance) also showed only modest effect sizes. Job satisfaction was a moderator between emotion management and job performance. The relationship between emotion management and customer-related antecedents and outcomes showed modest to small effect sizes. These results contribute to the ATE and highlight the knowledge gap on how customer emotions and behaviors affect front-line employees’ ability to manage their emotions and provide quality customer service and, in turn, how employee’s transference of emotions might influence customer satisfaction, loyalty, and trust. Managers should provide necessary support to mitigate the impact of affective events that may influence how front-line employees manage emotions at work to increase job satisfaction and performance.

Can generative AI infer thinking style from language? Evaluating the utility of AI as a psychological text analysis tool.

Generative AI, short for Generative Artificial Intelligence, a class of artificial intelligence systems, is not currently the choice technology for text analysis, but prior work suggests it may have some utility to assess dynamics like emotion. The current work builds upon this empirical foundation to consider how analytic thinking scores from a large language model chatbot, ChatGPT, were linked to analytic thinking scores from dictionary-based tools like Linguistic Inquiry and Word Count (LIWC). Using over 16,000 texts from four samples and tested against three prompts and two large language models (GPT-3.5, GPT-4), the evidence suggests there were small associations between ChatGPT and LIWC analytic thinking scores (meta-analytic effect sizes: .058 < rs < .304; ps < .001). When given the formula to calculate the LIWC analytic thinking index, ChatGPT performed incorrect mathematical operations in 22% of the cases, suggesting basic word and number processing may be unreliable with large language models. Researchers should be cautious when using AI for text analysis.

Individual-level risk factors for suicide mortality in the general population: an umbrella review

Deaths by suicide remain a major public health challenge worldwide. Identifying and targeting risk factors for suicide mortality is a potential approach to prevention. We aimed to summarise current knowledge on the range and magnitude of individual-level risk factors for suicide mortality in the general population and evaluate the quality of the evidence. In this umbrella review, five bibliographic databases were systematically searched for articles published from database inception to Aug 31, 2022. We included meta-analyses of observational studies on individual-level risk factors for suicide mortality in the general population. Biological, genetic, perinatal, and ecological risk factors were beyond the scope of this study. Effect sizes were synthesised and compared across domains. To test robustness and consistency of the findings, evidence for small-study effects and excess significance bias (ie, the ratio between the overall meta-analysis effect size and that of its largest included study) was examined, and prediction intervals were calculated. Risk of bias was assessed by the Risk of Bias in Systematic Reviews instrument. The protocol was pre-registered with PROSPERO (CRD42021230119). We identified 33 meta-analyses on 38 risk factors for suicide mortality in the general population. 422 (93%) of the 454 primary studies included in the meta-analyses were from high-income countries. A previous suicide attempt and suicidal ideation emerged as strong risk factors (with effect sizes ranging from 6 to 16). Psychiatric disorders were associated with a greatly elevated risk of suicide mortality, with risk ratios in the range of 4-13. Suicide risk for physical illnesses (such as cancer and epilepsy) and sociodemographic factors (including unemployment and low education) were typically increased two-fold. Contact with the criminal justice system, state care in childhood, access to firearms, and parental death by suicide also increased the risk of suicide mortality. Among risk factors for which sex-stratified analyses were available, associations were generally similar for males and females. However, the quality of the evidence was limited by excess significance and high heterogeneity, and prediction intervals suggested poor replicability for almost two-thirds of identified risk factors. A wide range of risk factors were identified across various domains, which underscores suicide mortality as a multifactorial phenomenon. Prevention strategies that span individual and population approaches should account for the identified factors and their relative strengths. Despite the large number of risk factors investigated, few associations were supported by robust evidence. Evidence of causal inference will need to be tested in high-quality study designs. Wellcome Trust.

Wet cupping with rehabilitation training for upper-limb poststroke spasticity: A systematic review and meta-analysis of randomized controlled trials

BackgroundUpper-limb poststroke spasticity (PSS) negatively impacts on patients’ quality of life. An increasing number of clinical trials have indicated that wet cupping with rehabilitation training is conductive to alleviate spastic muscle tone, thereby to improve upper-limb function. However, related evidence base is insufficient. This study systematically investigates the efficacy and safety of wet cupping with rehabilitation training on stroke patients with upper-limb spasticity. MethodsEight separate databases and two clinical trial registries were searched from their inception to December 6, 2022. Two reviewers extracted the data and assessed the quality of the literature, independently. The mean difference (MD) or risk ratio (RR) were used as measure of effect size in meta-analysis. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) was used for the certainty of evidence. ResultsEight randomized controlled trials (RCTs) were quantified for meta-analysis. The results indicated that in comparison with the control group, wet cupping with rehabilitation training was more effective in reducing modified Ashworth scale score (MD = −0.60, 95% CI: −0.74, −0.46; P < 0.00001) and the integral electromyography value of biceps muscle (MD = −4.71, 95% CI: −6.74, −2.67; P < 0.00001), but improving effective rate (RR = 1.28, 95% CI: 1.15, 1.41; P < 0.00001), Fugl-Myer Assessment score (MD = 4.84, 95% CI: 3.05, 6.64; P < 0.00001) as well as Barthel Index score (MD = 6.38, 95% CI: 2.20, 10.57; P = 0.003). However, no significant difference was found regarding the integral electromyography value of triceps muscle between groups (MD = 1.72, 95% CI: −2.05, 5.48; P = 0.37). ConclusionWet cupping with rehabilitation training should be included in a comprehensive therapeutic regimen for stroke patients with upper-limb spasticity. However, these results need to be further verified by more RCTs with rigorous design and large sample size.

Assessing the Evidential Value of Mental Fatigue and Exercise Research

It has often been reported that mental exertion, presumably leading to mental fatigue, can negatively affect exercise performance; however, recent findings have questioned the strength of the effect. To further complicate this issue, an overlooked problem might be the presence of publication bias in studies using underpowered designs, which is known to inflate false positive report probability and effect size estimates. Altogether, the presence of bias is likely to reduce the evidential value of the published literature on this topic, although it is unknown to what extent. The purpose of the current work was to assess the evidential value of studies published to date on the effect of mental exertion on exercise performance by assessing the presence of publication bias and the observed statistical power achieved by these studies. A traditional meta-analysis revealed a Cohen’s dz effect size of − 0.54, 95% CI [− 0.68, − 0.40], p < .001. However, when we applied methods for estimating and correcting for publication bias (based on funnel plot asymmetry and observed p-values), we found that the bias-corrected effect size became negligible with most of publication-bias methods and decreased to − 0.36 in the more optimistic of all the scenarios. A robust Bayesian meta-analysis found strong evidence in favor of publication bias, BFpb > 1000, and inconclusive evidence in favor of the effect, adjusted dz = 0.01, 95% CrI [− 0.46, 0.37], BF10 = 0.90. Furthermore, the median observed statistical power assuming the unadjusted meta-analytic effect size (i.e., − 0.54) as the true effect size was 39% (min = 19%, max = 96%), indicating that, on average, these studies only had a 39% chance of observing a significant result if the true effect was Cohen’s dz = − 0.54. If the more optimistic adjusted effect size (− 0.36) was assumed as the true effect, the median statistical power was just 20%. We conclude that the current literature is a useful case study for illustrating the dangers of conducting underpowered studies to detect the effect size of interest.

How Do Different Weighting Methods Affect the Overall Effect Size in Meta-Analysis? : An Example of Science Attitude in Türkiye Sample

There is increasing interest in meta-analysis in different fields due to the need to combine the results of primary research. One of the crucial concepts in combining results is weighting. This study examines how Hunter and Schmidt's method, weighting by sample size; Hedges and Vevea's method, weighting by inverse variance; and Osburn and Callender's method, unweighting, affect the overall effect size in meta-analysis. In this context, for meta-analysis, the search was done for studies examining the effects of alternative measurement and assessment techniques and methods in science education on science attitudes. The databases of the HEI National Thesis Center, Web of Science, ERIC, EBSCO, Google Scholar, and DergiPark were searched between 2010 and 2021. Eleven studies (with 14 effect sizes) that met the criteria were included in the meta-analysis. In line with the study's findings, it was observed that the overall effect sizes were significant and did not change much in the weighting methods. Besides, it was found that the method with the lowest standard error was unweighted. The weighting methods of Hunter and Schmidt and Hedges and Vevea gave similar results in terms of standard error. When the correlation coefficient between the weighting methods was examined, it was seen that all correlation coefficients were greater than 0.90.

The impact of innate immunity on malaria parasite infection dynamics in rodent models.

Decades of research have probed the molecular and cellular mechanisms that control the immune response to malaria. Yet many studies offer conflicting results on the functional impact of innate immunity for controlling parasite replication early in infection. We conduct a meta-analysis to seek consensus on the effect of innate immunity on parasite replication, examining three different species of rodent malaria parasite. Screening published studies that span four decades of research we collate, curate, and statistically analyze infection dynamics in immune-deficient or -augmented mice to identify and quantify general trends and reveal sources of disagreement among studies. Additionally, we estimate whether host factors or experimental methodology shape the impact of immune perturbations on parasite burden. First, we detected meta-analytic mean effect sizes (absolute Cohen's h) for the difference in parasite burden between treatment and control groups ranging from 0.1475 to 0.2321 across parasite species. This range is considered a small effect size and translates to a modest change in parasitaemia of roughly 7-12% on average at the peak of infection. Second, we reveal that variation across studies using P. chabaudi or P. yoelii is best explained by stochasticity (due to small sample sizes) rather than by host factors or experimental design. Third, we find that for P. berghei the impact of immune perturbation is increased when young or female mice are used and is greatest when effector molecules (as opposed to upstream signalling molecules) are disrupted (up to an 18% difference in peak parasitaemia). Finally, we find little evidence of publication bias suggesting that our results are robust. The small effect sizes we observe, across three parasite species, following experimental perturbations of the innate immune system may be explained by redundancy in a complex biological system or by incomplete (or inappropriate) data reporting for meta-analysis. Alternatively, our findings might indicate a need to re-evaluate the efficiency with which innate immunity controls parasite replication early in infection. Testing these hypotheses is necessary to translate understanding from model systems to human malaria.

The impact of paternal mental illness on child development: an umbrella review of systematic reviews and meta-analyses.

Whilst there is growing evidence highlighting the importance of paternal mental illness (PMI) on child development, this relationship still remains under-studied and often over-looked. Considering the increasingly active role of fathers in their children's upbringing, a comprehensive overview of the impact of PMI on child development is overdue. This study aimed to combine and synthesise currently available evidence on the relationship between PMI and multiple domains of child development. Narrative synthesis of systematic reviews and meta-analyses on the relationship between PMI and child development (mental health and social, emotional, language, cognitive or adaptive behaviour), published between 1980 to December 2021, was conducted in line with PRISMA 2020 guidelines. Review quality was assessed based on AMSTAR-2 criteria and respective study confidence was interpreted in line with GRADE scoring. All relevant meta-analytic effect sizes were converted to odds ratios (OR) and grouped using a random effects model. Grouped meta-analyses saw PMI to have a significant, detrimental effect on all studied domains of child development [OR: 1.54; 95% CI (1.36-1.74)]. Subgroup analyses saw PMI affecting both internalising [OR: 1.62; 95% CI (1.27-2.08)] and externalising [OR: 1.63; 95% CI (1.28-2.08)] child behaviours to a similar extent. However, included reviews were of poor methodological quality, demonstrating either low or critically low confidence. These results show a consistent and influential effect of PMI on child development. The relationship between fathers' mental illness and child development warrants further investigation, as current research is limited in scope, particularly regarding cognitive domains of child development and non-affective PMI diagnoses.

The therapeutic effect of pelvic floor muscle training on stress urinary incontinence following prostatectomy: a systematic review and meta-analysis.

Prostatectomy often causes urinary incontinence, especially stress Urinary incontinence, which has a serious impact on the quality of life of patients. Previous studies have proved that pelvic floor muscle training can help restore pelvic floor function and reduce Urinary incontinence, but the quantitative evaluation and systematic analysis of its effect have not yet been clear. This meta-analysis aimed to systematically evaluate the therapeutic effect of pelvic floor muscle training on managing stress urinary incontinence after prostatectomy. The literature on pelvic floor muscle training for patients after prostatectomy was searched in PubMed, Web of Science, EMBASE, CNKI, VIP, Wanfang, and China Biology Medical Literature Database (CBM) from database establishment up to January 30th, 2023. Risk bias assessment was conducted using RoB1, a risk assessment tool recommended by Cochrane for evaluating RCTs literature. Publication bias was evaluated through funnel plots. Meta-analysis of effect size was performed using R 4.2.2. Eleven randomized controlled studies were included. The risk of bias assessment showed that three studies had a moderate risk of bias and eight had a low risk. The meta-analysis results showed that the patient-reported incontinence was improved after one month [odds ratio (OR): 2.71, 95% 95% confidence interval (CI): 1.86-3.94, P<0.01]; improved after three months (OR: 3.42, 95% CI: 1.96-5.98, P<0.01); improved after six months (OR: 3.77, 95% CI: 1.51-9.41, P<0.01); improved after 12 months (OR: 1.21, 95% CI: 1.11-1.31, P<0.01); and the International Consultation on Incontinence Questionnaire-Simple Form (ICIQ-SF) score decreased [mean difference (MD): -2.74, 95% CI: -4.96 to -0.52, P=0.02]. Subgroup analysis showed that the ICIQ-SF score decreased after one month (MD: -0.61, 95% CI: -0.81 to -0.40) and three months (MD: -3.43, 95% CI: -6.85 to -0.02). Pelvic floor muscle training significantly improves stress urinary incontinence after prostatectomy, which can be improved by 2.77 times at most. However, due to the limited number of studies included, further validation is needed.

Meta-analyses in psychology often overestimate evidence for and size of effects

Adjusting for publication bias is essential when drawing meta-analytic inferences. However, most methods that adjust for publication bias do not perform well across a range of research conditions, such as the degree of heterogeneity in effect sizes across studies. Sladekovaet al. 2022 (Estimating the change in meta-analytic effect size estimates after the application of publication bias adjustment methods.Psychol. Methods) tried to circumvent this complication by selecting the methods that are most appropriate for a given set of conditions, and concluded that publication bias on average causes only minimal over-estimation of effect sizes in psychology. However, this approach suffers from a ‘Catch-22’ problem—to know the underlying research conditions, one needs to have adjusted for publication bias correctly, but to correctly adjust for publication bias, one needs to know the underlying research conditions. To alleviate this problem, we conduct an alternative analysis, robust Bayesian meta-analysis (RoBMA), which is not based onmodel-selectionbut onmodel-averaging. In RoBMA, models that predict the observed results better are given correspondingly larger weights. A RoBMA reanalysis of Sladekovaet al.’s dataset reveals that more than 60% of meta-analyses in psychology notably overestimate the evidence for the presence of the meta-analytic effect and more than 50% overestimate its magnitude.