Bias In Meta-analysis Research Articles

Sensitivity analysis for publication bias in meta-analysis of sparse data based on exact likelihood.

Meta-analysis is a powerful tool to synthesize findings from multiple studies. The normal-normal random-effects model is widely used to account for between-study heterogeneity. However, meta-analyses of sparse data, which may arise when the event rate is low for binary or count outcomes, pose a challenge to the normal-normal random-effects model in the accuracy and stability in inference since the normal approximation in the within-study model may not be good. To reduce bias arising from data sparsity, the generalized linear mixed model can be used by replacing the approximate normal within-study model with an exact model. Publication bias is one of the most serious threats in meta-analysis. Several quantitative sensitivity analysis methods for evaluating the potential impacts of selective publication are available for the normal-normal random-effects model. We propose a sensitivity analysis method by extending the likelihood-based sensitivity analysis with the $t$-statistic selection function of Copas to several generalized linear mixed-effects models. Through applications of our proposed method to several real-world meta-analyses and simulation studies, the proposed method was proven to outperform the likelihood-based sensitivity analysis based on the normal-normal model. The proposed method would give useful guidance to address publication bias in the meta-analysis of sparse data.

Trial Number and Sample Size Do Not Affect the Accuracy of the I2-Point Estimate for Testing Selection Bias Risk in Meta-Analyses.

Aim This study aims to establish the test sensitivity and specificity of the I2-point estimate for testing selection bias in meta-analyses under the condition of large versus small trial sample size and large versus small trial number in meta-analyses and to test the null hypotheses that the differences are not statistically significant. Material and methods Simulation trials were generated in MS Excel (Microsoft Corp., Redmond, WA), each consisting of a sequence of subject ID (accession) numbers representing trial subjects, a random sequence of allocation to group A or B, and a random sequence of a simulated baseline variable ("age") per subject, ranging from 50 to 55. These simulation trials were included infive types of meta-analyses with large/small numbers of trials, as well as trials with large and small sample sizes. Half of the meta-analyses were artificially biased. All meta-analyses were tested using the I2-point estimate. The numbers of true positive (TP), false positive (FP), false negative (FN), and true negative (TN) test results were established. From these, the test sensitivity and specificity were computed for each of the meta-analysis types and compared. Results All non-biased meta-analyses yielded true negative, and all biased meta-analyses yielded true positive test results, regardless of trial number and trial sample size. No false positive or false negative test results were observed. Accordingly, test sensitivities and specificities of 100% for all meta-analysis types were established, and thus, both null hypotheses failed to be rejected. Conclusion The results suggest that trial number and sample size in a baseline variable meta-analysis do not affect the test accuracy of the I2-point estimate.

Approaches for identifying and managing publication bias in meta-analysis

The consequences of publication bias in meta-analysis pose significant risks, potentially leading to erroneous conclusions within the meta-analytic framework. The objective of this article was to explore the methodologies for identifying publication bias and approaches for mitigating its effects. The techniques employed to detect publication bias can generally be distinguished into two major categories: graphical and statistical methodologies. Graphical approaches utilize techniques such as funnel plots and meta-plots, which visually depict the distribution of effect sizes and standard errors across studies. Statistical methods encompass various computations, including Fail-Safe N, rank correlation, Egger regression, tests for excess significance (TES), and selection models tailored for evaluating publication bias through quantitative analyses. The combination of these methods is recommended for a more comprehensive assessment, rather than relying on individual approaches. Methods for addressing publication bias include the trim and fill (T&F) method, Publication Error and True Effect Size Estimation (PET-PEESE) method, and the Weight-Function Model, each offering unique strategies for adjusting effect size estimates. The selection of these methods should consider the specific characteristics of the meta-analysis under consideration, ensuring the most appropriate approach is employed. Publication bias poses a significant risk in the field of meta-analysis, and selecting methods for its identification and mitigation requires comprehensive consideration

LFK index does not reliably detect small-study effects in meta-analysis: A simulation study.

The LFK index has been promoted as an improved method to detect bias in meta-analysis. Putatively, its performance does not depend on the number of studies in the meta-analysis. We conducted a simulation study, comparing the LFK index test to three standard tests for funnel plot asymmetry in settings with smaller or larger group sample sizes. In general, false positive rates of the LFK index test markedly depended on the number and size of studies as well as the between-study heterogeneity with values between 0% and almost 30%. Egger's test adhered well to the pre-specified significance level of 5% under homogeneity, but was too liberal (smaller groups) or conservative (larger groups) under heterogeneity. The rank test was too conservative for most simulation scenarios. The Thompson-Sharp test was too conservative under homogeneity, but adhered well to the significance level in case of heterogeneity. The true positive rate of the LFK index test was only larger compared with classic tests if the false positive rate was inflated. The power of classic tests was similar or larger than the LFK index test if the false positive rate of the LFK index test was used as significance level for the classic tests. Under ideal conditions, the false positive rate of the LFK index test markedly and unpredictably depends on the number and sample size of studies as well as the extent of between-study heterogeneity. The LFK index test in its current implementation should not be used to assess funnel plot asymmetry in meta-analysis.

Meta-analyses of partial correlations are biased: Detection and solutions.

We demonstrate that all meta-analyses of partial correlations are biased, and yet hundreds of meta-analyses of partial correlation coefficients (PCCs) are conducted each year widely across economics, business, education, psychology, and medical research. To address these biases, we offer a new weighted average, UWLS+3 . UWLS+3 is the unrestricted weighted least squares weighted average that makes an adjustment to the degrees of freedom that are used to calculate partial correlations and, by doing so, renders trivial any remaining meta-analysis bias. Our simulations also reveal that these meta-analysis biases are small-sample biases (n < 200), and a simple correction factor of (n - 2)/(n - 1) greatly reduces these small-sample biases along with Fisher's z. In many applications where primary studies typically have hundreds or more observations, partial correlations can be meta-analyzed in standard ways with only negligible bias. However, in other fields in the social and the medical sciences that are dominated by small samples, these meta-analysis biases are easily avoidable by our proposed methods.

Sensitivity analysis for the interactive effects of internal bias and publication bias in meta-analyses.

Meta-analyses can be compromised by studies' internal biases (e.g., confounding in nonrandomized studies) as well as publication bias. These biases often operate nonadditively: publication bias that favors significant, positive results selects indirectly for studies with more internal bias. We propose sensitivity analyses that address two questions: (1) "For a given severity of internal bias across studies and of publication bias, how much could the results change?"; and (2) "For a given severity of publication bias, how severe would internal bias have to be, hypothetically, to attenuate the results to the null or by a given amount?" These methods consider the average internal bias across studies, obviating specifying the bias in each study individually. The analyst can assume that internal bias affects all studies, or alternatively that it only affects a known subset (e.g., nonrandomized studies). The internal bias can be of unknown origin or, for certain types of bias in causal estimates, can be bounded analytically. The analyst can specify the severity of publication bias or, alternatively, consider a "worst-case" form of publication bias. Robust estimation methods accommodate non-normal effects, small meta-analyses, and clustered estimates. As we illustrate by re-analyzing published meta-analyses, the methods can provide insights that are not captured by simply considering each bias in turn. An R package implementing the methods is available (multibiasmeta).

Meta-analyzing the multiverse: A peek under the hood of selective reporting.

Researcher degrees of freedom refer to arbitrary decisions in the execution and reporting of hypothesis-testing research that allow for many possible outcomes from a single study. Selective reporting of results (p-hacking) from this "multiverse" of outcomes can inflate effect size estimates and false positive rates. We studied the effects of researcher degrees of freedom and selective reporting using empirical data from extensive multistudy projects in psychology (Registered Replication Reports) featuring 211 samples and 14 dependent variables. We used a counterfactual design to examine what biases could have emerged if the studies (and ensuing meta-analyses) had not been preregistered and could have been subjected to selective reporting based on the significance of the outcomes in the primary studies. Our results show the substantial variability in effect sizes that researcher degrees of freedom can create in relatively standard psychological studies, and how selective reporting of outcomes can alter conclusions and introduce bias in meta-analysis. Despite the typically thousands of outcomes appearing in the multiverses of the 294 included studies, only in about 30% of studies did significant effect sizes in the hypothesized direction emerge. We also observed that the effect of a particular researcher degree of freedom was inconsistent across replication studies using the same protocol, meaning multiverse analyses often fail to replicate across samples. We recommend hypothesis-testing researchers to preregister their preferred analysis and openly report multiverse analysis. We propose a descriptive index (underlying multiverse variability) that quantifies the robustness of results across alternative ways to analyze the data. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A Note on Cherry-Picking in Meta-Analyses.

We study selection bias in meta-analyses by assuming the presence of researchers (meta-analysts) who intentionally or unintentionally cherry-pick a subset of studies by defining arbitrary inclusion and/or exclusion criteria that will lead to their desired results. When the number of studies is sufficiently large, we theoretically show that a meta-analysts might falsely obtain (non)significant overall treatment effects, regardless of the actual effectiveness of a treatment. We analyze all theoretical findings based on extensive simulation experiments and practical clinical examples. Numerical evaluations demonstrate that the standard method for meta-analyses has the potential to be cherry-picked.

The clinical application of Chinese herbal medication to depression: A narrative review.

Depression severely impairs psychosocial functioning and quality of life, which places a huge burden on patients and their families. However, the physiological mechanism of depression remains unknown. Treatment with existing antidepressant medications is effective in around 50% of patients according to various studies, but is associated with severe side effects including nausea and headaches. Chinese herbal medicine (CHM) has been approved and widely used for depression as an alternative medicine in Chinese culture for decades. It has certain advantages and potential in the prevention and treatment of depression. In this review, we summarize the currently available evidence for the efficacy of CHM for the treatment of depression and physiological diseases comorbid with depression. We further discuss the possible mechanisms of action of CHM and the relationships to our current understanding of depression. The majority of current evidence has suggested that the combined treatment with CHM and mainstream antidepressants improves the response rate and reduces the side effects, while CHM alone could be more effective than placebo. However, the results should be carefully interpreted due to the shortcomings of existing clinical trials and a high risk of bias in meta-analyses. Our review provides a summary of the current applications and understanding of widely used CHMs for depression.

Publication bias in meta-analyses of the therapeutic efficacy of remdesivir interventions for patients with COVID-19

Purpose No comprehensive statistical assessment of publication bias has been conducted in remdesivir-based intervention research for COVID-19 patients. This study aims to examine all meta-analyses of the efficacy of remdesivir interventions in COVID-19 patients and perform a statistical assessment of publication bias. Design/methodology/approach This is an analytic study conducted to assess the impact of publication bias on the results of meta-analyses of remdesivir-based interventions in patients infected with COVID-19. All English full-text meta-analyses published in peer-reviewed journals in 2019–2021 were included. A computerized search of PubMed and Web of Science electronic databases was performed on December 24, 2021. The trim-and-fill method calculated the number of missing studies and the adjusted cumulative effect sizes. Findings The final analysis comprised 21 studies with 88 outcomes. The investigation revealed missing studies in 46 outcomes (52%). Seventy-six missing studies were replaced in the outcomes using the trim-and-fill procedure. The adjusted recalculated effect sizes of the 27 outcomes increased by an average of 0.04. In comparison, the adjusted effect size of 18 outcomes fell by an average of 0.036. Only 14 out of 46 outcomes with publication bias were subjected to a gray literature search (30%). To discover related research, no gray literature search was conducted in most outcomes with publication bias (n = 32; 70%). In conclusion, the reported effect estimates regarding the effect of remdesivir in COVID-19 patients are only slightly affected by publication bias and can be considered authentic. Health-care decision-makers in COVID-19 should consider current research results when making clinical decisions. Research limitations/implications Most health decisions are based on the effect sizes revealed in meta-analyses. When deciding on remdesivir-based treatment for COVID-19 patients, therefore, the outcomes of this investigation may be of paramount importance to health policymakers, leading to better treatment strategies. Practical implications According to the results, no major publication bias and missing studies were detected on average. Therefore, the calculated effect sizes of remdesivir-based interventions on meta-analyses can be used as authentic and unbiased benchmarks by health-care decision-makers in treating patients with COVID-19. Originality/value This is the first study to examine the effect of publication bias and gray literature searches on the results of meta-analyses of treatment with COVID-19 (remdesivir).

An artificial neural network (ANN) model for publication bias: a machine learning-based study on PubMed meta-analyses

PurposeNo study has investigated the effects of different parameters on publication bias in meta-analyses using a machine learning approach. Therefore, this study aims to evaluate the impact of various factors on publication bias in meta-analyses.Design/methodology/approachAn electronic questionnaire was created according to some factors extracted from the Cochrane Handbook and AMSTAR-2 tool to identify factors affecting publication bias. Twelve experts were consulted to determine their opinion on the importance of each factor. Each component was evaluated based on its content validity ratio (CVR). In total, 616 meta-analyses comprising 1893 outcomes from PubMed that assessed the presence of publication bias in their reported outcomes were randomly selected to extract their data. The multilayer perceptron (MLP) technique was used in IBM SPSS Modeler 18.0 to construct a prediction model. 70, 15 and 15% of the data were used for the model's training, testing and validation partitions.FindingsThere was a publication bias in 968 (51.14%) outcomes. The established model had an accuracy rate of 86.1%, and all pre-selected nine variables were included in the model. The results showed that the number of databases searched was the most important predictive variable (0.26), followed by the number of searches in the grey literature (0.24), search in Medline (0.17) and advanced search with numerous operators (0.13).Practical implicationsThe results of this study can help clinical researchers minimize publication bias in their studies, leading to improved evidence-based medicine.Originality/valueTo the best of the author’s knowledge, this is the first study to model publication bias using machine learning.

Adjusting for publication bias in meta-analysis via inverse probability weighting using clinical trial registries.

Publication bias is a major concern in conducting systematic reviews and meta-analyses. Various sensitivity analysis or bias-correction methods have been developed based on selection models, and they have some advantages over the widely used trim-and-fill bias-correction method. However, likelihood methods based on selection models may have difficulty in obtaining precise estimates and reasonable confidence intervals, or require a rather complicated sensitivity analysis process. Herein, we develop a simple publication bias adjustment method by utilizing the information on conducted but still unpublished trials from clinical trial registries. We introduce an estimating equationfor parameter estimation in the selection function by regarding the publication bias issue as a missing data problem under the missing not at random assumption. With the estimated selection function, we introduce the inverse probability weighting (IPW) method to estimate the overall mean across studies. Furthermore, the IPW versions of heterogeneity measures such as the between-study variance and the I2 measure are proposed. We propose methods to construct confidence intervals based on asymptotic normal approximation as well as on parametric bootstrap. Through numerical experiments, we observed that the estimators successfully eliminated bias, and the confidence intervals had empirical coverage probabilities close to the nominal level. On the other hand, the confidence interval based on asymptotic normal approximation is much wider in some scenarios than the bootstrap confidence interval. Therefore, the latter is recommended for practicaluse.

Let's decide what would be convincing, conduct randomized trials with rigorous comparison conditions, and report tests of moderation and publication bias in meta-analyses.

We appreciate Jacobson and colleagues' thoughtful commentary on our meta-review of mobile phone-based interventions for mental health. In this response, we address two issues raised: requiring low to moderate heterogeneity (I2<50%) and requiring no evidence of publication bias for evidence to be classified as "convincing." While we agree these represent a high bar, we disagree that these requirements are destined to fail. Other effect sizes reported in the literature, including effect sizes related to mental health interventions and effect sizes related to mobile health (mHealth) interventions (although not their combination) have met requirements for convincing evidence. Jacobson and colleagues argue that features of the mHealth interventions may produce heterogeneity when meta-analyses combine across intervention types. However, several of the effect sizes we reviewed were based on relatively homogeneous portions of the literature and many of the effect sizes we reviewed showed low to moderate heterogeneity. Ideally, future meta-analyses will examine intervention features as moderators of treatment effects. While an absence of publication bias may be a stringent criterion, all but two of the 34 effect sizes we reviewed did not report formal tests of publication bias. Clearly there is a need to reach consensus on how the strength of evidence for mHealth interventions can be evaluated. From our perspective, convincing evidence will ultimately come from large-scale randomized controlled trials employing rigorous comparison conditions along with meta-analyses that do not combine across control condition types, that examine theoretically important moderators, and report formal tests of publication bias. It is this kind of evidence that the public, clinicians, and the scientific community may need to encourage adoptions of mHealth interventions for mental health treatment and prevention.

Associations Between Wearing Masks and Respiratory Viral Infections: A Meta-Analysis and Systematic Review.

BackgroundRespiratory viral infections (RVIs) are a major health concern, and some previous studies have shown that wearing masks was effective in preventing RVIs, while others failed to show such effect. Therefore, a systematic review and meta-analysis was conducted to investigate the effectiveness of wearing masks.MethodsPubMed, ScienceDirect, Web of Science, the Cochrane Library, EMBASE, MEDLINE, China National Knowledge Infrastructure (CNKI), and Chinese Scientific Journal Database (VIP database) were searched for studies evaluating the effectiveness of wearing masks. The risk ratio (RR) was used to measure the effectiveness of wearing masks in preventing RVIs for randomized controlled trials (RCTs) and cohort studies, and the odds ratio (OR) was used for case-control studies. Forest plots were used to visually assess pooled estimates and corresponding 95% CIs. The I2 test was used to examine the heterogeneity, and subgroup analysis was used to explore the possible explanations for heterogeneity or compare the results between subgroups. Sensitivity analysis was conducted to assess robustness of the synthesized results. Begg's test and Egger's test were used to assess the publications bias.ResultsThirty-one studies (13,329 participants) were eligible for meta-analyses. Overall, the results showed that wearing masks was effective in preventing RVIs. The sensitivity analysis showed that the results of those meta-analyses were robust and reliable. There was no significant publication bias in meta-analysis of case-control studies and most subgroup analyses.ConclusionsWearing masks might be effective in preventing RVIs. To reduce their RVI risk, people should wear masks when they go out in public.Systematic Review Registrationhttps://www.crd.york.ac.uk/PROSPERO/, identifier: CRD42021296092.

Antiepileptic drugs for the primary and secondary prevention of seizures after stroke.

Seizures after stroke are an important clinical problem and may result in poor outcomes. The indications of antiepileptic drugs (AEDs) for seizure prophylaxis after stroke remain unclear. This is an updated version of the Cochrane Review previously published in 2014. To assess the effects of AEDs for the primary and secondary prevention of seizures after stroke. For primary prevention, we aimed to assess whether AEDs reduce the likelihood of seizures in people who have a stroke but do not have a seizure. For secondary prevention, we aimed to assess whether AEDs reduce the likelihood of further seizures in people who have a stroke and at least one post-stroke seizure. We searched the following databases on 9 March 2021: Cochrane Register of Studies (CRS Web), MEDLINE (Ovid, 1946 to March 08, 2021). CRS Web includes randomised or quasi-randomised controlled trials from PubMed, Embase, ClinicalTrials.gov, the World Health Organisation International Clinical Trials Registry Platform (ICTRP), the Cochrane Central Register of Controlled Trials (CENTRAL), and the Specialised Registers of Cochrane Review Groups including Epilepsy and Stroke. We also checked the reference lists of articles retrieved from these searches. We selected randomised and quasi-randomised controlled studies that recruited participants with a clinical diagnosis of stroke, either ischaemic or haemorrhagic. We excluded studies that only recruited participants with subarachnoid haemorrhage, subdural haemorrhage, extradural haemorrhage, or other non-stroke diagnoses such as tumour- or infection-related infarction or haemorrhage. We also excluded studies that recruited only participants who had undergone neurosurgery. We included participants of all ages suffering any seizure type who were assigned to AEDs or placebo groups. In accordance with standard methodological procedures expected by The Cochrane Collaboration, two review authors independently assessed trials for inclusion before evaluating trial risk of bias and extracting relevant data. The primary outcome assessed was the proportion of participants who experienced seizures in the follow-up period. We presented results as summary risk ratios (RRs) with 95% confidence intervals (CIs) for dichotomous outcomes and mean differences (MDs) with 95% CIs for continuous outcomes. Where we had sufficient data, we calculated random-effects (Mantel-Haenszel) meta-analyses for dichotomous outcomes; otherwise, we reported results narratively. We used the I2 statistic to analyse statistical heterogeneity. We planned to use funnel plots to assess publication bias in meta-analyses with at least 10 included studies. We used the GRADE approach to assess the certainty of the evidence. Two studies with a total of 856 subjects were included. AEDs were not shown to be effective in primary prophylaxis of post-stroke seizure (RR 0.65, 95% CI 0.34 to 1.26; 2 studies, 856 participants; moderate-certainty evidence). The first study was a randomised double-blind study comparing valproic acid with placebo for primary seizure prevention up to one year after stroke. The study included 72 adults with intracerebral haemorrhage. There was no difference in the risk of post-stroke seizures (RR 0.88, 95% CI 0.35 to 2.16) or of death (RR 1.20, 95% CI 0.40 to 3.58). The second study was a substudy on the use of diazepam in acute stroke. It was a randomised double-blind study, comparing a three-day diazepam treatment versus placebo for primary seizure prevention up to three months after stroke in 784 adults with acute stroke. There was no evidence of a difference in the risk of post-stroke seizures for all stroke or subgroups of haemorrhagic or ischaemic stroke (RR for all stroke 0.47, 95% CI 0.18 to 1.22). In a subgroup analysis of anterior circulation cortical infarcts, primary prophylaxis with diazepam was associated with a reduced risk of post-stroke seizures (RR 0.21, 95% CI 0.05 to 0.95). Risks of mortality did not differ between the diazepam and the placebo group at two weeks (RR 0.84, 95% CI 0.56 to 1.26) and three months follow-up (RR 0.95, 95% CI 0.72 to 1.26). We assessed both studies to be at a low overall risk of bias. Using the GRADE approach, we assessed the overall certainty of the evidence as low to moderate. There is insufficient evidence to support the routine use of AEDs on the primary and secondary prevention of seizures after stroke. Further well-conducted studies are warranted for this important clinical problem.

Herpes simplex virus type 1 in subgingival plaque and periodontal diseases. Meta-analysis of observational studies.

This meta-analysis of observational studies (PROSPERO registration number CRD42021236054) sought to investigate strength and generalizability of the association of herpes simplex virus type 1 (HSV-1) in subjects with plaque-induced gingivitis and periodontitis, since the data from literature are contrasting. Case-control and cross-sectional studies, investigating HSV-1 in subgingival plaque/crevicular fluid and periodontal status, were searched through MEDLINE via PubMed, Scopus, Web of Science and Google Scholar. From each study the crude odds ratio (OR) with 95% confidence interval (95CI) was extracted, and the pooled OR was assessed for periodontitis, chronic and aggressive, and gingivitis. The meta-analytic method was chosen based on the level of heterogeneity. The generalizability of results, determined by the meta-analysis bias, was investigated through secondary analyses including sensitivity analyses for study quality, publication bias, and study inclusion, and subgroup analyses for quality of scientific journals that published the primary studies, world Region, subgingival plaque sampling method and study design. Twelve studies were included (738 cases, 551 controls). The pooled ORs were 4.4 (95CI, 1.9-10.2) for any periodontitis; specifically, 2.8 (95CI, 1.0-8.3) for chronic periodontitis, 11.8 (95CI, 5.4-25.8) for aggressive periodontitis and 4.8 (95CI, 2.1-11.0) for gingivitis. These estimates were statistically significant, excluding for chronic periodontitis, resulting marginally significant (p=.05). Secondary analyses on any and aggressive periodontitis, and, partly, chronic periodontitis corroborated the results, while the material was insufficient for secondary analyses on gingivitis. The results obtained indicated that HSV-1 is associated with periodontitis, while data about gingivitis are inconclusive. HSV-1 investigation in subgingival plaque could help assess periodontitis risk and severity and, if causal association were confirmed, could contribute to its control.

Acne and insulin resistance: A systematic review and meta-analysis

To the Editor: Acne is more common in westernized societies, suggesting that environmental factors likely contribute to acne pathogenesis.1Cordain L. Lindeberg S. Hurtado M. Hill K. Eaton S.B. Brand-Miller J. Acne vulgaris: a disease of western civilization.Arch Dermatol. 2002; 138: 1584-1590Crossref PubMed Google Scholar Metabolic abnormalities may influence key factors in the development of acne, such as the proliferation of basal keratinocytes in the sebaceous-pilosebaceous unit and abnormal desquamation of follicular corneocytes.2Del Prete M. Mauriello M.C. Faggiano A. et al.Insulin resistance and acne: a new risk factor for men?.Endocrine. 2012; 42: 555-560Crossref PubMed Scopus (50) Google Scholar Therefore, we hypothesized that patients with acne would demonstrate higher levels of insulin resistance. In this systematic review and meta-analysis, we examined the relationship between insulin resistance and acne, as it may have important therapeutic and diagnostic implications, such as exploring insulin modifiers as an adjunctive acne treatment and screening acne patients for metabolic syndrome. PubMed and Embase were systematically searched for studies published prior to March 17, 2021 that evaluated insulin resistance in patients with acne. In PubMed and Embase (acne OR acne vulgaris) AND (insulin resistance OR diabetes OR blood glucose) were searched as Medical Subject Heading terms and keywords, respectively. Only clinical studies measuring values for Homeostatic Model Assessment for Insulin Resistance (HOMA-IR), a validated laboratory measure of insulin resistance, in acne patients and control subjects were included.3Muniyappa R. Madan R. Assessing insulin sensitivity and resistance in humans.in: Feingold K.R. Anawalt B. Boyce A. Endotext [Internet]. MDText.com, Inc, 2000Google Scholar Studies were excluded if they were unpublished, abstracts, not written in English, or included patients with polycystic ovarian syndrome in order to exclude hyperandrogenism as a potential confounding variable. Database searches yielded 136 unique results. Thirteen articles met inclusion criteria and were included in the analysis (Supplementary Fig 1, available via Mendeley at https://data.mendeley.com/datasets/bfrm4wbdrz/2). The following data were extracted from each article: number of acne subjects, mean age of acne subjects, mean body mass index of acne subjects, acne severity, other acne qualifiers (postadolescent acne or treatment nonresponders), control subjects, mean HOMA-IR value in both acne and control groups, and quality assessment score (Supplementary Table 1, available via Mendeley at https://data.mendeley.com/datasets/bfrm4wbdrz/2). Meta-analysis compared the differences between acne and control patients. Hedges' g standardized mean difference between HOMA-IR values in acne patients and controls was calculated as the summary statistic. Results are reported from a random effects model. An I2 statistic was calculated to estimate heterogeneity across studies.4Egger M. Davey Smith G. Schneider M. Minder C. Bias in meta-analysis detected by a simple, graphical test.BMJ. 1997; 315: 629-634Crossref PubMed Scopus (32351) Google Scholar Moderating variables were studied in a mixed-effects model. Statistical analyses were conducted with R statistical software (version 4.0.5) using the meta package.5Balduzzi S. Rücker G. Schwarzer G. How to perform a meta-analysis with R: a practical tutorial.Evid Based Ment Health. 2019; 22: 153-160Crossref PubMed Scopus (952) Google Scholar Acne patients had higher HOMA-IR values than controls (standardized mean difference = 0.60; 95% CI, 0.33-0.88; P < .001) (Supplementary Fig 2, available via Mendeley at https://data.mendeley.com/datasets/bfrm4wbdrz/2). Onset of acne in the postadolescent period, average age of participants, sex of participants, average body mass index, publication year, or quality score were not significant modifying variables in our model. There was heterogeneity between studies (P < .001), which was a limitation of our study. Publication bias was assessed visually using a funnel plot and quantitatively with Egger's regression test of asymmetry (Supplementary Fig 3, available via Mendeley at https://data.mendeley.com/datasets/bfrm4wbdrz/2). Although some asymmetry exists, the large effect size of study 13 appears to mitigate the asymmetry (P = .47). However, due to the small number of studies with a large variation in effect sizes and standard errors, we do not have enough evidence that publication bias does not exist in this study. Dermatologists should be aware of the potential relationship between acne and insulin resistance and consider referring acne patients to primary care for further evaluation. Early treatment of insulin resistance can prevent the development of diabetes and cardiovascular disease. None disclosed.

Publication bias: A “bird's-eye view” of meta-analytic practice in criminology and criminal justice

ObjectivesThe aims of the current study were 2-folded. This study examined the recent meta-analyses in the criminology and criminal justice (CCJ) field to capture a snapshot of the current state of publication bias practices and provided practical guidelines and recommendations on how to address the issues of publication bias in meta-analytic reviews. MethodsThe content analysis reviewed 64 meta-analyses published in top-tier journals in the CCJ field in 2019–2020. The narrative review of previous simulation studies in medical science and psychology fields was performed to synthesize practical guidelines on publication bias in meta-analyses. ResultsRecent CCJ meta-analytic studies have at least partially addressed the issue of publication bias by employing systematic search and statistical methods. However, the current state of CCJ meta-analyses does not meet the expectation required in medical science and psychology that all meta-analytic reviews report the range of effect size estimates across multiple publication bias detection and correction tests. The statistical methods commonly used for assessing publication bias are applied without testing and interpreting assumptions about the missing studies. ConclusionsThere is a need to continue monitoring the quality of meta-analyses to gain a comprehensive picture of how bias leaves a potential imprint in CCJ research.

Publication bias in meta-analysis

Publication bias in meta-analysis

Methods to Address Confounding and Other Biases in Meta-Analyses: Review and Recommendations.

Meta-analyses contribute critically to cumulative science, but they can produce misleading conclusions if their constituent primary studies are biased, for example by unmeasured confounding in nonrandomized studies. We provide practical guidance on how meta-analysts can address confounding and other biases that affect studies' internal validity, focusing primarily on sensitivity analyses that help quantify how biased the meta-analysis estimates might be. We review a number of sensitivity analysis methods to do so, especially recent developments that are straightforward to implement and interpret and that use somewhat less stringent statistical assumptions than do earlier methods. We give recommendations for how these newer methods could be applied in practice and illustrate using a previously published meta-analysis. Sensitivity analyses can provide informative quantitative summaries of evidence strength, and we suggest reporting them routinely in meta-analyses of potentially biased studies. This recommendation in no way diminishes the importance of defining study eligibility criteria that reduce bias and of characterizing studies' risks of bias qualitatively.