Mini Meta-analysis Research Articles

Meta-Analysis within a Single Paper Using Individual-Participant Data: A Simpler Alternative

More researchers recognize the importance of meta-analyzing findings within a single paper (i.e., mini meta-analysis). These analyses have been primarily conducted by meta-analyzing aggregated data at the study level, particularly the effect size of each study. We highlight an alternative way of achieving the same goal by meta-analyzing data at the individual-participant level, which combines all the raw data into one dataset. We outline the procedure of conducting mini meta- analysis using the individual-level approach. In a series of 21 simulations, we compared the results between individual-level and the study-level meta-analyses in terms of the accuracy of effect size estimates, Type I errors, and Type II errors. We found that although the results are not identical, they are generally consistent, with the overall outcomes of mini meta-analyses at the individual-level being at least as good as those conducted at the study-level. We recommend more mini meta-analyses at the individual-level not only because individual-level meta-analysis has been regarded as the gold standard of meta-analysis, but also that it is a simple and convenient way to achieve the same goal.

On Whether to Meditate Before a Negotiation: Mindfulness Slightly Impairs Negotiation Performance

What little prior empirical research there is that investigated the effects of mindfulness meditation on negotiation performance was conducted in Singapore and the UK and finds benefits. This research reports a mini meta-analysis of ten studies (N > 1100) we conducted in the US on the effect of a brief mindfulness meditation induction on negotiation outcomes and finds a small detriment in terms of value claimed. We had initially hypothesized that mindfulness meditation would help individuals obtain better objective outcomes by claiming more value for themselves. However, the first study we ran found a moderately strong result in the opposite direction – participants who had just meditated obtained worse objective outcomes by claiming less value than participants in the control condition who had not meditated. In terms of subjective negotiation outcomes, participants in the mindfulness condition reported less satisfaction with the instrumental outcome and their sense of self compared to participants in the control condition. Then we ran nine more experiments and never obtained a significant effect of mindfulness on objective outcomes again. The meta-analysis of the total effect on objective outcomes across these ten studies was significant (p = .021), negative, and very small (aggregated d = -.139, 95% confidence interval [-.256, -.021]). If Americans wonder whether they should meditate before a negotiation, our suggestion is that they probably should not, as it is unlikely to help performance, and may weakly harm performance.

Cybersickness Variability by Race: Findings From 6 Studies and a Mini Meta-analysis.

BackgroundWith the influx of medical virtual reality (VR) technologies, cybersickness has transitioned from a nuisance experienced during leisure activities to a potential safety and efficacy concern for patients and clinicians. To improve health equity, it is important to understand any potential differences in cybersickness propensity among demographic groups, including racial groups.ObjectiveThis study aims to explore whether cybersickness propensity differs across racial groups.MethodsWe collected self-reported cybersickness ratings from 6 racially diverse independent samples within 1 laboratory group (N=931). In these studies, the participants were asked to perform tasks in VR such as traversing environments, pointing at and selecting objects, and interacting with virtual humans.ResultsSignificant racial differences in cybersickness were found in 50% (3/6) of studies. A mini meta-analysis revealed that, on average, Black participants reported approximately one-third of SD less cybersickness than White participants (Cohen d=−0.31; P<.001), regardless of the nature of the VR experience. There was no overall difference in reported cybersickness between the Asian and White participants (Cohen d=−0.11; P=.51).ConclusionsRacial differences in cybersickness indicate that researchers, practitioners, and regulators should consider patient demographics when evaluating VR health intervention outcomes. These findings lay the groundwork for future studies that may explore racial differences in cybersickness directly.

Depression is not related to deficits in emotional reasoning skills

Several lines of research have examined whether people with depressive symptoms have deficits in social-cognitive abilities, such as emotional reasoning skills. While many depressed people report having such deficits, it is less clear whether depressive symptoms are related to actual objective performance deficits. We examined the relationship between emotional reasoning skills (as assessed with the TEMINT) and depressive symptoms (i.e., BDI-II) in a mini meta-analysis of 11 studies with data from 1503 participants with varying levels of depression, from healthy people to clinical samples with severely depressed people. Using a random effects approach, we found a small but significant correlation between depressive symptoms and TEMINT performance (mean rz = 0.065), indicating that depressive symptoms were associated with higher emotional reasoning skills. These findings suggest that depression is unrelated to deficits in emotional reasoning, though assessed with only one test. If anything, depressive symptoms are associated with improved performance in the TEMINT. The current results point to a discrepancy between depressed people's self-evaluation of their abilities (as shown in previous research) and their actual performance. Our findings also have practical implications as they suggest that clinicians may focus on modifying depressed people's negative views of themselves, rather than on improving their skills.

Do Empathetic People Have Strong Religious Beliefs? Survey Studies with Large Japanese Samples

ABSTRACT The exploration of personality factors to explain individual differences in religiosity has demonstrated a link between empathic concern and religious beliefs using the Empathic Concern subscale of the Interpersonal Reactivity Index (IRI-EC). Research in the cognitive science of religion emphasized the role of empathizing ability related to mentalizing in acquisition of religious belief and has demonstrated the relationship between the Empathy Quotient (EQ) and religious belief. The current study was designed to compare the strength of relationships between religious belief and two representative measures of empathy (the IRI-EC and the EQ). Study 1 aimed to statistically evaluate the strength of the relationship between the EQ/IRI-EC and religious belief with four Japanese samples (Ns = 207, 155, 208, 183). The mini meta-analysis results with random effect model indicated that the effect size (semi partial correlation, r sp) of the IRI-EC (r sp = .120, 95%CI [.0002, .237]) was larger than that of the EQ (r sp = .074, 95%CI [−.0001, .147]). Moreover, these results were confirmed by Study 2 (N = 1440). Thus, the present study provided reliable evidence of the link between empathy and religious belief in non-Western samples. We discuss how empathic concern and mentalizing-related empathy contribute to acquiring religious beliefs. Abbreviations: EQ: Empathy Quotient; IRI-EC: Interpersonal Reactivity Index-Empathic Concern; WEIRD: Western, educated, industrialized, rich, and democratic

True-false tests enhance retention relative to rereading.

Testing with various formats enhances long-term retention of studied information; however, little is known whether true-false tests produce this benefit despite their frequent use in the classroom. We conducted four experiments to explore the retention benefits of true-false tests. College students read passages and reviewed them by answering true-false questions or by restudying correct information from the passages. They then took a criterial test 2 days later that consisted of short-answer questions (Experiments 1 and 2) or short-answer and true-false questions (Experiments 3 and 4). True-false tests enhanced retention compared to rereading correct statements and compared to typing those statements while rereading (the latter in a mini meta-analysis). Evaluating both true and false statements yielded a testing effect on short-answer criterial tests, whereas evaluating only true statements produced a testing effect on true-false criterial tests. Finally, a simple modification that asked students to correct statements they marked as false on true-false tests improved retention of those items when feedback was provided. True-false tests can be an effective and practical learning tool to improve students' retention of text material. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Watching Videos on a Smartphone: Do Small Screens Impair Narrative Transportation?

ABSTRACT Smartphones are a preferred platform to access audiovisual stories. Prior theory and research suggest that using smaller screens could lead to a shallower narrative experience. In three experiments we examined the influence of screen size (smartphone vs. computer screen) on the experience of being transported into the world of the story (narrative transportation). We further examined interaction effects with manipulations meant to change transportation by means of reviews (Experiment 1, N = 120), consistency of main character information (Experiment 2, N = 139), and prior information meant to facilitate comprehension (Experiment 3, N = 129). Because our series of studies involved theoretically and practically relevant null hypotheses (i.e., screen size does not influence transportation), we added Bayes factor analyses to standard frequentist statistics. A mini meta-analysis was conducted to summarize the results. Taken together, the three experiments indicate that smaller screen size does not impair narrative transportation. Implications and future research are discussed.

Confidence ratings are better predictors of future performance than delayed judgments of learning

ABSTRACT What is the best way to predict future memory performance? The intuitive answer is through judgments of learning (JOLs), in which people estimate how likely they are to remember something in the future. Recent theory, however, suggests that a retrospective confidence rating made just after a retrieval attempt might be a better predictor in some situations. In three preregistered experiments, we compared delayed JOLs to confidence ratings. People studied paired associates (E1) or psychology vocabulary terms (E2 & E3), then took a practice cued-recall test in which they made either a JOL or confidence rating after each response. They then took a final test. In Experiment 1, confidence ratings offered higher resolution (metacognitive accuracy) of memory for paired associates than did JOLs, but in Experiments 2 and 3, the advantage of confidence ratings was much smaller. A mini meta-analysis indicated that confidence ratings have a small advantage in predicting future performance over delayed JOLs. We argue that the two judgments rely on similar cues, and that even though JOLs explicitly ask people to predict future performance, doing so does not enhance prediction accuracy. Rather, the presence of a retention interval in the JOL cue adds variability to the judgment process.

Caring About (COVID-19 Related) Social Issues Signals Trustworthiness: Direct and Conceptual Replication of Zlatev (2019)

With three convenient samples (n = 1,087) and one sample representative for the German population in terms of age and gender (n = 210), we replicate research by Zlatev (2019) showing that perceived benevolence-based and perceived integrity-based trustworthiness increase with a target’s level of caring about a social issue. We show that these results generalize to various issues ranging from environmental issues (i.e., installation of wind turbines in the North Sea) to issues related to the COVID-19 pandemic (i.e., online teaching to prevent the spread of the virus). Furthermore, we provide initial behavioral evidence for this effect by showing that transfers in a trust game increase with a target’s caring about a social issue. All results are robust for age, gender, and social issue. To provide best estimates for the effect of a target’s level of caring on perceived trustworthiness, we report results of three mini meta-analyses including our findings as well as the findings of the original research. Policy implications are discussed.

An appetite for meat? Disentangling the influence of animal resemblance and familiarity

Consumers in modern society are often less exposed to meat that resembles the animal, and thus are less familiar with it, making it difficult to disentangle the influence of these two inputs (familiarity vs. animal resemblance) on meat appetite. Across three studies, we sought to systematically disentangle the impact of familiarity and animal resemblance on meat appetite using inductive (Study 1) and experimental (Studies 2a-2b) approaches. In Study 1 (N = 229) we separated familiarity and animal resemblance into orthogonal dimensions using 28 meat products. Participants provided free associations and rated the products on familiarity, animal resemblance, and appetitive appeal. In Studies 2a and 2b (N = 514) we experimentally examined the independent contributions of familiarity and animal resemblance, using stimuli normed in Study 1. We hypothesized that animal resemblance has its most pronounced influence on appetite when meat products are unfamiliar. Participants’ free associations and ratings of the products were in line with this conditional hypothesis (Study1), as were the experimental manipulations of familiarity and animal resemblance (Studies 2a-2b), confirmed by a mini meta-analysis. In all three studies, familiarity had a pervasive influence on appetite. These findings suggest that product familiarity can attenuate the psychological impact that animal reminders have on appetite. Thus, interventions aimed at eliciting animal associations with meat should consider the familiarity of the products employed.

Difference in Analgesic Effects of Repetitive Transcranial Magnetic Stimulation According to the Site of Pain.

High-frequency repetitive transcranial magnetic stimulation (rTMS) of the primary motor cortex for neuropathic pain has been shown to be effective, according to systematic reviews and therapeutic guidelines. However, our large, rigorous, investigator-initiated, registration-directed clinical trial failed to show a positive primary outcome, and its subgroup analysis suggested that the analgesic effect varied according to the site of pain. The aim of this study was to investigate the differences in analgesic effects of rTMS for neuropathic pain between different pain sites by reviewing our previous clinical trials. We included three clinical trials in this mini meta-analysis: a multicenter randomized controlled trial at seven hospitals (N = 64), an investigator-initiated registration-directed clinical trial at three hospitals (N = 142), and an exploratory clinical trial examining different stimulation parameters (N = 22). The primary efficacy endpoint (change in pain scale) was extracted for each patient group with pain in the face, upper limb, or lower limb, and a meta-analysis of the efficacy of active rTMS against sham stimulation was performed. Standardized mean difference (SMD) with 95% confidence interval (CI) was calculated for pain change using a random-effects model. The analgesic effect of rTMS for upper limb pain was favorable (SMD = −0.45, 95% CI: −0.77 to −0.13). In contrast, rTMS did not produce significant pain relief on lower limb pain (SMD = 0.04, 95% CI: −0.33 to 0.41) or face (SMD = −0.24, 95% CI: −1.59 to 1.12). In conclusion, these findings suggest that rTMS provides analgesic effects in patients with neuropathic pain in the upper limb, but not in the lower limb or face, under the conditions of previous clinical trials. Owing to the main limitation of small number of studies included, many aspects should be clarified by further research and high-quality studies in these patients.

Keeping One’s Distance: Mask Wearing is Implicitly Associated With Psychological Distance

Mask wearing plays a vital role in the fight against the novel coronavirus (COVID-19). Despite its ubiquity in everyday social life, it is still unknown how masked faces are mentally represented. Drawing on construal-level theory, we test the hypothesis that masked faces and unmasked faces are implicitly associated with psychological distance and proximity in memory, respectively. Four preregistered, high-powered experiments (N = 354 adults) using the Implicit Association Test lend convergent support to this hypothesis across all four dimensions of psychological distance: social distance, spatial distance, temporal distance, and hypothetical distance. A mini meta-analysis validates the reliability of the findings (Hedge’s g = 0.46). The present work contributes to the growing literature on construal-level effects on implicit social cognition and enriches the current discussion on mask wearing in the pandemic and beyond.

The cost of attributing moral blame: Defensiveness and resistance to change when raising awareness to animal suffering in factory farming.

Social change campaigns often entail raising awareness of harm caused by people's behavior. For example, campaigns to reduce meat eating frequently highlight the suffering endured by animals. Such messages may simultaneously attribute moral blame to individuals for causing the harm described. Given people's motivation to protect their moral self-image, we expected that information about the suffering of animals in the meat industry presented with a blaming (versus absolving) frame would generate greater defensiveness and correspondingly resistance to change in support of veg*nism (veganism/vegetarianism). We ran three studies to test this expectation. In two studies, we found that raising awareness of animal suffering using a blaming frame increased defensiveness, leading to lower veg*n-supporting attitudes and behavioral intentions. In one study, our hypothesis was not supported, however, a mini-meta analysis across the three studies suggests the overall pattern is robust. This work expands our understanding of the role of moral self-image preservation in defensiveness and resistance to change, and has applied relevance for the development of effective communication strategies in social and moral campaigns.

Cognitive Control and the Implicit Association Test: A Replication of Siegel, Dougherty, and Huber (2012)

The implicit association test (IAT) is widely used to measure evaluative associations towards groups or the self but is influenced by other traits. Siegel, Dougherty, and Huber (2012, Journal of Experimental Social Psychology) found that manipulating cognitive control via false feedback (Study 3) changed the degree to which the IAT was related to cognitive control versus evaluative associations. We conducted two replications of this study and a mini meta-analysis. Null-hypothesis tests, meta-analysis, and a small telescope approach demonstrated weak to no support for the original hypotheses. We conclude that the original findings are unreliable and that both the original study and our replications do not provide evidence that manipulating cognitive control affects IAT scores.

Preparing Silver Linings for a Cloudy Day: The Consequences of Preemptive Benefit Finding.

Waiting for important news is stressful. In four studies, we assess the utility of preemptive benefit finding, a coping strategy in which people seek silver linings in bad news before receiving news, for emotional well-being across several waiting periods (waiting for bar exam results, the outcome of political elections, and results of a fictitious health risk assessment). Our findings support the effectiveness of preemptive benefit finding while waiting, such that identifying benefits in bad news while waiting predicts more positive emotions during the wait (Studies 3 and 4) and buffers people against the emotional consequences of bad news by boosting post-news positive emotions (Studies 2-4). Importantly, engaging in preemptive benefit finding does not backfire if a person ultimately receives good news (Studies 1, 3, and 4). We discuss results from a mini meta-analysis and consider implications of our findings for interventions to improve well-being while waiting and after news arrives.

Housing Policy Reframed? How Conceptual Framing Affects Support for Social Housing Policies in Germany

Housing policy is a growing problem in many countries. Three online experiments were conducted to test framing effects in the context of housing policies in Germany. In experiment 1 (N = 303), urban growth was either described as ‘rural exodus’ or ‘influx’ of the rural population into urban areas. Experiment 2 (N = 300) focused on framing effects in the current expropriation debate in Germany. A reacquisition policy of private housing was either framed as a measure of ‘expropriation’ or ‘repurchase’. In experiment 3 (N = 311), a vacancy taxation was either labelled to ‘punish’ or ‘overcome’ actual problems with housing shortage. A mini meta-analysis across all three studies shows that conceptual framing affects policy support in a consistent way. The data supports our hypotheses that framing has the power to influence mental representations of policy and solutions. Moreover, moderation analyses were conducted to systematically test the influence of political ideology and perceived issue relevance on framing effects. Results indicate that participants’ ideological self-placement partly affects the effectiveness of linguistic framing in some contexts. Thus, the results expand existing knowledge about the effectiveness of political framing, by making visible not only the possibilities but also the boundaries of framing strategies.

The relation between disgust sensitivity and risk-taking propensity: A domain specific approach

AbstractDisgust is a cross-culturally recognized emotion that is characterized by avoidant or cautious tendencies. Accordingly, greater sensitivity to disgust may be related to less willingness to take risks. Relatively little research has examined the association between disgust sensitivity and risk-taking propensity. Further, no research to date has taken a domain specific approach to understanding the association between these constructs. Across two studies (N1=98, N2=390) and a mini-meta analysis utilizing two additional datasets (total N=1981), we assessed the extent to which domain specific disgust sensitivity (i.e., Pathogen, Sexual, and Moral) were related to domain specific risk-taking propensity (i.e., Social, Recreational, Health/Safety, Ethical, and Financial). We conducted two cross-sectional studies, with a community and a student sample. Participants completed surveys that included measures of disgust sensitivity and risk-taking propensity. Bivariate correlations across the two studies indicated that greater disgust sensitivity was related to lower risk-taking propensity across almost all domains. However, when controlling for covariance among the disgust sensitivity domains, regression analyses suggested variability in the association between disgust sensitivity and risk-taking propensity depending on domains. Based on mini meta-analysis, sexual disgust sensitivity had the strongest relationships with social risk-taking propensity and health/safety risk-taking propensity, and moral disgust sensitivity had the strongest relationship with ethical risk-taking propensity. These findings suggest the presence of domain specific relations between disgust sensitivity and risk-taking propensity. The domain specific focus may help in utilizing disgust as a factor to decrease certain risky behaviors (e.g., moral disgust for decreasing plagiarism).

Reducing defensive responding to implicit bias feedback: On the role of perceived moral threat and efficacy to change

The last decade has seen a rush to address the causes and consequences of bias in applied contexts across the world. When and why might these initiatives promote attitudes and behavior that align with egalitarian goals? A common assumption is that increasing awareness of bias can motivate control over prejudiced responding. However, learning that one's actions are biased is threatening, and often motivates a range of self-protective responses to buffer that threat. In the current research, we tested a strategy for reducing such defensive responding and increasing the kind of awareness central to contemporary theories of prejudice regulation and egalitarian behavior. Four experiments (N > 2500) and a mini meta-analysis demonstrate that interventions that (a) decrease perceived moral blameworthiness for having bias and (b) increase the perceived ability to control bias, can reduce defensive responding and increase awareness both in the short-term and approximately 6 months later. Interventions that minimize threat and facilitate efficacy can motivate increased bias awareness and commitment to egalitarian values.

Predictors of Change in Mental Health during the COVID-19 Pandemic

Background: Cross-sectional data suggest that depression, anxiety, and stress have increased during the COVID-19 pandemic. However, longitudinal research is needed to test changes in mental health and determine factors that contribute to change. The purpose of this study was to compare anxiety, depression, and stress pre-pandemic to during the pandemic within the same sample and identify predictors of change (i.e., disease threat, changes to daily life, social isolation, financial worries). Method: Three national samples of U.S. adults were recruited through an online platform (Amazon's Mechanical Turk). Participants completed online surveys pre-pandemic (September – December 2019) and during the pandemic(April - June 2020). Results: Across the three samples, mini-meta analyses revealed significant increases in anxiety and stress (Cohen's ds = 0.17, 0.16, respectively; ps < 0.01). Financial concern and effects of COVID-19 on daily life predicted higher levels of depression, anxiety, and stress during the pandemic across all three samples (ds = 0.24, 0.40, and 0.40, respectively; ps < 0.001). Limitations: Response rates for follow-up surveys were relatively low, with some noted differences between those who did and did not complete both surveys. Conclusions: Significant increases in anxiety and stress were observed across three samples of U.S. adults from before to during the COVID-19 pandemic. Financial concern and effects of the pandemic on daily life emerged as the most consistent predictors of psychological distress across these samples.

The ‘me’ in meat: Does affirming the self make eating animals seem more morally wrong?

People typically extend limited moral standing to animals reared for food. Prominent perspectives in the literature on animal-human relations characterize this phenomenon as an outcome of moral disengagement: in other words, a strategy that protects people from moral self-condemnation. To provide a direct test of this hypothesis, we exposed people to a self-affirmation manipulation, and hypothesized that this would lead them to be more critical of their own meat eating and be more appreciative of animals' minds and suffering. Three experiments tested this idea in meat-eaters from the United Kingdom. Two initial experiments (n = 244, n = 247) found that affirming the self made eating animals seem more morally wrong. However, a subsequent pre-registered experiment (n = 719) failed to replicate this effect. In addition, this experiment found no effects of the affirmation procedure on specific beliefs about eating animals that participants consume compared to animals they do not consume. A mini-meta analysis of all the experiments found only weak evidence in support of the idea that affirming the self makes eating meat seem more morally wrong. There was no evidence that the affirmation procedure affected beliefs about animal minds.