Population-based Survey Experiment Research Articles

Audience evaluations of news videos made with various levels of automation: A population-based survey experiment

The use of automation in news content creation is expanding from the written to the audio-visual medium with news organizations including Reuters turning to video automation services provided by companies such as Wibbitz. Although researchers have explored audience perceptions of text-based news automation, to date no published study has examined how news consumers perceive automated news videos. We conducted a between-subjects online survey experiment to compare how a socio-demographically representative sample ( n = 4200) of online news consumers in the UK perceived human-made, partly automated, and highly automated short-form online news videos on 14 different story topics. Our findings show that human-made videos received on average more favourable responses on some evaluation variables, although the differences were not large. We also found some significant differences in the relative evaluation of automated and human-made news videos across different individual stories. For practitioners our results suggest partly automated news videos with post-automation human editing can be well received. For researchers our results show the need to use reasonably large sets of experimental stimuli, and suggest that ensuring socio-demographic variation within samples of respondents is worthwhile.

Sugar tax and product reformulation proposals reduce the perceived legitimacy of health-promotion institutions: a randomized population-based survey experiment.

Structural nutrition interventions like a sugar tax or a product reformulation are strongly supported among the public health community but may cause a considerable backlash (e.g. inspiring aversion to institutions initiating the interventions among citizens). Such a backlash potentially undermines future health-promotion strategies. This study aims to uncover whether such backlash exists. We fielded a pre-registered randomized, population-based survey experiment among adults from the Longitudinal Internet Studies for the Social Sciences panel (n = 1765; based on a random sampling of the Dutch population register). Participants were randomly allocated to the control condition (brief facts about health-information provision/nudging), or one of two experimental groups (the same facts, expanded with either a proposed sugar tax on or reformulation of sugar-sweetened beverages). Ordinary least squares regression was used to estimate the proposed interventions' effects on four outcome variables: trust in health-promotion institutions involved; perceptions that these institutions have citizens' well-being in mind (i.e. benevolence); perceptions that these institutions' perspectives are similar to those of citizens (i.e. alignment of perspectives); and attitudes toward nutrition information. Trust, perceived benevolence and perceived alignment of perspectives were affected negatively by a proposed sugar tax (-0.24, 95% CI -0.38 to -0.10; -0.15, -0.29 to -0.01; -0.15, -0.30 to 0.00) or product reformulation (-0.32, -0.46 to -0.18; -0.24, -0.37 to -0.11; -0.18, 0.33 to -0.03), particularly among the non-tertiary educated respondents. Sugar taxes or product reformulations may delegitimize health-promotion institutions, potentially causing public distancing from or opposition to these bodies. This may be exploited by political and commercial parties to undermine official institutions. https://osf.io/qr9jy/?view_only=5e2e875a1fc348f3b28115b7a3fdfd90. Registered 3 February 2022.

Does the prospect of further sovereignty loss fuel Euroscepticism? A population-based survey experiment

ABSTRACT Whether further European integration is desirable is an ongoing question in public opinion research: both extant research and the outcome of various EU-related referendums show that many citizens hold negative views on proposals that lead to further loss of national sovereignty. An important issue thus arises: does the prospect of such additional sovereignty loss increase negativity towards the EU? This study attempts to answer this question using a pre-registered original survey experiment conducted among members of a nationally-representative high-quality Dutch panel. Our focus is on how exposure to proposed abolishment of EU member states’ right of veto affects EU attitudes. In addition, we analyse whether the exposure effect is shaped by 1) citizens’ prior populist attitudes and 2) the Eurosceptic character of the medium. Concerning the former, informed by recent in-depth qualitative research, we hypothesise that populist attitudes aggravate the extent to which exposure to potential loss of national sovereignty leads to more negative EU attitudes. Concerning the latter, we hypothesise that exposure via a Eurosceptic medium might either aggravate or abate the extent to which the newspaper message leads to more negative EU attitudes. We discuss our findings and provide suggestions for further research.

Does informing citizens about the non-meritocratic nature of inequality bolster support for a universal basic income? Evidence from a population-based survey experiment

ABSTRACT Despite citizens’ precarization and policymakers’ enthusiasm for a universal basic income (UBI), this alternative to targeted welfare has, curiously, received limited popular support. We theorize that this is due to people overestimating society’s meritocratic nature. Accordingly, we field a randomized survey experiment with a representative sample of the Dutch population (n = 1,630) to investigate the impact of information provision about the non-meritocratic nature of wealth and ethnic inequality on support for a UBI. Informed by extant research indicating that citizens respond differently to the same information because of material circumstances or different worldviews, we further estimate conditional average treatment effects to explore moderation by (1) income, (2) economic egalitarianism, (3) welfare chauvinism and (4) institutional trust. We find that support for a UBI is higher among individuals with lower incomes and those who are more egalitarian and less welfare chauvinistic. Nonetheless, while exposure to our factual treatment makes participants more concerned about inequality and supportive of economic redistribution in general, it neither directly nor conditionally affects their support for a UBI. Our findings suggest that a UBI may be deemed too radical an approach to addressing inequality. We discuss theoretical and policy implications and provide suggestions for future research.

Public acceptability of policies to reduce urban air pollution: A population-based survey experiment

Low-emission zones, congestion charges and pedestrian areas have the potential to improve urban air quality. However, the lack of public and political support for these policies is often a significant obstacle to their introduction. In this article, we present the results of two studies. First, we investigated the public acceptability of three policies to reduce urban air pollution: pedestrianization, low emission zones and congestion charges. Second, based on an experimental online survey, we examined the effect of various framing conditions on the acceptability of congestion charges. The results show, first, that public acceptability is higher for pedestrianization and low emission zones and lower for congestion charging. Second, we find a positive effect on acceptability of information emphasizing the benefits of congestion charges and of positive labelling, comparing the wording “environmental contribution” with “urban toll”; and a negative effect of making political ideology prominent. Our findings suggest that while some interventions aimed at mitigating urban air pollution may be more readily supported than others, their acceptability can be enhanced (or diminished) through careful design and effective communication.

Evaluating the impact of the EU’s normative message of the “EU as a model” on external public perceptions: an experimental study in Israel

ABSTRACT This article analyses whether the EU’s normative idea, as embedded in the idea of the “EU as a model”, influences external public perceptions of its normative message; in other words, does exposure to the story of how the EU emerged from the ashes of World War II to become a community of peace and security impact public views of the EU as a normative power? To examine these questions, a population-based survey experiment was conducted in Israel – a country with contradictory relations with the EU. The findings are clear and straightforward: exposure to the “EU as model” message increased recognition of the normative actorness of the EU. This impact holds when controlling and interacting with the exposure to various cultural filters which are key factors in shaping views of the EU like political attitudes. The implications of these findings for EU studies and public diplomacy as well as the potential contribution of the approach proposed in this article are discussed.

Explaining cybercrime victimization using a longitudinal population-based survey experiment. Are personal characteristics, online routine activities, and actual self-protective online behavior related to future cybercrime victimization?

ABSTRACT With the increasing prevalence of cybercrime victimization there is a growing need for prevention. Previous studies have attempted to uncover risk factors associated with cybercrime victimization in the areas of personal characteristics and online routine activities. This article aims to take the field a step further by including actual self-protective online behavior, obtained through a population-based survey experiment (N = 1886), as a risk factor for cybercrime victimization. In wave 1 of our longitudinal design, personal characteristics, online routine activities, and actual self-protective online behavior concerning password strength, clicking behavior, sharing personal information, and handling phishing emails were measured. In wave 2, cybercrime victimization of several types of cyber-enabled and cyber-dependent cybercrimes was measured one year later. Results indicate that few personal characteristics, online routine activities, and self-protective online behaviors are related to the odds of becoming a cybercrime victim. This furthermore illustrates the heterogeneity of cybercrime victimization, since most significant factors only seem to be related to the risk of one particular type of cybercrime. These results indicate that to explain cybercrime victimization, the research field needs to shift its focus and adapt to new online developments.

Social engineering and the disclosure of personal identifiable information: Examining the relationship and moderating factors using a population-based survey experiment

People tend to disclose personal identifiable information (PII) that could be used by cybercriminals against them. Often, persuasion techniques are used by cybercriminals to trick people to disclose PII. This research investigates whether people can be made less susceptible to persuasion by reciprocation (i.e., making people feel obligated to return a favour) and authority, particularly in regard to whether information security knowledge and positive affect moderate the relation between susceptibility to persuasion and disclosing PII. Data are used from a population-based survey experiment that measured the actual disclosure of PII in an experimental setting ( N = 2426). The results demonstrate a persuasion–disclosure link, indicating that people disclose more PII when persuaded by reciprocation, but not by authority. Knowledge of information security was also found to relate to disclosure. People disclosed less PII when they possessed more knowledge of information security. Positive affect was not related to the disclosure of PII. And contrary to expectations, no moderating effects were found of information security knowledge nor positive affect on the persuasion–disclosure link. Possible explanations are discussed, as well as limitations and future research directions.

Acceptability of mandatory vaccination: a survey experiment on the effects of thresholds and justifications

ABSTRACT Mandatory vaccination mightseem like a straightforward solution for reaching a sufficient vaccine coverage during a pandemic. However, the effectiveness of mandatory vaccination is undermined if the public does not perceive such a compulsory policy as acceptable. We report a population-based survey experiment (n = 1,131), conducted in Finland, that addresses the effects that different ways of framing a mandatory vaccination program have on three outcome variables: (1) the perceived acceptability of the program, (2) the willingness to take the vaccine voluntarily, and (3) the preparedness to refuse the mandatory vaccine. In the 2 × 3 factorial experiment, the respondents were presented with one of three justification alternatives that stressed the benefits of the program for either the economy, health, or basic rights. The justification was accompanied with one of two thresholds for herd immunity (70 or 90 percent). We found that the justification and the threshold for herd immunity interact when it comes to the acceptability of the program and the willingness to take the vaccine voluntarily. Importantly, justifications drawing on health were ineffective at the lower threshold level but very effective when the threshold was high. The preparedness to refuse a mandatory vaccine was low and reacted weakly with the experimental treatments.

Audience Evaluations of News Videos Made with Various Levels of Automation: A Population-Based Survey Experiment

Audience Evaluations of News Videos Made with Various Levels of Automation: A Population-Based Survey Experiment

The expression effects of uncivil disagreement: the mechanisms of cognitive dissonance and self-perception

Abstract Political incivility is pervasive and still on the rise. Although empirical studies have examined the effects of exposure to political incivility in different contexts, few have attempted to investigate the expression effects of incivility on its senders. This study proposes two mechanisms—cognitive dissonance and self-perception—to explain the expression effects of political incivility on anger, perceptions of incivility, and political participation. The study conducts a population-based online survey experiment (N = 413) in Hong Kong. Participants were either forced to express uncivil or civil disagreements or did so voluntarily. The results suggest that expressing uncivil disagreement increases anger and perceptions of incivility. However, no difference is found between the forced and self-selection conditions, indicating that self-perception is more applicable than cognitive dissonance. In addition, the study finds that expressing uncivil disagreement influences political participation via both anger and perceptions of incivility, though the effects run in opposite directions.

Receptive to an authoritative voice? Experimental evidence on how patronizing language and stressing institutional sources affect public receptivity to nutrition information

Common strategies to make official nutrition information more persuasive include highlighting its institutional sources and using simple and direct language. However, such strategies may be counterproductive, as institutions are no longer self-evidently deemed to be legitimate in contemporary societies and such language can be viewed as patronizing. Our preregistered, population-based survey experiment fielded among a high-quality Dutch probability sample in February 2022 (n = 1947) 1) examines whether these dominant strategies hold up when tested against suggestions of psychological reactance and source derogation, and 2) scrutinizes if such responses are stronger among less-educated citizens. Our experiment mirrored real-life examples of health-information campaigns concerning healthy and unhealthy beverages, with data collected on seven outcome measures to discern receptivity toward the information and its sources. We found that just highlighting institutional sources in the information did not lead to it being perceived more negatively. This was also the case when the language used could be perceived as patronizing, with reactance only present for one outcome measure. Moreover, while less-educated citizens were generally less receptive to nutrition information (six of seven outcome measures), versions that could possibly be perceived as patronizing or/and highlighted institutional sources did not make them less receptive systematically. Importantly, therefore, while our results show that the dominant health-communication strategies do not increase receptivity either, their use will probably not have a negative effect on the general public and so do not need to be discarded.

Environmental Concern Leads to Trade Skepticism on the Political Left and Right

Abstract The environmental implications of international trade appear to be associated with public backlash against trade liberalization and efforts at greening international trade. Because public support is essential to environmental and trade policy-making alike, we examine the trade–environment nexus from a public opinion perspective. We investigate whether negative attitudes toward trade are in fact fueled by concern over its environmental consequences. We argue that environmental concern affects how citizens evaluate the costs and benefits of trade, and that such evaluation is moderated by political ideology. The empirical analysis relies on a large representative survey and a population-based survey experiment in Switzerland, a small open economy. We show that environmental concern leads to decreasing appreciation of and support for international trade, with different manifestations of trade skepticism on the political left and right. This suggests (i) that policy-makers should focus more on greening global supply chains, and thus trade, if they wish to sustain public support for liberal international trade policy; and (ii) that the public follows informational cues on the environmental impacts of trade.

Effect of visualising and re-expressing evidence of policy effectiveness on perceived effectiveness: a population-based survey experiment

Abstract Communicating evidence that a policy is effective can increase public support although the effects are small. In the context of policies to increase healthier eating in out-of-home restaurants, we investigate two ways of presenting evidence for a policy's effectiveness: (i) visualising and (ii) re-expressing evidence into a more interpretable form. We conducted an online experiment in which participants were randomly allocated to one of five groups. We used a 2 (text only vs visualisation) × 2 (no re-expression vs re-expression) design with one control group. Participants (n = 4500) representative of the English population were recruited. The primary outcome was perceived effectiveness and the secondary outcome was public support. Evidence of effectiveness increased perceptions of effectiveness (d = 0.14, p < 0.001). There was no evidence that visualising, or re-expressing, changed perceptions of effectiveness (respectively, d = 0.02, p = 0.605; d = −0.02, p = 0.507). Policy support increased with evidence but this was not statistically significant after Bonferroni adjustment (d = 0.08, p = 0.034, α = 0.006). In conclusion, communicating evidence of policy effectiveness increased perceptions that the policy was effective. Neither visualising nor re-expressing evidence increased perceived effectiveness of policies more than merely stating in text that the policy was effective.

Vote-selling as unethical behavior: Effects of voter’s inhibitory self-control, decision toward vote-buying money, and candidate’s quality in Indonesia election

This study examines two stages of vote-selling among voters: to accept or refuse money, and then to vote for the vote-buyer or competitor. Using unethical behavior and money-as-temptation as a framework, we predict that the amount of the money will influence the decision to accept, but that the effect will be lower among voters with higher inhibitory self-control (ISC). We also predict that accepting money will influence the vote choice, but that the influence will be lower when the competitor has higher levels of integrity and leadership. Overall, the voters decision on money offered will moderate the relations between amount of money and vote choice. A population-based survey experiment with 1,220 participants of Indonesian voters was conducted to test these predictions. Participants were randomly assigned to one of 12 groups using a 4 (money offered: IDR50,000 vs. 100,000 vs. 150,000 vs. none) × 3 (candidate's personal quality: high leadership competitor vs. high integrity competitor vs. equal) between-group design. They made two hypothetical decisions: to accept or refuse the money, and to vote for the vote buyer or the competitor. The results showed that participants were signicantly tempted by larger amount of money, but the effect was moderated by ISC. However, there were no significant interaction between decision on money offered and candidates’ quality in influencing vote choice. Results from mediation analysis showed that accepting money was significantly mediate the influence of money on vote choice. Theoretical implications for understanding the practice of vote-buying and vote-selling are discussed.

Effects of grade, smiley, and text on Danish and Finnish consumers’ perceptions of food safety inspection reports

Publicly accessible food safety inspection reports are a standard procedure to inform consumers on restaurants' food safety levels in many countries. This study examined how different formats of food safety inspection report are associated with consumer perceptions related to food safety, as well as other perceptions about the restaurant. The study was conducted in Denmark and Finland with similar inspection grade systems but differences in the distribution of awarded grades. We conducted a population-based survey experiment with a between-subjects design on nationally representative samples of the 18–70 years old Danish (n = 978) and Finnish (n = 907) populations. Respondents received one of six food safety inspection reports with different combinations of inspection grade with a smiley and/or text. According to the results, both Danish and Finnish consumers’ food safety perceptions of the same grade were more positive when the report included a smiley, and more negative when the report included a text. Finnish respondents perceived a good food safety inspection grade more positively than Danish respondents but there were no country differences when the grade was poor. In addition, food safety inspection results elicited perceptions that were not related to food safety in both countries. The results suggest that if the grade is poor, the inclusion of text is effective in eliciting perceptions of increased food safety risk. If the grade is good, a standalone smiley may be most effective in eliciting positive perceptions of a high food safety level. Moreover, these results indicate the importance of carefully evaluating how to develop public accessible inspection grades to ensure they are correctly interpreted by consumers in different countries.

Public Opinion on Institutional Designs for the United Nations: An International Survey Experiment

AbstractScholars and policy makers have intensely debated institutional reforms of the United Nations (UN) since its creation. Yet, relatively little attention has been given to institutional design preferences among the public in UN member states. This study examines two questions: Which possible rules concerning UN authority and representation do citizens prefer? Which personal and country characteristics are associated with their varying institutional preferences? A population-based conjoint survey experiment conducted in Argentina, China, India, Russia, Spain, and the United States is used to identify public preferences on nine distinct institutional design dimensions figuring prominently in UN reform debates. We find widespread support for increasing or at least maintaining UN authority over member states and for handing control over its decision-making to UN organs that would represent the citizens of every member state more directly. Citizens’ institutional preferences are associated with their political values and vary depending on whether their home countries would gain or lose influence from a specific reform.

Acceptability of policies to reduce consumption of red and processed meat: A population-based survey experiment

Policies to reduce meat consumption are needed to help achieve climate change targets, and could also improve population health. Public acceptability can affect the likelihood of policy implementation. This study estimated the acceptability of policies to reduce red and processed meat consumption, and whether acceptability differed when policies were framed as benefitting health or the environment. In an online experiment, 2215 UK adults rated the acceptability of six policies, presented in a randomised order. Prior to rating policies, participants were randomised to one of two framing conditions, with policy outcomes described either as benefitting health or the environment. Regression models examined differences in the primary outcome – policy acceptability (rated on a 7-point scale) – by framing. Labels were the most accepted policy (48% support), followed by a media campaign (45%), reduced availability (40%) and providing incentives (38%). Increasing price (27%) and banning advertising (26%) were the least accepted. A substantial proportion of participants neither supported nor opposed most policies (26–33%), although this fell to 16% for increasing price. There was no evidence that framing policy benefits from a health or environment perspective influenced acceptability (−0.06, 95%CIs: 0.18,0.07). Fewer than half of the UK sample expressed support for any of six policies to reduce meat consumption, regardless of framing measures as benefitting health or the environment. Conversely, fewer than half expressed opposition, with the exception of price, suggesting considerable scope to influence public opinion in support of meat reduction measures to meet environmental and health goals.

Saving the environment from the internet: A polynomial mitigation model of reducing Individual Internet Consumption through Internet Pricing and Environmental Awareness

The ever-increasing internet usage purports to make substantial damage to the environment as a result of the emissions arising from the internet supporting infrastructures like data centers. However, there has been much less discussion on creating an awareness of the damage that the internet does to the environment. The purpose of this study is to investigate whether the internet use can be reduced at the individual level by calibrating internet price and one’s environmental awareness. The study employs a population-based survey experiment and conducts a polynomial regression analysis using a sample of 326 individuals to understand the conjoined relationship between internet price and environmental awareness. The results indicate that internet price affordability moderated the relationship between environmental awareness and internet usage. Although, before inducing awareness, pricing plays a major role in changing consumption, after inducing awareness the significance of pricing decreases, and awareness tends to determine the consumption. Moreover, the multi-group analysis infers that age does not show any significance on price affordability and environmental awareness, although males display a high price sensitivity and responsiveness towards awareness than their female counterparts.

When Do Work-Family Policies Work? Unpacking the Effects of Stigma and Financial Costs for Men and Women

Work-family policies—such as parental leave and flextime—can help to facilitate gender equality in workplaces and in families. But policy use is typically low, varies significantly from one workplace to another, and is often more prevalent among women than men. Extant research suggests that flexibility stigma—workplace norms that penalize workers for utilizing policies that facilitate non-work demands—as well as the financial costs associated with policy use, contribute to this pattern. However, previous studies have been largely correlational in nature, and have had difficulty assessing how these factors may interact with one another to shape gendered patterns of policy use. In this study, we offer novel causal traction on this set of issues. Using an original, population-based survey experiment, we examine how the salience of flexibility stigma and financial costs affect men's and women's intentions to use work-family policies. We find that these factors exert a large direct effect on men's and women's intentions to use work-family policies. Moreover, the gender gap in parental leave use intentions is large in workplace contexts with high flexibility stigma and high financial costs, but this gap narrows significantly under more favorable conditions. Findings point to the importance of organizational contexts and policy design in shaping work-family policy use and, in turn, gender inequality.