Prepaid Incentives Research Articles

The effect of prepaid incentives on panelists’ response across survey modes in a sequential mixed-mode design

AbstractWhile the use of prepaid incentives and data collection in a sequential mixed-mode survey design is standard in cross-sectional and longitudinal studies, little is known about the interaction of these strategies and how this affects target persons’ survey participation. Therefore, based on a panel study, this study investigates whether such unconditional prepayments—monetary or non-monetary incentives—increase the efficiency of a sequential mixed-mode survey design in regard to boosted response rates and speed of return after receipt of an invitation to participate in a survey. While the survey design is always identical, the different incentives vary across the waves considered. Different prepayments, such as cash or in-kind incentives, have different effects on invitees’ participation in different survey modes. In particular, non-monetary incentives seem to work only in a sequential mixed-mode design, while cash, as a universal medium, always works in the expected way, across different survey modes. In sum, the study finds that when a single sequential mixed-mode design is used across panel waves the overall response rates are rather similar, even when different incentives are provided.

Does it Pay to Send Multiple Pre-Paid Incentives? Evidence from a Randomized Experiment

To encourage survey participation and improve sample representativeness, the Survey of Consumer Finances (SCF) offers an unconditional pre-paid monetary incentive and separate post-paid incentive upon survey completion. We conducted a pre-registered between-subject randomized control experiment within the 2022 SCF, with at least 1,200 households per experimental group, to examine whether changing the pre-paid incentive structure affects survey outcomes. We assess the effects of: (1) altering the total dollar value of the pre-paid incentive (“incentive effect”), (2) giving two identical pre-paid incentives holding the total dollar value fixed (“reminder effect”), and (3) offering multiple pre-paid incentives of different amounts holding the total dollar value fixed (“slope effect”) on survey response rates, interviewer burden, and data quality. Our evidence indicates that a single $15 pre-paid incentive increases response rates and maintains similar levels of interviewer burden and data quality, relative to a single $5 pre-paid incentive. Splitting the $15 into two pre-paid incentives of different amounts increases interviewer burden though lengthening time in the field without improving response rates, reducing the number of contact attempts needed for a response, or improving data quality, regardless of whether the first pre-paid is larger or smaller than the second.

Peekaboo! The Effect of Different Visible Cash Display and Amount Options During Mail Contact When Recruiting to a Probability-Based Panel

Abstract Recent studies consistently showed that making cash visible with a windowed envelope during mail contact increases response rates in surveys. The visible cash aims to pique interest and encourage sampled households to open the envelope. This article extends prior research by examining the effect of additional interventions implemented during mail recruitment to a survey panel on recruitment rates and costs. Specifically, we implemented randomized experiments to examine size (small, large) and location (none, front, back) of the window displaying cash, combined with what part of the cash is shown through the window envelope (numeric amount, face/image), and various prepaid incentive amounts (two $1, one $2, one $5). We used the recruitment effort for NORC’s AmeriSpeak Panel as the data source for this study. The probability-based AmeriSpeak Panel uses an address-based sample and multiple modes of respondent contact, including mail, phone, and in-person outreach during recruitment. Our results were consistent with prior research and showed significant improvement in recruitment rates when cash was displayed through a window during mail contact. We also found that placing the window on the front of the envelope, showing $5 through the envelope compared to $2 and $1, and showing the tender amount compared to the image on the cash through the window were more likely to improve the recruitment rates. Our cost analyses illustrated that the cost difference in printing window versus no window envelope is small. There is no difference in printing cost between front window and back window as they both require custom manufacturing. There is also no cost difference in printing envelopes with small windows versus large windows. Lastly, we found no evidence of mail theft based on our review of the United States Postal Service’s “track and trace” reports, seed mailings sent to staff, and undeliverable mailing rates.

Incorporating Adaptive Survey Design in a Two-Stage National Web or Mail Mixed-Mode Survey: An Experiment in the American Family Health Study.

This article presents the results of an adaptive design experiment in the recruitment of households and individuals for a two-stage national probability web or mail mixed-mode survey, the American Family Health Study (AFHS). In the screening stage, we based the adaptive design's subgroup differentiation on Esri Tapestry segmentation. We used tailored invitation materials for a subsample where a high proportion of the population was Hispanic and added a paper questionnaire to the initial mailing for a subsample with rural and older families. In the main-survey stage, the adaptive design targeted the households where a member other than the screening respondent was selected for the survey. The adaptations included emailing and/or texting, an additional prepaid incentive, and seeking screening respondents' help to remind the selected individuals. The main research questions are (i) whether the adaptive design improved survey production outcomes and (ii) whether combining adaptive design and postsurvey weighting adjustments improved survey estimates compared to performing postsurvey adjustments alone. Unfortunately, the adaptive designs did not improve the survey production outcomes. We found that the weighted AFHS estimates closely resemble those of a benchmark national face-to-face survey, the National Survey of Family Growth, although the adaptive design did not additionally change survey estimates beyond the weighting adjustments. Nonetheless, our experiment yields useful insights about the implementation of adaptive design in a self-administered mail-recruit web or mail survey. We were able to identify subgroups with potentially lower response rates and distinctive characteristics, but it was challenging to develop effective protocol adaptations for these subgroups under the constraints of the two primary survey modes and the operational budget of the AFHS. In addition, for self-administered within-household selection, it was difficult to obtain contact information from, reach, and recruit selected household members that did not respond to the screening interview.

Guaranteed Incentives and Prize Drawings: Effects on Participation, Data Quality, and Costs in a Web Survey of College Students on Sensitive Topics

Many studies rely on traditional web survey methods in which all contacts with sample members are through email and the questionnaire is administered exclusively online. Because it is difficult to effectively administer prepaid incentives via email, researchers frequently employ lotteries or prize draws as incentives even though their influence on survey participation is small. The current study examines whether a prize draw is more effective if it is divided into a few larger amounts versus several smaller amounts and compares prize draws to a small but guaranteed postpaid incentive. Data are from the 2019 Campus Climate Survey on Sexual Assault and Sexual Misconduct. Sample members include 38,434 undergraduate and graduate students at a large Midwestern university who were randomly assigned to receive: a guaranteed $5 Amazon gift card; entry into a high-payout drawing for one of four $500 prizes; or entry into a low-payout drawing for one of twenty $100 prizes. Results indicate the guaranteed incentive increased response rates, with no difference between the prize draws. While results from various data quality outcomes show the guaranteed incentive reduced break-off rates and the high-payout drawing increased item nonresponse, there were no differences across incentive conditions in rates of speeding, reporting of sensitive data, straightlining, or sample representativeness. As expected, the prize draws had much lower overall and per complete costs.

The interplay of incentives and mode-choice design in self-administered mixed-mode surveys

Self-administered mixed-mode surveys are increasingly used as an alternative to face-to-face surveys for collecting data from the general population. However, little is known about how decisions regarding the incentive scheme and the mode-choice design jointly affect key outcomes such as response rates, net sample composition, and survey costs. To study this, we drew a probability sample of the residential population of the city of Mannheim, Germany ( N = 2,980) and randomly assigned target persons to one of four incentive schemes (€0, €1, or €2 prepaid incentive on first contact, and €2 delayed prepaid incentive) and one of two mode-choice designs (concurrent or sequential [web-push]). Our results indicate that small prepaid monetary incentives work better in concurrent than in sequential designs. Moreover, a €2 prepaid incentive in a concurrent design proved particularly successful for older target persons, probably reinforcing their sense of trust and reciprocity, while also fitting better with their survey-mode preferences. Finally, a €2 delayed prepaid incentive in a sequential design primarily motivated target persons aged under 50 years. This combination of incentive scheme and mode-choice design also proved to be most cost-effective in that age group. Based on our results, we recommend using sampling frame information on age to address different age groups with different combinations of incentive scheme and mode-choice design. This may help to maximize response rates, achieve a balanced net sample composition, and minimize survey costs.

The Visible Cash Effect with Prepaid Incentives: Evidence for Data Quality, Response Rates, Generalizability, and Cost

Abstract A prior study found that mailing prepaid incentives with $5 cash visible from outside the envelope increased the response rate to a mail survey by 4 percentage points compared to cash that was not externally visible. This “visible cash effect” suggests opportunities to improve survey response at little or no cost, but many unknowns remain. Among them: Does the visible cash effect generalize to different survey modes, respondent burdens, and cash amounts? Does it differ between fresh samples and reinterview samples? Does it affect data quality or survey costs? This article examines these questions using two linked studies where incentive visibility was randomized in a large probability sample for the American National Election Studies. The first study used $10 incentives with invitations to a long web questionnaire (median 71 minutes, n = 17,849). Visible cash increased response rates in a fresh sample for both screener and extended interview response (by 6.7 and 4.8 percentage points, respectively). Visible cash did not increase the response rate in a reinterview sample where the baseline reinterview response rate was very high (72 percent). The second study used $5 incentives with invitations to a mail-back paper questionnaire (n = 8,000). Visible cash increased the response rate in a sample of prior nonrespondents by 4.0 percentage points (from 31.5 to 35.5), but it did not increase the response rate in a reinterview sample where the baseline reinterview rate was very high (84 percent). In the two studies, several aspects of data quality were investigated, including speeding, non-differentiation, item nonresponse, nonserious responses, noncredible responses, sample composition, and predictive validity; no adverse effects of visible cash were detected, and sample composition improved marginally. Effects on survey costs were either negligible or resulted in net savings. Accumulated evidence now shows that visible cash can increase incentives’ effectiveness in several circumstances.

Incentive delivery timing and follow-up survey completion in a prospective cohort study of injured children: a randomized experiment comparing prepaid and postpaid incentives

BackgroundRetaining participants over time is a frequent challenge in research studies evaluating long-term health outcomes. This study’s objective was to compare the impact of prepaid and postpaid incentives on response to a six-month follow-up survey.MethodsWe conducted an experiment to compare response between participants randomized to receive either prepaid or postpaid cash card incentives within a multisite study of children under 15 years in age who were hospitalized for a serious, severe, or critical injury. Participants were parents or guardians of enrolled children. The primary outcome was survey response. We also examined whether demographic characteristics were associated with response and if incentive timing influenced the relationship between demographic characteristics and response. We evaluated whether incentive timing was associated with the number of calls needed for contact.ResultsThe study enrolled 427 children, and parents of 420 children were included in this analysis. Follow-up survey response did not differ according to the assigned treatment arm, with the percentage of parents responding to the survey being 68.1% for the prepaid incentive and 66.7% with the postpaid incentive. Likelihood of response varied by demographics. Spanish-speaking parents and parents with lower income and lower educational attainment were less likely to respond. Parents of Hispanic/Latino children and children with Medicaid insurance were also less likely to respond. We found no relationship between the assigned incentive treatment and the demographics of respondents compared to non-respondents.ConclusionsPrepaid and postpaid incentives can obtain similar participation in longitudinal pediatric critical care outcomes research. Incentives alone do not ensure retention of all demographic subgroups. Strategies for improving representation of hard-to-reach populations are needed to address health disparities and ensure the generalizability of studies using these results.

An Experimental Comparison of Three Strategies for Converting Mail Respondents in a Probability-Based Mixed-Mode Panel to Internet Respondents

AbstractIn recent years, web-push strategies have been developed in cross-sectional mixed-mode surveys to improve response rates and reduce the costs of data collection. However, pushing respondents into the more cost-efficient web mode has rarely been examined in the context of panel surveys. This study evaluates how a web-push intervention affects the willingness of panel members to switch survey modes from mail to web. We tested three web-push strategies in a German probability-based mixed-mode panel by randomly assigning 1,895 panelists of the mail mode to one of three conditions: (1) the web option was offered to panelists concurrently with the paper questionnaire including a promised €10 incentive for completing the survey on the web, (2) the web option was presented sequentially two weeks before sending the paper questionnaire and respondents were also promised an incentive of €10, or (3) same sequential web-first approach as for condition 2, but with a prepaid €10 incentive instead of a promised incentive. The study found that a sequential presentation of the web option significantly increases the web response in a single survey but may not motivate more panelists to switch to the web mode permanently. Contrary to our expectation, offering prepaid incentives neither improves the web response nor the proportion of mode switchers. Overall, all three web-push strategies show the potential to effectively reduce survey costs without causing differences in panel attrition after five consecutive waves. Condition 2, the sequential web-first design combined with a promised incentive was most effective in pushing respondents to switch to the web mode and in reducing costs.

Effects of Sequential Prepaid Incentives on Response Rates, Data Quality, Sample Representativeness, and Costs in a Mail Survey of Physicians.

While collecting high quality data from physicians is critical, response rates for physician surveys are frequently low. A proven method for increasing response in mail surveys is to provide a small, prepaid monetary incentive in the initial mailing. More recently, researchers have begun experimenting with adding a second cash incentive in a follow-up contact in order to increase participation among more reluctant respondents. To assess the effects of sequential incentives on response rates, data quality, sample representativeness, and costs, physicians (N = 1,500) were randomly assigned to treatments that crossed the amount of a first ($5 or $10) and second ($0, $5, or $10) incentive to form the following groups: Group $5/$5; Group $5/$10; Group $10/$0; Group $10/$5; and Group $10/$10. Overall, second incentives were associated with higher response rates and lower costs per completed survey, and while they had no effect on item nonresponse, they increased sample representativeness.

Cost effectiveness of pre- and post-paid incentives for mail survey response

While incentives improve survey response rates, they entail significant costs for the researcher. A mail survey study randomized across three incentive conditions ($2 pre-paid, $10 post-paid, and $20 post-paid), found that households in the $2 pre-paid condition were more likely to respond than the $10 post-paid condition (48.5% vs. 33.5%, β = -0.536, SE = 0.154, p < .001) and the $20 post-paid condition (48.5% vs. 42.4%, β = -0.305, SE = 0.10, p = 0.11). The $2 pre-paid condition was also more cost-effective than the $10 and $20 post-paid conditions (cost-per-complete of $15.35 vs. $26.67 and $34.40, respectively). The results of this study show that a small pre-paid incentive is generally most effective for maximizing both response rates and costs. The data from the study is also used to construct a model to forecast cost-efficiency for variable eligibility conditions, through a factor-adjusted response rate.

What Works, What Doesn’t? Three Studies Designed to Improve Survey Response

Over the past few decades, the survey industry has experienced a steady decline in response rates, which has posed numerous challenges for researchers, most notably concerns about nonresponse bias. We present results from three studies conducted in an attempt to increase response rates and reduce nonresponse bias for a U.S. national household survey. We examined design features related to mode (i.e., mail-only vs. web-push), incentive, reminder type, nonresponse follow-up, and messaging. Results showed that a combination of a mail-only approach and the use of US$2 prepaid incentives increased response rates. However, using incentives did not bring into the respondent pool sample members who were less likely to respond. Additionally, we found that the use of a seventh follow-up contact only slightly increased response rates. Finally, we found that the use of a friendly message not only suppressed response rates but also increased potential nonresponse bias for sample members aged 35–48.

Assessing Health and Safety Concerns and Psychological Stressors among Agricultural Workers in the U.S. Midwest

There is limited research exploring agricultural workers' own perspectives on the relative importance of the hazards and stressors they experience. There is also a lack of evidence on whether this reporting differs by method of elicitation. Finally, very little research exists on how to improve mail survey response rates among agricultural workers. We examined health and safety concerns and psychological stressors among Midwestern farmers. We assessed whether these reports varied by survey mode (mail survey versus in-person survey). The efficacy of two different types of incentives to enhance mail survey response rates among agricultural workers was also investigated. In 2018, a needs assessment survey was developed and mailed to a random sample of farm owner-operators in Iowa, Ohio, and Missouri, with randomly assigned prepaid or promised monetary incentives. In-person surveys were conducted among farm owner-operators and hired workers at three regional farm shows in Iowa, Minnesota, and Nebraska. The mail survey response rates were compared by incentive type. Content analysis was used to generate themes associated with health and safety concerns and psychological stressors, which were then ranked by frequency counts. Chi-square tests were used to analyze variation in the distribution of these themes by survey mode. The response rate for the $1 prepaid incentive was double that of the $10 promised incentive. Content analysis identified 13 health and safety concerns and eight psychological stressors. Chemicals, equipment/tools, and health outcomes were the most frequently noted health and safety concerns. Finances, climate/weather, and farm workload and management were the most frequently noted psychological stressors. Although there was considerable overlap in survey responses across mail and in-person respondents, important differences by sample and survey mode characteristics were observed. The results can support a variety of stakeholders in prioritizing and developing interventions and educational resources to address health and safety concerns and psychological stressors among Midwestern farmers. Our findings also contribute to the evidence base on primary data collection methods for agricultural workers.

Improving Survey Response Rates with Visible Money

AbstractIn mail surveys and in advance letters for surveys in other modes, it is common to include a prepaid incentive of a small amount such as $5. However, when letters are addressed generically (such as to “Resident”), advance letters may be thrown away without being opened, so the enclosed cash is wasted and the invitation or advance letter is ineffective. This research note describes results of an experiment using a nationally representative sample of 4,725 residential addresses to test a new way of letting mail recipients know their letter contains cash and is therefore worth opening: an envelope with a window revealing $5, so the cash is clearly visible from outside the sealed envelope. We also tested the USPS for evidence of theft, and we compared First Class and Priority Mail postage. We found no evidence of theft. We found no difference in response rates between Priority Mail and First Class, making First Class much more cost-effective, and we found that visible money increased the response rate to a mail survey from 42.6 to 46.9 percent, at no significant cost.

Effect of prepaid and promised financial incentive on follow-up survey response in cigarette smokers: a randomized controlled trial

BackgroundMonetary incentive is often used to increase response rate in smokers’ survey, but such effect of prepaid and promised incentives in a follow-up survey is unknown. We compared the effect of different incentive schemes on the consent and retention rates in a follow-up survey of adult cigarette smokers.MethodsThis was a randomized controlled trial (RCT) in Hong Kong, China. Smokers who completed a non-incentivized baseline telephone smoking survey were invited to a 3-month follow-up, with randomization into (1) the control group (no incentive), (2) a promised HK$100 (US$12.8) incentive upon completion, (3) a promised HK$200 (US$25.6) incentive upon completion, or (4) a prepaid HK$100 incentive plus another promised HK$100 incentive (“mixed incentive”). Crude risk ratios from log-binomial regression models were used to assess if the 3 incentive schemes predicted higher rates of consent at baseline or retention at 3-month than no incentive.ResultsIn total, 1246 smokers were enrolled. The overall consent and retention rates were 37.1 and 23.0%, respectively. Both rates generally increased with the incentive amount and offer of prepaid incentive. The mixed incentive scheme marginally increased the retention rate versus no incentive (26.8% vs 20.3%; risk ratio (RR) = 1.32; 95% CI: 1.00–1.76; P = 0.053), but not the consent rate (RR = 1.13; 95% CI: 0.93–1.38; P = 0.22). Among the consented participants, approximately 50% in the mixed incentive group received the mailed prepaid incentive, who achieved a higher retention rate than the group without incentives (82.8% vs 56.1%; RR = 1.48; 95% CI: 1.21–1.80; P < 0.01).ConclusionThe mixed incentive scheme combining the prepaid and promised incentive was effective to increase the follow-up retention rate by 48%. We recommend this mixed incentive scheme to increase the follow-up retention rate. More efficient methods of delivering the incentive are needed to maximize its effects.Trial registrationU.S. Clinical Trials registry (clinicaltrials.gov, retrospectively registered, reference number: NCT03297866).

A $2 Bill or Two $1 Bills: An Experiment that Challenges Standard Protocol

The $2 bill is commonly used as a prepaid incentive in survey administration. Despite the widespread use of this incentive, there is no peer-reviewed research into how providing a single $2 bill rather than two $1 bills affects survey response rate. The results are from a mail survey of 2,970 rural landowners in Michigan with identical survey protocols. Half were randomly assigned to receive a single $2 bill and the other half to receive two $1 bills accompanying the questionnaire. I found no statistical difference in the response rates of the two groups. While there may be logistical advantages to using a single $2 bill compared to two $1 bills, this research suggests that the $2 bill is not more effective at inducing survey participation.

Cash vs. vouchers vs. gifts in web surveys of a mature panel study––Main effects in a long-term incentives experiment across three panel waves

In this study, we evaluate short- and long-term effects of three different prepaid incentives: a ballpoint-pen (gift worth approximately 2 Swiss francs), a voucher (cash card worth 10 Swiss francs) and cash (a 10-Swiss-francs’ banknote) on young panellists’ cooperation and response rate in three waves of a mature panel study with a sequential multi-mode design (web-based online survey, CATI, and PAPI). The survey experiment involved an alternative procedure to analyse the effect of different types of prepaid incentives, taking selective attrition into account as well as considering problems related to causal inference. The subjects were students, from randomly-selected school classes, who had finished their compulsory school in 2013. The findings are clear: cash provides the strongest direct, positive effect on the overall response rate and also on the latency until response after first contact. The other incentives did not work as efficiently as did cash. Additionally, cash is the most likely to minimise social selectivity in response. Finally, cash provides the potential to convert refusals in previous waves into cooperation.

Risks and rewards of using prepaid vs. postpaid incentive checks on a survey of physicians

BackgroundSurvey researchers use monetary incentives as a strategy to motivate physicians’ survey participation. Experiments from general population surveys demonstrate that prepaid incentives increase response rates and lower survey administration costs relative to postpaid incentives. Experiments comparing these two incentive strategies have rarely been attempted with physician samples.MethodsA nationally representative sample of oncologists was recruited to participate in the National Survey of Precision Medicine in Cancer Treatment. To determine the optimal strategy for survey incentives, sample members were randomly assigned to receive a $50 prepaid incentive check or a $50 promised (postpaid) incentive check. Outcome measures for this incentives experiment include cooperation rates, speed of response, check-cashing behavior, and comparison of hypothetical costs for different incentive strategies.ResultsCooperation rates were considerably higher for sample members in the prepaid condition (41%) than in the postpaid condition (29%). Similar differences in cooperation rates were seen for physicians when stratified by region, size of the physician’s metropolitan statistical area, specialty, and gender by age. Survey responders in the prepaid condition responded earlier in the field period than those in the postpaid condition, thus requiring fewer contacts. In the prepaid group, 84% of sample members who responded with a completed survey cashed the incentive check and only 6% of nonresponders cashed the check. In the postpaid condition, 72% of survey responders cashed the check; nonresponders were not given a check. The relatively higher cooperation rates and earlier response of the responders in the prepaid condition was associated with a 30% cost savings for the prepaid condition compared to the postpaid incentive condition.ConclusionsThe results of this study suggest that the rewards of offering physicians a prepaid incentive check outweigh the possible risks of nonresponders cashing the check. The relative cost benefit of this strategy is likely to vary depending on the amount of the incentive relative to the costs of additional contact attempts to nonresponders.

Exploring the Feasibility of Mail Surveys in Urban China

While mail surveys are common in many developed countries, it is still a new method in other, often less-developed countries. This study explores the feasibility of mail surveys in urban China. We conducted an experiment manipulating delivery method (regular mail vs. courier delivery service) and incentive payment (prepaid vs. no incentive) using an area-probability sample of 2,400 households in Shanghai. Concerns about the effectiveness of prepaid incentives in China arise given the prevalence of scams. Courier delivery is generally faster and more reliable than regular mail, but there was uncertainty whether courier could outperform regular delivery when recipients’ name and contact information are unavailable. Findings show that mail surveys delivered using regular postal service with a prepaid incentive achieved the most cost-effective return per completed interview in Shanghai.

Psychological influence on survey incentives: valuing climate change adaptation benefits in agriculture

Psychological influences affect the way people value the environment. However, traditional economic valuation models often do not account for how people are asked about valuing the environment. We examined how valuations by Nepalese farmers differ based on how the questions are asked and which incentives are provided. In a face-to-face choice experiment, incentive receivers spent more time than incentive non-receivers answering the survey, but were not more likely to choose a status quo option. Prepaid survey incentives had minimal effect on the stated welfare measures. The results suggest that prepaid incentives increase response rates, but do not increase welfare estimates. The findings also strengthen the methodological validity of our results, which indicated that farmers are willing to pay a substantial amount to secure climate change adaptation benefits on their land.