Falling Response Rates Research Articles

Evaluation of the effectiveness of an incentive strategy on the questionnaire response rate in parents of premature babies: a randomised controlled Study Within A Trial (SWAT) nested within SIFT

BackgroundLoss to follow-up resulting in missing outcomes compromises the validity of trial results by reducing statistical power, negatively affecting generalisability and undermining assumptions made at analysis, leading to potentially biased and misleading results. Evidence that incentives are effective at improving response rates exists, but there is little evidence regarding the best approach, especially in the field of perinatal medicine. The NIHR-funded SIFT trial follow-up of infants at 2 years of age provided an ideal opportunity to address this remaining uncertainty.MethodsParticipants: parents of infants from participating neonatal units in the UK and Ireland followed up for SIFT (multicentre RCT investigating two speeds of feeding in babies with gestational age at birth < 32 weeks and/or birthweight < 1500 g). Interventions: parents were randomly allocated to receive incentives (£15 gift voucher) before or after questionnaire return. The objective was to establish whether offering an unconditional incentive in advance or promising an incentive on completion of a questionnaire (conditional) improved the response rate in parents of premature babies. The primary outcome was questionnaire response rate. Permuted block randomisation was performed (variable size blocks), stratified by SIFT allocation (slower/faster feeds) and single/multiple birth. Multiple births were given the same incentives allocation. Parents were unaware that they were in an incentives SWAT; SIFT office staff were not blinded to allocation.ResultsParents of 923 infants were randomised: 459 infants allocated to receive incentive before, 464 infants allocated to receive incentive after; analysis was by intention to treat. Allocation to the incentive before completion led to a significantly higher response rate, 83.0% (381/459) compared to the after-completion group, 76.1% (353/464); adjusted absolute difference of 6.8% (95% confidence interval 1.6% to 12.0%). Giving an incentive in advance is the more costly approach, but the mean difference of ~£3 per infant is small given the higher return.ConclusionsAn unconditional incentive in advance led to a significantly higher response rate compared to the promise of an incentive on completion. Against a backdrop of falling response rates to questionnaires, incentives can be an effective way to increase returns.Trial registrationSIFT (ISRCTN76463425). Registered on March 5, 2013.; SWAT registration (SWAT 69 available from http://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/FileStore/Filetoupload,864297,en.pdf). Registered on June 27, 2016.

Assessing consent for and response to health survey components in an era of falling response rates: National Health and Nutrition Examination Survey, 2011-2018.

Response rates for national population-based surveys have declined, including the National Health and Nutrition Examination Survey (NHANES). Declining response to the initial NHANES interview may impact consent and participation in downstream survey components such as record linkage, physical exams, storage of biological samples and phlebotomy. Interview response rates dropped from 68% in 2011-2012 to 53% in 2017-2018 for adults age 18 and older. Response was higher for children (1-17 years) but with a similar downward trend (2011-2012, 81%; 2017-2018, 65%). Despite declining interview response rates, changes in consent and response rates for downstream components over time have been mixed. Among those interviewed, the examination response rate was over 93%, consent for record linkage was over 90%, and consent for storage of specimens for future research was over 99%. The availability of a blood sample for storage ranged between 60%-65% for children and 78%-85% for adults.

Evaluating Nonresponse Bias in Survey Research Conducted in the Rural Midwest

An understanding of private landowner’s perceptions can guide decisions about conservation in rural landscapes. However, mailed surveys that evaluate landowner views are increasingly plagued by falling response rates and nonresponse bias. Using survey data from research conducted in the Midwestern United States in 2007 and 2017, we adapted a framework for testing nonresponse bias across demographics, attitudes, and land-use practices inferred from aerial imagery. We compared respondents and nonrespondents, early and late-respondents, low- and high-interest respondents, and our sample and published data. Across all comparisons, we found little consistent evidence of nonresponse bias, except a possible under-sampling of Amish landowners and conflicting results on gender. Many discrepancies were likely related to the sampling methodology. Because comparisons yielded conflicting results, we recommend that researchers engaged in survey research with rural communities use multiple methods to triangulate nonresponse bias to determine whether subgroups of rural populations are under-sampled.

Exploring New Statistical Frontiers at the Intersection of Survey Science and Big Data: Convergence at “BigSurv18”

Held in October 2018, The Meets Survey Science conference, also known as BigSurv18, provided a first-of-its-kind opportunity for survey researchers, statisticians, computer scientists, and data scientists to convene under the same roof. At this conference, scientists from multiple disciplines were able to exchange ideas about their work might influence and enhance the work of others. This was a landmark event, especially for survey researchers and statisticians, whose industry has been buffeted of late by falling response rates and rising costs at the same time as a proliferation of new tools and techniques, coupled with increasing availability of data, has resulted in Big Data approaches to describing and modelling human behavior.

All's Not Well on the Marketing Frontlines: Grasping the Challenges of Adverse TechnologyyConsumer Interactions

The dominant worldview among marketers is one of technology optimism, which holds that technological advances influence consumers and businesses in positive ways. In direct contrast to this perspective, I advance the thesis that at the organizational frontlines where marketers interact with consumers by observing, informing, persuading, negotiating and co-creating with, and entertaining them, technology commonly produces unforeseen and unexpected effects on consumers with significant negative implications for marketers. The result is Adverse Technology-Consumer Interactions (ATCIs). Marketing practitioners play an instrumental role in producing and exacerbating ATCIs. Yet, I argue they have few incentives to fully investigate the underlying reasons, understand their scope, or find solutions to these potentially troublesome phenomena. Academic researchers, however, are uniquely poised to identify ATCIs, investigate them in depth by considering their industry-wide and society-wide import, develop appropriate theoretical frameworks, and design and test solutions to alleviate their effects. I develop these ideas by considering two ATCIs, falling response rates to customer surveys and customer reactance to frequent price changes. I also point out promising research opportunities for both these phenomena.

Redesign in a statistical office - sociodemographic statistics in the Office for National Statistics

The Office for National Statistics (ONS) has developed a Strategy setting out the areas in which it aims to make progress in the coming years in order to be well-placed to take advantage of the opportunities offered by technology and new data sources, and to meet the challenges of anticipated budget constraints and falling response rates. One key part of this is a project investigating options for future provision of population and small area sociodemographic statistics. I will outline the way the options in this project interact with ONS's existing social survey programme and the system for constructing population estimates as an example of how the strategy is leading the redesign process in the ONS.

Patient Characteristics of Provider Survey Respondents

Surveys of physicians are an important tool to assess opinions and self-reported behaviors of this policy-relevant population. However, this population is notoriously difficult to survey and plagued with low and falling response rates. In order to evaluate the potential import of response rate, we examine the presence of nonresponse bias in a survey of physicians providing diabetes care that achieved a 36% response rate. Unlike other studies examining differences in individual characteristics for responding and nonresponding physicians, we also assess differences with respect to aggregate patient demographic, clinical, and behavioral characteristics. We are unable to demonstrate nonresponse bias, even with what could be construed as a relative low response rate. Nonetheless as the threat of nonresponse bias can never be completely assuaged, we believe that it should be monitored as a matter of course in physician surveys and offer a new dimension by which it can be evaluated.

Would you mind completing this survey: Assessing the state of survey research in supply chain management

In supply chain management, survey research is one of the primary methodologies used by researchers to generate empirical data. Critical to its effectiveness is an acceptable response rate. Recently, concerns have been raised regarding perceived falling response rates. Using data taken from 464 survey-based studies, as drawn from five representative journals that publish survey-based supply chain studies, this study assesses whether these concerns are valid. Furthermore, it assesses whether the various tactics recommended for improving response rates have been successful. The findings indicate that (1) responses rates have been falling (with a significant decrease beginning in 2001); (2) it is difficult to evaluate the effectiveness of the various techniques because of unevenness in reporting of techniques used; and, (3) response rates are significantly influenced by factors such as the number of questions, source of survey population, method of survey delivery, specific respondents targeted, and use of prequalification/pre-notification.

Survey Methodology: International Developments

Falling response rates and the advancement of technology have shaped the discussion in survey methodology in the last few years. Both led to a notable change in data collection efforts. Survey organizations try to create adaptive recruitment and survey designs and increased the collection of non-survey data for sampled cases. While the first strategy is an attempt to increase response rates and to save cost, the latter is part of efforts to reduce possible bias and response burden of those interviewed. To successfully implement adaptive designs and alternative data collection efforts researchers need to understand error properties of mixed-mode and multiple-frame surveys. Randomized experiments might be needed to gain that knowledge. In addition close collaboration between survey organizations and researchers is needed, including the possibility and willingness to shared data between those organizations. Expanding options for graduate and post-graduate education in survey methodology might help to increase the possibility for high quality surveys.

Student Engagement: The Key to Improving Survey Response Rates

Over the last decade, universities have paid increasing attention to stakeholder needs, which include the needs and expectations of students. To identify student needs and expectations, tertiary institutions have primarily employed the mechanism of evaluation surveys. In recent years, international research has shown falling survey response rates among students. Monash University, Australia, identified the issue of falling response rates as critical, particularly regarding the graduate destination survey and the course experience questionnaire distributed to Australian university graduates. To address the issue, Monash University adopted a strategy to increase student engagement. The strategy involved setting up a call centre for the purpose of contacting recent Monash University graduates who had not completed the graduate destination survey and course experience questionnaire surveys to explain the significance of the graduate feedback to the institution. This paper highlights the significance of enhancing the engagement of tertiary institutions with students and proposes enhancement of student engagement as one of the more effective and efficient ways of addressing declining student survey response rates.

What Respondents Really Expect From Researchers

This article addresses the issue of falling response rates in telephone surveys. To better understand and maintain respondent goodwill, concepts of psychological contract and respondent expectations are introduced and explored. Results of the qualitative study show that respondent expectations are not only socially contingent but also ego-expressive, utilitarian, pleasurable, and epistemic by nature. Although results are reassuring in terms of commercialization of the psychological contract, they indicate some radical changes that are needed for the respondents to accept its continuation. The article discusses several practical and theoretical implications of such changes and suggests a series of corresponding propositions aimed at facilitating and inspiring future developments in this field.

Parameter estimation validity and relationship robustness: A comparison of telephone and internet survey techniques

With the expansion of telecommunication and online technologies for the purpose of survey administration, the issue of measurement validity has come to the fore. The proliferation of automated audio services and computer-based survey techniques has been matched by a corresponding denigration of the quality of traditional phone survey data, most notably as an outcome of falling response rates. This trend, combined with the introduction of screening technologies and answering machines, represents a barrier to the proper execution of survey research. Whereas the question was once, “can technology-assisted surveys achieve the same level of validity as traditional phone surveys?”, the question now becomes, “what are the relative advantages and disadvantages of technology-assisted and phone surveys?” Each has its own challenges and opportunities, and this paper begins to explore these. The present study provides further insight into the validity of telephone and Internet survey data, and explores whether or not the robustness of relationships between variables varies by survey mode. Study data were provided by two surveys, the first of which was conducted in a metropolitan area of the Midwestern US, with interviews of 505 adults using a computer-aided telephone-interviewing (CATI) system. The second was a national survey of 2172 respondents conducted over the Internet by a commercial research firm that sends requests to a diverse set of potential respondents, who logged onto the survey site to participate. Results suggest that weighting in an attempt to achieve parametric matching does seem to increase robustness of relationships and, in this age of poor response rates, this seems to demand an increased use of parametric weightings. Implications of study findings for telematic survey practitioners are discussed.

Consumer Sentiment Surveys

Consumer sentiment surveys are regularly conducted in at least forty-five countries. The surveys are based on the premise that data on consumer expectations represent a leading indicator of future changes in the macro economy. A series of Granger causality tests indicated that measures of consumer sentiment both predict and are predicted by a wide range of economic variables, although the data exhibit substantial differences across countries and across variables. The predictive performance was not found to be directly related to differences in sampling, question wording, or index construction. Several recommendations are given to enhance the predictive performance of consumer sentiment. First, a new conceptualization of consumer sentiment is now warranted based on changes in the knowledge and sophistication of consumers, the increased availability of economic information, shifts in the composition of demand, and changes due to an aging population. New survey measurement techniques are needed as well. Preliminary data on numeric probability scales indicate that there are advantages and disadvantages to a shift away from the now commonly used verbal likelihood scales. Second, new analysis models are proposed that would disaggregate consumer spending into its components as well as disaggregate the consumer data into separate demographic and economic subgroups. Third, changes in the survey administration are proposed. The shift away from telephone surveys will be due to falling response rates, especially among younger adults; the shift toward internet surveys will be due to their greater capacity to handle much more complex measurement strategies. Finally, the very notion of worldwide harmonization must also evolve; shifting its main focus from the input questions to the output indices so as to encompass a greater range of diversity in measurement methodologies. The basic premise of the consumer sentiment surveys has been affirmed by past measurements, and with appropriate developments, the consumer sentiment surveys can provide even more robust and timely forecasts of trends in the macro economy in the future.

Not another questionnaire! Maximizing the response rate, predicting non-response and assessing non-response bias in postal questionnaire studies of GPs.

Non-response is an important potential source of bias in survey research. With evidence of falling response rates from GPs, it is of increasing importance when undertaking postal questionnaire surveys of GPs to seek to maximize response rates and evaluate the potential for non-response bias. Our aim was to investigate the effectiveness of follow-up procedures when undertaking a postal questionnaire study of GPs, the use of publicly available data in assessing non-response bias and the development of regression models predicting responder behaviour. A postal questionnaire study was carried out of a random sample of 600 GPs in Wales concerning their training and knowledge in palliative care. A cumulative response rate graph permitted optimal timing of follow-up mailings: a final response rate of 67.6% was achieved. Differences were found between responders and non-responders on several parameters and between sample and population on some parameters: some of these may bias the sample data. Logistic regression analysis indicated medical school of qualification and current membership of the Royal College of General Practitioners to be the only significant predictors of responders. Late responders were significantly more likely to have been qualified for longer. This study has several implications for future postal questionnaire studies of GPs. The optimal timing of reminders may be judged from plotting the cumulative response rate: it is worth sending at least three reminders. There are few parameters that significantly predict GPs who are unlikely to respond; more of these may be included in the sample, or they may be targeted for special attention. Publicly available data may be used readily in the analysis of non-response bias and generalizability.