Mixed-mode Survey Design Research Articles

On the Relative Advantage of Mixed-Mode versus Single-Mode Surveys

Survey researchers increasingly use mixed-mode surveys for general population data collection because mixed-mode surveys are argued to provide lower selection error at constant budgets or lower variable costs at constant selection error. Nevertheless, the advantage of lower selection error and variable costs might be counteracted by higher measurement error and by higher fixed costs. This trade-off between selection error, measurement error, variable costs and fixed costs has hardly been studied within the existing literature about mixed-mode surveys. This paper discusses a possible procedure for evaluating this trade-off by comparing the performance (mean squared error) of mixed-mode survey designs against single-mode survey designs. The procedure is further illustrated by real example data stemming from a mixed-mode mail---face-to-face survey. This illustration yields smaller errors for single-mode designs under low budgets but smaller errors for mixed-mode designs under large budgets or, alternatively, a budgetary advantage of single-mode designs when the allowed error is relatively high but a budgetary advantage of mixed-mode designs when the allowed error is relatively small. However, the validity of these results depend on several modelling assumptions which may be topics for future research.

Coverage Error in Internet Surveys: Can Fixed Phones Fix It?

The internet is increasingly being used for cross-sectional surveys and online panels. Although internet accessibility is growing across developed and developing countries, it seems unlikely that the internet alone will ever provide complete coverage of the general population. Given the incomplete coverage and imbalanced penetration rate of the internet across segments of the population, it is pertinent both for survey companies and academics to assess the potential of mixing the internet with other survey modes as part of a strategy to assure validity of inferential samples when surveying general populations. The purpose of this research was to evaluate to what extent coverage error in internet surveys can be reduced by surveying the offline population via telephone. We use data from Eurobarometer collected in the EU27 member states to simulate first an internetbased survey and then a mixed-mode survey combining the internet with the telephone. Comparisons are made to identify differences in the socio-demographic characteristics of internet households and those of non-internet households with telephone. Coverage error is also estimated in each survey design. Findings reveal significant socio-demographic differences and although the coverage error is reduced in the mixed-mode survey design, it cannot be completely eliminated. Moreover, the outcomes are not homogeneous across countries.

The outsourcing of health, sport and physical educational work: a state of play

Background: The outsourcing of health, sport and physical educational (HSPE) work has been a feature of physical education (PE) ‘futures talk’ for over 20 years. However, HSPE work outsourcing has been the focus of little empirical research and only occasional commentary. That small amount of empirical research that has been conducted has been narrow in scope, focusing exclusively on primary schools and at times presented data that are partial and fragmentary. Purpose: The purpose of this paper was to investigate HSPE work outsourcing in the Australian state of Queensland. Specifically, we examined the prevalence of such outsourcing among Queensland schools, the ways that these schools were using it, and the reasons why they had done so. Methods: A random, proportionately stratified sample of 846 Queensland schools was invited to participate in this study. This sample included government and non-government, primary, secondary, combined primary-secondary and special schools. Data were collected via a mixed-mode survey design that was implemented using Internet and paper-copy forms of a context specific 21-item questionnaire. Results: The questionnaire's response rate was 32%. Simple relational analyses of the data demonstrated that: (a) 85% (n = 230) of schools reported outsourcing some form of HSPE work in the previous twelve months; (b) many schools used outsourced HSPE work for outdoor adventure activities and extra-curricular activities; (c) 75% (n = 203) of all respondents engaged in some form of fee-based outsourcing and 83% of all outsourcing arrangements were fee-based; (d) the majority of schools paid for outsourced HSPE work using school funds or by charging participating students; and (e) the most frequently reported reason for outsourcing HSPE work was to access external suppliers' expertise. Conclusions: These data replicate some findings of previous research in a different research context as well as presenting forms of data on HSPE work outsourcing hitherto unreported. Furthermore, they demonstrate the ways in which HSPE work is embedded in broader social, political and economic shifts, particularly changing relations between capital, education and the state. Finally they also serve to prompt critical questions about why HSPE work is outsourced, how it is outsourced and the impact it has on the educational labour process and all those involved in it. We conclude that HSPE work outsourcing is a practice with the potential to significantly disturb labouring, learning and the politics of expertise as they relate to health, sport and physical education.

Diaspora tourism: Using a mixed-mode survey design to document tourism behavior and constraints of people of Turkish extraction resident in Germany

Despite their growing numbers as consumers and citizens, and in spite of the ongoing discussion about immigrants and integration in many countries, ethnic minorities remain almost invisible in the tourism literature on consumer behavior, tourism rights and enjoyment, and the effects of tourism on integration. Ethnic minorities are not a homogenous group and a better understanding may be achieved by focusing on one group. This study used a mixed-mode research design to examine the tourism behavior of people of Turkish background living in Germany, with an emphasis on identifying and explaining their tourism constraints. Marginality, ethnicity, discrimination, and acculturation theories were used with the concepts of acculturation, familism and religion as the background to the study. Unlike other studies, in this study discrimination and marginality hypotheses were not supported as factors explaining tourism behavior and constraints of this group. Rather, ethnicity tainted with familism, religion and tradition appeared to interact to shape this group’s travel behavior and subsequently their negotiation of perceived constraints. This complexity begs for similar studies to inform our understanding of ethnic minorities’ travel behavior and constraints.

Comparison of an Internet Versus Mail Survey: A Case Study

The benefits of Internet-based surveys are often exaggerated with claims that Internet surveys are always faster, better, cheaper, and easier for researchers than conventional survey methods. This case study compares two side-by-side surveys of South Dakota resident spring turkey hunters (traditional mail survey vs. by e-mail with link to the Web-based survey: sample size of 1,200 each). With approximate equal effort to contact hunters the Internet survey received a 44% return rate compared to a 75% return rate from the mail survey. Until the issue of sample validity is solved and response rates increased an Internet survey by itself cannot be relied on to collect valid, scientific human dimensions information. However, a mixed-mode survey design that incorporates Internet surveys with traditional survey methods is discussed as a way to solve for sample validity and non-response methodological issues associated with Internet surveys while achieving some of the cost saving benefits of Internet surveys.

Impact of perceived innovation characteristics on adoption of pharmacy-based in-house immunization services

An in-house immunization service in which staff pharmacists administer vaccines was conceptualized as an innovation. Prior to making adoption decisions, community pharmacies evaluated characteristics of in-house immunization services. This study examined the impact of three specific characteristics (perceived benefit, perceived compatibility and perceived complexity) of in-house immunization services on community pharmacies' adoption decisions. A multi-stage mixed-mode survey design was used to collect data from key informants of community pharmacies in Washington State, USA. Key informants included pharmacy managers or pharmacists-on-duty who were able to answer questions related to immunization activities in their pharmacies. Perceived characteristics of in-house immunization services and pharmacy adoption decisions were measured in 2004 and in 2006-2007, respectively. Each perceived characteristic individually predicted adoption of in-house immunization services. When all three characteristics were included in logistic regression, perceived benefit was the only significant predictor of in-house immunization service adoption. Appropriate strategies, particularly promoting the benefit of in-house immunization services, should be implemented. The proposed model and findings may be applicable to other pharmacy-based innovative practices or other public health initiatives. We recommend that organizational leaders, researchers and practitioners consider the impact of perceived benefit and incorporate it when they design strategies to foster adoption of innovative practices. Doing this may increase the number of adopters and also increase diffusion rates for innovative services.

Surveying Political Activists: The Effectiveness of a Mixed Mode Survey Design

This article reports on the used of a mixed mode survey – mail and internet – to study 2004 New Mexico Democratic presidential caucus voters. Potential respondents were sent a letter asking them to answer on‐line or to request a mail survey. We tested for sample representativeness, possible differences between survey modes, and the effectiveness of reminder postcards on survey response. We find that the mixed mode survey does a relatively good job of representing the sample, but that web responders were younger, whiter, wealthier, and more educated. We find no difference by survey mode for answers to our substantive survey questions once we control for demographics. Postcards were very effective in increasing the response rate. In addition, we find that respondents had a difficult time understanding how to enter the survey web address on their web browser and suggest that information on browser use be included in the initial contact letter and follow‐up postcards.

Mixed-mode administration reduced bias and enhanced poststratification adjustments in a health behavior survey

Objective To assess whether a mixed-mode survey design reduced bias and enhanced methods commonly used to correct for bias (poststratification weighting). Study Design and Setting The data for this paper are from a study of 1,900 adult patients enrolled in a randomized controlled trial to promote repeat treatment for relapsed smokers at five Veteran's Affairs Medical Centers. A sequential mixed-mode design was used for data collection whereby the initial attempt was conducted using phone administration, with mail follow-up for nonresponders. Analyses examined demographic, health, and smoking cessation treatment seeking differences between telephone responders, mail responders, and nonresponders and compared the relative effectiveness of global vs. targeted poststratification weighting adjustments for correcting for response bias. Results The findings suggest (1) that responders to the additional survey mode (mail) did not significantly differ from responders to the first mode (phone) or nonresponders and (2) that poststratification weighting adjustments that take this additional information into account perform better than the standard global adjustments. Conclusions A mixed-mode design can improve survey representativeness and enhance the performance of poststratification weighting adjustments.

Comparing web and mail responses in a mixed mode survey in college alcohol use research

Objective This exploratory study examined potential mode effects (web versus U.S. mail) in a mixed mode design survey of alcohol use at eight U.S. colleges. Methods Randomly selected students from eight U.S. colleges were invited to participate in a self-administered survey on their alcohol use in the spring of 2002. Data were collected initially by web survey ( n = 2619) and non-responders to this mode were mailed a hardcopy survey ( n = 628). Results College students who were male, living on-campus and under 21 years of age were significantly more likely to complete the initial web survey. Multivariate analyses revealed few substantive differences between survey modality and alcohol use measures. Conclusions The findings from this study provide preliminary evidence that web and mail surveys produce comparable estimates of alcohol use in a non-randomized mixed mode design. The results suggest that mixed mode survey designs could be effective at reaching certain college sub-populations and improving overall response rate while maintaining valid measurement of alcohol use. Web surveys are gaining popularity in survey research and more work is needed to examine whether these results can extend to web surveys generally or are specific to mixed mode designs.

Telefonsurveys

Telephone surveys have gained considerable importance in recent years. The present article describes advantages and drawbacks of this mode of data collection. Particular attention is devoted to a number of aspects involved: costs, computer-assisted fieldwork, response rates, special demands on sampling, use of incentives, survey participation and the role of telephone surveys in flexible mixed-mode survey designs. A concluding section offers some practical recommendations on carrying out telephone surveys.

Mixed-mode survey of female veterinarians yields high response rate.

In a recent study of female veterinarians, a subgroup of health professionals growing rapidly in number, the authors employed a mixed-mode survey design in targeting the cohort of women graduating from all US veterinary colleges during the 11-year period 1970-80 (n = 2,997). The questionnaire elicited information on a variety of health and occupational factors and required 35 minutes on average to complete. In the first stage, a modified version of Dillman's Total Design Method for mailed, self-administered questionnaires was employed, yielding a response rate of 82.9%. In the second stage, a telephone interview of all mail non-respondents was attempted, yielding a response rate here of only 30.1%, but increasing the overall response rate among those contacted to 90.2%. Non-respondents differed little from mail (early) or telephone (late) respondents with respect to year of graduation and geographic region of veterinary college attendance. Gentle probing of telephone non-respondents suggested the personal nature of some questions and the amount of time required to answer all questions were the main reasons they chose not to participate. It therefore appears that conventional survey techniques may be successfully employed in health studies of health professionals, particularly if issues of great concern to the target population are addressed.

A Mean Squared Error Model for Dual Frame, Mixed Mode Survey Design

Abstract An error model for dual frame survey designs is developed. It includes components of error for sampling variance, interviewer variance, and bias in each frame. A cost model that attempts to capture the complexity of a full scale dual frame survey is presented. The error and cost models are applied to a large national survey, the National Crime Survey, and the effect that alternative levels of bias in both frames have on the optimal allocation of sample to the two frames is examined for two types of crime.

A Mean Squared Error Model for Dual Frame, Mixed Mode Survey Design

A Mean Squared Error Model for Dual Frame, Mixed Mode Survey Design