Mixed-mode Survey Design Research Articles

Optimizing Response Rates to Examine Health IT Maturity and Nurse Practitioner Care Environments in US Nursing Homes: Mixed Mode Survey Recruitment Protocol.

Survey-driven research is a reliable method for large-scale data collection. Investigators incorporating mixed-mode survey designs report benefits for survey research including greater engagement, improved survey access, and higher response rate. Mix-mode survey designs combine 2 or more modes for data collection including web, phone, face-to-face, and mail. Types of mixed-mode survey designs include simultaneous (ie, concurrent), sequential, delayed concurrent, and adaptive. This paper describes a research protocol using mixed-mode survey designs to explore health IT (HIT) maturity and care environments reported by administrators and nurse practitioners (NPs), respectively, in US nursing homes (NHs). The aim of this study is to describe a research protocol using mixed-mode survey designs in research using 2 survey tools to explore HIT maturity and NP care environments in US NHs. We are conducting a national survey of 1400 NH administrators and NPs. Two data sets (ie, Care Compare and IQVIA) were used to identify eligible facilities at random. The protocol incorporates 2 surveys to explore how HIT maturity (survey 1 collected by administrators) impacts care environments where NPs work (survey 2 collected by NPs). Higher HIT maturity collected by administrators indicates greater IT capabilities, use, and integration in resident care, clinical support, and administrative activities. The NP care environment survey measures relationships, independent practice, resource availability, and visibility. The research team conducted 3 iterative focus groups, including 14 clinicians (NP and NH experts) and recruiters from 2 national survey teams experienced with these populations to achieve consensus on which mixed-mode designs to use. During focus groups we identified the pros and cons of using mixed-mode designs in these settings. We determined that 2 mixed-mode designs with regular follow-up calls (Delayed Concurrent Mode and Sequential Mode) is effective for recruiting NH administrators while a concurrent mixed-mode design is best to recruit NPs. Participant recruitment for the project began in June 2023. As of April 22, 2024, a total of 98 HIT maturity surveys and 81 NP surveys have been returned. Recruitment of NH administrators and NPs is anticipated through July 2025. About 71% of the HIT maturity surveys have been submitted using the electronic link and 23% were submitted after a QR code was sent to the administrator. Approximately 95% of the NP surveys were returned with electronic survey links. Pros of mixed-mode designs for NH research identified by the team were that delayed concurrent, concurrent, and sequential mixed-mode methods of delivering surveys to potential participants save on recruitment time compared to single mode delivery methods. One disadvantage of single-mode strategies is decreased versatility and adaptability to different organizational capabilities (eg, access to email and firewalls), which could reduce response rates. DERR1-10.2196/56170.

The effect of prepaid incentives on panelists’ response across survey modes in a sequential mixed-mode design

AbstractWhile the use of prepaid incentives and data collection in a sequential mixed-mode survey design is standard in cross-sectional and longitudinal studies, little is known about the interaction of these strategies and how this affects target persons’ survey participation. Therefore, based on a panel study, this study investigates whether such unconditional prepayments—monetary or non-monetary incentives—increase the efficiency of a sequential mixed-mode survey design in regard to boosted response rates and speed of return after receipt of an invitation to participate in a survey. While the survey design is always identical, the different incentives vary across the waves considered. Different prepayments, such as cash or in-kind incentives, have different effects on invitees’ participation in different survey modes. In particular, non-monetary incentives seem to work only in a sequential mixed-mode design, while cash, as a universal medium, always works in the expected way, across different survey modes. In sum, the study finds that when a single sequential mixed-mode design is used across panel waves the overall response rates are rather similar, even when different incentives are provided.

Recent Innovations and Advances in Mixed-Mode Surveys

Abstract The use of mixed-mode surveys has grown rapidly in recent years, due to both technological advances and the COVID-19 pandemic. The increased mixing of modes (and the adoption of newer digital modes like web and text messaging) necessitates an evaluation of the impact of these newer designs on survey errors and costs, as well as new techniques for disaggregating and adjusting for nonresponse and measurement errors. This special issue highlights recent innovations, applications, and evaluations of mixed-mode survey designs and identifies areas where additional research is required.

When to Text? How the Timing of Text Message Contacts in Mixed-Mode Surveys Impacts Response

Abstract Using multiple modes of contact has been found to increase survey participation over a single contact mode. Text messaging has emerged as a new mode to contact survey participants in mixed-mode survey designs, especially for surveys that include web and/or phone data collection. However, it is unclear how to best combine text messages with mailings and other outreach contacts to improve response rates and data quality. To explore the effectiveness of using text messaging as a contact mode, we conducted a full factorial experiment that varies the sequencing of text messages with mailing contacts (early versus late reminder) and the time text messages were sent (morning versus afternoon). The experiment was implemented in a follow-up wave of a mixed-mode nationally representative longitudinal survey with two sample groups (Cooperative versus Other Respondents). For Cooperative Respondents, text reminders seemed to be effective at increasing completion rates, with the early text reminder being somewhat more effective than the late text reminder, at least early in the field period. For Other Respondents, text invitations were effective at improving the completion rate, but effects diminished quickly once the invitation letter was sent. Additionally, the early text reminder appears to be more effective than the late text reminder at increasing completion rates for Other Respondents. The sequencing of text messages did not affect data quality across sample groups or substantially impact nonresponse. The time of day the text messages were sent did not affect any of the outcome measures examined.

Lessons Learned From a Sequential Mixed-Mode Survey Design to Recruit and Collect Data From Case-Control Study Participants: Formative Evaluation.

Sequential mixed-mode surveys using both web-based surveys and telephone interviews are increasingly being used in observational studies and have been shown to have many benefits; however, the application of this survey design has not been evaluated in the context of epidemiological case-control studies. In this paper, we discuss the challenges, benefits, and limitations of using a sequential mixed-mode survey design for a case-control study assessing risk factors during the COVID-19 pandemic. Colorado adults testing positive for SARS-CoV-2 were randomly selected and matched to those with a negative SARS-CoV-2 test result from March to April 2021. Participants were first contacted by SMS text message to complete a self-administered web-based survey asking about community exposures and behaviors. Those who did not respond were contacted for a telephone interview. We evaluated the representativeness of survey participants to sample populations and compared sociodemographic characteristics, participant responses, and time and resource requirements by survey mode using descriptive statistics and logistic regression models. Of enrolled case and control participants, most were interviewed by telephone (308/537, 57.4% and 342/648, 52.8%, respectively), with overall enrollment more than doubling after interviewers called nonresponders. Participants identifying as female or White non-Hispanic, residing in urban areas, and not working outside the home were more likely to complete the web-based survey. Telephone participants were more likely than web-based participants to be aged 18-39 years or 60 years and older and reside in areas with lower levels of education, more linguistic isolation, lower income, and more people of color. While there were statistically significant sociodemographic differences noted between web-based and telephone case and control participants and their respective sample pools, participants were more similar to sample pools when web-based and telephone responses were combined. Web-based participants were less likely to report close contact with an individual with COVID-19 (odds ratio [OR] 0.70, 95% CI 0.53-0.94) but more likely to report community exposures, including visiting a grocery store or retail shop (OR 1.55, 95% CI 1.13-2.12), restaurant or cafe or coffee shop (OR 1.52, 95% CI 1.20-1.92), attending a gathering (OR 1.69, 95% CI 1.34-2.15), or sport or sporting event (OR 1.05, 95% CI 1.05-1.88). The web-based survey required an average of 0.03 (SD 0) person-hours per enrolled participant and US $920 in resources, whereas the telephone interview required an average of 5.11 person-hours per enrolled participant and US $70,000 in interviewer wages. While we still encountered control recruitment challenges noted in other observational studies, the sequential mixed-mode design was an efficient method for recruiting a more representative group of participants for a case-control study with limited impact on data quality and should be considered during public health emergencies when timely and accurate exposure information is needed to inform control measures.

Improving Gerontological Survey Participation With Mixed-Mode Response Options, Multiple Contacts, and Noncontingent Incentives.

Surveys are a prominent tool in gerontological research but are hindered by low response rates. This study examined methods to improve response rate among social workers, an understudied cohort in survey methodology literature. In this brief report, we analyzed factors that influence response rate using mixed-mode survey design (mail vs. online response) and randomized monetary ($10 gift card) and in-kind (continuing education coupon) noncontingent incentives with three reminder postcards among social workers (N = 307). Response rate was 34.1% with more surveys returned via mail than online. Participants who received a gift card responded at a higher rate (46.33%) than those who received continuing education coupons (28.0%). The results add to the literature of survey methodology to improve response rates among social workers and support noncontingent gift cards and mixed-mode survey return procedures as best practices for reducing bias and improving the quality of gerontological research.

Dynamic Analysis of the Timing of Survey Participation: An Application of Event History Analysis of the Stochastic Process of Response in a Probability-Based Multi-Wave Panel With Computer-Assisted Interview Modes

The response patterns across the fieldwork period are analyzed in the context of a panel study with a sequential mixed-mode design including a self-administered online questionnaire and a computer-assisted telephone interview. Since the timing of participation is modelled as a stochastic process of individuals’ response behaviour, event history analysis is applied to reveal time-constant and time-varying factors that influence this process. Different distributions of panelists’ propensity for taking part in the web-based survey or, alternatively, in the computer-assisted telephone interview can be considered by hazard rate analysis. Piecewise constant rate models and analysis of sub-episodes demonstrate that it is possible to describe the time-related development of response rates by reference to individuals’ characteristics, resources and abilities, as well as panelists’ experience with previous panel waves. Finally, it is shown that exogenous factors, such as a mixed-mode survey design, the incentives offered to participants and the reminders that are sent out, contribute significantly to time-related response after the invitation to participate in a survey with a sequential mixed-mode design. Overall, this contribution calls for a dynamic analysis of response behaviour instead of the categorization of response groups.

Understanding preferences for bicycling and bicycle infrastructure

Using survey data collected in New Jersey, we analyze the frequency of bicycling and respondent perceptions of the safety of various bicycling facilities. Data was collected via a mixed-mode survey design, including intercepts, bicycle hangers, flyers in bicycle shops, and a Facebook advertisement targeted toward bicyclists in New Jersey (N = 1937). This provided us with a reasonable sample of respondents that included bicycle commuters and non-cyclists. Data on cycling frequency was collected for recreational and commute trips. Respondents ranked the relative safety of images of bicycle facilities, ranging from cycling on-street to off-road trails. We also collected attitudinal data on risk perceptions and world views linked to political perspectives. Our analysis suggested that feeling safer with on-street bicycle lanes and off-street bicycle paths is not associated with the frequency of bicycling, while feeling safer on-street in traffic is associated with the frequency of bicycling. In analyzing correlates associated with our images of bicycling infrastructure, we found those with more liberal/egalitarian world views prefer on-street bicycle lanes and off-street bicycle paths, while those with traditional community world views tend to not feel safe with on-street bicycle lanes and bicycling in traffic. Those who are risk takers also feel safer bicycling in traffic. Most other demographic controls in our models give us the expected results. Policy implications suggest that bicycle infrastructure will be less controversial when world views are more liberal/egalitarian, but that making all streets safer might be a useful approach for increasing the frequency of bicycling.

Strategies to recruit people with selected nationalities for the interview survey GEDA Fokus

Abstract Background People with migration history (PMH) are underrepresented within health monitoring at Robert Koch Institute (RKI). To better describe the health status of PHM, the RKI is currently conducting the health interview survey GEDA Fokus with different migrant groups. Aim of this contribution is to present which sub-groups in this sequential mixed-mode survey design are reached by which mode. Methods People aged 18-70 years with Croatian (hr), Italian (it), Polish (pl), Syrian (sy) or Turkish (tr) nationality were drawn out of 99 residents’ registration offices all over Germany (N = 33,436). Study persons were invited sequentially to participate online (saq-web), paper-based(saq-paper) and in person (CAPI) or by telephone (CATI) in Arabic, Croatian, German, Italian, Polish or Turkish. Saq-web was available in German only or bilingual. Data collection took place from November 2021 to April 2022. Per nationality, 1,200 participants were recruited. Results As of 25th of April, 6,197 people took part, most often per saq-web (49%), which was less often used among participants aged ≥ 50 years (35% vs. 56%). Pl participants used saq-web most often in German only (55%), sy participants most often used the bilingual version (77%). Saq-paper was more often used by participants aged ≥ 50 years (41% vs. 22%) and less often by tr (21%) and sy (24%) participants. Participants with it nationality most often took part on their own initiative (saq) (92%), while tr (33%) and pl (28%) participants were recruited via personal contact (CAPI/ CATI) more often. Conclusions Preliminary results show that offering multiple modes of administration helps to reach different sub-groups. Personal contact contributes to reach those not directly taking part on their own initiative. The opportunity to utilize the bilingual version of the questionnaire was well accepted, especially among those with presumably shorter duration of residence in Germany. Key messages

Monitoring Mode Effects in A Survey of Consumer Buying Behaviour

In an effort to increase response rates and control survey costs, survey designers have come to increasingly consider mixed mode survey designs. However, the impact of differential administration by mode is less clear for ongoing data collection efforts that are primarily designed for a single mode of administration, where alternative modes have emerged over time upon respondent. This is the case for the Consumer Expenditures Quarterly Interview Survey, a personal visit household survey with a non-ignorable percentage of cases interviewed by telephone. This research draws on six months data to assess variation in mode of administration and evaluate the size of mode effects in key survey estimates. The results of the study suggest that mode itself matters less than does the respondent behavior typically associated with mode. While telephone interviewing may impact the quality of Consumer Expenditures data collected relative to that of personal visit interviewing, Keywords: Mode equivalence, Consumer Expenditure, mixed mode designs, Survey costs; Respondent Behaviour DOI: 10.7176/JMCR/86-02 Publication date: September 30 th 2022

A Model to Foster Records Use in Evidence-Based Decision-Making: Neither Restricted by Time nor Space

The study is derived from a doctoral dissertation by Momoti that triangulated the Continuum of Evidence Use and the Records Continuum Models to determine the extent to which records as sources of evidence were used for evidence-based decision-making by senior managers in Western Cape governmental bodies in South Africa. The investigation was descriptive, cross-sectional, and used a mixed-mode survey design. The mixed-mode survey addressed low response challenges encountered during data collection. Data was collected from a stratified random sample of 163 senior managers from 31 Western Cape governmental bodies. Descriptive statistical analysis was used to ensure that an accurate description of the findings was achieved. The study revealed that knowledge of evidence sources is crucial for their use. Senior managers acknowledged the importance of using records for evidence-based decision-making and always used them as sources of evidence to decide on service delivery programmes. The study revealed 83 such service delivery improvement programmes. The study proposed a model to foster the use of records as sources of evidence in decision-making.

Nurses’ and nurse managers’ perceived transformational leadership behavioural practices: a survey

PurposeThe purpose of this paper is to compare the perceived transformational leadership practices of charge and staff nurses. Transformational leadership is effective in promoting change in organisations, with the leader guiding followers towards a common vision.Design/methodology/approachA quantitative, descriptive, comparative survey design was used. All charge nurses (N = 151) and staff nurses (N = 1,950) in six health entities in Malta were included. A mixed mode survey design was used. Data were collected using the Leadership Practices Inventory and analysed using ANOVA and the Kruskal–Wallis test.FindingsAn overall response rate of 15% (n = 315) was achieved. Both staff and charge nurses perceived transformational leadership to be practiced. Charge nurses scored consistently higher than staff nurses. In long-term care environments, charge nurses are more likely to “model the way”, while in acute settings, they were more likely to “enable others to act”.Research limitations/implicationsTransformational leadership appears to be applied by charge nurses in Malta. The response rate achieved was low and may limit the generalisability of the results of the study.Practical implicationsNurse managers need to adapt their transformational leadership style based on the context in which they work.Originality/valueRegular feedback from nursing staff should be sought for charge nurses to be aware of the extent to which they are implementing transformational leadership.

Do Physicians Prefer to Complete Online or Mail Surveys? Findings From a National Longitudinal Survey.

Survey response rates for physicians are falling generally, and surveys of physicians tend to have lower response rates than those of the general population. To maximize response, respondents are often given a choice of modes in which to respond. The aim of this article is to describe mode response patterns and identify factors related to physicians' decisions to complete surveys online rather than by mail. The data are from the fifth annual wave of the Medicine in Australia: Balancing Employment and Life longitudinal survey of physicians, in which there was a 43.5% response rate (10,746/24,711) and 33.7% of respondents completed the survey online. Online completion was more likely when the physician had completed the survey online in the previous wave, was a general practitioner rather than other medical specialist or doctor-in-training, worked in a remote location, and was young and male. Free-text spontaneous comments from respondents indicated that mode choice was based on a combination of preference, previous experience, and feasibility. These results provide support for the use of mixed mode survey designs, which can accommodate doctors with different mode preferences and cast doubt over the possibility of tailoring mode based on respondent characteristics.

Mode Equivalence of Health Indicators Between Data Collection Modes and Mixed-Mode Survey Designs in Population-Based Health Interview Surveys for Children and Adolescents: Methodological Study.

BackgroundThe implementation of an Internet option in an existing public health interview survey using a mixed-mode design is attractive because of lower costs and faster data availability. Additionally, mixed-mode surveys can increase response rates and improve sample composition. However, mixed-mode designs can increase the risk of measurement error (mode effects).ObjectiveThis study aimed to determine whether the prevalence rates or mean values of self- and parent-reported health indicators for children and adolescents aged 0-17 years differ between self-administered paper-based questionnaires (SAQ-paper) and self-administered Web-based questionnaires (SAQ-Web), as well as between a single-mode control group and different mixed-mode groups.MethodsData were collected for a methodological pilot of the third wave of the "German Health Interview and Examination Survey for Children and Adolescents". Questionnaires were completed by parents or adolescents. A population-based sample of 11,140 children and adolescents aged 0-17 years was randomly allocated to 4 survey designs—a single-mode control group with paper-and-pencil questionnaires only (n=970 parents, n=343 adolescents)—and 3 mixed-mode designs, all of which offered Web-based questionnaire options. In the concurrent mixed-mode design, both questionnaires were offered at the same time (n=946 parents, n=290 adolescents); in the sequential mixed-mode design, the SAQ-Web was sent first, followed by the paper questionnaire along with a reminder (n=854 parents, n=269 adolescents); and in the preselect mixed-mode design, both options were offered and the respondents were asked to request the desired type of questionnaire (n=698 parents, n=292 adolescents). In total, 3468 questionnaires of parents of children aged 0-17 years (SAQ-Web: n=708; SAQ-paper: n=2760) and 1194 questionnaires of adolescents aged 11-17 years (SAQ-Web: n=299; SAQ-paper: n=895) were analyzed. Sociodemographic characteristics and a broad range of health indicators for children and adolescents were compared by survey design and data collection mode by calculating predictive margins from regression models.ResultsThere were no statistically significant differences in sociodemographic characteristics or health indicators between the single-mode control group and any of the mixed-mode survey designs. Differences in sociodemographic characteristics between SAQ-Web and SAQ-paper were found. Web respondents were more likely to be male, have higher levels of education, and higher household income compared with paper respondents. After adjusting for sociodemographic characteristics, only one of the 38 analyzed health indicators showed different prevalence rates between the data collection modes, with a higher prevalence rate for lifetime alcohol consumption among the online-responding adolescents (P<.001).ConclusionsThese results suggest that mode bias is limited in health interview surveys for children and adolescents using a mixed-mode design with Web-based and paper questionnaires.

Testing the Impact of Mixed-Mode Designs (Mail and Web) and Multiple Contact Attempts within Mode (Mail or Web) on Clinician Survey Response.

To compare response rate and nonresponse bias across two mixed-mode survey designs and two single-mode designs. This experiment was embedded in a clinician survey of knowledge and attitudes regarding HPV vaccination (n=275). Clinicians were randomly assigned one of two mixed-mode (mail/web or web/mail) or single-mode designs (mail-only/web-only). Differences in response rate and nonresponse bias were assessed. Using a multiple-contact protocol increased response, and sending a web survey first provided the more rapid response. Overall, the mixed-mode survey designs generated final response rates approximately 10 percentage points higher than their single-mode counterparts, although only the final response differences between the mail-only and web/mail conditions attained statistical significance (32.1 percent vs. 48 percent, respectively; p=.005). Observed differences did not result in nonresponse bias. Results support mixing modes of survey administration and web-based data collection in a multiple contact survey data collection protocol.

Mixed mode survey design and panel repetition – findings from the German Mobility Panel

Abstract The German Mobility Panel is a multi-day and multi-period panel survey, which has been collecting data on travel behavior in Germany since 1994. Declining survey participation rates in the last decade in various socio-demographic groups resulted in the implementation of a mixed-mode design for the German Mobility Panel, both in the sampling stage (landline and mobile phone recruitment) and in the data collection stage (paper and web). This study deals with two questions: does the new mixed-mode design affect a person’s willingness to continue participating in the consecutive survey waves? Is there a causal relation between data quality and completeness of survey repeaters and survey mode in the data collection stage? Our results show that survey mode and drop out propensity are not linked but web participants are more likely to switch their survey mode in the consecutive year. Overall, our study makes a case of how adaptations to the survey modes in a longitudinal panel survey on travel behavior affect the survey outcomes.

Analyses of mixed survey mode effects in the German national travel survey 2017

Abstract The paper analyzes the survey mode effects observed in the recent German national household travel survey “MiD 2017 – Mobility in Germany” with focus on the observed mobility behavior and survey mode preferences. In order to disentangle survey mode selection and survey mode measurement effects, multivariate regression models where used to control for socio-demographic factors. The results strengthen the assumption that hard facts like daily travel show no substantial mode measurement effects and mixed mode surveys improve the coverage of the target population. But beside the envisaged mode selection effects that result in a better coverage of the target population we also identified indications for survey mode measurement effects concerning the number of reported trips. The authors intend to raise awareness of the advantages of elaborated mixed mode survey designs and to contribute to a sound discussion of mode measurement effects in transport surveys.

The Rental Housing Needs and Experiences of Veterans With Disabilities

This article presents an overview of the rental housing search and occupancy experiences of Veterans with disabilities. Utilizing a mixed-mode survey design, 83 usable responses were obtained. Measures of discrimination, accessibility, and demographics were used as independent variables to explore relationships with housing satisfaction using one-way ANOVA and multiple regression. In addition, chi-square tests were conducted to determine the association between perceived discrimination and independent variables. Findings suggest that Veterans with disabilities are facing challenges in locating and occupying suitable rental housing with factors such as discrimination impacting housing satisfaction. Implications for policy-makers, Veterans, and researchers are discussed.

Same, Same but Different

The audience plays an important role in the journalistic context. The audience implications of journalistic work and output, business matters and policy making requires knowledge of public preferences and whether and how they take part in the news. Mixed-mode survey designs including Web and mail are becoming increasingly common in audience research, but it is unclear how mixing the survey modes affects the results. With the possibility of increasing response rates, is the risk of degrading the quality of answers. As audience research constitutes an important cornerstone of journalism studies, and because survey design including the Web is becoming more and more attractive, it is relevant to study the implications of merging audience data from different collection modes. This large-N survey provides a unique opportunity to analyse representative data on the matter. Altogether, 6603 respondents completed the survey, of whom 7.5 per cent chose the Web mode. Both habitual questions and opinion questions on different media were included in the study. The largest differences were found for item non-responses, whereas response distribution among respondents only differed to a small extent due to mode. There is no urgent need for caution when combining mail and Web responses in one data-set. However, thorough analyses of item non-response are necessary to control measurement errors when surveying media audiences and habits of using digital and other media in mixed-mode designed surveys.

Mode Effects on Variances, Covariances, Standard Deviations, and Correlations

The European Social Survey (ESS) program currently includes an investigation of mixed-mode survey designs, in which different respondents complete the survey by different survey modes. Such mixed-mode designs may lower systematic and random selection error if selection effects occur between the modes. Nonetheless, the advantage of selection effects may be counteracted by measurement effects between the modes. In the existing literature, selection and measurement effects in mixed-mode designs have already been extensively researched for mean and proportion parameters. Surprisingly, both effects have hardly been researched for covariance parameters, even though covariances are sufficient statistics for widely used parameters such as correlations, regression coefficients, or and factor loadings. This paper analyzes measurement and selection effects on variances, covariances, standard deviations, and correlations for different items from MTMM experiments within the Estonian European Social Survey Round 6 data by using the instrumental variable method. This analysis yields small to moderate selection and measurement effects, which means that mixed-mode surveys may perform slightly better than single-mode surveys, but also that analysis of mixed-mode data may require small error correction for measurement effects.