Mixed-mode Design Research Articles

Health in Germany – A new panel infrastructure for Public Health Research in Germany: study design

Abstract Background The Robert Koch Institute (RKI) is developing a new panel infrastructure, ‘Health in Germany,’ to enhance public health research through continuous, rapid data collection from questionnaire surveys, measurements, and laboratory analyses. Using online and offline collection methods (CAWI and PAPI), the project aims to have 30,000 registered panelists by Spring 2024. Methods The sample of the recruitment study is based on address data from Germany’s local registration offices, whereby a mixed-mode design is used for data collection. Individuals aged 16-69 receive an initial online-only invitation, followed by postal reminders that include a paper questionnaire. For those aged 70 and above, both online and paper options are available from the outset. Incentives include an unconditional €5 cash payment for all invitees and an additional €10 for successful registration into the panel infrastructure. Results The initial response rate of the recruitment study is 35%. Of the CAWI participants, approx. 90% expressed willingness to participate again, with approx. 80% completing the online registration. To date, over 42,000 panelists have been recruited. Further detailed results on response rates, non-response bias, and sample composition will be presented at the conference. Conclusions RKI’s ‘Health in Germany’ is one of the largest probability panels in public health research, effectively gathering and analyzing data. The recruitment success and strong participant engagement highlight the infrastructure’s efficiency and its significant contribution to epidemiological research in Germany. Key messages • Successful recruitment of over 42,000 panelists demonstrates robust engagement and effective implementation. • The push-to-web strategy (or web-push strategy) underscores the infrastructure’s adaptability and efficiency.

Recruiting people with selected citizenships for the health interview survey GEDA Fokus throughout Germany: evaluation of recruitment efforts and recommendations for future research

BackgroundGermany is the second most common country of immigration after the US. However, people with own or familial history of migration are not represented proportionately to the population within public health monitoring and reporting. To bridge this data gap and enable differentiated analyses on migration and health, we conducted the health interview survey GEDA Fokus among adults with Croatian, Italian, Polish, Syrian, or Turkish citizenship living throughout Germany. The aim of this paper is to evaluate the effects of recruitment efforts regarding participation and sample composition.MethodsData collection for this cross-sectional and multilingual survey took place between 11/2021 and 5/2022 utilizing a sequential mixed-mode design, including self-administered web- and paper-based questionnaires as well as face-to-face and telephone interviews. The gross sample (n = 33436; age range 18–79 years) was randomly drawn from the residents’ registers in 120 primary sampling units based on citizenship. Outcome rates according to the American Association for Public Opinion Research, the sample composition throughout the multistage recruitment process, utilization of survey modes, and questionnaire languages are presented.ResultsOverall, 6038 persons participated, which corresponded to a response rate of 18.4% (range: 13.8% for Turkish citizenship to 23.9% for Syrian citizenship). Home visits accounted for the largest single increase in response. During recruitment, more female, older, as well as participants with lower levels of education and income took part in the survey. People with physical health problems and less favourable health behaviour more often took part in the survey at a later stage, while participants with symptoms of depression or anxiety more often participated early. Utilization of survey modes and questionnaire languages differed by sociodemographic and migration-related characteristics, e.g. participants aged 50 years and above more often used paper- than web-based questionnaires and those with a shorter duration of residence more often used a translated questionnaire.ConclusionMultiple contact attempts, including home visits and different survey languages, as well as offering different modes of survey administration, increased response rates and most likely reduced non-response bias. In order to adequately represent and include the diversifying population in public health monitoring, national public health institutes should tailor survey designs to meet the needs of different population groups considered hard to survey to enable their survey participation.

Optimizing Response Rates to Examine Health IT Maturity and Nurse Practitioner Care Environments in US Nursing Homes: Mixed Mode Survey Recruitment Protocol.

Survey-driven research is a reliable method for large-scale data collection. Investigators incorporating mixed-mode survey designs report benefits for survey research including greater engagement, improved survey access, and higher response rate. Mix-mode survey designs combine 2 or more modes for data collection including web, phone, face-to-face, and mail. Types of mixed-mode survey designs include simultaneous (ie, concurrent), sequential, delayed concurrent, and adaptive. This paper describes a research protocol using mixed-mode survey designs to explore health IT (HIT) maturity and care environments reported by administrators and nurse practitioners (NPs), respectively, in US nursing homes (NHs). The aim of this study is to describe a research protocol using mixed-mode survey designs in research using 2 survey tools to explore HIT maturity and NP care environments in US NHs. We are conducting a national survey of 1400 NH administrators and NPs. Two data sets (ie, Care Compare and IQVIA) were used to identify eligible facilities at random. The protocol incorporates 2 surveys to explore how HIT maturity (survey 1 collected by administrators) impacts care environments where NPs work (survey 2 collected by NPs). Higher HIT maturity collected by administrators indicates greater IT capabilities, use, and integration in resident care, clinical support, and administrative activities. The NP care environment survey measures relationships, independent practice, resource availability, and visibility. The research team conducted 3 iterative focus groups, including 14 clinicians (NP and NH experts) and recruiters from 2 national survey teams experienced with these populations to achieve consensus on which mixed-mode designs to use. During focus groups we identified the pros and cons of using mixed-mode designs in these settings. We determined that 2 mixed-mode designs with regular follow-up calls (Delayed Concurrent Mode and Sequential Mode) is effective for recruiting NH administrators while a concurrent mixed-mode design is best to recruit NPs. Participant recruitment for the project began in June 2023. As of April 22, 2024, a total of 98 HIT maturity surveys and 81 NP surveys have been returned. Recruitment of NH administrators and NPs is anticipated through July 2025. About 71% of the HIT maturity surveys have been submitted using the electronic link and 23% were submitted after a QR code was sent to the administrator. Approximately 95% of the NP surveys were returned with electronic survey links. Pros of mixed-mode designs for NH research identified by the team were that delayed concurrent, concurrent, and sequential mixed-mode methods of delivering surveys to potential participants save on recruitment time compared to single mode delivery methods. One disadvantage of single-mode strategies is decreased versatility and adaptability to different organizational capabilities (eg, access to email and firewalls), which could reduce response rates. DERR1-10.2196/56170.

Development of Administrators’ Creative Leadership Structural Equation Model Under the Department of Local Administration Promotion

This study was planned to examine the factors and indicators of administrators’ creative leadership for the Department of Local Administration Promotion in Thailand. The researchers employed a mixed-mode design using checklist and questionnaire as research instruments. They conceptualized creative leadership factors and indicators by analyzing documents and past studies to develop a creative leadership structural equation model. The assessment model was then tested the goodness of fit of the identified factors and indicators for creative leadership with the empirical data. The findings indicated that a total of 13 indicators resulting from the four factors in a creative leadership model of administrators were found in line with the empirical data.

The effect of prepaid incentives on panelists’ response across survey modes in a sequential mixed-mode design

AbstractWhile the use of prepaid incentives and data collection in a sequential mixed-mode survey design is standard in cross-sectional and longitudinal studies, little is known about the interaction of these strategies and how this affects target persons’ survey participation. Therefore, based on a panel study, this study investigates whether such unconditional prepayments—monetary or non-monetary incentives—increase the efficiency of a sequential mixed-mode survey design in regard to boosted response rates and speed of return after receipt of an invitation to participate in a survey. While the survey design is always identical, the different incentives vary across the waves considered. Different prepayments, such as cash or in-kind incentives, have different effects on invitees’ participation in different survey modes. In particular, non-monetary incentives seem to work only in a sequential mixed-mode design, while cash, as a universal medium, always works in the expected way, across different survey modes. In sum, the study finds that when a single sequential mixed-mode design is used across panel waves the overall response rates are rather similar, even when different incentives are provided.

Community-Based View on Diagnostic Imaging at the End of COVID-19 Pandemic: Online Survey-Assisted Study.

(1) Background: An online survey-based observational cross-sectional study aimed at elucidating the experience and attitudes of an unstructured population regarding diagnostic imaging. (2) Methods: Invitations to participate were distributed using mixed-mode design to deidentified residents aged 18 years and older. Main outcome measures included morbidity structure and incidence of diagnostic imaging administrations. (3) Results: Respondents (n = 1069) aged 44.3 ± 14.4 years; 32.8% suffered from cardiovascular diseases (CVD); 9.5% had chronic respiratory pathology; 28.9% considered themselves healthy. Respondents with COVID-19 history (49.7%) reported higher rates of computed tomography (CT) (p < 0.0001), magnetic resonance imaging (MRI) (p < 0.001), and ultrasound (p < 0.05). COVID-19 history in CVD respondents shifted imaging administrations towards CT and MRI (p < 0.05). Every tenth respondent received MRI, CT, and ultrasound on a paid basis; 29.0% could not pay for diagnostic procedures; 13.1% reported unavailable MRI. Professional status significantly affected the pattern of diagnostic modalities (p < 0.05). MRI and CT availability differed between respondents in urban and rural areas (p < 0.0001). History of technogenic events predisposed responders to overestimate diagnostic value of fluorography (p < 0.05). (4) Conclusions: Preparedness to future pandemics requires the development of community-based outreach programs focusing on people's awareness regarding medical imaging safety and diagnostic value.

NATIONAL EXAMINATION SURVEY ON BLOOD PRESSURE OF OLDER AND VERY OLD PEOPLE IN GERMANY 2022-2023: WHY SELF-REPORTS, MEASUREMENTS AND MEDICATION DATA ARE NEEDED

Objective: The National Study on Health of Older People in Germany “Gesundheit 65+” aimed to provide up-to-date information on the health of older adults in Germany including blood pressure (BP) measurements at home by a study nurse. Design and method: Nationally representative examination survey of the population aged 65 years and older residing in Germany and with sufficient German language skills, using two-stage population registry sampling from 128 sample points from June 2022 to April 2023. Participation of physically or cognitively impaired persons was possible via proxy interviews and consent from legal representatives. Recruitment and data collection used a mixed-mode design. Three standardized blood pressure measurements at two-minute intervals were taken after 5 minutes rest with an oscillometric device (Mobil-O-Graph) using one of five cuffs according to upper arm circumference. The mean of the second and third measurement was used for analysis. Hypertension was defined as mean systolic BP (SBP) at or above140 mm Hg, or mean diastolic BP (DBP) at or above 90 mm Hg, or the use of antihypertensives (ATC-codes C02 antihypertensives, C03 diuretics, C07 beta-blockers, C08 calcium-channel blockers or C09 ACEI and ARB) in participants with self-reported hypertension diagnosis. Provisional weights account for unequal sampling and participation probabilities. Results: The “Gesundheit 65+” examination survey included 1,493 participants and 1,475 (98.8%) had successful BP measurements (763 men and 712 women aged 65 to 100 years). External quality control showed very good standardization of BP measurements in the home environment. Central BP data were available from 1,356 participants. Based on self-reported diagnoses alone the prevalence of hypertension would be 58.3% [95% CI 53.2-63.3] in men and 59.5% [53.8-64.9] in women. However, overall hypertension prevalence was estimated at 70.4% [65.8-74.7] in men and 72.3% [66.9-77.1] in women based on the study definition which combines self-reported physician diagnoses, standardized BP measurements and medication. Conclusions: Our study shows very good feasibility, acceptability and data completeness of standardized BP measurements using home visits. The prevalence of hypertension would be greatly underestimated in the absence of examination survey data with standardized measurements and detailed recording of medication.

Three Approaches to Improve Inferences Based on Survey Data Collected with Mixed-mode Designs

Abstract Mixed-mode designs have become increasingly common in survey data collection. Although different modes often have different measurement properties, the standard practice is to treat mixed-mode data as if they had been collected with a single mode, neglecting the potential impact of mode effects. To account for potential mode effects when making inferences for mixed-mode samples, we propose (i) a Testimator approach, (ii) a Bayesian approach, and (iii) a model averaging approach. In the Testimator approach, we test whether the means and the variances of mixed-mode samples are the same. If the means are the same, we take the average of mode-specific estimates. If the means are different, we take the average when we have no prior information about preferred modes and take the smaller (or larger) estimate when we have prior information about preferred modes (e.g., a smaller estimate is better). In the Bayesian approach, we assume some prior information. We use a data-driven method to determine whether there are mode effects. If there are no mode effects, we draw inferences using a common mean model. If there are mode effects, we draw inferences using the data collected with the mode that produces smaller estimates. In the model averaging approach, we combine estimates of different models (characterized by whether assume same means and variances across modes) using marginal posteriors as weights. We evaluate the approaches in simulation studies and find that they achieve robust inferences compared to the standard approach. We apply the methods to the Arab Barometer study, which employs a randomized mixed-mode design.

Sequential and Concurrent Mixed-Mode Designs: A Tailored Approach

Abstract Due to rising costs and declining response rates, surveys are increasingly moving from face-to-face interviewing to a self-administered mixed-mode design. Mixed-mode surveys can be conducted using a concurrent or a sequential design. A sequential design in which the web mode is offered first is a common strategy for mixed-mode surveys as it reduces survey costs. However, when deciding which mode choice sequence to use, sample balance should also be considered. One approach to achieving a balanced sample might be to tailor the sequence of the choice of modes, or the mode choice sequence. For this purpose, we use an indicator that assigns the sampled persons to the different mode choice sequences to minimize the variability of response probabilities. In this study, we compare the sample composition achieved with a concurrent and a sequential design. Additionally, we investigate whether indicator-based tailoring of the two mode choice sequences can improve sample composition. We implemented a randomized experiment in the 2021 German General Social Survey (ALLBUS), which surveyed the general population aged 18 and older in private households (N = 5,342) using a mixed-mode design (web and mail). In a first step, respondents were randomly assigned to a concurrent or a sequential design. We find that the two mode choice sequences lead to a similar sample composition. Next, we identify age as the best available single indicator of the variables known before the survey to tailor the mode choice sequence. Our analyses show that a tailored approach based on age improves the sample composition slightly.

Text Messages to Facilitate the Transition to Web-First Sequential Mixed-Mode Designs in Longitudinal Surveys

Abstract This article is concerned with the transition of a longitudinal survey from a single-mode design to a web-first mixed-mode design and the role that text messages to sample members can play in smoothing that transition. We present the results of an experiment that investigates the effects of augmenting the contact strategy of letters and emails with text messages, inviting the sample members to complete a web questionnaire and reminding them of the invite. The experiment was conducted in a subsample of Understanding Society, a household panel survey in the United Kingdom, in the wave that transitioned from a CAPI-only design to a sequential design combining web and CATI. In the experiment, a quarter of the sample received letters and emails, while the rest received between one and three text messages with a personalized link to the questionnaire. We examine the effect of the text messages on response rates, both at the web phase of a sequential design and at the end of the fieldwork after a CATI follow-up phase, and explore various mechanisms that might drive the increase in response rates. We also look at the effects on the device used to complete the survey and field efforts needed at the CATI stage. The findings indicate that text messages did not help to significantly increase response rates overall, although some subgroups benefited from them, such as panel members who had not provided an email or postal address before. Likewise, the text messages increased web completion among younger panel members and those with an irregular response pattern. We only found a slight and nonsignificant effect on smartphone use and no effect on the web household response rate, a proxy for fieldwork efforts.

Lessons Learned From a Sequential Mixed-Mode Survey Design to Recruit and Collect Data From Case-Control Study Participants: Formative Evaluation.

Sequential mixed-mode surveys using both web-based surveys and telephone interviews are increasingly being used in observational studies and have been shown to have many benefits; however, the application of this survey design has not been evaluated in the context of epidemiological case-control studies. In this paper, we discuss the challenges, benefits, and limitations of using a sequential mixed-mode survey design for a case-control study assessing risk factors during the COVID-19 pandemic. Colorado adults testing positive for SARS-CoV-2 were randomly selected and matched to those with a negative SARS-CoV-2 test result from March to April 2021. Participants were first contacted by SMS text message to complete a self-administered web-based survey asking about community exposures and behaviors. Those who did not respond were contacted for a telephone interview. We evaluated the representativeness of survey participants to sample populations and compared sociodemographic characteristics, participant responses, and time and resource requirements by survey mode using descriptive statistics and logistic regression models. Of enrolled case and control participants, most were interviewed by telephone (308/537, 57.4% and 342/648, 52.8%, respectively), with overall enrollment more than doubling after interviewers called nonresponders. Participants identifying as female or White non-Hispanic, residing in urban areas, and not working outside the home were more likely to complete the web-based survey. Telephone participants were more likely than web-based participants to be aged 18-39 years or 60 years and older and reside in areas with lower levels of education, more linguistic isolation, lower income, and more people of color. While there were statistically significant sociodemographic differences noted between web-based and telephone case and control participants and their respective sample pools, participants were more similar to sample pools when web-based and telephone responses were combined. Web-based participants were less likely to report close contact with an individual with COVID-19 (odds ratio [OR] 0.70, 95% CI 0.53-0.94) but more likely to report community exposures, including visiting a grocery store or retail shop (OR 1.55, 95% CI 1.13-2.12), restaurant or cafe or coffee shop (OR 1.52, 95% CI 1.20-1.92), attending a gathering (OR 1.69, 95% CI 1.34-2.15), or sport or sporting event (OR 1.05, 95% CI 1.05-1.88). The web-based survey required an average of 0.03 (SD 0) person-hours per enrolled participant and US $920 in resources, whereas the telephone interview required an average of 5.11 person-hours per enrolled participant and US $70,000 in interviewer wages. While we still encountered control recruitment challenges noted in other observational studies, the sequential mixed-mode design was an efficient method for recruiting a more representative group of participants for a case-control study with limited impact on data quality and should be considered during public health emergencies when timely and accurate exposure information is needed to inform control measures.

Ascertaining adequacy of fire suppression systems: the perspective of Accra central of Ghana

Purpose Fire is the fundamental element of most people’s lives, and when not controlled, the same fire can lead to several catastrophes in homes, offices, schools, lives and other public places with severe repercussions. Hence, this study aims to examine the adequacy and extent of the application of fire suppression systems in residential and commercial property in Ghana. Design/methodology/approach This study adopts a sequential mixed-mode design comprising quantitative and qualitative research strategies to analyse factors to produce findings. The target population for this study includes shop occupiers, end users of office buildings, and residents in the Accra Central of Ghana. Systematic random sampling was used for the quantitative research, and a sample size of 385 was obtained using a multi-stage and cluster sampling method. A structured survey and semi-structured interviews were used to collect the primary data. The quantitative data were analysed using descriptive and inferential statistics, whereas the qualitative data were analysed using content analysis. Findings From an empirical literature review and the analysis, the three main factors contributing to fire breakouts are equipment malfunction, improper use of heat sources and human mistakes. According to the respondents, fire suppression systems were also inadequate, as most of the suppression systems prescribed in the building code were unavailable. Regarding the ability to manually operate fire suppression systems, most property occupiers stated that they are generally unaware of these suppression systems. Practical implications This study will aid policymakers in developing interventions for fire safety enforcement by ensuring that fire safety regulations are consistently followed by design team members and property developers, resulting in a positive effect on public building structures performing their required functions. It is also critical to provide end users with education and training on how to operate the fire suppression system as well as effective handling of firefighting installations in the event of a fire. Originality/value The findings of this investigation contribute to knowledge and comprehension of the effect of fire suppression systems on building users and may serve as a precursor to the development of a “As Built” certification system for ascertaining the adequacy of fire suppression systems for new and existing residential and commercial property.

Text Messages to Incentivise Response in a Web-First Sequential Mixed-Mode Survey

This article reports research exploring the benefits of adding text messages to the contact strategy in the context of a sequential mixed-mode design where telephone interviewer administration follows a web phase. In a web-first mixed-mode survey, supplementing the contact strategy with text messages can help increase the response rate at the web phase and, consequently, reduce fieldwork efforts at the interviewer-administered phase. We present results from a survey experiment embedded in wave 11 of Understanding Society in which the usual contact strategy of emails and letters was supplemented with text messages. Effects of the text messages on survey response and fieldwork efforts were assessed. In addition, we also investigated the impact of SMS on the device selected to complete the survey, time to response, and sample balance. The results show a weak effect of the SMS reminders on response during the web fieldwork. However, this positive effect did not significantly reduce fieldwork effort.

The association between patient experiences in outpatient care and their characteristics in Slovenia

Abstract Background The aim of the cross-sectional study was to measure adult patients’ experiences in the outpatient specialist medical settings in the fields of neurology and neurosurgery, cardiology, ophthalmology, gynaecology and obstetrics, orthopaedics and oncology with radiotherapy, and explore the association between the general score of the healthcare service given by patients and their socio-demographic characteristics and health status. Methods The survey used mixed mode design (i.e. paper questionnaire and online survey), took into account various aspects of healthcare such as waiting period to the visit, admission, waiting time in the waiting room, physician work, nurse work, privacy, patient rights information, and familiarity with continuation of treatment. As part of the study, patients evaluated overall care at a specialist outpatient clinic on a scale of 0 to 10. The questionnaire used has been recently developed, based on a review of the literature and focus groups with patients. The questionnaire was also cognitively tested and piloted before national rollout. Multi-level regression analysis was used as the main method of statistical analysis. Results The national survey was conducted in autumn 2022. A total of 10,533 valid completed questionnaires from 210 specialist outpatient clinics were received. 82.4% of the surveyed patients rated the medical treatment with a score of 9 or 10 with the average rating 9.34. Preliminary statistical analysis indicates the presence of association between socio-demographic/health status variables on one hand and overall assessment of treatment on the other. Conclusions A possible association between socio-demographic and health status factors with overall score on specialist outpatient clinic rating calls for caution when comparing outpatient clinics that care for patients with different socio-demographic characteristics and health status. Key messages • There is an association between sociodemographic factors and health status of patients with overall score of healthcare experiences in specialist outpatient clinics. • The association calls for caution when comparing outpatient clinics.

Study on Health of Older People in Germany (Gesundheit 65+): objectives, design and implementation.

The longitudinal population-based study Gesundheit 65+ aimed to close data gaps on health and well-being of older adults in Germany in times of the COVID-19 pandemic. The target population comprised persons 65 years and older permanently residing in Germany and with sufficient German language skills. Proxy interviews were possible and consent from legal representatives was obtained as necessary in order to enable participation of physically or cognitively impaired persons. A two-stage sampling process, was used to draw 128 primary sample points (PSUs) and within these PSUs sex- and age-stratified random samples were drawn from population registries. A mixed-mode design was applied to contact the study population and for data collection. Data were collected between June 2021 and April 2023. Participants were surveyed a total of four times at intervals of four months. At month 12 participants were offered a home visit including a non-invasive examination. Data on all-cause mortality and information on neighborhood social and built environment as well as health insurance data will be linked to primarily collected data at the individual level. Results will inform health politicians and other stakeholders in the care system on health and health care needs of older people in Germany.

Dynamic Analysis of the Timing of Survey Participation: An Application of Event History Analysis of the Stochastic Process of Response in a Probability-Based Multi-Wave Panel With Computer-Assisted Interview Modes

The response patterns across the fieldwork period are analyzed in the context of a panel study with a sequential mixed-mode design including a self-administered online questionnaire and a computer-assisted telephone interview. Since the timing of participation is modelled as a stochastic process of individuals’ response behaviour, event history analysis is applied to reveal time-constant and time-varying factors that influence this process. Different distributions of panelists’ propensity for taking part in the web-based survey or, alternatively, in the computer-assisted telephone interview can be considered by hazard rate analysis. Piecewise constant rate models and analysis of sub-episodes demonstrate that it is possible to describe the time-related development of response rates by reference to individuals’ characteristics, resources and abilities, as well as panelists’ experience with previous panel waves. Finally, it is shown that exogenous factors, such as a mixed-mode survey design, the incentives offered to participants and the reminders that are sent out, contribute significantly to time-related response after the invitation to participate in a survey with a sequential mixed-mode design. Overall, this contribution calls for a dynamic analysis of response behaviour instead of the categorization of response groups.

Problem gambling in Germany: results of a mixed-mode population survey in 2021

ABSTRACT Since 2006, representative telephone surveys among the German general population have not detected any significant changes with respect to gambling-related problems. This is remarkable, as in the meantime not only the availability of gambling services and the turnover and gross gambling revenues of gambling providers have increased, but also the demand for treatment from people with gambling problems has grown. Not only in the gambling field have population surveys of this type recently been associated with certain methodological problems. These include a decreasing willingness to participate among and limited access to specific subgroups. The prevalence study at hand is based on a mixed-mode design which combines both a telephone and an online survey. The weighting of both samples in a ratio of 2 to 1 was checked against plausibility data. The sample consists of 12,303 complete interviews (telephone: 61%, online: 39%). The results show that 2.3% of the German population aged 18–70 years are identified as having a ‘gambling disorder’ according to DSM-5 (mild disorder: 1.1%, moderate disorder: 0.7%, and severe disorder: 0.5%). The prevalence detected in this study is significantly higher than that of previous surveys. Finally, we discuss the results primarily within the context of the new survey methodology.

Transitioning an employee panel survey from telephone to online and mixed-mode data collection

Employee panel surveys, which are essential for measuring ongoing labor market developments, are facing significant challenges of respondent recruitment and retention. Even interviewer-administered panel surveys, historically considered the gold standard form of data collection, are facing high costs and nonresponse issues that threaten their sustainability and inferential capabilities. Supplementing interviewer-administration with online data collection is a popular method of reducing costs and may improve contactability and reduce nonresponse in employee surveys. However, the effects of introducing online data collection in an ongoing panel survey of the employed population have received little attention. We address this research gap by analyzing a mode design experiment embedded in the fourth wave of a German employee panel survey. Individuals were randomly assigned to the standard telephone-only design, or a sequential web-telephone mixed-mode design. An invitation letter experiment was also conducted to test the effect of mentioning the telephone follow-ups in the web survey invitation. Introducing the mixed-mode design led to a higher response rate (59.9% vs. 50.1%), similar levels of nonresponse bias, and lower costs compared to the single-mode design. Mentioning the telephone follow-ups had no effect on participation in the web starting mode or the full mixed-mode design. Implications of these findings for survey practice are discussed.

Effects of mixing modes on nonresponse and measurement error in an economic panel survey

Numerous panel surveys around the world use multiple modes of data collection to recruit and interview respondents. Previous studies have shown that mixed-mode data collection can improve response rates, reduce nonresponse bias, and reduce survey costs. However, these advantages come at the expense of potential measurement differences between modes. A major challenge in survey research is disentangling measurement error biases from nonresponse biases in order to study how mixing modes affects the development of both error sources over time. In this article, we use linked administrative data to disentangle both nonresponse and measurement error biases in the long-running mixed-mode economic panel study “Labour Market and Social Security” (PASS). Through this study design we answer the question of whether mixing modes reduces nonresponse and measurement error biases compared to a single-mode design. In short, we find that mixing modes reduces nonresponse bias for most variables, particularly in later waves, with only small effects on measurement error bias. The total bias and mean-squared error are both reduced under the mixed-mode design compared to the counterfactual single-mode design, which is a reassuring finding for mixed-mode economic panel surveys.

The Impact of Mixing Survey Modes on Estimates of Change: A Quasi-Experimental Study

Abstract Longitudinal surveys are a key data collection tool used to estimate social change. Recent developments have accelerated the move from traditional single-mode longitudinal designs to mixed-mode designs. Nevertheless, there are concerns that mixing survey modes may affect coefficients of change at the individual level. We investigate the impact of mixing survey modes on estimates of change using a quasi-experimental design implemented in a long-running UK panel study. Two types of comparisons are carried out: single-mode (face-to-face) design versus sequential mixed-mode (Web–face-to-face) design, and Web versus face to face. Across 41 variables, we find no differences in estimates of individual-level change across modes (designs). However, correlations between intercepts and slopes, an estimate of convergence of respondents, were significantly different for most variables, which led to some biases in estimates of change. Applied researchers are encouraged to do sensitivity checks to ensure their results are robust to mode effects.