Multi-mode Survey Research Articles

Access to transition-related health care among transmasculine people in India: A mixed-methods investigation.

Little research has examined the health care experiences of transmasculine people in India, where government initiatives to improve access to transition-related (also called gender-affirmative) care have recently been announced. We draw on data from 'Our Health Matters', a mixed-methods community-based participatory research project, to characterize the transition-related care experiences of transmasculine people in India. Peer researchers conducted 40 virtual qualitative interviews in Hindi or Marathi from July to September 2021. Between November 2022 and January 2023, 377 transmasculine people participated in a multi-mode survey available in five languages. Qualitative data were analysed with a combination of framework analysis and grounded theory techniques. Data were mixed using a convergent parallel approach. Transmasculine persons' care journeys began with information-seeking, relying on peers and internet searches. In choosing between the public and private healthcare systems, they weighed issues of quality and affordability: the public system was perceived as lower-quality and difficult to access but most could not afford private care, leading to delays in care. Indeed, unmet need was common; 36.4% of survey participants were planning but had not begun to receive transition-related care and 80.2% wanted at least one transition-related surgery. Although some participants encountered stigma and refusal of care when seeking hormones, survey participants reported largely positive experiences with their hormone prescribers, which may reflect the influence of peer referrals. Participants underwent psychological assessments prior to transition-related care, which some experienced as disempowering and a barrier to disclosing mental health challenges. Finally, participants who were able to access care reported improved well-being, although surgical dissatisfaction was not uncommon (26.2%). Trans-inclusive medical training and continuing education are critical to enhancing access to high-quality transition-related care. Transmasculine people generally relied on peers and grassroots organizations for information, system navigation, and financial assistance. Strengthening these existing community resources may improve access to care.

Transitioning an In-Person Survey of Older Adults to Multi-Mode Data Collection.

The first three rounds of the National Social Life, Health, and Aging Project (NSHAP) were in-person. Preparing for Round Four (R4), NSHAP began developing ways to collect complex questionnaire and biomeasure data remotely. R4 was scheduled to begin in 2020, but due to the coronavirus pandemic, NSHAP delayed R4 data collection and instead conducted a study on respondents' experiences during the pandemic, as well as pretests to strengthen NSHAP's remote data collection capability. This paper describes the methodology, results, and lessons learned from these efforts which were undertaken as a bridge between NSHAP's all in-person past and multimode future. The Covid-19 Study was a multimode survey of NSHAP respondents to assess the impact of the pandemic. The multimode approach allowed evaluation of the feasibility of using different modes of data collection with older adults. NSHAP adapted its in-person questionnaire for phone and web administration and conducted pretests of the full phone questionnaire and sections of the web questionnaire. The project developed and tested a "BioBox," a kit containing all the supplies and instructions for respondents to self-collect biomeasures remotely. The BioBox was tested through an in-lab and in-home pilot, followed by two larger-scale pretests. The Covid-19 Study and pretests achieved NSHAP respondent participation in remote questionnaire and biomeasure collection, despite being accustomed to fully in-person data collection. Our findings and experiences will inform the collection of NSHAP data in future rounds and could inform other panel studies of older adults considering multimode data collection.

A Second Chance at Response: A Multimode Reinterview Analysis Using Data from the National Survey on Drug Use and Health

Traditionally administered in-person, restrictions imposed by the COVID-19 pandemic prompted the National Survey on Drug Use and Health to transition to a multimode survey that offers a self-administered web option prior to in-person data collection. A relatively small portion of respondents who began the survey on the web but broke off prior to completing were later readministered the survey (from the beginning) in-person. We refer to these as “second chance” respondents. The aim of this in-brief note is to report the individual-level mode discrepancies these second chance respondents exhibited with respect to lifetime use of cigarettes, alcohol, and marijuana. In addition to higher-than-anticipated discrepancy rates overall, we found directional differences that run contrary to what would be expected from the literature on survey mode effects. Second chance respondents are more than 15 percentage points more likely to report ever smoking cigarettes or using marijuana than web-only respondents.

Survey Protocols, Response Rates, and Representation of Underserved Patients

Surveys often underrepresent certain patients, such as underserved patients. Methods that improve their response rates (RRs) would help patient surveys better represent their experiences and assess equity and equity-targeted quality improvement efforts. To estimate the effect of adding an initial web mode to existing Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey protocols and extending the fielding period on RR and representativeness of underserved patient groups. This randomized clinical trial included 36 001 patients discharged from 46 US hospitals from May through December 2021. Data analysis was performed from May 2022 to September 2023. Patients were randomized to 1 of 6 survey protocols: 3 standard HCAHPS protocols (mail only, phone only, mail-phone) plus 3 web-enhanced protocols (web-mail, web-phone, web-mail-phone). RR and number of respondents per 100 survey attempts (yield) were calculated and compared for each of the 6 survey protocols, overall, and by patient age, service line, sex, and race and ethnicity. A total of 34 335 patients (median age range, 55-59 years; 59.3% female individuals and 40.7% male individuals) were eligible and included in the study. Of the respondents, 6.9% were Asian American or Native Hawaiian or Other Pacific Islander, 0.7% were American Indian or Alaska Native, 11.5% were Black, 17.4% were Hispanic, 61.0% were White, and 2.6% were multiracial. Of the 6 protocols, RRs were highest in web-mail-phone (36.5%), intermediate for the 3 two-mode survey protocols (mail-phone, web-mail, web-phone, 30.3%-31.1%), and lowest for the 2 single-mode protocols (mail only, phone only, 22.1%-24.3%). Web-mail-phone resulted in the highest yield for 3 racial and ethnic groups (Black, Hispanic, and White patients) and second highest for another (multiracial patients). Otherwise, the highest or second highest yield was almost always a 2-mode protocol. Mail only was the lowest-yield protocol for Black, Hispanic, and multiracial patients and phone-only was the lowest-yield protocol for White patients; these 2 protocols tied for lowest-yield for Asian American or Native Hawaiian or Other Pacific Islander patients. Gains from multimode approaches were often 2 to 3 times as large for Asian American or Native Hawaiian or Other Pacific Islander, Black, Hispanic, and multiracial patients as for White patients. Web-mail-phone had the highest RR for 6 of 8 age groups and 4 of 5 combinations of service line and sex. In this randomized clinical trial, web-first multimode survey protocols significantly improved the RR and representativeness of patient surveys. The best-performing protocol based on RR and representativeness was web-mail-phone. Web-phone performed well for young and diverse patient populations, and web-mail for older and less diverse patient populations. The US Centers for Medicare & Medicaid Services will allow hospitals to use the web-mail, web-phone, and web-mail-phone protocols for HCAHPS administration beginning in 2025.

Conducting Small-Scale Multimethod Questionnaire Evaluation

When pretesting survey questionnaires, there are benefits to using a multimethod approach. However, using multiple methods can be cost prohibitive. In 2021, the National Agricultural Statistics Service (NASS) received feedback from stakeholders regarding concerns about double counting of grain stocks in the Agricultural Survey and the Grain Stocks Report. Data from these two surveys are used to produce the Grain Stocks publication, which is a principal Federal Economic Indicator publication. To address stakeholder feedback, NASS evaluated these two multimode surveys, in a short timeframe, with limited resources. Despite these limitations, NASS utilized expert review, cognitive testing, usability testing and behavior coding to evaluate these two survey questionnaires. This paper demonstrates that multimethod pretesting can be done effectively on a low-cost, small-scale basis.

Optimizing Data Collection Interventions to Balance Cost and Quality in a Sequential Multimode Survey

AbstractHigh-quality survey data collection is getting more expensive to conduct because of decreasing response rates and rising data collection costs. Responsive and adaptive designs have emerged as a framework for targeting and reallocating resources during the data collection period to improve survey data collection efficiency. Here, we report on the implementation and evaluation of a responsive design experiment in the National Survey of College Graduates that optimizes the cost-quality tradeoff by minimizing a function of data collection costs and the root mean squared error of a key survey measure, self-reported salary. We used a Bayesian framework to incorporate prior information and generate predictions of estimated response propensity, self-reported salary, and data collection costs for use in our optimization rule. At three points during the data collection process, we implement the optimization rule and identify cases for which reduced effort would have minimal effect on the mean squared error (RMSE) of mean self-reported salary while allowing us to reduce data collection costs. We find that this optimization process allowed us to reduce data collection costs by nearly 10 percent, without a statistically or practically significant increase in the RMSE of mean salary or a decrease in the unweighted response rate. This experiment demonstrates the potential for these types of designs to more effectively target data collection resources to reach survey quality goals.

Intersecting Inequalities in Access to Justice for Trans and Non-binary Sex Workers in Canada

IntroductionIn the context of renewed debate about Canada’s “end demand” sex work laws, we took an intersectional approach to characterize experiences with the criminal legal system and perceived access to justice among transgender (trans) and nonbinary sex workers in Canada.MethodsThe Trans PULSE Canada community-based study collected multimode survey data in fall 2019. Data were cross-stratified by lifetime sex work and sex assigned at birth, ethnoracial group, or household income. Age adjustment did not meaningfully impact results.ResultsOf 2012 included participants (median age = 30, 66.5% assigned female at birth, 48.9% non-binary), 280 (16.1%) had ever done sex work. While access to justice was limited for trans and non-binary people overall, sex workers were more likely to anticipate (72.1% vs. 50.5%) and experience (43.2% vs. 15.7%) police mistreatment. In the previous 5 years, sex workers were more likely to experience violence for any reason (61.4% vs. 27.4% of other participants) or due to being trans or non-binary (41.4% vs. 14.0%), and to have avoided calling 911 for police (51.4% vs. 18.1%). Few sex workers trusted that they would be treated fairly by police if they experienced physical (10.8% vs. 34.9%) or sexual (4.7% vs. 20.6%) violence. Intersectional inequalities included that sex workers assigned male at birth and street-based workers were most likely to have experienced gender-based violence, and that Indigenous and racialized sex workers reported higher levels of police mistreatment and 911 avoidance.ConclusionsOverall, trans and non-binary people in Canada reported high levels of violence and limited access to justice. Sex workers faced large inequities in these outcomes, which were exacerbated for transfeminine, Indigenous, racialized, and street-based workers.Policy ImplicationsThese findings challenge the notion that Canadian sex work laws protect sex workers and highlight the limitations of formal legal protections for trans people. These inequities must be addressed in sex work legal reform efforts.

Avoidance of primary healthcare among transgender and non-binary people in Canada during the COVID-19 pandemic

Transgender (trans) and non-binary people experience barriers to culturally competent healthcare and many have reported avoiding care. COVID-19 and related mitigation strategies may have exacerbated avoidance, and poor mental health may be bidirectionally related to avoiding care. This study estimated the prevalence of primary care avoidance during the pandemic in a national sample of trans and non-binary people in Canada with a primary care provider and examined the association between poorer self-rated mental health and avoidance. In Fall 2019, Trans PULSE Canada collected multi-mode survey data from trans and non-binary people. In September to October 2020, 820 participants completed a COVID-19-focused survey. In this cross-sectional analysis, multivariable logistic regression models estimated odds ratios adjusted for confounders and weighted to the 2019 sample. The analysis included 689 individuals with a primary healthcare provider, of whom 61.2% (95% CI: 57.2, 65.2) reported fair or poor mental health and 25.7% (95% CI: 22.3, 29.2) reported care avoidance during the pandemic. The most common reason for avoidance was having a non-urgent health concern (72.7%, 95% CI: 65.9, 79.5). In adjusted analyses, those with fair or poor mental health had higher odds of avoiding primary care as compared to those with good to excellent mental health (adjusted odds ratio [AOR] = 2.37; 95% CI: 1.50, 3.77). This relationship was similar when excluding COVID-related reasons for avoidance (AOR = 2.52; 95% CI: 1.52, 4.17). Expansion of virtual communication may enhance primary care accessibility, and proactively assessing mental health symptoms may facilitate connections to gender-affirming mental health services.

An Adaptive Mode Adjustment for Multimode Household Surveys

Abstract Multimode data collection has emerged as a common approach for conducting household surveys in the United States. A number of different data collection schemes have been investigated, with an emphasis on collecting as many respondents by the Web prior to going to paper data collection to reduce costs. Despite this, little research has been conducted on the approaches to weighting data from multimode surveys. The typical approach assumes that all respondents should be treated the same regardless of mode even though it is well known that the response patterns by mode vary substantially. We examine an adaptive mode adjustment to address these differences and propose an imbalance measure to help determine the adjustment factor using ideas from responsive design. We then compare the effects of the alternative weighting method in two recent sequential mixed-mode surveys and show it appears to reduce bias while only slightly increasing variances of the estimates.

Immigration Policy Vulnerability Linked to Adverse Mental Health Among Latino Day Laborers

Latino day laborers in the United States are socially and economically vulnerable due to exclusionary immigration policies. Using data from a multi-mode survey, we examine the relationship between immigration policy legal vulnerability and mental health outcomes among 138 Latino, male day laborers (mean age = 45.65, SD = 12.05). Multivariable linear and logistic regression models separately estimated the effect of legal vulnerability, as measured by the Perceived Immigration Policy Effects Scale, on anxiety and depression symptoms and a positive depression and anxiety screening, respectively. Approximately 26.1% and 27.9% of day laborers reported depression and anxiety symptoms, respectively. In each adjusted model, we find a positive relationship between legal vulnerability and adverse mental health. Immigration policy legal vulnerability is associated with more depression and anxiety symptoms among Latino day laborers. Policies to reduce legal vulnerability, such as pathways to citizenship and employment authorization, may support Latino day laborers' mental health outcomes.

Does Survey Nonresponse Bias Estimates of Religious Service Attendance? Evidence from an Address-Based Sample from the Boston Area

Abstract This study investigates what role, if any, nonresponse plays in inflating survey estimates of religious behavior, using a multimode survey designed to allow estimation of nonresponse bias. A sample of 3,000 Boston-area households drawn from an address-based frame was randomly divided into two subsamples, contacted by mail, and invited to participate in a survey. The first subsample was asked to complete an interactive voice response interview. The second subsample was asked to complete a survey by telephone if a number was available for the address or by personal interview if not. Finally, random samples of nonrespondents were recontacted for a personal interview. Comparison of attendance estimates from initial interviews with nonrespondent interviews within sample segments yields minor or minimal differences that are not statistically significant. Findings suggest that the mechanism generating survey nonresponse is unlikely to be a major cause of bias in religious service attendance estimates in this study.

Workshop Synthesis: Making the transition to new methods for travel survey sampling and data retrieval

Abstract Adopting new methods in large longstanding surveys, such as National Travel Surveys, is associated with specific challenges. Aside from methodological issues, the continuation of time series and data that is usable for many, are important concerns. Multi-mode surveys are becoming common place and new methods are run in parallel to established methods. This improves survey representativeness but there are new challenges, for example mode effects. The introduction and spread of smartphone use in surveys and emerging methods for recruiting survey participants will add to this complexity. As survey methods evolve, data post-processing and statistical methods have to keep up as well.

Estimating Survey Questionnaire Profiles for Measurement Error Risk

Surveys differ in their topics, language, style, and design, and consequently, in their sensitivity to measurement error. Survey literature presents a range of characteristics of survey items that are assumed to be related to the magnitude and frequency of measurement error. In terms of questionnaire design and testing, it would be very useful to have a questionnaire profile that is a summary of the characteristics of the items contained in a questionnaire. This holds especially true in the context of multi-mode surveys where the detection of measurement error is crucial. The questionnaire profiles may be derived from scores that coders assign to the items in a questionnaire. Given that agreement among coders may be relatively low, as we observe, the number of coders must be large to ensure sufficient precision of the profiles. For multiple surveys, the coding workload may then become infeasible. In this paper, we propose methodology for the estimation of questionnaire profiles when a pool of coders is randomly allocated to a series of surveys. The methodology is based on multiple imputation and applied to 11 general purpose surveys in the Netherlands.

Testing the Representativeness of a Multimode Survey in South Korea: Results from KAMOS

The Korean Academic Multimode Open Survey (KAMOS) is a national survey first conducted in 2016. Stratified cluster random sampling was used in an initial face-to-face survey during which panel members were recruited. The second survey allowed invited panel members to answer online or by phone. KAMOS includes both longitudinal items and omnibus items, i.e., researchers can propose questions to include on KAMOS. This paper seeks to establish that KAMOS is representative of the South Korean adult population. The demographic variables from the first survey were comparable to demographic variables from two well-respected surveys in South Korea: the KOSTAT Social Survey and the Gallup Korea Omnibus Survey. To ensure that there was no substantial difference between those who answered the first survey and those who answered the second survey, we compared the results of 22 items from the first survey. The 2,000 panel members who were invited to participate in the second survey had similar responses to the 1,008 of those who responded to the second survey. Based on our findings, KAMOS can be considered a representative sample.

Social Resources and Community Resilience in the Wake of Superstorm Sandy.

Recovery efforts after natural disasters typically focus on physical infrastructure. In general less attention is paid to the social infrastructure that might impact the capacity of the community to rebuild. We examine perceptions of preparedness and recovery (markers of resilience at the community level) in the wake of Superstorm Sandy with a novel data set that includes a multi-mode survey of twelve neighborhoods severely affected by the storm. With these data, we suggest that social resources are associated with beliefs about neighborhood resilience. People who live in communities with higher social cohesion (coefficient = .73, p <.001), informal social control (coefficient = .53, p <.001), and social exchange (coefficient = .69, p <.001) are more likely to believe their neighborhoods are well prepared for a disaster. Likewise, people living in communities with higher social cohesion (coefficient = .35, p <.01), informal social control (coefficient = .27, p <.05), and social exchange (coefficient = .42, p <.001) are more likely to be confident their neighborhoods will recover quickly from a disaster. However, the effects of social resources on beliefs about resilience vary based on neighborhood socioeconomic status (SES) and the impact of the storm. Informal social control and social exchange lead to a greater increase in confidence in recovery in low, as compared to high, SES neighborhoods. Social resources tend to have more impact on perceptions of recovery in communities less affected by the storm. In sum, these findings suggest the potential value of various forms of social intervention to better equip communities to respond when disaster strikes.

State of transition: Marijuana use among young adults in the San Francisco Bay Area

BackgroundCalifornia may vote on marijuana legalization in 2016. Young adults have the highest rates of marijuana use, but little is known about the correlates of use in this age group, including factors that may be affected by policy change. We investigated whether there are differences in marijuana use by sociodemographic characteristics, psychological distress, loneliness and social support, controlling for risk factors such as alcohol and cigarette use as well as perceived harm of marijuana. MethodsBivariate and multivariable analysis of past 30day marijuana use using the 2014 San Francisco Bay Area Young Adult Health Survey, a probabilistic multi-mode survey of (N=1324) young adults (aged 18–26years) residing in Alameda and San Francisco Counties, stratified by race/ethnicity. Results291 (27%) sample participants reported current marijuana use. Compared to non-Hispanic Whites (referent) Asian/Pacific Islander respondents were less likely to use marijuana (AOR, 0.42; 95% CI, 0.22–0.80) while multiracial participants were twice as likely (AOR, 2.27; 95% CI, 1.06–4.85). Psychological distress was not related to marijuana use, but social support (AOR, 1.42; 95% CI, 1.08–1.88) and loneliness (AOR, 1.42; 95% CI, 1.09–1.86) were. Perceived harm of marijuana was inversely related to marijuana use (AOR, 0.60; 95% CI, 0.51–0.70), while smoking cigarettes (AOR, 3.95; 95% CI, 2.28–6.84) and binge drinking (AOR, 1.13; 95% CI, 1.03–1.24) were positively related. ConclusionsLegalization policies should include public education campaigns addressing potential harms of marijuana use particularly targeting multiracial young adults who also engage in other risk behaviors, such as cigarette smoking and binge drinking.

Mode Matters: Evaluating Response Comparability in a Mixed-Mode Survey

This paper examines the effects of survey mode on patterns of survey response, paying special attention to the conditions under which mode effects are more or less consequential. We use the Youth Participatory Politics survey, a study administered either online or over the phone to 2920 young people. Our results provide consistent evidence of mode effects. The internet sample exhibits higher rates of item non-response and “no opinion” responses, and considerably lower levels of differentiation in the use of rating scales. These differences remain even after accounting for how respondents selected into the mode of survey administration. We demonstrate the substantive implications of mode effects in the context of items measuring political knowledge and racial attitudes. We conclude by discussing the implications of our results for comparing data obtained from surveys conducted with different modes, and for the design and analysis of multi-mode surveys.

AN EVALUATION OF PRIMARY DATA-COLLECTION MODES IN AN ADDRESS-BASED SAMPLING DESIGN.

As address-based sampling becomes increasingly popular for multimode surveys, researchers continue to refine data-collection best practices. While much work has been conducted to improve efficiency within a given mode, additional research is needed on how multimode designs can be optimized across modes. Previous research has not evaluated the consequences of mode sequencing on multimode mail and phone surveys, nor has significant research been conducted to evaluate mode sequencing on a variety of indicators beyond response rates. We conducted an experiment within the Racial and Ethnic Approaches to Community Health across the U.S. Risk Factor Survey (REACH U.S.) to evaluate two multimode case-flow designs: (1) phone followed by mail (phone-first) and (2) mail followed by phone (mail-first). We compared response rates, cost, timeliness, and data quality to identify differences across case-flow design. Because surveys often differ on the rarity of the target population, we also examined whether changes in the eligibility rate altered the choice of optimal case flow. Our results suggested that, on most metrics, the mail-first design was superior to the phone-first design. Compared with phone-first, mail-first achieved a higher yield rate at a lower cost with equivalent data quality. While the phone-first design initially achieved more interviews compared to the mail-first design, over time the mail-first design surpassed it and obtained the greatest number of interviews.

Workshop Synthesis: Comparing and Combining Survey Modes

This paper summarizes the discussions held during an in-depth six-hour workshop on the challenges of combining data from different survey modes with the anticipated aim of identifying current research needs. The main theme of the workshop was mixing survey modes as a way to meet the challenge of low response rates. However, the use of multi-mode surveys introduces new sources of bias: not all households have access to certain survey media (coverage bias); the response rate using one or another of the survey modes is correlated with social demographics (non-response bias); the sampling frame is dependent on the mode (sampling bias) or the instrument itself may affect the responses (measurement bias). The aim of this report is present the workshop's discussion on the identification of research needs with related to combining data from different survey modes.

How Much Do Additional Mailings and Telephone Calls Contribute to Response Rates in a Survey of Medicare Beneficiaries?

Surveys often spend substantial money on multiple mailings and telephone calls to ensure high overall response rates and adequate representation of hard-to-reach demographic subgroups. We examine the extent to which an additional mailing and additional sets of telephone calls are effective in attaining these goals across a variety of subgroups in a large, national multimode survey of Medicare beneficiaries. We also examine the relative data quality of the responses that come with each level of extra effort. We find that additional mailings appear more effective in some groups, while additional telephone calls appear more effective in others. Tailoring the fielding strategy differently by subgroup may improve response rates at a lower cost per complete than using the same fielding protocol for all potential respondents, although data quality is likely to decline with additional efforts in either mode.