Mode Of Survey Administration Research Articles

Overview and Methods for the Youth Risk Behavior Surveillance System - United States, 2023.

The Youth Risk Behavior Surveillance System (YRBSS) is a set of surveys that tracks a broad range of behaviors, experiences, and conditions that can lead to poor health among high school students. The system includes a nationally representative Youth Risk Behavior Survey (YRBS) and separate school-based YRBSs conducted by states, tribes, territories, and local school districts. For the 2023 national YRBS, CDC made changes to the sampling method, survey administration mode, and questionnaire. Specifically, the sampling design added an American Indian or Alaska Native (AI/AN) supplemental sample so that separate, precise estimates could be made for AI/AN high school students, in addition to the usual sample designed to provide nationally representative data for the population of students in grades 9-12. To decrease the time needed to collect and process data, CDC changed the survey administration mode from paper-and-pencil scannable booklets to a tablet-based electronic survey. To provide national data on topics of emerging interest, CDC added new questions to the questionnaire. These new questions assessed social media use, experiences of racism at school, adverse childhood experiences, transgender identity, consent for sexual contact, and unfair discipline at school. Public health practitioners and researchers can use YRBSS data to examine the prevalence of youth health behaviors, experiences, and conditions; monitor trends; and guide interventions. This overview report describes 2023 YRBSS survey methodology, including sampling, data collection, data processing, weighting, and data analyses. The 2023 YRBS participation map, survey response rates, and a detailed examination of student demographic characteristics are included in this report. During 2023, in addition to the national YRBS, 68 site-level surveys were administered to high school students in 39 states, three tribal governments, five territories, and 21 local school districts. These site-level surveys use site-specific questionnaires that are similar to the national YRBS questionnaire but are modified to meet sites' needs. This overview and methods report is one of 11 featured in this MMWR supplement, which reports results from the 2023 national YRBS but does not include data from the 68 site-level surveys. Each report is based on data collected using methods presented in this overview report. A full description of YRBSS results and downloadable data are available (https://www.cdc.gov/yrbs/index.html).

Recruiting people with selected citizenships for the health interview survey GEDA Fokus throughout Germany: evaluation of recruitment efforts and recommendations for future research

BackgroundGermany is the second most common country of immigration after the US. However, people with own or familial history of migration are not represented proportionately to the population within public health monitoring and reporting. To bridge this data gap and enable differentiated analyses on migration and health, we conducted the health interview survey GEDA Fokus among adults with Croatian, Italian, Polish, Syrian, or Turkish citizenship living throughout Germany. The aim of this paper is to evaluate the effects of recruitment efforts regarding participation and sample composition.MethodsData collection for this cross-sectional and multilingual survey took place between 11/2021 and 5/2022 utilizing a sequential mixed-mode design, including self-administered web- and paper-based questionnaires as well as face-to-face and telephone interviews. The gross sample (n = 33436; age range 18–79 years) was randomly drawn from the residents’ registers in 120 primary sampling units based on citizenship. Outcome rates according to the American Association for Public Opinion Research, the sample composition throughout the multistage recruitment process, utilization of survey modes, and questionnaire languages are presented.ResultsOverall, 6038 persons participated, which corresponded to a response rate of 18.4% (range: 13.8% for Turkish citizenship to 23.9% for Syrian citizenship). Home visits accounted for the largest single increase in response. During recruitment, more female, older, as well as participants with lower levels of education and income took part in the survey. People with physical health problems and less favourable health behaviour more often took part in the survey at a later stage, while participants with symptoms of depression or anxiety more often participated early. Utilization of survey modes and questionnaire languages differed by sociodemographic and migration-related characteristics, e.g. participants aged 50 years and above more often used paper- than web-based questionnaires and those with a shorter duration of residence more often used a translated questionnaire.ConclusionMultiple contact attempts, including home visits and different survey languages, as well as offering different modes of survey administration, increased response rates and most likely reduced non-response bias. In order to adequately represent and include the diversifying population in public health monitoring, national public health institutes should tailor survey designs to meet the needs of different population groups considered hard to survey to enable their survey participation.

Measuring Cognitive Function In-Person and Remotely in Round 4 of the National Social Life, Health, and Aging Project (NSHAP).

This paper describes the changes made to the collection of cognitive measures when the National Social Life, Health, and Aging Project (NSHAP) introduced remote modes of data collection. In Round 4 (2021-23), the longitudinal study transitioned from being conducted in-person to collecting data via multiple modes including in-person and remote modes: web, phone, and paper-and-pencil. The team began with the measures used in Rounds 2 and 3 of NSHAP-the survey-adapted Montreal Cognitive Assessment (MoCA-SA)-and evaluated which measures could be administered remotely, introducing new measures for each cognitive subdomain, as needed, to compensate for items that could not be administered remotely. Cognitive items used in Rounds 2 and 3 that could not be administered remotely were dropped from the respective modes, and items selected from the Rush Alzheimer's Disease Center's (RADC) global cognition battery were added as substitutes. For comparison, the RADC substitute items were added to the in-person mode making it longer in R4. The changes in cognitive measures resulted in different numbers of cognitive items across the four modes of survey administration in Round 4. Analysts should be aware of these changes when creating a single global cognition score for the entire NSHAP sample in Round 4, and aware that there may be mode effects that could impact cognition scores.

SA53 Does Mode Matter? Impact of Survey Administration Mode on Health-Related Contingent Valuation: Face-to-Face and Telephone Interviews

SA53 Does Mode Matter? Impact of Survey Administration Mode on Health-Related Contingent Valuation: Face-to-Face and Telephone Interviews

Reach, Acceptability, and Patient Preferences of a Mobile Health-Based Survey to Assess COVID-19 Vaccine Hesitancy Among Patients Receiving Dialysis

Rationale & ObjectiveThe majority of patients with kidney failure receiving dialysis own mobile devices, but the use of mobile health (mHealth) technologies to conduct surveys in this population is limited. We assessed the reach and acceptability of a short message service (SMS) text message-based survey that assessed COVID-19 vaccine hesitancy among patients receiving dialysis. Study DesignCross-sectional SMS-based survey in January 2021. Setting& Participants: Patients receiving in-center hemodialysis, peritoneal dialysis, or home hemodialysis in a non-profit dialysis organization in New York City. Outcomes1) Reach of the SMS survey, 2) Acceptability using the 4-item Acceptability of Intervention Measure (AIM), and 3) Patient preferences for modes of survey administration. Analytical ApproachWe used Fisher’s exact tests and multivariable logistic regression to assess sociodemographic and clinical predictors of SMS survey response. Qualitative methods were used to analyze open-ended responses capturing patient preferences. ResultsAmong 1008 patients, 310 responded to the SMS survey (response rate 31%). In multivariable adjusted analyses, participants who were age 80 and above (aOR 0.49, 95% CI 0.25 – 0.96) were less likely to respond to the SMS survey compared with those aged 18 to 44. Non-Hispanic Black (aOR 0.58, 95% CI 0.39 – 0.86), Hispanic (aOR 0.31, 95% CI 0.19 – 0.51), and Asian or Pacific Islander (aOR 0.46, 95% CI 0.28 – 0.74) individuals were less likely to respond compared with non-Hispanic White participants. Participants residing in census tracts with higher Social Vulnerability Index (SVI), indicating greater neighborhood-level social vulnerability, were less likely to respond to the SMS survey (5th vs. 1st quintile aOR 0.61, 95% CI 0.37 – 0.99). Over 80% of a sample of survey respondents and non-respondents completely agreed or agreed with the Acceptability of Intervention Measure. Qualitative analysis revealed 4 drivers of patient preferences for survey administration: 1) convenience (subtopics: efficiency, multi-tasking, comfort, synchronicity); 2) privacy; 3) interpersonal interaction; and 4) accessibility (subtopics: vision, language, fatigue). LimitationsGeneralizability, length of survey. ConclusionsAn SMS text message-based survey had moderate reach among patients on dialysis and was highly acceptable, but response rates were lower in older (age ≥ 80), non-White individuals, and those with greater neighborhood-level social vulnerability. Future research should examine barriers and facilitators to mHealth among patients on dialysis to ensure equitable implementation of mHealth-based technologies.

Survey Mode and Item Non-Substantive Response: Insights from a Replication Study and Other Recent Trials

This is a replication of a Survey Practice article that was published in September 2022 that explored survey administration mode and non-substantive response. It revisits the findings from the first study using a randomized controlled design. In this study, we found that on average phone respondents provided more non-substantive responses than web survey respondents, though the portion of phone and web respondents with one or more non-substantive response was similar. Another paper we published in Survey Practice in 2023 also explored mode effects with random assignment to text-to-web and phone groups. Thus, this is the third article by the authors on this topic released in Survey Practice. As part of this paper, we discuss findings from all three articles and contextualize them in relationship to other research on item non-substantive response and survey mode (specifically web and phone).

Does parent report of child physical abuse and supervisory neglect differ by method of study recruitment or mode of survey administration?

AbstractGeneral population studies find that rates of child maltreatment are higher than those identified in the child welfare system. Some modes of administration may result in higher disclosure rates of child maltreatment by respondents. This study assesses differences in characteristics of parents who are recruited using telephone sampling techniques or via the internet and whether the survey was administered by telephone or the web. We have 1599 parent respondents recruited from listed samples and Craigslist advertisements. Surveys were administered via the telephone or online. Survey included measures of parenting behaviours, psychosocial characteristics and demographics for the parent, child and family. Data were analysed using logit multilevel models. We found no differences between mode of administration and our child maltreatment and parenting measures, including physical abuse, corporal punishment and supervisory neglect. Recruitment strategy was not related to leaving a child home alone and corporal punishment. However, we did find that parents recruited by Craigslist use physical abuse more frequently. That we found no difference in use of punitive and neglectful parenting practices by survey mode of administration and recruitment method provides some degree of confidence that survey methods other than random‐digit‐dialling telephone surveys may result in generalizable samples in this age of declining landline use.

Does a digitalized single-mode survey lead to similar participation as a mixed-mode survey?

Abstract Background Choice of survey administration mode in surveys mapping public health may influence the participation rates and hence the generalizability of the results. The aim of the study was, in a randomized design, to examine whether three digital invitation letters (i.e., single-mode administration) lead to a similar participation as two digital letters and three physical letters (mixed-mode administration) mailed with a shorter time interval between reminders. Methods The study utilizes data on 10,000 individuals (≥20 years) who participated in The Danish Capital Region Health Survey in 2017 and were re-invited in 2021. Citizens who were registered to receive digital mail from public authorities were randomized into two groups; the intervention group that received three digital letters (day 0, 21, 46; N = 4,745), and the control group that received two digital letters (day 0 and 7) and three physical letters (day 23, 44, 68; N = 4,744). Participation rate in the intervention and control group was compared after the entire period. Furthermore, the two groups were compared in terms of increase in participation rate between first reminder and 14 days later, as well as after second reminder and 19 days later. Results The overall response rate was 69%, divided into 61% in the intervention group and 78% in the control group. After the first reminder, the response rate increased from 39% to 53% in the intervention group and from 34% to 53% in the control group, while the response rate after the second reminder increased from 54% to 60% in the intervention group and from 53% to 68% in the control group. Conclusions A digitalized single-mode survey with three digital letters led to a significantly lower participation rate compared with a mixed-mode survey with two digital and three physical letters. The results can create a better basis for choice of survey administration mode in future surveys. Key messages • Mixed-mode survey administration with multiple reminders leads to significantly higher response rates compared with single-mode surveys with fewer reminders. • Although it is more expensive and time-consuming, mixed-mode survey administration with multiple reminders should be considered in surveys to increase generalizability of data.

The effect of survey administration mode on youth mental health measures: Social desirability bias and sensitive questions

AimResearch on trends in youth mental health is used to inform government policy and service funding decisions. It often uses interviewer-administered surveys, which may be affected by mode effects related to social desirability bias. This study sought to determine the impact of survey administration mode on mental health measures, comparing mode effects for sensitive mental health measures (psychological distress and wellbeing) and non-sensitive (physical activity) measures. MethodsData were from two large national community samples of young Australians aged 12–25 years conducted in 2020 (N = 6238) and 2022 (N = 4122), which used both interviewer-administered and self-report modes of data collection. ResultsResults showed participants reported lower psychological distress and higher wellbeing in the interviewer-assisted compared with the self-report mode. No mode effects were found for the non-sensitive physical activity measures. No interaction between mode and gender was found, but an age group by mode interaction revealed that those in the 18–21 and 22–25-year age groups were more strongly affected than younger adolescents. ConclusionsThese findings suggest underestimates of mental health issues from interview survey formats, particularly for young adults. The results show how even a weak mode effect can have a large impact on mental health prevalence indicators. Researchers and policy makers need to be aware of the impact social desirability bias can have on mental health measures and consider taking steps to mitigate this effect.

A Web-Based Audio Computer-Assisted Self-interview Application With Illustrated Pictures to Administer a Hepatitis B Survey Among a Myanmar-Born Community in Perth, Australia: Development and User Acceptance Study.

Self-administered paper or electronic surveys can create accessibility issues for people with language barriers and limited literacy, whereas face-to-face interviews can create privacy issues and give rise to reporting biases, particularly in the context of sensitive subject matters. An audio computer-assisted self-interview (ACASI) offers an alternative mode of survey administration, and its use has been tested against other survey modes to determine whether the presence of a background narration helps overcome literacy and privacy issues. There are still gaps with the ACASI survey administration because audio narration alone does not assist respondents with limited literacy in choosing response options. To overcome literacy issues, a few studies have used illustrated pictures for a limited number of response options. This study aimed to illustrate all the questions and response options in an ACASI application. This research is part of a larger study comparing different modes of survey administration (ACASI, face-to-face interviews, and self-administered paper surveys) to collect data on hepatitis B knowledge, attitudes, and practices among the Myanmar-born community in Perth, Australia. This study describes the 2-phase process of developing a web-based ACASI application using illustrated pictures. The first phase was the preparation of the ACASI elements, such as questionnaire, pictures, brief descriptions of response options, and audio files. Each element was pretested on 20 participants from the target population. The second phase involved synchronizing all the elements into the web-based ACASI application and adapting the application features, in particular, autoplay audio and illustrated pictures. The preprototype survey application was tested for user acceptance on 5 participants from the target population, resulting in minor adjustments to the display and arrangement of response options. After a 12-month development process, the prototype ACASI application with illustrated pictures was fully functional for electronic survey administration and secure data storage and export. Pretesting each element separately was a useful approach because it saved time to reprogram the application at a later stage. Future studies should also consider the participatory development of pictures and visual design of user interfaces. This picture-assisted ACASI survey administration mode can be further developed and used to collect sensitive information from populations that are usually marginalized because of literacy and language barriers.

MODE EFFECTS ON COGNITIVE FUNCTIONING ASSESSMENTS IN THE HEALTH AND RETIREMENT STUDY

Abstract As the population of the US ages, there is interest in assessing health conditions associated with age and longevity, such as age-related decline in cognitive functioning. As a result, there is an increased focus on measuring cognitive functioning in surveys of older populations. One challenge relates to conducting comparable measurement across survey modes (e.g., phone vs. web). Compounding this is that mode of survey administration is often not assigned randomly making inter-group comparison more difficult. This paper addresses these issues using a novel experiment embedded within the Health and Retirement Study (HRS). The HRS, a US-based cohort of people over 50, has measured cognition since its inception using both in-person and telephone modes. In 2018, a sample of approximately 3700 respondents was identified as web-eligible based on a prior report of internet access along with other selection criteria. Of these, 60% were randomly selected for the web sample with the remainder serving as controls, assigned to telephone mode for comparison purposes. We deploy techniques from item response theory (IRT) and differential item functioning (DIF) to estimate the difference in cognitive functioning between web and phone respondents in 2018 based on longitudinal cognition data collected prior to 2018. Second, we estimate the overall effect of taking the survey via the web as compared to the phone. Third, we examine item-level variation in the magnitude of the mode effect and suggest possible methods for adjustment to support longitudinal consistency. These results are important in guiding future research that utilizes web-based cognitive measures.

Does survey administration mode relate to non-substantive responses? A comparison of email versus phone administration of a residential utility-sponsored energy efficiency program survey

Program evaluation research often relies heavily on surveys and increasingly self-administered online tools. Often, however, a portion of program participants or customers do not have or do not provide email addresses. This study investigates if the type of contact information that members of a population provide relates to non-substantive responses (choosing "don’t know or “prefer not to answer”). We found evidence that survey administration mode relates to non-substantive responses, even while controlling for demographic factors.

Variation in Hospice Experiences by Care Setting for Patients With Dementia.

Use of hospice care among patients with dementia has been steadily increasing. Our objectives were to characterize quality of hospice care experiences among decedents who had a primary diagnosis of dementia and their caregivers and investigate differences across settings of hospice care. We analyzed Consumer Assessment of Healthcare Providers and Systems (CAHPS) Hospice Survey data from caregiver respondents whose family members received hospice care. Data from 96,845 caregiver respondents whose family members had a primary diagnosis of dementia and died in 2017 or 2018 while receiving hospice care in 2829 hospices. We calculated quality measure scores overall and stratified by setting, adjusting for mode of survey administration and differences in case mix, and examined variability in hospice-level scores among decedents with dementia. Mean quality measure scores ranged from 69.0 (Getting Hospice Care Training) to 90.9 (Getting Emotional Support). Measure scores varied significantly across settings, with caregivers of decedents who received care in a nursing home (NH), acute care hospital (ACH), or assisted living facility (ALF) consistently reporting poorer quality of care. Hospice-level scores varied substantially, with a wide range between the 10th and 90th percentiles of hospice performance (eg, 25 points). There are important opportunities to improve hospice care for patients with dementia and their caregivers, particularly with respect to caregiver training, symptom management, and across all dimensions within the NH, ACH, and ALF settings. Variability in care experiences across hospices, as well as long lengths of stay for those with dementia, highlight the importance of informed and timely hospice referral.

Attitudes toward epilepsy assessed by the SAPE questionnaire in Germany – Comparison of its psychometric properties and results in a web-based vs. face-to-face survey

ObjectiveThe aims of this study were (1) to investigate psychometric properties of a new questionnaire (SAPE, Scales of the Attitudes toward People with Epilepsy) that assesses attitudes toward people with epilepsy (PWE) (2) to compare the effects of mode of survey administration (web-based vs. face-to-face) on attitudes, and (3) to identify predictors of attitudes. MethodsA face-to-face and a web-based survey were performed in Germany. Weighting factors were used to achieve representative samples of the German population. Reliability and validity of the 6 scales of the SAPE (social distance, stereotypes, personal concerns, and emotional reactions differentiated by fear, anger, and pity) were evaluated and compared for both surveys. Epilepsy knowledge was also assessed. General linear models were performed to investigate predictors of attitudes toward PWE including the type of survey. ResultsIn total, 1001 participants of the web-based survey and 1026 participants of face-to-face survey were included. Psychometric analyses indicated satisfactory reliability and validity of the scales and differed only slightly between modes of survey. In both surveys, fears and concerns were more pronounced than stereotypes and social distance. However, mean values of two scales were slightly or moderately higher in the face-to-face survey indicating more negative attitudes toward PWE (p < 0.001). Fewer participants of the face-to-face survey reported personal experience with PWE, claiming to know what to do in case of a seizure and claiming to know that seizures can be treated successfully (p < 0.001). These variables proved to be important predictors of positive attitudes toward PWE, besides demographic factors (e.g. age < 65, female gender). When controlling for them, the differences remained significant only for the scales Social Distance (moderately) and Fear (slightly). SignificanceIn total, psychometric analyses show that web-based surveys using the SAPE may be an alternative to face-to-face surveys to assess attitudes toward PWE. This applies also to the scales Social Distance and Emotional Reactions that allow comparisons with other diseases, e.g. psychiatric disorders. Most scales differ only slightly between survey modes, except social distance. This indicates that single components of attitudes toward PWE may be dependent on the mode of survey or different characteristics of respondents.

Building a Framework for Mode Effect Estimation in United States Presidential Election Polls

Abstract As public confidence in polling has been waning in the wake of recent elections (Narea, N. 2016. After 2016, Can we Ever Trust the Polls Again? New Republic), many researchers have been seeking to diagnose the shortcomings in these data (Gelman, A., and J. Azari. 2017. “19 Things We Learned from the 2016 Election.” Statistics and Public Policy 4 (1): 1–10; Kennedy, C., M. Blumenthal, S. Clement, J. D. Clinton, C. Durand, C. Franklin, K. McGeeney, L. Miringoff, K. Olson, D. Rivers, L. Saad, G. E. Witt, and C. Wlezien. 2018. “An Evaluation of the 2016 Election Polls in the United States.” Public Opinion Quarterly 82 (1): 1–33; Mercer, A., C. Deane, and K. McGeeney. 2016. Why 2016 Election Polls Missed Their Mark. Pew Research Center. Also available at https://www.pewresearch.org/fact-tank/2016/11/09/why-2016-election-polls-missed-their-mark/). One conjecture stems from observed differences between polling results based on the methodological choice between live and non-live modes of survey administration (Enten, H. 2015. The Future of Polling May Depend on Donald Trumps Fate. FiveThirtyEight. Also available at https://fivethirtyeight.com/features/the-future-of-polling-may-depend-on-donald-trumps-fate/). While it has become commonplace to discuss “mode effect” on surveys, it reemerged in the political zeitgeist as the “Shy Trump” supporter hypothesis leading up to the 2016 U.S. Presidential Election (Edsall, T. B. 2016. How Many People Support Trump but Dont want to Admit it. New York Times). Motivated by the conflicting evidence for (Enns, P. K., J. Lagodny, and J. P. Schuldt. 2017. “Understanding the 2016 US Presidential Polls: The Importance of Hidden Trump Supporters.” Statistics, Politics, and Policy 8 (1): 41–63) and against (Coppock, A. 2017. “Did Shy Trump Supporters Bias the 2016 Polls? Evidence from a Nationally-Representative List Experiment.” Statistics, Politics, and Policy 8 (1): 29–40) this hypothesis, we built a complex statistical model that pools together results across multiple pollsters and throughout the election cycle while accounting for the nuances of these data. Specifically, we explored election data for the presence of mode effect using time series with a general additive mixed model (GAMM). We estimated mode effect at state and national levels to perform statistical mode adjustments, which we then compared to observed election results. In this paper, we utilized polling results from the United States Presidential Elections in 2016 (4208 polls) and 2020 (4133 polls). Using these data, we identified spatial trends and areas where mode effect was statistically significant at a 0.05 level. In summary, we make three contributions to the literature on mode effect adjustment in the poll aggregation setting. First, we present a straightforward and flexible statistical approach to estimating mode effect using time series data. In doing so, we help to bridge the gap between theory-focused statistical work and the social sciences. Second, we apply this method to two recent presidential elections, providing insight into the significance of mode effect. Third, we provide evidence for spatial mode effect trends suggesting regional voting behaviors that future scholars can explore.

Stroke survivors\u2019 preferences regarding study participation in rehabilitation research

BackgroundTo pursue high quality research, successful participant recruitment is essential, but recruitment rates are often low. This is specifically true in target populations with impairments, for instance, among stroke survivors. Previous studies focusing on recruitment have mainly relied on information from professionals, and there is therefore a need to contribute with new methodological insights to how potential rehabilitation research participants describe their interest and preferences to participate in research. The purpose of this study was to generate knowledge about stroke survivors’ interest in participating in rehabilitation research, reasons for being interested or not, and preferred forms and foci of rehabilitation interventions. An additional aim was to describe preferences regarding survey administration modes and processes for recruitment to studies.MethodThis cross-sectional study recruited Swedish residents who had sustained a stroke, initially by using advertisement on the National Stroke Association’s website, flyers posted at local occupational and physical therapy offices and at local stroke/senior organization meetings. Secondly, participants were recruited through a local stroke register. The survey, administered either in a paper form returned by postal mail; online or as a phone interview with 128 stroke survivors.ResultsMost of the participants were interested in participating in rehabilitation research, particularly younger persons (p = 0.001) and those closer to stroke onset (p = 0.047). Contribution to research, possibility to try new rehabilitation interventions and meeting others in the same situation were reasons that attracted an interest to participate. Other important aspects were related to motivation, individual needs, as well as how skilled the people who provided the intervention were. Participants preferred group-based programs, and programs focusing on regaining lost functions were highly requested. A majority wanted to be contacted through postal mail (70%) and most of them (90%) used the paper form to respond to the survey.ConclusionsA range of personal and external aspects, including challenges related to digitized administration modes, should be considered to achieve high participation rates in rehabilitation research targeting stroke survivors. The importance of addressing individual needs and prerequisites in an individualized manner should not be underestimated and might be a useful strategy to recruitment success.

Adjusting for Patient Characteristics to Compare Quality of Care Provided by Serious Illness Programs.

Background:To compare serious illness programs (SIPs) using recently developed patient experience measures, adjustment must be made for patient characteristics not under control of the programs.Objectives:To develop a case-mix adjustment model to enable fair comparison of patient experience between SIPs by investigating the roles of patient characteristics, proxy response, and mode of survey administration (mail-only vs. mail with telephone follow-up) in survey responses.Methods:Using survey data from 2263 patients from 32 home-based SIPs across the United States, we fit regression models to assess the association between patient-level variables and scores for seven quality measures (Communication, Care Coordination, Help for Symptoms, Planning for Care, Support for Family and Friends, and two global assessments of care). Characteristics that are not consequences of the care the program delivered were considered as adjustors.Results:Final recommended case-mix adjustors are age, education, primary diagnosis, self-reported functional status, self-rated physical health, self-rated mental health, proxy respondent use, and response percentile (a measure of how soon a person responded compared with others in the same program and mode). Age, primary diagnosis, self-rated mental health, and proxy respondent use had the most impact on program-level scores. We also recommend adjusting for mode of survey administration. We find that up to 12 percent of pairs of programs would have their rankings reversed by adjustment.Conclusions:To ensure fair comparison of programs, scores should be case-mix adjusted for variables that influence patients' reports about care quality, but are not under the control of the program administering care.

Is sex lost in translation? Linguistic and conceptual issues in the translation of sexual and reproductive health surveys

Translated questionnaires are increasingly used in population health research. Nevertheless, translation is often not conducted with the same rigour as the process of survey development in the original language. This has serious limitations and may introduce bias in question relevance and meaning. This article describes and reflects on the process of translating a large and complex sexual and reproductive health survey from English into Simplified Chinese. We interrogated assumptions embedded in taken-for-granted translation practice to locate the sociocultural origins of these assumptions. We discuss how terminology and expression related to sexual and reproductive health may lose their conceptual or linguistic significance during translation in three different ways. Firstly, meanings can be lost in the negotiation of meanings associated with linguacultural and geographical variations of terminology. Secondly, meanings can be lost in the clash between everyday and professional sexual and reproductive health discourses. Thirdly, meanings can be lost due to the design of the source questionnaire and the intended mode of survey administration. We discuss ways to help overcome the unavoidable translation challenges that arise in the process of translating English sexual and reproductive health surveys for migrants from non-English speaking backgrounds.

Adjustment of Patient Experience Surveys for How People Respond.

Patient surveys are the primary tool to measure patient experiences of care. Caution must be taken when analyzing these data, as responses can be influenced by factors that do not reflect the quality of care received. To provide a practical overview of adjusting patient experience survey results to address bias related to patient case-mix, extreme response tendency, and mode of survey administration. We discuss options for adjustment for biases in how people respond to patient experience surveys. Case-mix adjustment (CMA) aims to compare provider performance that would have been observed if all providers had treated the same set of patients by removing the effects of patient characteristics that vary across providers. Extreme response tendency can bias the measurement of the disparities in patient experiences even after typical CMAs, since differences in patients' use of extreme response options may affect patient experience scores when they have a skewed distribution. Survey mode may affect scores for the provider entity being evaluated (eg, hospital) more than CMA if survey mode differs at the provider level. It is best practice to evaluate known source of bias when analyzing patient experience surveys. Failure to adjust for patient case-mix, extreme response tendency, and survey mode in patient experience surveys may lead to erroneous comparisons of providers.

Negligible Effects of the Survey Modes for Patient-Reported Outcomes: A Report From the Childhood Cancer Survivor Study.

This study compared the measurement properties for multiple modes of survey administration, including postal mail, telephone interview, and Web-based completion of patient-reported outcomes (PROs) among survivors of childhood cancer. The population included 6,974 adult survivors of childhood cancer in the Childhood Cancer Survivor Study who completed the Brief Symptom Inventory-18 (BSI-18), which measured anxiety, depression, and somatization symptoms. Scale reliability, construct validity, and known-groups validity related to health status were tested for each mode of completion. The multiple indicators and multiple causes technique was used to identify differential item functioning (DIF) for the BSI-18 items that responded through a specific survey mode. The impact of the administration mode was tested by comparing differences in BSI-18 scores between the modes accounting for DIF effects. Of the respondents, 58%, 27%, and 15% completed postal mail, Web-based, and telephone surveys, respectively. Survivors who were male; had lower education, lower household income, or poorer health status; or were treated with cranial radiotherapy were more likely to complete a telephone-based survey compared with either a postal mail or Web-based survey (all P < .05). Scale reliability and validity were equivalent across the 3 survey options. One, 2, and 5 items from the anxiety, depression, and somatization domains, respectively, were identified as having significant DIF among survivors who responded by telephone (P < .05). However, estimated BSI-18 domain scores, especially depression and anxiety, between modes did not differ after accounting for DIF effects. Certain survivor characteristics were associated with choosing a specific mode for PRO survey completion. However, measurement properties among these modes were equivalent, and the impact of using a specific mode on scores was minimal.